Top all-time · #373
colah's blog
Personal blog · colah.github.io · on HN as colah3aimathscience
colah's blog is a personal blog at colah.github.io about ai, math and science. Since 2014, 146 different Hacker News readers have linked it 178 times, most often “Understanding LSTM Networks (2015)”. The last post was in 2019, but the archive is still widely read.
Often recommended · 20 of 40 sampled comments recommend it?
Why HN links it
HN readers link colah's blog for its visual explanations of neural networks, manifolds, LSTMs and information theory, often as an introduction to machine learning.
- Recommended for · 20 comments
- Commenters call its posts fantastic, awesome and very interesting, and praise the visualisations.
Summarised by Claude from a sample of 40 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Understanding LSTM Networks (2015)14 people · 14 mentions · first linked 2015-08-27
- Deep Learning, NLP, and Representations13 people · 13 mentions · first linked 2014-07-08
- Home – colah's blog13 people · 13 mentions · first linked 2015-03-23
- Neural Networks, Types, and Functional Programming11 people · 12 mentions · first linked 2015-09-03
- Neural Networks, Manifolds, and Topology11 people · 11 mentions · first linked 2014-04-09
- Visual Information Theory (2015)9 people · 11 mentions · first linked 2016-07-12
- Neural Networks, Manifolds, and Topology (2014)9 people · 9 mentions · first linked 2014-04-11
- Understanding LSTM Networks (2015)8 people · 8 mentions · first linked 2019-12-09
- Visual Information Theory8 people · 8 mentions · first linked 2015-10-14
- Neural Networks, Types, and Functional Programming – Colah's Blog8 people · 8 mentions · first linked 2016-05-10
What Hacker News says?
I haven’t done much with neural networks, but Christopher Olah wrote a fantastic post on visualizing low-dimensional ones: http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
Comment linking /posts/2014-03-NN-Manifolds-Topology/
on “Visualizing Algorithms”Submitted Visual Information Theory (2015) as a story · 543 points
HN discussionHi! I lead interpretability research at Anthropic. I also used to do a lot of basic ML pedagogy ( https://colah.github.io/ ). I think this post and its children have some important questions about modern deep learning and how it relates to our present research, and wanted to take the opportunity to try and clarify a few things. When people talk about models "just predicting the next word", this is a popularization o…
Comment linking /
on “Tracing the thoughts of a large language model”Submitted I've decided to move on to Distill as a story · 353 points
HN discussionVarious announcements: Google Research: https://research.googleblog.com/2017/03/distill-supporting-c... DeepMind: https://deepmind.com/blog/distill-communicating-science-mach... OpenAI: https://openai.com/blog/Distill/ YC Research: http://blog.ycombinator.com/distill-an-interactive-visual-jo... Chris Olah: http://colah.github.io/posts/2017-03-Distill/
Comment linking /posts/2017-03-Distill/
on “Distill: a modern machine learning journal”- cf · 2014-04-09
Submitted Neural Networks, Manifolds, and Topology as a story · 263 points
HN discussion Neural networks are a relatively simple mathematical model. They don't actually "think" or have a conscience. Neural networks are also regularly fed books, in order to model some properties of natural language. Here's a good introduction: http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...
Comment linking /posts/2014-07-NLP-RNNs-Representations/
on “Inceptionism: Going Deeper into Neural Networks”Submitted Neural Networks, Types, and Functional Programming as a story · 218 points
HN discussionAlthough probably not sufficient for AGI, network architecture is essentially guaranteed to be important, because of both ample empirical evidence of the importance of architectures and ample reason, from facts about numerics, to believe that it is important. In the first category (empirical evidence), - The discrete leap from non-LSTM RNN to LSTM network performance on NLP was essentially due to a "better factoring…
Comment linking /posts/2015-09-NN-Types-FP/
on “Deep image prior 'learns' on just one image”Submitted Do I need to go to university? as a story · 217 points
HN discussionI'm glad to see the shout out to http://colah.github.io/ (even if the linked post is described as "monster stories")
Comment linking /
on “Learn TensorFlow and deep learning, without a Ph.D.”Submitted Neural Networks, Types, and Functional Programming (2015) as a story · 179 points
HN discussionNot the author, but I wrote an article introducing conv nets you might find helpful: http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ I also have an article on RNNs, although it's focused on explaining a special version, called an LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ If you have experience with functional programming, you might find this a nice way to think about Conv Nets/RNNs/etc:…
Comment linking /posts/2015-09-NN-Types-FP/
on “Neural Networks Demystified”Submitted Calculus on Computational Graphs: Backpropagation (2015) as a story · 170 points
HN discussionWe need more people teaching math through visual intuition. Life a friend of mine said, "if you want to do computation fast, phrase it as a problem for your GPU, er, visual cortex". Here is a tool you can play with to visualize this: http://playground.tensorflow.org If you liked this video, try this different visual intuition of what a neural network does that I find even better: http://colah.github.io/posts/2014-03…
Comment linking /posts/2014-03-NN-Manifolds-Topology/
on “What is backpropagation and what is it doing? [video]”Submitted Visual Information Theory as a story · 151 points
HN discussionThe following is a concise and good explanation of necessary knowledge of information theory: http://colah.github.io/posts/2015-09-Visual-Information/
Comment linking /posts/2015-09-Visual-Information/
on “Ask HN: What maths are critical to pursuing ML/AI?”Submitted Understanding Convolutions (2014) as a story · 129 points
HN discussionLove this article. It makes statistics enjoyable and accessible. Most of Olah's old stuff is also really good, especially the one on manifolds and neural networks [1] [1] https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
Comment linking /posts/2014-03-NN-Manifolds-Topology/
on “Visual Information Theory (2015)”Submitted Neural Networks, Manifolds, and Topology (2014) as a story · 129 points
HN discussionincredible guy, he really is the 10x everything type: https://colah.github.io/cv.pdf
Comment linking /cv.pdf
on “Visual Information Theory (2015)”Submitted Deep Learning, NLP, and Representations as a story · 122 points
HN discussionVanishing (or exploding) gradients affected all deep architectures, not just RNNs. They were solved by LSTMs first proposed in 1997. See: https://www.semanticscholar.org/paper/Long-Short-Term-Memory... I find it interesting that this knowledge seems to be all but forgotten now. Back in the day, ca. 2014, LSTMs were all the rage, e.g. see: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://colah.github.…
Comment linking /posts/2015-08-Understanding-LSTMs/
on “Were RNNs all we needed?”Submitted Understanding LSTM Networks (2015) as a story · 106 points
HN discussionSince this post is based on my 2014 blog post ( https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ ), I thought I might comment. I tried really hard to use topology as a way to understand neural networks, for example in these follow ups: - https://colah.github.io/posts/2014-10-Visualizing-MNIST/ - https://colah.github.io/posts/2015-01-Visualizing-Representa... There are places I've found the topological pe…
Comment linking /posts/2015-01-Visualizing-Representations/
on “Deep Learning Is Applied Topology”Submitted Understanding LSTM networks as a story · 99 points
HN discussionI say this as someone who has been in deep learning for over a decade now: this is pretty wrong, both on the merits (data obviously lives on a manifold) and on its applications to deep learning (cf chris olah's blog as an example from 2014, which is linked in my post -- https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ ). Embedding spaces are called 'spaces' for a reason. GANs, VAEs, contrastive losses --…
Comment linking /posts/2014-03-NN-Manifolds-Topology/
on “Deep Learning Is Applied Topology”Submitted Neural Networks, Manifolds, and Topology (2014) as a story · 98 points
HN discussionNo, you don't need the degree. Denny Britz, He drops school (his phd) to go into industries. Christopher Olah: Same story, at least he doesn’t communicate about his education ( http://colah.github.io/cv.pdf ). Both are well-known for their expertise, and followed by a large amount of people in the deep learning community. You can def become an great machine learning engineer who used deep learning technique, without…
Comment linking /cv.pdf
on “Carnegie Mellon Launches Undergraduate Degree in Artificial Intelligence”Submitted Visualizing Representations: Deep Learning and Human Beings as a story · 76 points
HN discussion
