Top all-time · #1656
Hamel Husain
Personal blog · hamel.dev/blog · on HN as hamelsmuaiprogrammingdata
Hamel Husain writes Hamel’s Blog, a personal blog at hamel.dev about ai, programming and data. Since 2022, 43 different Hacker News readers have linked it 63 times, most often “AI Product Needs Evals”. It is still writing: the last post was on 2026-10-20, and you can follow it by feed.
Too few sampled comments (21) to say how often HN recommends it.?
Why HN links it
HN readers link Hamel Husain's posts on LLM evals and LLM-as-judge, plus his inference benchmarks and Dokku deployment guide. Commenters use them as practical references for building AI products.
- Recommended for · 11 comments
- Evals and LLM-as-judge posts called excellent, the best guide seen, and extremely helpful; inference benchmarks the best comparison; landscape post awesome; Dok
Summarised by Claude from a sample of 21 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Notes on applied AI engineering, machine learning, and data science.
Most-linked posts?
- AI Product Needs Evals14 people · 14 mentions · first linked 2024-03-29
- Show me the prompt5 people · 5 mentions · first linked 2024-02-14
- About AI Evals4 people · 4 mentions · first linked 2025-06-30
- Optimizing LLM Latency3 people · 3 mentions · first linked 2023-09-08
- An Open Course on LLMs, Led by Practitioners2 people · 2 mentions · first linked 2024-07-29
- "It's Hard to Eval" Is a Product Smell2 people · 2 mentions · first linked 2026-06-30
- Stop Saying RAG Is Dead2 people · 2 mentions · first linked 2025-07-14
- The Revenge of the Data Scientist2 people · 2 mentions · first linked 2026-03-28
- How to Use Decision Models for Evals2 people · 2 mentions · first linked 2026-10-20
- Dokku: My favorite personal serverless platform2 people · 2 mentions · first linked 2024-08-26
What Hacker News says?
agy is good for those cases where you are willing to put the effort into the harness specifically for a task or family of tasks. The full suite, with evals, monitoring, hooks, custom tools, custom verifiers, etc,. It is not good if you want a "general coding assistant" like codex or claudecode. The reality is that if you optimise a harness for a family of tasks[1], then most of these models give successful output. A…
Comment linking /notes/llm/evals/
on “Gemini 3.8 Flash and 3.8 Flash Cyber”Submitted Dokku: My favorite personal serverless platform as a story · 922 points
HN discussionThe word "simply" is doing a bit too much heavy lifting there - I'd love to see an article about that solution that's as detailed as Hamel's Dokku one https://hamel.dev/blog/posts/dokku/
Comment linking /blog/posts/dokku/
on “Dokku: My favorite personal serverless platform”Submitted About AI Evals as a story · 189 points
HN discussionAbout .ml The first demo Jeremy put out was called "Build Applications For LLMs in Python",¹ as part of the "Mastering LLMs" conference by Hamel Husain and Dan Becker.² (You can see a few PoC demos by the end of that video when Johno takes over, it looks a lot like what Gradio or Streamlit can do). So I think your .ml angle is definitely part of the original ethos of FastHTML (which isn't surprising coming from the…
Comment linking /blog/posts/course/
on “FastHTML – Modern web applications in pure Python”Submitted Show me the prompt as a story · 142 points
HN discussion- open-source vs closed fork of vscode - transparent assistant panel vs opaque composer. you control your own prompts (cf. [0]) - in Zed the assistant panel is "just another editor", which means you can inline-assist when writing prompts. super underrated feature imo - Zed's assistant is pretty hackable as well, you can add slash commands via native Zed extensions [1] or non-native, language-agnostic Context Servers…
Comment linking /blog/posts/prompt/
on “Zed AI”Submitted An Open Course on LLMs, Led by Practitioners as a story · 113 points
HN discussionLots of people can get impressive demos up and running, but if you want to run AI products in production, you're going to have to do system evals. System evals make sure your product is doing what it says on the box with unquantifiable qualities. We wrote a zine on system evals without jargon: https://forestfriends.tech Eugene Yan has written extensively on it https://eugeneyan.com/writing/evals/ Hamel has as well.…
Comment linking /blog/posts/evals/
on “Ask HN: Recommendation for a SWE looking to get up to speed with latest on AI”Submitted Creating a LLM-as-a-Judge That Drives Business Results as a story · 82 points
HN discussionIn the LLM engineering community we call those "evals", and they're critical to deploying useful solutions. More on that here: https://hamel.dev/blog/posts/evals/
Comment linking /blog/posts/evals/
on “Chrome is adding `window.ai` – a Gemini Nano AI model right inside the browser”Submitted Fuck you, show me the prompt as a story · 20 points
HN discussionhere's an awesome post on the landscape https://hamel.dev/blog/posts/prompt/
Comment linking /blog/posts/prompt/
on “Show HN: Fructose – LLM calls as strongly typed functions”Submitted Frequently Asked Questions About AI Evals as a story · 8 points
HN discussionHamel wrote a whole lot more about the "LLM as a judge" pattern (where you use LLMs to evaluate the output of other LLMs) here: https://hamel.dev/blog/posts/llm-judge/
Comment linking /blog/posts/llm-judge/
on “About AI Evals”Submitted ML serving is about optimization and portability as a story · 4 points
HN discussionHere's another article by the same author with more background on AI Evals: https://hamel.dev/blog/posts/evals/ I've appreciated Hamel's thinking on this topic.
Comment linking /blog/posts/evals/
on “About AI Evals”Submitted Claude's new auto eval tool as a story · 3 points
HN discussionThis is the best guide I've seen to the LLM-as-judge pattern: https://hamel.dev/blog/posts/llm-judge/index.html
Comment linking /blog/posts/llm-judge/index.html
on “Principles for production AI agents”Submitted "It's Hard to Eval" Is a Product Smell as a story · 3 points
HN discussionThis is so hard ! I don't yet have a great solution for this myself, but I've been collecting notes about this on my "evals" tag for a while: https://simonwillison.net/tags/evals/ The best writing I've seen about this is from Hamel Husain - https://hamel.dev/blog/posts/llm-judge/ and https://hamel.dev/blog/posts/evals-faq/ are both excellent.
Comment linking /blog/posts/evals-faq/
on “Context engineering”Submitted Why I Stopped Using Nbdev as a story · 3 points
HN discussionHamel Husain hasn't done testing vs llama.cpp, but this still might be of interest (includes mlc which is roughly in line w/ llama.cpp batch=1 perf): https://hamel.dev/notes/llm/inference/03_inference.html He has benchmarks on an A6000 which should be roughly in line w/ a 3090 if you want to compare to my numbers (I test mlc as well, although my 3090 results are slower since I'm testing a llama2-7b @ 4K context and…
Comment linking /notes/llm/inference/03_inference.html
on “Show HN: Ollama for Linux – Run LLMs on Linux with GPU Acceleration”Submitted Stop Saying RAG Is Dead as a story · 3 points
HN discussion> Can anyone direct me towards how to ... make one? https://hamel.dev/blog/posts/evals/ > What are "obvious" things that are important to get right - temperature set to 0? At least ~10 or 20 attempts at the same problem for each llm? LLMs are actually pretty deterministic, so there is no need to do more than one attempt with the exact same data. > Finally, any known/commonly used frameworks to do this, or any toolin…
Comment linking /blog/posts/evals/
on “Thoughts on Evals”Submitted Building an Audience Through Technical Writing: Strategies and Mistakes as a story · 3 points
HN discussionThis is the best comparison I've found that benchmarks the current OSS inference solutions: https://hamel.dev/notes/llm/inference/03_inference.html IME the streaming API in text-generation-inference works fine in production. (Though some of the other solutions may be better). I've used it with Starcoder (15B) and the time-to-first-token / tokens per second all seem quite reasonable out of the box.
Comment linking /notes/llm/inference/03_inference.html
on “Releasing Persimmon-8B”Submitted AI Evals: Everything You Need to Know as a story · 2 points
HN discussionThat's a big topic :D First thing about building AI or LLM powered apps is that 90% of the time you don't need to train models from scratch. LLMs are models that have the world's knowledge, so you can "train them" by prompting them and giving them the right context. Where to start? I think the first thing is to play around with the OpenAI playground to get an idea of what's possible with prompts. You'll find tons of…
Comment linking /notes/llm/officehours/erroranalysis.html
on “What's your AI development path?”Submitted It's Hard to Eval Is a Product Smell as a story · 2 points
HN discussion
