Top all-time · #1389
phiresky's blog
Personal blog · phiresky.github.io/blog · on HN as phireskyprogrammingwebdata
phiresky's blog is a personal blog at phiresky.github.io about programming, web and data. Since 2019, 50 different Hacker News readers have linked it 75 times, most often “Hosting SQLite databases on GitHub Pages or any static file hoster”. It is still writing: the last post was on 2025-01-31, and you can follow it by feed.
Sometimes recommended · 11 of 40 sampled comments recommend it?
Why HN links it
HN readers link this blog for its technique of hosting read-only SQLite databases on static file hosts using HTTP range requests. Also linked are TypeScript SQL library comparisons and a book visualization.
- Recommended for · 11 comments
- Called absolutely brilliant and the neatest SQLite-over-HTTP approach; a TypeScript SQL libraries overview was called very helpful.
Summarised by Claude from a sample of 40 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Most-linked posts?
- Hosting SQLite databases on GitHub Pages or any static file hoster29 people · 37 mentions · first linked 2021-05-02
- SQLite Performance Tuning6 people · 6 mentions · first linked 2022-04-16
- Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz5 people · 6 mentions · first linked 2019-06-16
- SQLite-ZSTD: Transparent dictionary-based row-level compression for SQLite5 people · 5 mentions · first linked 2022-07-31
- Visualizing all books of the world in ISBN-Space3 people · 4 mentions · first linked 2025-02-01
- An overview of typed SQL libraries for TypeScript (2020)2 people · 7 mentions · first linked 2021-02-22
What Hacker News says?
If you just do an occasional key/value lookup, you don't need 1.2 MiB of WebAssembly. [1] That might already exceed your total database size. I'd solve it via sharding: divide the database into N pieces via range- or hash-sharding. [1] Choose an N that's large enough for each piece to be reasonably small. When you look up a key, fetch the shard of interest. You can put each piece into separate files (a little simple…
Comment linking /youtube-sponsorship-stats/sql-wasm-dd22a976e2e149886e6e.wasm
on “Hosting SQLite databases on GitHub Pages or any static file hoster”Submitted Hosting SQLite databases on GitHub Pages or any static file hoster as a story · 1812 points
HN discussionYeah, that was one of the inspirations for this. That one does not work in the browser though, would be a good project to do that same thing but with sqlite in wasm and integrated with WebTorrent instead of a native torrent program. I actually did also implement a similar thing fetching data on demand from WebTorrent (and in turn helping to host the data yourself by being on the website): https://phiresky.github.io/…
Comment linking /tv-show-ratings/
on “Hosting SQLite databases on GitHub Pages or any static file hoster”Submitted Rga: Ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz as a story · 684 points
HN discussionIt's the same trick that was described here - it's absolutely brilliant: https://phiresky.github.io/blog/2021/hosting-sqlite-database...
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Show HN: Static.wiki – read-only Wikipedia using a 43GB SQLite file”Submitted Visualizing all books of the world in ISBN-Space as a story · 486 points
HN discussionDon't say server-"less" if you mean in-process (which may well run on a server). Here's Sqlite without a server, running in the client and backed by static file hosting: https://phiresky.github.io/blog/2021/hosting-sqlite-database...
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Neon – Serverless Postgres”Submitted Show HN: Rga: ripgrep, but also search in PDFs, Office documents, zip, tar.gz as a story · 284 points
HN discussionI'll start off by quoting the winning submission. Libraries have been trying to collect humanity’s knowledge almost since the invention of writing. In the digital age, it might actually be possible to create a comprehensive collection of all human writing that meets certain criteria. That’s what shadow libraries do - collect and share as many books as possible. One shadow library, Anna’s Archive (which I will not li…
Comment linking /blog/2025/visualizing-all-books-in-isbn-space/
on “Winners of the $10k ISBN visualization bounty”Submitted Hosting SQLite databases on any static file hoster (2021) as a story · 199 points
HN discussionThe winning submission [0] was discussed on HN recently [1]. It's highly impressive from both technical decisions and graphic design viewpoints, it somehow elegantly visualizes 2 billion books (in a way that resembles a bookcase no less). [0]: https://phiresky.github.io/blog/2025/visualizing-all-books-i... [1]: https://news.ycombinator.com/item?id=42897120
Comment linking /blog/2025/visualizing-all-books-in-isbn-space/
on “Winners of the $10k ISBN visualization bounty”Submitted Hosting SQLite Databases on GitHub Pages (2021) as a story · 86 points
HN discussionIt fails on other browsers as well (e.g. Firefox on Linux) and someone has discovered[1] that it may well be a Netlify bug (sending more data than it's supposed to). I agree with your point that that Apple could do better but in this case it does seem that Apple's browser is doing the right thing and ignoring invalid data provided by Netlify (which is the same behaviour as Firefox at the very least). In any case goi…
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Hosting SQLite Databases on GitHub Pages”Submitted Show HN: Url2cite: Transparently convert links to citations in markdown papers as a story · 64 points
HN discussionLooks like Netlify changed something since I wrote this article regarding what headers they send. Detecting support for Range-requests is kinda tricky and relies on heuristics [1]. Not sure why it still works in Chrome though. You can go to this version of my blog, it should work there: https://phiresky.github.io/blog/2021/hosting-sqlite-database... Maybe the link could be updated? Except for the DOM demo, since tho…
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Hosting SQLite Databases on GitHub Pages”Submitted ISBN Visualization as a story · 17 points
HN discussionhttps://casual-effects.com/markdeep/ has a similar idea but different execution. I use something similar for my blog (I don't have a WYSIWYG editor for the widgets though, that's pretty neat): The posts are written in markdown, but then there's a few "widgets" that are basically markdown code blocks with a special "language" tag set that will render a React component with the text content as a input prop. I also use…
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Plain text, with lines”Submitted SQLite-ZSTD: Transparent dictionary-based row-level compression for SQLite as a story · 5 points
HN discussionOh that’s an interesting idea. I saw someone built SQLite over HTTP with the Range header: https://phiresky.github.io/blog/2021/hosting-sqlite-database... So presumably in a similar manner as https://github.com/phiresky/sql.js-httpvfs you could map SQLite pages to leaf torrents too and get the chunking you are looking for.
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Transmission 4.0.0 beta 1”Submitted sqlite-zstd: Transparent dictionary-based row-level compression for SQLite as a story · 5 points
HN discussionIt is admittedly pretty noisy, which is somewhat intentional because the focus was on high data density. Here's an a bit more minimalistic view (less color, only one text level simultaneously): https://phiresky.github.io/isbn-visualization/?dataset=all&g... It could probably be tweaked further to not show some of the texts (the N publishers part), less stuff on hover, etc.
on “Visualizing all books of the world in ISBN-Space”Submitted Rga: Ripgrep, but also search in PDFs, E-Books, Office documents (2022) as a story · 4 points
HN discussionYes! You can go here to have group texts be horizontal, it was just a stylistic choice to better show the layout: https://phiresky.github.io/isbn-visualization/?groupTextVert...
Comment linking /isbn-visualization/?groupTextVertical=false
on “Visualizing all books of the world in ISBN-Space”Submitted SQLite-zstd: Transparent dictionary-based row-level compression for SQLite as a story · 4 points
HN discussionIt's this one: https://phiresky.github.io/blog/2021/hosting-sqlite-database...
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “How to circumvent Sci-Hub ISP block”Submitted rga: ripgrep, but also search PDFs, ebooks, Office docs, zip, tar.gz, etc (2019) as a story · 3 points
HN discussionthis is the one: https://phiresky.github.io/blog/2021/hosting-sqlite-database... HN submission: https://news.ycombinator.com/item?id=27016630
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “How to circumvent Sci-Hub ISP block”Submitted rga: ripgrep, but also search in PDFs, E-Books, Office documents, zip, tar.gz as a story · 3 points
HN discussionIt's probably just a few steps away to finally build an IPFS version of sci-hub. Sadly, I'm not a fan of the site's method of searching the libgen index by using sqlite's partial load feature [1] mainly because of the possible limited available storage issue. [1]: https://phiresky.github.io/blog/2021/hosting-sqlite-database...
Comment linking /blog/2021/hosting-sqlite-databases-on-github-pages/
on “Show HN: Fully-searchable Library Genesis on IPFS”Submitted Hosting SQLite databases on GitHub Pages (2021) as a story · 2 points
HN discussion
