Top all-time · #324
Chris Stucchio
Personal blog · chrisstucchio.com · on HN as yummyfajitasdataeconomicsprogramming
Chris Stucchio's personal blog at chrisstucchio.com covers data, economics and programming. Since 2012, 162 different Hacker News readers have linked it 453 times, most often “Don't use Hadoop when your data isn't that big (2013)”. The last post was in 2022, but the archive is still widely read.
Sometimes recommended · 11 of 40 sampled comments recommend it?
Why HN links it
HN readers link Stucchio's posts mostly as sources against Hadoop for small data and on bandits, A/B testing, HFT and basic income. Some call them worth reading.
- Recommended for · 11 comments
- Called 'articulated this well', 'really interesting', 'well worth a read', 'the most helpful' tutorial, 'this gem'
Summarised by Claude from a sample of 40 HN comments that link here. Each comment is marked recommending, critical or neutral (citing without judging); how this works.
Chris Stucchio
Most-linked posts?
- Don't use Hadoop when your data isn't that big (2013)15 people · 19 mentions · first linked 2015-01-19
- AI ethics, impossibility theorems, and tradeoffs [pdf]14 people · 15 mentions · first linked 2018-10-31
- Alien Intelligences and discriminatory algorithms10 people · 24 mentions · first linked 2016-03-21
- How to lie without statistics edition9 people · 23 mentions · first linked 2016-05-25
- A High Frequency Trader's Apologia, Part 19 people · 12 mentions · first linked 2012-04-17
- A High Frequency Trader's Apology8 people · 9 mentions · first linked 2015-01-02
- Don't use Bandit Algorithms – they probably won't work for you (2015)7 people · 9 mentions · first linked 2015-07-01
- /blog/2012/hft_whats_broken.html6 people · 11 mentions · first linked 2015-02-13
- /blog/2014/why_xkcd_style_graphs_are_important.html6 people · 7 mentions · first linked 2014-12-15
- A High Frequency Trader's Apology, Pt 25 people · 6 mentions · first linked 2012-04-26
What Hacker News says?
I did some arithmetic on this a while back. Paying BI to people who work utterly destroys any cost advantage. https://www.chrisstucchio.com/blog/2013/basic_income_vs_basi... This is why most BI proponents are so averse to arithmetic.
Comment linking /blog/2013/basic_income_vs_basic_job.html
on “Moving Forward on Basic Income”Submitted Don't use Hadoop when your data isn't that big as a story · 711 points
HN discussionThis is very misleading reporting. First: All studies so far show a pretty consistent ~10% work disincentive. This is what all the detractors say when they say it disincentivizes work. So how about this one? From actually reading the study's conclusion: > Slightly less than one-fifth were employed before but unemployed during the pilot (17%) So even worse than what we've seen so far. 17% dropping out of the labor ma…
Comment linking /blog/2019/basic_income_reduces_employment.html
on “People kept working, became healthier while on basic income: report”Submitted Don't use Hadoop when your data isn't that big (2013) as a story · 598 points
HN discussionA similar article that articulated this well. https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html
Comment linking /blog/2013/hadoop_hatred.html
on “Command-line tools can be faster than your Hadoop cluster”Submitted A High Frequency Trader's Apologia, Part 1 as a story · 375 points
HN discussionThere's good reason to use it for a beta build (or an alpha) - it changes the type of feedback you get. While my google-fu is failing me as I write this, there was a blog post I read long ago about a making things look not done (I want to say it was in the context of an AWT look and feel). Related: xkcd-style Plots https://mathematica.stackexchange.com/questions/11350/xkcd-s... ( https://news.ycombinator.com/item?id…
Comment linking /blog/2014/why_xkcd_style_graphs_are_important.html
on “UI elements with a hand-drawn, sketchy look”Submitted Why xkcd-style graphs are important as a story · 296 points
HN discussionDo you have any evidence that guns cause any significant amount of harm? I've never seen studies that conclude much of anything. Also, there are many things significantly more dangerous than guns. Should we ban those things too? If not, why not? https://www.chrisstucchio.com/blog/2016/are_gays_or_guns_mor...
Comment linking /blog/2016/are_gays_or_guns_more_dangerous.html
on “Ask HN: What is the biggest untapped opportunity for startups?”Submitted A High Frequency Trader's Apology, Pt 2 as a story · 274 points
HN discussionYou have been arguing against basic income for years now, and with some really good points[1], but I think you consistently miss the main attraction of the idea to the HN crowd. Many here believe that if people weren't forced to work for income, they would be doing things that are more valuable but also more difficult to monetize (at least initially but perhaps generally), like open source development. There is an u…
Comment linking /blog/2013/basic_income_vs_basic_job.html
on “A Basic Income Should Be the Next Big Thing”Submitted The Mathematics of Paul Graham's Bias Test as a story · 243 points
HN discussionSo I'm a blogger with posts that occasionally go semi-viral [1], and I've been subject to unpleasant comments on a few occasions. Here are a couple of posts which have gotten me quite a bit of hate: http://www.chrisstucchio.com/blog/2013/basic_income_vs_basic... http://www.chrisstucchio.com/blog/2014/equal_weights.html In both cases they are posts about mathematics which use other topics (economics, dating) as concr…
Comment linking /blog/2014/equal_weights.html
on “"Open Source is awful in many ways, and people should be aware of this"”Submitted My adventures in medical tourism as a story · 210 points
HN discussionReminds me of this gem from the previous decade: https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html (Don't use Hadoop - your data isn't that big) Which, of course, was discussed on HN back then: https://news.ycombinator.com/item?id=6398650
Comment linking /blog/2013/hadoop_hatred.html
on “Big data is dead (2023)”Submitted Why Multi-armed Bandit algorithms are superior to A/B testing (with Math) as a story · 209 points
HN discussionIt's simply not possible to "short ahead of you". If I place an order via ETrade an HFT doesn't know about it until it hits their FIX/ITCH/OUCH feed (i.e., after it's already on the order book and possibly after it executed). They cannot jump ahead of you except by offering a better price. I wrote a blog post a while back that explains the mechanics of matching engines, you might find it helpful: http://www.chrisstu…
Comment linking /blog/2012/hft_apology.html
on “For the Love of Money”Submitted One hiring filter that works as a story · 208 points
HN discussionBasic job might have a better cost/benefit tradeoff. For example, https://www.chrisstucchio.com/blog/2013/basic_income_vs_basi...
Comment linking /blog/2013/basic_income_vs_basic_job.html
on “The rise of the pointless job”Submitted Spaced Repetition is Awesome as a story · 188 points
HN discussionOf course, in reality this bias can be corrected for. I don't know if specific crime-stat software does it, but it's certainly doable. Here's an example where I solve literally the same problem (better measurement => more events detected) in a different scenario. Is it remotely controversial that I can do this for a sensor network? https://www.chrisstucchio.com/blog/2016/delayed_reactions.ht... You might also be int…
Comment linking /blog/2016/delayed_reactions.html
on “Deep-Fried Data”Submitted Microservices for the Grumpy Neckbeard as a story · 144 points
HN discussionOf course data contains biases. But again, please read the article I linked; algorithms will have a tendency to correct that bias. The examples in the article you link to are not algorithmic bias at all. They consist of: 1) Humans at Facebook manipulating trending results. 2) Google's keyword algorithm (accurately) reflecting the fact that people with black names are more likely to have arrest records. Lets distingu…
Comment linking /blog/2016/delayed_reactions.html
on “Deep-Fried Data”Submitted Don't use Scatterplots as a story · 135 points
HN discussionThis is simply not true. Most algorithms can and will correct for biases in their inputs. See this (somewhat technical) article where I go into explicit (simulations in numpy) levels of detail: https://www.chrisstucchio.com/blog/2016/alien_intelligences_... The best analogy I've come up with for the non-technical is that algorithms are like humans trying to draw inferences on octopus society. Some octopi might have…
Comment linking /blog/2016/alien_intelligences_and_discriminatory_algorithms.html
on “Deep-Fried Data”Submitted How to leave academia (for science PhDs) as a story · 113 points
HN discussionWingify Labs | Pune (Kalyani Nagar) and Delhi (Pitampura), India. Onsite Wingify (wingify.com) is the company behind Visual Website Optimizer (vwo.com). Wingify Labs is focused on building new worldclass products, both SAAS and consumer facing. If you want to level up your skills and build things you've never thought you could build, this is the place for you. We've got a lot of projects cooking - from a SAAS focusi…
Comment linking /blog/2016/bayesian_calibration_of_mobile_phone_compass.html
on “Ask HN: Who is hiring? (May 2016)”Submitted Modelling a Basic Income with Python and Monte Carlo Simulation as a story · 111 points
HN discussionHere are 2 good articles that follow up on the arguments presented by VWO in that article. From the first link below: "They do make a compelling case that A/B testing is superior to one particular not very good bandit algorithm, because that particular algorithm does not take into account statistical significance. However, there are bandit algorithms that account for statistical significance." * https://www.chrisstu…
Comment linking /blog/2015/dont_use_bandits.html
on “20 lines of code that beat A/B testing (2012)”Submitted A Fervent Defense of Front-running HFTs as a story · 107 points
HN discussion
