> Let’s say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while.
This is one of my favorite blog posts, and it can basically be encapsulated in the idea of "innovation tokens." It is one of the most useful concepts I have had as a PM / eng leader in my career. It helps actually make the the right tradeoffs, and helps even more in explaining those tradeoffs to colleague of all levels. Highly recommend.
I like the general concept, but I think framing it as a small number of discrete tokens isn't quite right. I'd treat the whole thing in terms of debt and risk. Using "non-boring" technology [0] is really just subtracting some amount from your balance. You don't want your balance to go too negative, but carrying some debt is sometimes fine. And some risky bets might turn out to have a huge ROI!
The amounts clearly aren't discrete. Writing your entire app on a new language runtime might be very risky (and also might have large potential ROI!), but choosing a new email provider might not be (random example, but presuming that you can swap out providers fairly easily).
[0] My bigger complaint is really about the vagueness of even deciding what is "boring." How does a new technology transition from being "non-boring" to being "boring"? Apparently that requires a lot of people to ignore this article's advice for a long time, until we collectively decide that those people have had good enough results to consider that technology "boring."
There was another, much older, post similar to this about, I think, "beans" that engineers use to solve problems. If I remember correctly, it was something like, solving a problem costs "beans", and engineers will always use most all of their "beans" to solve a given problem, because it's somewhat "easy" until you run out of them.
Maybe it wasn't beans? But, I've been looking for it for years.
I’m vaguely recalling something - and I think the beans might’ve also represented vetos, so one had a limited number of objections to other people’s ideas… quick search revealing nothing so far, though.
His example of a web crawler stinks. I can get behind the philosophy, but his specific web crawler example at a minimum needs to retry, respect robots.txt, and rate limit per domain. So no, xargs + curl is a dumb example.
This is where efficiency can be applied in two distinct ways. On the one hand, if using more beans means you can solve the problem more quickly, then it is more efficient to the problem solver to do so.
But if beans are scarce, and that's where you can try to find ways to use less beans per function, then the goal is to make the function more efficient by using less beans.
I wish there was a jobs board for companies that are somehow vetted for this type of engineering culture.
So many jobs are sold as “we’re pragmatist's” and when you show up there’s 5 devs, 50 repos and most of the work is discussing if x requirement should be a new micro service. The product is usually an web app with 10 entities and and an API.
I love this post. It’s also interesting to revisit in the age of agents.
Using the language of the article, I’d say “push all your innovation tokens into agents” is probably a good move. This means the tech your agents work with should all be boring tech.
Another way of saying this is “use in-distribution technology”. If agents are substantially better at Rust than Zig, probably you should use Rust, even if Zig is “better”. The amount that Zig is better is going to get swamped by the amount that in-distribution agents are better.
(This is not a claim that Rust actually is better, just a hypothetical fact pattern for discussion.)
It's a total tangent but AI being so good at developing rust means that my first impression of seeing a rust project has totally changed.
4 or 5 years ago if I had to weigh up two options and one was written in rust it was almost a sure thing that the program was snappy, fast, reliable and that the author was competant.
Now it's a sure fire sign that the project was probably vibe coded. Not saying it can't still be good, I just have a different first impression now
I admit I have a hard time following this logic. If an agent has a baseline intelligence, you can... instead of using boring technology, just DIY/NIH the simplest custom solution which fulfills your requirements. And you'll usually do a better job than a general-purpose software library.
Even if in the other approach, the LLM knows the stack better, you're just going to push the complexity into your client code, so it's not like it's a huge gain to use boring tech stack.
That’s definitely not true, and I’ll give two incredibly common examples:
1: arraylist. A library will probably do better than you. It’s already written and common.
2: webservers. Sure, you can write one yourself with sockets and a bunch of buffers and string management… or, you could use one of the 100s of existing ones. Which one? Probably a stable and established one.
One thing I learned about AI is that, e.g., if you're making websites you're much better off going for php/ruby/elixir, even if you don't like the languages much.
Pipelines and deploys gets much easier and faster than the very common TypeScript monorepo, and so does communication between server and client.
And I say that as a TypeScript and Effect aficionado who has no particular love for neither php or ruby, but they are extremely solid choices to move fast, well, and get excellent performance and tooling out of the box.
Great point! I'd also add that Django's another solid choice for boring tech that LLM agents will know very well and will very likely continue to do so. A fair bit of SWE Bench and other Python benchmarks are Django related tests, which the LLM vendors care very deeply about keeping their scores up. Also, Django's docs are excellent, so strategically pointing an LLM to them in a prompt can often produce great results.
PHP has had a lot of conventions so the training data is all over the place. Ruby is a nightmare in that regard. Can't speak to Elixir but I'm surprised to hear someone say LLMs producing good code in either language.
On the other hand, Go code from 2012 and Go code from 2026 looks virtually the same. Conventions are respected, go fmt is the one single formatter, "use the stdlib" is a popular mantra and the code is readable by design.
If I were to codegen a project I wouldn't use anything but Go at this point.
I've been using elixir professionally and for hobby work for the past 7 years now. Language models are damn good at it, and have been for some time.
The language isn't huge, the documentation has always been very good, and the patterns extremely consistently across the community. I almost wonder whether it's smaller community has been a benefit, as LLMs have been trained on a tighter set of code samples.
Alternatively, if AI is doing most of the work, why not pick a platform that performs? If the cost of the code (and preferences of the developers) are factored out of the equation, pick based on operational cost.
Serving traffic with Ruby and Python is significantly more expensive than serving traffic with Java or Go.
> Pipelines and deploys gets much easier and faster than the very common TypeScript monorepo
Can you elaborate more on the slowness you've seen with deploying TypeScript codebases? My experience is that it's improved significantly over the past decade or so. tsc is still kinda slow for large projects, but the rewrite to Go should improve things significantly. Pre-install/post-install scripts can also be slow, but I've always disabled those and haven't yet run into issues. The JS-written bundlers can also be slow, but I use the non-JS ones (esbuild, parcel, bun, etc.) and they're fast enough for me to not care. The main build bottleneck I can think of for a modern TypeScript project would be build plugins, since those are still written in JS/TS.
Is that what's happening in this example, though? Ditching TypeScript in favor of PHP seems to be a net positive in favor of having a reliable, performant tech stack. From certain perspectives, TypeScript is itself a form of technical debt: writing code in one language just to transpile it into another interpreted language just to add type safety to correct for human error seems like a heavy layer of complexity just to make development teams work faster.
If we get to the point where AI tools are able to consistently able to produce desired results within strict performance and security constraints, without having to make the same tradeoff between delivery velocity and final quality, why would we not have them target the lowest level feasible for implementation, and cut out all of the middleware cruft that makes everything slow and take up ten times the RAM it actually needs to?
I disagree. We've been doing this for two decades already. A lot of monoliths were rewritten in a way that best fit the AWS pricing model of the time with the software itself seemingly being an after afterthought. (I'm not here to discuss whether monoliths or micro services are better, I'm just saying the choice of how they were rewritten was too often mostly for AWS pricing)
I think you're completely misreading what I wrote.
PHP (lets use this as an example) brings already many benefits over a common solution like a TypeScript monorepo mostly operational simplicity. That was already true before AI.
It's boring, very fast and easy to deploy, offers straightforward horizontal scaling, no need to orchestrate containers and/or multiple runtime processes, has excellent html rendering (nothing in JS-land really does), and has very solid framework solutions like Laravel where everything works out of the box.
Those merits existed before AI already.
The issue was that you had to buy into PHP as a language, which was a horrible experience.
But if AI writes most of the code? Suddenly PHP becomes an excellent candidate to choose for many use cases.
If anything, AI makes the choice of programming and languages and software about finding the right tool for the job. Somehow the industry instead keeps vomiting React/Tailwind slop which are the right tool for 1% of the jobs.
I'll push back against this, despite it being so popular. I dislike the arbitrary "innovation tokens" and I think this entire concept really blurs the lines and feels sort of unserious.
Engineers should understand requirements, risks, tradeoffs, and potential gains. New technology may be right for that. Novel approaches may be right for that. "Novel" or "New" are only proxies and they're weak.
For example, I may think "New" means untested, but is that true? What if a new project has Jepsen testing, a fuzzing suite, massive compute running tons of oracle tests, etc? I should just say "Choose well tested" instead of "Choose old" - lots of old software is very poorly tested.
Maybe I think that "Old" implies better documentation, but does it? Lots of older projects have insane cruft and weird edge cases that are undocumented and accumulated over years.
Why do we need a metaphor? Why is "innovation token" helpful?
If you're incapable of evaluating a technology in terms of these properties, you aren't a serious developer and "boring" will not save you.
Sit down, write our your requirements, determine candidate solutions, and choose them based on their fit. "Boring" means nothing, it's a vague proxy term. "Well tsted", "performant for our use case", "developers know it", etc mean something.
> MySQL is boring. Postgres is boring. PHP is boring. Python is boring. Memcached is boring. Squid is boring. Cron is boring.
Literally every one of these has caused hilarious and disastrous failures for me in my career. But yep, boring.
> If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens.
What if you know NodeJS really well? Or MongoDb? What if you have empirical, verifiable reasons for why they fit better?
I'm a bit tired of "simple" and "boring" and other nonsense words in this field taking up the air in the room that should be spent evaluating solutions on their actual merits.
I don't think that context is relevant to my comment. I didn't say "in hindsight, those technologies are great!", I pointed out that "boring" is meaningless, and any meaning you attribute to it like "defined as the tech you know the sharp edges of" is better substituted in.
That is, if someone said two sentences, I would only care about the second one:
1. "We should use this because it is boring"
2. "We should use this because we understand the sharp edges"
I wouldn't care at all about (1) and I'd have a real conversation based on (2).
Any productive conversation that starts with (1) immediately has to follow "can you clarify what that means", so the term is useless at best and thought terminating at worst.
Isn't this somewhat semantical? Boring implies a lot of the things you said, especially if it means your team's experience is largely pooled in a particular dev environment. Assuming most boring tech is ubiquitous, it's probably rare that your team, statistically, is deeply literate in some obscure tech - they most likely are experts in some definition of "boring".
TLDR; #1 and #2 are essentially implying the same thing.
insanitybit's point is that "boring" can imply those things, but it can imply other things too. If you have to bring the correct context to make the word make sense, and there's other contexts that could apply that don't make sense, the word isn't helping. You bringing the correct context is doing the heavy lifting.
I guess context clues and the way the post was written had obvious implications - that was my point. I do understand it leaves a little open to interpretation, which is the angle he decided to comment on.
I see innovation as a guardrail against CV driven development. More, I think you need to consider the context of when this was written. It was a period of rapid innovation/evolution - I remember more than a handful of projects failing (either undelivered or rewritten well under their expected lifecycle) around this time because teams had taken bets on new tech either they didn't know how to use well or the tech didn't take off and was a dead end.
> What if you know NodeJS really well?
Then you consider it boring. I'm sure Node and MongoDB were singled out by the author because at the time of writing they were still relatively new and undergoing periods of rapid development and change.
CV driven development is just as well guarded against by asking someone to justify their technical decisions based on the requirements and how the solution meets them. So "boring" does nothing to further that.
> More, I think you need to consider the context of when this was written. It was a period of rapid innovation/evolution
It's linked today, people feel it's relevant today. This isn't a historic piece about how the tech industry used to be, people reference this post today.
> around this time because teams had taken bets on new tech either they didn't know how to use well or the tech didn't take off and was a dead end.
Yes, they should have had a discussion about their requirements and which technologies would have solved them.
> Then you consider it boring.
Then "boring" is useless and you should just say "I know this technology well and it maps to our use case well" and be able to justify that.
> CV driven development is just as well guarded against by asking someone to justify their technical decisions based on the requirements and how the solution meets them. So "boring" does nothing to further that.
It doesn't though. It's very easy to manufacture reasons why "shiny new thing" is the objectively right fit for something, even when it's not.
The word "boring" is well-chosen because it's addressing a bias most engineers have towards the interesting and new. It's a reminder: that fun new tech you really want to try here may not be, probably isn't, the cold-hearted best choice, if you're being practical and business-minded.
I don't agree with the article though and I dislike the influence it has had. I have seen engineers use "Boring" to justify "I know this technology" for situations where that technology is a bad fit.
Conversations about technical solutions are bespoke, there is no one term that can or should be used to guide them.
Poor choices whether influenced by this article or engineers chasing CV points are no different. Boring technology is a tool or communication device like any other. I hope in your situation you were able to influence the engineers utilising it poorly to reconsider.
I will say one of my career regrets was kinda being dogmatic about boring tech instead of being pragmatic e.g with new tech
there was a period around 2016-17s when people were building stuff with Node.js | Mongo. but it was risky tech though I had experience in it & turned some of those opportunities down cz I wanted to work with boring ruby/swift(iOS stuff) lol. rookie mistake.
fortunately the market made me wise up - you go where you're wanted.
> That's quite hard to do for solutions that you don't know the details.
That's a great thing to discuss when deciding on the technology. Maybe you should aim for solutions that you know well, or a solution that makes migrating away easy, or maybe you need to do some discovery work, etc.
> You have an objection to something. It's clearly not to the article's point, though.
It's an objection to the nature of the article itself - that technical decisions should work this way, that metaphors like "innovation tokens" are useful, that "boring" is a good proxy word.
Then it's useless. If I have to take the context into account then I should be prepared to have a conversation about the requirements and how the technology fits it, which "boring" does not facilitate (and discourages).
Weird take. It's clearly useful as a communication tool. You talk to your team, you say "Let's use boring technology. Read the essay, then we can discuss what boring technology means to us first."
I obviously don't agree that it's useful as a communication tool though. Why not "Use the technology that's appropriate for our use case"? That seems radically better and doesn't suffer from weird misinterpretations or vague terms.
Because people LOVE COMING UP with excuses to try a new technology under the basis that "this is appropriate for our use-case", and if you don't introduce a concept similar to innovation tokens you may find that six months later your project is combining three different unproven new technologies and doesn't actually work yet.
Encouraging your team to be selective in where they place their new bets - and use "boring" aka already-understood technology for the bits that are not going to help solve unique problems - can help avoid expensive mistakes.
It does happen, but having the ability to pivot quickly is almost always more valuable than dealing with "the devil you know".
There is nothing wrong with evaluating something new and discovering it doesn't work out as expected. That's still a meaningful outcome. Try it, if it doesn't work, replace it. You've learned something in the process, and it's a mistake you won't make again.
At some point you have to trust in the scientific method. The hypothesis->test->evaluate->adjust hypothesis cycle hasn't been beaten for thousands of years.
Well your points are making sense in isolation whereas this article is making more sense in general.
> I'm a bit tired of "simple" and "boring" and other nonsense words in this field...
This is hilarious in sense millions more will be tired and exhausted by evaluating new and exciting technology endlessly appearing all the time.
People go by these rule of thumbs which may not be perfect in every single case but they do increase success chances for even sub-par teams as opposed to "rigorously evaluating latest technology"
I don't think it's exhausting to determine if a solution fits your requirements and I don't think much about the people who would find it exhausting. I'm not suggesting some insane formal verification, but you really can't just answer basic questions about how technologies can address problems? Then what is your role? To choose mysql irrespective of requirements?
> People go by these rule of thumbs which may not be perfect in every single case but they do increase success chances for even sub-par teams as opposed to "rigorously evaluating latest technology"
Rule of thumb. And it's not a rule. It's a bias based on a vague term.
Well mysql likely will turnout to be better choice than choosing "Cloud scale nosql DBs" when evaluators have rather limited hands-on knowledge about either of them.
> Well mysql likely will turnout to be better choice than choosing "Cloud scale nosql DBs" when evaluators have rather limited hands-on knowledge about either of them.
Obviously a straw-man, but also... justify it then? That's the point. You should be able to justify your position. "Cloud scale nosql db" doesn't tell me why you shouldn't choose it.
If it's understood then it's pointless. I also reject that it's understood.
I have no idea why you're talking about Prinicipia Mathematica as if I'm advocating for some sort of formal verification or extraordinary rigor as opposed to my suggestion that people just use their words and have reasons behind their decisions.
> I'll push back against this, despite it being so popular. I dislike the arbitrary "innovation tokens"
It’s a cute way of saying that you can only do 2-3 new things.
The post is written for an engineer at a startup as a reminder that although it’s green field development, you only have so much runway, so it’s better to focus on what matters instead of trying some new tech because it seems cool.
If you’ve ever had to estimate your stories/tickets/etc in “story points” or “T-shirt sizes” then “innovation tokens” is roughly the same.
If you haven’t had to do that, you’ve lived a charmed life.
> Engineers should understand requirements, risks, tradeoffs, and potential gains.
Ideally. But I’ve worked with plenty of engineers who get far too excited by shiny new tech and overvalue its potential while undervaluing its risk.
Hell, I’ve been that engineer in my misspent youth. The post resonates with many of us because it describes hard-won wisdom of our mistakes.
> New technology may be right for that. Novel approaches may be right for that. "Novel" or "New" are only proxies and they're weak.
Yeah maybe, but unless you’re working on a problem that the tech directly solves, it’s pretty unlikely.
> What if you know NodeJS really well? Or MongoDb? What if you have empirical, verifiable reasons for why they fit better?
In 2015, MongoDB was a dumpster fire (which is still kind of true) and node.js was still kind of new and had enough rough edges that most teams were probably better off choosing some other language/framework.
> I'm a bit tired of "simple" and "boring" and other nonsense words in this field taking up the air in the room that should be spent evaluating solutions on their actual merits.
“Boring” and “simple” are ways to convey that it’s good to be risk averse. It’s a bit of rhetorical flourish that helps drive the point home: choose what you work on carefully because you have limited runway and should spend that runway working on the problems that matter for your business, not new tech that’s orthogonal to it.
I'm reminded of my father in law, who when asked what superpower he'd want, it was flight, so he could get to work without waiting in traffic.
If all of your employees are AI's, clarity of vision trumps choosing boring. And if you're having boring visions... well the world is already full of competition for you, so good luck I guess.
It's better than saying invisibility as that always leads to someone being creepy. Laser beam eyes is of limited use. Moving through walls seems kind of meh to me as that's not a really limiting factor in my day to day. Shooting webs from my writs maybe kind of cool, but if its main use is for moving around it is much more limited than just being able to fly. Lasso of truth isn't really a super power but an interesting artifact. Of all of the super powers, the ability to fly does feel the most utilitarian to me. Plus, think of how environmentally friendly that super power would be if everyone could do it
Boring isn't bad. Ideas are cheap: often the execution is what counts. Doing a boring thing to near-perfection looks like the Japanese Shinkansen trains that average 1.6 minutes of delay annually and have operated with zero fatal accidents over decades of service.
This advice holds up, but there are caveats to keep in mind. Two off the top of my head:
1. I once worked for a company that had a large Cassandra cluster that was primarily serving the role of a distributed append-only log. This role is as perfect a fit for Cassandra as I can imagine. When we needed a distributed database for authentication, we decided to use Cassandra for it since we had in-house expertise (it was "boring technology") and a cluster we could piggy-back on for a while. When we needed a distributed database for a key-value service, we used the same reasoning and chose Cassandra again. There were many problems with Cassandra that I won't go into, but I will say that straying too far from Cassandra's comfort zone stretched it to a breaking point. And we had some downtime and rough nights because of it. Sometimes boring technology isn't enough, you also need boring workloads for it.
2. I was an early adopter for Rust. Before 1.0, it wasn't clear to most people whether the language would mount to anything. To me, I saw a formal proof assistant being put into programmer's hands and could tell right away it had a bright future. Rust was not a boring technology back then. You might say it is now (for some use cases). Adopting shiny new technologies that measurably improve confidence is not a risk. What's foolish is getting comfortable and complacent with familiar, aka boring, technologies that are difficult to use correctly.
It's bad for building your resume. Managers and employees are rewarded for popular thing. Popularity is mostly a function of novelty. So the (social and therefore economic) incentive structure is inversely correlated with choosing boring technology.
I've been noticing this as a solo dev. I need to put lots of work into infra and security, and all I get for it is... the thing not blowing up (hopefully!).
The work is invisible, and impossible to "show off", so it's a bit unrewarding and demoralizing. But hey, someone needs to do it!
Try to explain that part of an imagined "Agentic AI Everything" product being built would be cheaper and more efficient as a simple function call. Good luck!
Some of this may have been a reaction to the era of Javascript framework churn. There were way too many different technologies for doing roughly the same job.
They all more or less worked.
On the other hand, IBM was late getting into integrated circuits. They had Solid Logic Technology, automated machinery for putting transistors into ceramic substrates to make tiny but discrete circuits. That's what powered the IBM System/360. Worked, but kept mainframe prices high and made IBM late to minicomputers.
Innovation when the problem is hard is more interesting. Look at the history of US long range bombers. The B-29 was effective, but underpowered, and had a lot of trouble getting off the ground fully loaded. So, after WWII, the next development was the B-36, which answered the question "what if we scaled up the B-29?"[1] Six propellers, and four jet engines (added late in the design cycle). Was a sky barge, but it worked, as long as no one was trying hard to shoot it down. No flyable aircraft remain. That was the boring technology approach.
After that came the B-47, which answered the question "what if we scaled up a jet fighter to bomber size?"[2] The B-47 was all jets, no props. It needed solid-fuel rocket boosters (!) to help it get off the ground, and a drag chute to slow it down on landing. It was terrible to fly; its operating speed and altitude were too near the "coffin corner" where stall and Mach buffet meet.[3] No flyable aircraft remain.
Then came the B-52. New engines. New airframe. New design. [4] That's the high-risk approach. It worked. Hundreds are still in active service. Outlasted most of its successors, the B-58 Hustler (a supersonic intercontinental bomber), the B-70, the F-111, etc.
On the commercial side, we have Boeing, whose most successful airliner is a variant of the B-737, which first flew in 1967. But that's another story.
Software that works year-after-year has never been a commodity. It's boring on the surface. It doesn't get the flashy posts. But I'll choose reliable over new in almost all cases.
Does nobody ever think of trying more than one technology? It’s like people buying a car based on what everyone else is driving rather than actually driving different cars.
nawh. 50x node modules, typescript out the wazoo, all the state in the client (where you can't see it in prod), the most over-complicated UI, paired with async callback spaghetti is what you do these days.
We're "scalable" over here. It's a sexy problem to have.
I read a similar blog post years ago about restricting the technologies you use, and making do with a slightly worse option if it means reusing the stack you already have.
The example they gave was something like they wanted to use rabbitmq for a new side project (I might be misremembering) but they were forced to make it work with redis instead. The author said that years later he found out that that side project had exploded in popularity and it had coped with it fine because the infra team were already handling the stack and it wasn't some snowflake deployment.
> A good example of this from my experience is Etsy’s activity feeds. When we built this feature, we were working pretty hard to consolidate most of Etsy onto PHP, MySQL, Memcached, and Gearman (a PHP job server). It was much more complicated to implement the feature on that stack than it might have been with something like Redis (or maybe not). But it is absolutely possible to build activity feeds on that stack.
> An amazing thing happened with that project: our attention turned elsewhere for several years. During that time, activity feeds scaled up 20x while nobody was watching it at all. We made no changes whatsoever specifically targeted at activity feeds, but everything worked out fine as usage exploded because we were using a shared platform. This is the long-term benefit of restraint in technology choices in a nutshell.
I can't believe I was asking about the same article. The flow of time made it feel different and I forgot this was one of my all time favourites that I've been trying to re-find for years. Thank you
I remember reading this back in 2015, but how is this not just the age old conservative (don't do more than needed) vs progressives (lets try some new risk things) debate? As much as I can relate to a more conservative choice when choosing tech that might power a giant consumer company, sometimes using riskier tech in a startup makes more sense to get the real innovation flowing...
and the idea of "Choose New Technology, Sometimes" just feels like a little cheat to get away from the bigger issue with the overall thesis. In this way, the idea in this blog just feels so out of touch.
It’s not about conservative vs progressive but about “safe, battle-tested, stable” vs “could be amazing future-tech”; and the idea of having a small number of innovative technologies, in particular where they matter, is about allowing yourself to spend all those thought-tokens on areas that matter.
You don’t want to be innovating your logging tech or your database if the old will work because you might spend 90% of your time trying to get your new, creative logging solution working when you should be spending it on the creative part of your business.
I might be missing something but conceptually, “safe, battle-tested, stable” vs “could be amazing future-tech” IS about conservative vs progressive... am I missing something?
I feel like there's a common theme in business advice like this, the bullshit jobs thesis, general alienation of workers, that individuals have a lot of tunnel vision and don't understand the place of a firm in capitalism. The reality is we don't know what strategies will and won't succeed. The system overall works to the extent it works because a hundred thousand monkeys throwing shit at the wall are bound to hit a few targets at least a few times, even if the vast majority of them miss. The larger economy and investment ecosystems are set up under the assumption that most products will never see a single user and most companies will fail. Most employees will spend much of their careers working on projects that get scrapped. It's not much different than life itself. Evolution has no master plan. Most organisms never reproduce. Most species barely make a blip in any ecosystem before going extinct. It sucks for the individuals, but the larger collective goes on and thrives. Life itself does fine. The economy at large does fine, in spite of the ebbs and flows of the business cycle with its bubbles and recessions. Everything still works out in the long run. It's not only okay to fail but it's crucial to the long run success of our collective endeavors.
The vast majority of business strategy is just the lucky survivors assuming in retrospect they were smarter than everyone else. I've been seeing it my entire life, not just in business but in the military, in politics, in sports, in entertainment. Some project leader has one success and coasts off of that forever, failing over and over again afterward because they keep getting rehired, reelected, or re-invested in by inherently conservative investors, hiring committees, team owners, voters, party leaders, who blindly assume winning a Super Bowl means you must have known what you were doing, no matter how many random bounces going your way you had no control over it took.
Excellent post. Now, somewhat outdated, and in other ways, more relevant than ever. To whom it may concern: if you need a database, always choose postgresql.
i just did an AI hackathon and 90% of the submissions were written in TypeScript and Next.js which is mostly due to the training data. AI is skewed to use these tools by default vs the best for the job.
This blog post annoys me a little. Not the post itself - it’s great - but how it has been used to prop up thousands of lazy arguments against innovations over the years.
The problem with this is that the list of tech that gets boring changes all time, faster than people's opinions. Kubernetes is very boring tech, but if you go through the old discussion threads on this (even from the last year or two), Kubernetes is still cited as some brand new wizbang thing you shouldn't spend tokens on.
Apologies in advance this turned in to a bit of a rant of violent agreement.
For most places k8s is a complete boat anchor. Ends up being a huge complication and sap on product momentum. Can’t tell you how many outages I’ve seen from k8s misconfiguration and misunderstanding.
In no world would I consider k8s to be boring, I can’t understand how all of these shops have convinced themselves that k8s specifically is the level of abstraction at which they want to be interacting with their infrastructure.
I think k8s happens because a large number of devs look at Heroku-like platforms and bristle at the notion that they could ever be expected to intentionally constrain their brilliant system designs into preexisting, Heroku-style shapes. It’s ego on some level.
That, and a ton engineers are still incredibly bad at trading off hardware cost and compensation/complexity/organizational cost. “Heroku will cost us $2000/month, running k8s on our own hardware will only cost $400/month.” Ok, but your team’s total comp is costing the company on the order of $1MM/year. Is saving $1,600/month on infrastructure to spend $20k/month on dev time a good deal? This happens all the freaking time, it’s crazy how little devs value their own time.
Ideally a techthat is not core to the business goals should neither occupy dev/ops time managing it or be so complex that its failure modes are cryptic. It should get out of the devs way and be easy to grok in case of failures.
Lol. In 2015 boring technology is at the hands of your PM spinning up some agent spam. Bury this pile of outdated thinking. Go hard at the most difficult programming you can find or go home.
This is one of my favorite blog posts, and it can basically be encapsulated in the idea of "innovation tokens." It is one of the most useful concepts I have had as a PM / eng leader in my career. It helps actually make the the right tradeoffs, and helps even more in explaining those tradeoffs to colleague of all levels. Highly recommend.
The amounts clearly aren't discrete. Writing your entire app on a new language runtime might be very risky (and also might have large potential ROI!), but choosing a new email provider might not be (random example, but presuming that you can swap out providers fairly easily).
[0] My bigger complaint is really about the vagueness of even deciding what is "boring." How does a new technology transition from being "non-boring" to being "boring"? Apparently that requires a lot of people to ignore this article's advice for a long time, until we collectively decide that those people have had good enough results to consider that technology "boring."
Maybe it wasn't beans? But, I've been looking for it for years.
I've tried over the years, but can never find it. All the search engines moving to vector search makes finding something like this near impossible.
[0] https://widgetsandshit.com/teddziuba/2010/10/taco-bell-progr...
https://archive.uie.com/brainsparks/2011/07/08/beans-and-nos...
But if beans are scarce, and that's where you can try to find ways to use less beans per function, then the goal is to make the function more efficient by using less beans.
So many jobs are sold as “we’re pragmatist's” and when you show up there’s 5 devs, 50 repos and most of the work is discussing if x requirement should be a new micro service. The product is usually an web app with 10 entities and and an API.
Suppose it’s keeping people in jobs
Using the language of the article, I’d say “push all your innovation tokens into agents” is probably a good move. This means the tech your agents work with should all be boring tech.
Another way of saying this is “use in-distribution technology”. If agents are substantially better at Rust than Zig, probably you should use Rust, even if Zig is “better”. The amount that Zig is better is going to get swamped by the amount that in-distribution agents are better.
(This is not a claim that Rust actually is better, just a hypothetical fact pattern for discussion.)
4 or 5 years ago if I had to weigh up two options and one was written in rust it was almost a sure thing that the program was snappy, fast, reliable and that the author was competant.
Now it's a sure fire sign that the project was probably vibe coded. Not saying it can't still be good, I just have a different first impression now
Groan. That's sad.
Even if in the other approach, the LLM knows the stack better, you're just going to push the complexity into your client code, so it's not like it's a huge gain to use boring tech stack.
1: arraylist. A library will probably do better than you. It’s already written and common.
2: webservers. Sure, you can write one yourself with sockets and a bunch of buffers and string management… or, you could use one of the 100s of existing ones. Which one? Probably a stable and established one.
Pipelines and deploys gets much easier and faster than the very common TypeScript monorepo, and so does communication between server and client.
And I say that as a TypeScript and Effect aficionado who has no particular love for neither php or ruby, but they are extremely solid choices to move fast, well, and get excellent performance and tooling out of the box.
On the other hand, Go code from 2012 and Go code from 2026 looks virtually the same. Conventions are respected, go fmt is the one single formatter, "use the stdlib" is a popular mantra and the code is readable by design.
If I were to codegen a project I wouldn't use anything but Go at this point.
The language isn't huge, the documentation has always been very good, and the patterns extremely consistently across the community. I almost wonder whether it's smaller community has been a benefit, as LLMs have been trained on a tighter set of code samples.
I can't speak for Ruby or PHP.
I dislike macros and languages allowing developers to get creative with their own DSLs.
Serving traffic with Ruby and Python is significantly more expensive than serving traffic with Java or Go.
Can you elaborate more on the slowness you've seen with deploying TypeScript codebases? My experience is that it's improved significantly over the past decade or so. tsc is still kinda slow for large projects, but the rewrite to Go should improve things significantly. Pre-install/post-install scripts can also be slow, but I've always disabled those and haven't yet run into issues. The JS-written bundlers can also be slow, but I use the non-JS ones (esbuild, parcel, bun, etc.) and they're fast enough for me to not care. The main build bottleneck I can think of for a modern TypeScript project would be build plugins, since those are still written in JS/TS.
If we get to the point where AI tools are able to consistently able to produce desired results within strict performance and security constraints, without having to make the same tradeoff between delivery velocity and final quality, why would we not have them target the lowest level feasible for implementation, and cut out all of the middleware cruft that makes everything slow and take up ten times the RAM it actually needs to?
PHP (lets use this as an example) brings already many benefits over a common solution like a TypeScript monorepo mostly operational simplicity. That was already true before AI.
It's boring, very fast and easy to deploy, offers straightforward horizontal scaling, no need to orchestrate containers and/or multiple runtime processes, has excellent html rendering (nothing in JS-land really does), and has very solid framework solutions like Laravel where everything works out of the box.
Those merits existed before AI already.
The issue was that you had to buy into PHP as a language, which was a horrible experience.
But if AI writes most of the code? Suddenly PHP becomes an excellent candidate to choose for many use cases.
If anything, AI makes the choice of programming and languages and software about finding the right tool for the job. Somehow the industry instead keeps vomiting React/Tailwind slop which are the right tool for 1% of the jobs.
I hope to have cleared the concept.
Engineers should understand requirements, risks, tradeoffs, and potential gains. New technology may be right for that. Novel approaches may be right for that. "Novel" or "New" are only proxies and they're weak.
For example, I may think "New" means untested, but is that true? What if a new project has Jepsen testing, a fuzzing suite, massive compute running tons of oracle tests, etc? I should just say "Choose well tested" instead of "Choose old" - lots of old software is very poorly tested.
Maybe I think that "Old" implies better documentation, but does it? Lots of older projects have insane cruft and weird edge cases that are undocumented and accumulated over years.
Why do we need a metaphor? Why is "innovation token" helpful?
If you're incapable of evaluating a technology in terms of these properties, you aren't a serious developer and "boring" will not save you.
Sit down, write our your requirements, determine candidate solutions, and choose them based on their fit. "Boring" means nothing, it's a vague proxy term. "Well tsted", "performant for our use case", "developers know it", etc mean something.
> MySQL is boring. Postgres is boring. PHP is boring. Python is boring. Memcached is boring. Squid is boring. Cron is boring.
Literally every one of these has caused hilarious and disastrous failures for me in my career. But yep, boring.
> If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens.
What if you know NodeJS really well? Or MongoDb? What if you have empirical, verifiable reasons for why they fit better?
I'm a bit tired of "simple" and "boring" and other nonsense words in this field taking up the air in the room that should be spent evaluating solutions on their actual merits.
So yes, pick boring tech, defined as the tech you know the sharp edges of.
That is, if someone said two sentences, I would only care about the second one:
1. "We should use this because it is boring"
2. "We should use this because we understand the sharp edges"
I wouldn't care at all about (1) and I'd have a real conversation based on (2).
Any productive conversation that starts with (1) immediately has to follow "can you clarify what that means", so the term is useless at best and thought terminating at worst.
TLDR; #1 and #2 are essentially implying the same thing.
> What if you know NodeJS really well?
Then you consider it boring. I'm sure Node and MongoDB were singled out by the author because at the time of writing they were still relatively new and undergoing periods of rapid development and change.
> More, I think you need to consider the context of when this was written. It was a period of rapid innovation/evolution
It's linked today, people feel it's relevant today. This isn't a historic piece about how the tech industry used to be, people reference this post today.
> around this time because teams had taken bets on new tech either they didn't know how to use well or the tech didn't take off and was a dead end.
Yes, they should have had a discussion about their requirements and which technologies would have solved them.
> Then you consider it boring.
Then "boring" is useless and you should just say "I know this technology well and it maps to our use case well" and be able to justify that.
It doesn't though. It's very easy to manufacture reasons why "shiny new thing" is the objectively right fit for something, even when it's not.
The word "boring" is well-chosen because it's addressing a bias most engineers have towards the interesting and new. It's a reminder: that fun new tech you really want to try here may not be, probably isn't, the cold-hearted best choice, if you're being practical and business-minded.
However, it's influenced a lot of people for more than a decade and the snippy title might well have something to do with that.
Conversations about technical solutions are bespoke, there is no one term that can or should be used to guide them.
there was a period around 2016-17s when people were building stuff with Node.js | Mongo. but it was risky tech though I had experience in it & turned some of those opportunities down cz I wanted to work with boring ruby/swift(iOS stuff) lol. rookie mistake.
fortunately the market made me wise up - you go where you're wanted.
The article answers this, and the answer is "no". New technology is one you don't know the details of.
> determine candidate solutions, and choose them based on their fit
That's quite hard to do for solutions that you don't know the details.
You have an objection to something. It's clearly not to the article's point, though.
That's a great thing to discuss when deciding on the technology. Maybe you should aim for solutions that you know well, or a solution that makes migrating away easy, or maybe you need to do some discovery work, etc.
> You have an objection to something. It's clearly not to the article's point, though.
It's an objection to the nature of the article itself - that technical decisions should work this way, that metaphors like "innovation tokens" are useful, that "boring" is a good proxy word.
Then they're not boring. Boring isn't a universal trait, it has to be evaluated within the context of your own team.
Encouraging your team to be selective in where they place their new bets - and use "boring" aka already-understood technology for the bits that are not going to help solve unique problems - can help avoid expensive mistakes.
There is nothing wrong with evaluating something new and discovering it doesn't work out as expected. That's still a meaningful outcome. Try it, if it doesn't work, replace it. You've learned something in the process, and it's a mistake you won't make again.
At some point you have to trust in the scientific method. The hypothesis->test->evaluate->adjust hypothesis cycle hasn't been beaten for thousands of years.
> I'm a bit tired of "simple" and "boring" and other nonsense words in this field...
This is hilarious in sense millions more will be tired and exhausted by evaluating new and exciting technology endlessly appearing all the time.
People go by these rule of thumbs which may not be perfect in every single case but they do increase success chances for even sub-par teams as opposed to "rigorously evaluating latest technology"
> People go by these rule of thumbs which may not be perfect in every single case but they do increase success chances for even sub-par teams as opposed to "rigorously evaluating latest technology"
Rule of thumb. And it's not a rule. It's a bias based on a vague term.
Obviously a straw-man, but also... justify it then? That's the point. You should be able to justify your position. "Cloud scale nosql db" doesn't tell me why you shouldn't choose it.
I have no idea why you're talking about Prinicipia Mathematica as if I'm advocating for some sort of formal verification or extraordinary rigor as opposed to my suggestion that people just use their words and have reasons behind their decisions.
It’s a cute way of saying that you can only do 2-3 new things.
The post is written for an engineer at a startup as a reminder that although it’s green field development, you only have so much runway, so it’s better to focus on what matters instead of trying some new tech because it seems cool.
If you’ve ever had to estimate your stories/tickets/etc in “story points” or “T-shirt sizes” then “innovation tokens” is roughly the same.
If you haven’t had to do that, you’ve lived a charmed life.
> Engineers should understand requirements, risks, tradeoffs, and potential gains.
Ideally. But I’ve worked with plenty of engineers who get far too excited by shiny new tech and overvalue its potential while undervaluing its risk.
Hell, I’ve been that engineer in my misspent youth. The post resonates with many of us because it describes hard-won wisdom of our mistakes.
> New technology may be right for that. Novel approaches may be right for that. "Novel" or "New" are only proxies and they're weak.
Yeah maybe, but unless you’re working on a problem that the tech directly solves, it’s pretty unlikely.
> What if you know NodeJS really well? Or MongoDb? What if you have empirical, verifiable reasons for why they fit better?
In 2015, MongoDB was a dumpster fire (which is still kind of true) and node.js was still kind of new and had enough rough edges that most teams were probably better off choosing some other language/framework.
> I'm a bit tired of "simple" and "boring" and other nonsense words in this field taking up the air in the room that should be spent evaluating solutions on their actual merits.
“Boring” and “simple” are ways to convey that it’s good to be risk averse. It’s a bit of rhetorical flourish that helps drive the point home: choose what you work on carefully because you have limited runway and should spend that runway working on the problems that matter for your business, not new tech that’s orthogonal to it.
If all of your employees are AI's, clarity of vision trumps choosing boring. And if you're having boring visions... well the world is already full of competition for you, so good luck I guess.
That is tragically sad.
1. I once worked for a company that had a large Cassandra cluster that was primarily serving the role of a distributed append-only log. This role is as perfect a fit for Cassandra as I can imagine. When we needed a distributed database for authentication, we decided to use Cassandra for it since we had in-house expertise (it was "boring technology") and a cluster we could piggy-back on for a while. When we needed a distributed database for a key-value service, we used the same reasoning and chose Cassandra again. There were many problems with Cassandra that I won't go into, but I will say that straying too far from Cassandra's comfort zone stretched it to a breaking point. And we had some downtime and rough nights because of it. Sometimes boring technology isn't enough, you also need boring workloads for it.
2. I was an early adopter for Rust. Before 1.0, it wasn't clear to most people whether the language would mount to anything. To me, I saw a formal proof assistant being put into programmer's hands and could tell right away it had a bright future. Rust was not a boring technology back then. You might say it is now (for some use cases). Adopting shiny new technologies that measurably improve confidence is not a risk. What's foolish is getting comfortable and complacent with familiar, aka boring, technologies that are difficult to use correctly.
The work is invisible, and impossible to "show off", so it's a bit unrewarding and demoralizing. But hey, someone needs to do it!
I've found it extremely useful over the years, personally.
On the other hand, IBM was late getting into integrated circuits. They had Solid Logic Technology, automated machinery for putting transistors into ceramic substrates to make tiny but discrete circuits. That's what powered the IBM System/360. Worked, but kept mainframe prices high and made IBM late to minicomputers.
Innovation when the problem is hard is more interesting. Look at the history of US long range bombers. The B-29 was effective, but underpowered, and had a lot of trouble getting off the ground fully loaded. So, after WWII, the next development was the B-36, which answered the question "what if we scaled up the B-29?"[1] Six propellers, and four jet engines (added late in the design cycle). Was a sky barge, but it worked, as long as no one was trying hard to shoot it down. No flyable aircraft remain. That was the boring technology approach.
After that came the B-47, which answered the question "what if we scaled up a jet fighter to bomber size?"[2] The B-47 was all jets, no props. It needed solid-fuel rocket boosters (!) to help it get off the ground, and a drag chute to slow it down on landing. It was terrible to fly; its operating speed and altitude were too near the "coffin corner" where stall and Mach buffet meet.[3] No flyable aircraft remain.
Then came the B-52. New engines. New airframe. New design. [4] That's the high-risk approach. It worked. Hundreds are still in active service. Outlasted most of its successors, the B-58 Hustler (a supersonic intercontinental bomber), the B-70, the F-111, etc.
On the commercial side, we have Boeing, whose most successful airliner is a variant of the B-737, which first flew in 1967. But that's another story.
[1] https://www.youtube.com/watch?v=vKQYG_fA2uM
[2] https://www.youtube.com/watch?v=l1-urTRxeEM
[3] https://en.wikipedia.org/wiki/Coffin_corner_%28aerodynamics%...
[4] https://www.youtube.com/watch?v=k8EURBL53_k
I'm glad I finally did, I've been enjoying it.
Don't drive a new car, using new a new type of engine, with a new type of steering wheel, with a new type of entertainment system, .... all at once.
WHEN something goes wrong it will be intense trying to figure it out.
Highly likely to not get any other replies as it’s too much uncommon sense.
And generalizations in software development are always wrong.
We're "scalable" over here. It's a sexy problem to have.
The example they gave was something like they wanted to use rabbitmq for a new side project (I might be misremembering) but they were forced to make it work with redis instead. The author said that years later he found out that that side project had exploded in popularity and it had coped with it fine because the infra team were already handling the stack and it wasn't some snowflake deployment.
Does anyone remember the post I'm talking about?
> A good example of this from my experience is Etsy’s activity feeds. When we built this feature, we were working pretty hard to consolidate most of Etsy onto PHP, MySQL, Memcached, and Gearman (a PHP job server). It was much more complicated to implement the feature on that stack than it might have been with something like Redis (or maybe not). But it is absolutely possible to build activity feeds on that stack.
> An amazing thing happened with that project: our attention turned elsewhere for several years. During that time, activity feeds scaled up 20x while nobody was watching it at all. We made no changes whatsoever specifically targeted at activity feeds, but everything worked out fine as usage exploded because we were using a shared platform. This is the long-term benefit of restraint in technology choices in a nutshell.
You don’t want to be innovating your logging tech or your database if the old will work because you might spend 90% of your time trying to get your new, creative logging solution working when you should be spending it on the creative part of your business.
The vast majority of business strategy is just the lucky survivors assuming in retrospect they were smarter than everyone else. I've been seeing it my entire life, not just in business but in the military, in politics, in sports, in entertainment. Some project leader has one success and coasts off of that forever, failing over and over again afterward because they keep getting rehired, reelected, or re-invested in by inherently conservative investors, hiring committees, team owners, voters, party leaders, who blindly assume winning a Super Bowl means you must have known what you were doing, no matter how many random bounces going your way you had no control over it took.
This should not be new.
Make your shit work. Make it work well. Don't fuck with a thing that's working as desired.
In this way, you'll make everyone's lives at least a little easier.
This should be a mandatory teaching to anyone who wants to use the moniker of "engineer".
Sadly, that is real out-of-the-box thinking.
Excel for spreadsheets -_-
And Slack for communications -_-
Sometimes we use Google (yawn) Docs -_-
“I use Linux”
O_O
Don't let one affect the other.
For most places k8s is a complete boat anchor. Ends up being a huge complication and sap on product momentum. Can’t tell you how many outages I’ve seen from k8s misconfiguration and misunderstanding.
In no world would I consider k8s to be boring, I can’t understand how all of these shops have convinced themselves that k8s specifically is the level of abstraction at which they want to be interacting with their infrastructure.
I think k8s happens because a large number of devs look at Heroku-like platforms and bristle at the notion that they could ever be expected to intentionally constrain their brilliant system designs into preexisting, Heroku-style shapes. It’s ego on some level.
That, and a ton engineers are still incredibly bad at trading off hardware cost and compensation/complexity/organizational cost. “Heroku will cost us $2000/month, running k8s on our own hardware will only cost $400/month.” Ok, but your team’s total comp is costing the company on the order of $1MM/year. Is saving $1,600/month on infrastructure to spend $20k/month on dev time a good deal? This happens all the freaking time, it’s crazy how little devs value their own time.
edit: Node is boring, as far as the frontend goes -- a notoriously unstable ecosystem.