This isn’t about any of that.

It’s about the fact that none of those decisions show up in the metrics that matter to the people making hiring and delivery calls. The cost is immediate and visible. The return is delayed, quiet, and arrives in the form of things that didn’t happen — the investigation that took two hours instead of two days, the API change that didn’t bleed into the domain model, the bug that the structure caught before it shipped.

Sprint velocity captures the extra day. It doesn’t capture what that day bought.

This is not a new problem. Most engineers who’ve been around long enough have felt it from both sides - made the careful call and got measured on the slowness, or inherited the codebase built entirely for speed and paid the tax. The measurement system was already broken. It has been rewarding the appearance of velocity over the thing velocity is supposed to serve.

This was true long before anyone was generating code with AI. The PR process in a lot of teams was already largely theatrical — review comments on naming conventions while the architectural decisions slipped through unquestioned, approvals given because the diff was too large to meaningfully read. The gate was already not doing much. We brushed it under the carpet and moved on.

AI tooling is changing the volume of code moving through that process by an order of magnitude. The pressure to remove the gate entirely — to trust the output, to ship faster - is only growing. The faster-is-better incentive that was already making review ineffective is about to be handed a much larger surface to work on.

Many years ago, I pitched full redevlopment of a ticketing system from a PHP based system to a Java EE system because it was struggling to scale.

It probably needed a couple of years to build. They wanted it in six months. I accepted the challenge.

We built and deployed the system in eight months. We spent the next year fixing it.

The client then rebuilt it in-house.

When AI runs this experiment at scale, who takes it back?

Project Stargate: $500,000,000,000(i.e. $500b, the GDP of Singapore) in AI over the next four years, starting with $100,000,000,000 (i.e. $100b, the GDP of Bulgaria). https://lnkd.in/getadcFN

These numbers boggle the mind. The California fires have been estimated to cost between $250b and $275b, which is just about half the total investment into Project Stargate.

How much of this money is likely to be spent on mitigating bias? What about on exploring the risks of AI? I don’t mean AGI, Singularity, or about AI taking jobs. I mean the day-to-day risks to ordinary people: the spreading of misinformation and biases. We all saw the damage that Facebook and Cambridge Analytica brought to the world. How much more damage could this cause?

How might these new tools be used to manipulate, control and oppress? How might these tools further marginalise those who are already disaffected?

Yes, there might be fantastic breakthroughs, but at what cost?

I am somehow reminded of the poem “First they came…”:

First they came for the socialists, and I did not speak out—
Because I was not a socialist.

Then they came for the trade unionists, and I did not speak out—
Because I was not a trade unionist.

Then they came for the Jews, and I did not speak out—
Because I was not a Jew.

Then they came for me—
And there was no one left to speak for me.

The Manhattan Project, which built the first Nuclear bomb, cost $2 billion, which, adjusted for inflation, comes to $27 billion. America wants to spend almost 20 times that amount over roughly the same time on Project Stargate.

I apologise for the doom & gloom, but I have genuine concerns about the risks at play here. How about you? Are you worried, excited or both?