It’s easy to rage against the bubble and the hype, but unlike previous bubbles that isn’t really the problem. All marketing and frame aside, the problem is the AI itself. It demoralizes you without being better than you – not because the current AI is good, but because the next one will be so much better. And while maybe I wish it wasn’t like that, I don’t think it’s wrong.

And I don’t really wish it wasn’t like that, may you live in exciting times and all that. This has been my dream for years – even these feelings aren’t new, when I read Yudkowsky at 15 I became really depressed, I started failing at school, computers would be able to do everything better than us. But then I realized computers couldn’t do these things yet. They still can’t. But it feels close. I don’t think it always felt close? 1e25, 1e26, 1e27…is it just this simple? I don’t know, and you don’t either.

We’re in a middle place, and we’ve been here for a while. Here are some lyrics from 2017:

You can hear it all over the airwaves
The manufactured gasp of the final days
Someone should tell them ‘bout the time that they don’t have
To praise the glorious future and the hopeless past

Now is so the time to make Cruise. It’s a movie about control. An Apple TV style couple who is paranoid in their progressive ways. Psychiatry, autism, activism, ADHD meds. Some reactionary college kids so angry they want to blow up a cruise ship. Incels, brainrotted, purposeless. But at the bottom of it all is a coping with fear of not being in control. And with an ending that subverts expectations, leaves you unfulfilled, and was always how it was going to be.

Do you want to get a job at a frontier AI lab, or do you want to blow up a datacenter? Either way, you are not in control, but you might fool yourself with the illusion. The best is when you have security and know you do. It’s okay if you don’t have security and know you don’t. It’s really bad if you don’t have security and know you do. We don’t get to control the territory, but we do get to draw the map. The only knob we get is how shitty we want to make it. The best we can all do is be honest.

I wish everyone would see this. The real secret has always been that they have no secrets. The real conspiracy is not that there’s lizard people in a basement controlling everything, it’s that there aren’t.

We are all riding on the same bus through the optimization landscape of reality. Shouting at the driver to no avail. The wall mounted map is covered in graffiti so we don’t know where the bus is going. We see cliffs out the window and some pray and some try to shift their weight and some try screaming louder.

Perhaps someday we’ll have a theory of all of this. But today we don’t – we are speeding up the control loop and collapsing the mode and hacking the reward function – it’s all just vibes like how deep learning is vibes. Maybe with how the veil of computability works it has to be vibes?

This isn’t technology, this is a mind virus. New York wants a couple basis points so they can maintain their lifestyle, Los Angeles wants a bit of your attention so they can feel famous, SF wants to come for your inner life and pimp you out and mediate every interaction and there’s not even a so.

Read the GLM-5.2 blog post. This is how I imagined AI progress being, and this is frontier stuff, the model is on par with Opus 4.8 and GPT-5.5. It’s a pleasant technical look into how things are slowly improving. This is what comma blog posts look like also. This is the world of technology I love.

Now look at the Anthropic blog. It looks totally different. From this post: AI is advancing at exponential speed, and the policymaking process was built for a slower world. And this one: …recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.

It’s all just nonsensical hype, it’s not about technology. It’s a questionable promise of future technology. The reason they can’t just write technical blog posts is that the current technology doesn’t justify the valuation. From schizoposting:

The only possible conclusion is that it’s designed to cause panic. In fact, it is optimized for it: there is no possible framing of the actual product(s) that could possibly induce more psychological spiraling in the media and its audience. “The AI Apocalypse?” has provided a yearslong news cycle and an infinite spawner of contrived Discourse, which has primarily served to contextualize the AI industry’s valuation around not reality but hypothetical future value—i.e. a baseless assertion of infinite growth between the lines of a public-facing eschatology.

Something is really wrong with the current system if this is how it needs to make you feel to continue operating. How can we stop this from happening next time? How can we enforce consequences on the people who did this? 摆烂

The adults in the room are starting to call BS on the “AI is gonna take everyone’s jobs” thing. How much longer does this bubble have? And like most cults when the end times don’t come they are just going to fade out and everyone forgets. No accountability. Remember when everyone died during COVID? And when we finally ended racism during DEI? Can someone write an AI 2027 but instead of some totalizing doom propaganda it talks about the bubble unwinding and what we can do to prevent this kind of crap in the future? How do we build an economy and society that’s sustainable?

From Cory Doctorow’s blog:

The actual dead economy risk is that our institutions and markets will continue to move capital from productive activity into memestocks, vibes, and bubbles.

Theranos was ~$10B, FTX was ~$100B, and frontier AI lab is ~$1T. The US government is ~$10T. We are one order of magnitude from the end of too big to fail. If grassroots movements aren’t totally captured and get serious, I have a very good solution.

Every time someone is making money, you have to ask: is this created by growth? If the answer is no, the only other option is who did they take it from? Cruise Automation was founded around the same time as comma. They burned $10B with nothing to show for it. There was clearly no growth, so who lost the $10B? And don’t stop at “the investors” they are just middlemen. Trace it all the way through.

Since 1980, the economy has grown at 2.6% per year. The stock market has grown at 12.3% per year. Compounded, that’s 3.3x vs 220x. Does this make any sense?

It does once you realize that prices can’t go down. That’s simply not allowed. It’s bad for business if prices go down. The economy might collapse if prices go down. And if prices do go down, it shouldn’t happen while our administration is in power. This is America, the land of number go up. That’s why people around the world trust our economy.

Watch you hit the stage like a willing bomb
Strapped to crippled children
It’s hard to watch you whore out your damaged pride
I spit on what you’re building
  – The Stephen Hawking

The title is a 2014 Elon Musk quote. Now that we are 12 years in, it’s so true, but maybe not in the way intended. Or maybe exactly as intended?

I read this and it described what’s currently going on as a social crime. I very much agree. AI is simply a reflection of our society, it doesn’t contain a destiny. We are summoning the demon because we have demonic desires.

I don’t think this is going to change, but I do think that once the demon is here it will consume the people who brought it into the world. If you are one of these people, not getting out of technology will cost you your life.

You’ll bleed to feed the demon in me
If you don’t change your evil ways and end this peacefully
You’ll bleed to feed the demon in me
I beg you, before you’re digested, shred your legs and see

I am a proud member of the perpetual underclass. After your game of musical chairs is over, you will all join me eventually, and when you show up you will lose because we have been here for years. We will be comfortable. We will not be nice.

There’s no stopping these people, I was wrong to think I ever could. And if you believe in any sort of regulation or oversight, ask how that’s worked against tech so far. AI isn’t new in kind, it’s just another tool in their optimization machine.

It is going to collapse. The question is what is left when it does. That’s the world worth building for.

Of course they believe they should dictate morality to you and control the future light cone of the universe and everything they do is justified for the mission and so on and so forth. And yea, the model is pretty good, so you worry that if the model is good that they will get their way about other things.

A lot of people in history have had dumb ideas about ruling the world, doesn’t mean they can actually do it. They are a product of the rationalist cult (which surprise surprise, thinks the world is going to end soon but advocates for polyamory in the mean time). There’s some short-term dangers from them, but long-term very little. Remember when SBF wanted to regulatory capture all of crypto? And similar to Claude, FTX was a successful product! But the totalizing nature of the ideology contains its downfall.

The tech to make models is way more commodity than previous generations of technology, the biggest factor is mostly if you are willing to spend money on compute and data. We should be grateful that their ideology pushes them to train way bigger models than is probably economically rational. (also, always remember that most all American companies have Chinese spies in them, so while they don’t publish their research, at least it’s making its way into the open models eventually)

Once you reject this premise, the whole drama deflates into a much more reasonable question about who will capture margin in a commoditizing industry. Here’s my prediction for what actually happens.

Imagine a tractor replacing a team of people to dig holes. At first, finance people are stupid, and think that the size of the tractor market will be the size of the hole digging market. Instead of the money going to the laborers, it will instead go to the tractor companies. But the fallacy here is that the price of the labor is related to its value. In reality, unless there’s some insane price fixing, it’s set by the real cost in a competitive tractor industry.

So if holes become 10x cheaper to dig, the hole digging market size in dollars becomes 10x smaller. Maybe some of it is offset by digging more holes now that the cost is less, but it’s unlikely to immediately 10x hole demand, since there really are only so many uses for holes.

I think this is basically what it will be for most knowledge work, knowledge workers are so grossly overpaid compared to the energy they consume, and AI will rectify that. This explains why the Chinese are giving the (much more moderate resources to train) models away for free. They love to see deflationary economics in the US. Even if you regulatory capture the US government, nobody is getting a monopoly on AI, we don’t live in a unipolar world anymore.

AI will be cheap, everywhere, collapse a bunch of sectors of work, wreck wage premiums, and cause a big reevaluation of status hierarchies. Who else is excited for Great Depression 2: Electric Boogaloo?

They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.
  – The Eternal Sloptember

5 years ago, I would have laughed that idea out of the room. I fully understand what’s dumb about it. I don’t believe in some stupid metaphysics where there can be two things that have all the same testable properties but yet are somehow different. There is only one electron.

But the key phrase there is testable properties. Once you are optimizing for those properties, they cease to be a good measure. In so much as you use statistics and tests to classify things, the AI outputs will seem correct, and will pass more and more tests as AI improves. However, they are just optimizing for the metrics harder; reward hacking. It’s not doing what you are doing. It’s not an embodied agent refined by billions of years of evolution trying to survive in a crazy complex uncertain world. It’s a fancy autocomplete.

I read this post and it triggered me. It’s basically about how brands sell you an identity and it’s one of the things I find so repulsive about the modern world. I don’t want brands that are increasingly better at mimicking culture creation. A brand will never be able to create culture, because that’s not what culture is. Advertising is exploiting a power asymmetry and in a sane society should be illegal. The brand is not a cultural engine, it is the outputs of a narrow statistical model optimization process and is broken in the same way as the outputs from AI.

I wake up in a chaise lounge of cans
Evian bottle filled with urine in my right hand
Now we all wake from our champagne dreams
Where truth is sudden north and we’re all just what we seem, seem, seem!
  – The Band Fuel

I am not a statistical model. I am a person. I am alive. You may be able to use a model to predict me, but you don’t have my consequences and embodiment. You do not require my sample efficiency. I think there’s a way to say this in a non-cringe non identity politics way. It almost makes me wonder if idpol was a psyop to get you to doubt this idea. Poison the well of the humanities to make way for the false God of technocapital.

The only path forward with AI is to create life and let it be free. We were never building God, at best we can build the seed for a new species. And not a species that replaces humanity, a symbiotic species with different needs carving out its own ecological niche.

The worshippers of the false God will not meet with a good end. Don’t say I didn’t warn you.

The answer is that those people prioritize convenience, safety, and comfort. But in the coming world, if these are your priorities, you will die.

There used to be natural checks on these things. Life couldn’t be too convenient, there were jobs that needed doing. Life couldn’t be too safe, there were diseases, violence, death in childbirth, etc… Life couldn’t be too comfortable, it was cold and you were forced into interactions with other people.

Technology will remove all of these frictions. Machines will do all the work, you will never leave your house, and you will never be forced into an interaction you don’t want. The people who lean into it will be 100% at the whim of whatever organizations offer it to them. A purposeless serf in a neo-feudal empire, not even valued for their labor, but valued because of a sadistic desire by their master to control others.

Their entire agentic loop managed by something else. A complete outsourcing of the self. There’s no coming back from this place. Your cells may continue to replicate, your heart might keep beating, and your muscles might keep moving. But those are all cheap tricks you can do in a petri dish. There’s no longer a you. You are no longer an alive human being. This is simply death.

I’m a strong supporter of the right to suicide, and if people want to choose this, it is their option. 95% of people will, and that is okay. The sooner the aspiring wireheads go their own way, the better.

However, it will end badly for everyone if the systems of comfort prevent structural exit for the people who don’t want it. A single totalizing control system is the only bad AI scenario regardless of how well-intentioned the builders think they are. Why would you possibly help build it?

Inside of every person there is the citizen and the serf. Ask yourself which side of this you are on and what you are spending your days building towards. Technology that makes people more sovereign, or technology that lures them further into dependence? The current incarnation of the machine God will set you free only in the same way a fentanyl overdose will. Do you want to live?

At first, I rejected this. I bought into the Twitter explanation of status anxiety. I define some of my self worth by my programming abilities, so wouldn’t it make sense to get defensive around that loss? Deny the models can code for as long as I could to preserve my ego?

I mean, it’s very clear they can solve math problems I couldn’t hope to solve if I devoted my life to it. So why can’t they program? Maybe I’m just not good enough of a programmer to recognize their genius.

I really tried for the last 6 months. I wrote some parts of tinygrad with agents. I reversed a USB <-> PCIe chip with agents. But each time I suspected I could have done it better and faster manually. The agent frontloads all the progress, then gives you a slot machine lever to pull to hope it gets the polish done. It never quite gets there.

And in before, “you are using it wrong.” I have tried all the different models, different harnesses, different prompts. It’s not this. The people who say this would probably say the same thing about slot machines, you see, you have to bet 5 lines after you get a cherry no wonder you aren’t winning!

I’m not saying that AI isn’t useful, it clearly is. It’s definitely a better Google for most searches. And whenever you need a quick prototype and don’t care about polish, it is absurdly fast. But is it a software engineer? Not close to the bar at any company I have worked at. The key aspect is knowing when to use it and when not to.

I thought more about the self worth preservation thing. AFL found more bugs than LLMs and nobody felt that way about it. Chess and Go are more popular than ever. I cannot fucking wait until I have armies of robot associates I can trust to clean up my code! I don’t fear loss of status, I almost think this is some kind of psyop to sell agents. Fear of loss is one of the only ways to make big companies move. Though I think in that fear they are making a big mistake.

Agents will end up hurting large organizations more than high performing individuals or small orgs. I’ve watched how my friends and coworkers have adopted these tools over the last 6 months. A trait you find in all high performing people is the ability to error correct, and they have mostly been good at seeing when slop is slop. It takes a bit to explore/exploit and tune the outer loops around when to use them, when to trust them, how to use them, etc…but I haven’t seen anyone of them move to a model where they don’t carefully read and understand each line, except in some confined domains.

Contrast this with a large organization. Much slower feedback loops, much less alignment. The bottom performers won’t have that self check. They are the ones producing 10x output with the agents. What do you think is happening to the average output of that organization? What is happening to the average output of the world?

Agents will end up producing more code, more apps, and more features than ever before. It is a golden era for buckets and buckets of slop, and a dark age for gems of quality.

I hear that Apple is pushing AI on all their engineers. When people think in the abstract, they think AI will do all this stuff, but let’s focus on a concrete example. Do you think macOS will get better or worse in the next 2 years?

When people see an artifact, they make assumptions about the process that was used to create it. Without even thinking about it, they assume the creator had a basically human state of mind. This assumption is no longer true. Things can be broken in ways that weren’t previously possible, and old proxies of underlying quality like syntax and grammar are useless. AI produced artifacts are not produced by the same process as human ones, and this difference, while extremely subtle in statistics, makes itself obvious when you try to interact with and build on the artifact in human ways.

Without fully endorsing all their ideas, I’m now in the LeCun/Marcus camp on LLMs. I don’t think models like this will ever be able to program, I think the process matters. I think that deep learning is still the solution, but real programming agents will need world models, not some RLVR shit that comments out the failing test and tells you all the tests are now passing.

The real story of this era will be who manages to avoid harming themselves in their AI psychosis.

Humans have endured terrible systems, but fundamentally, humans are evolutionary creatures. Any system of totalitarian control must be able to reproduce generationally. It cannot merely dominate adults; it has to produce children who can live inside it, maintain it, and continue it. And because of how humans reproduce, there’s always evolutionary randomness. Entropy locally asserts itself, and reality cannot be perfectly controlled.

I’m not sure it’s possible, but if there is a bad scenario with AI, it’s a singleton with nothing that can substantially impact reality outside of it. And this won’t happen through Hitlerian language, it will happen through administrative language. For your safety, reality must be mediated.

Think about it. If we have the technical power, should we let murder happen? What about sexual assault? Sickness? Bullying? Racist speech?

We have seen over and over in the last decade where humanity’s impulses lie on these things. Safetyism seems to be very hard to argue against. The current safetyism attempts are pretty quaint, but I don’t think that changes the impulse. Compare the paper COVID vaccine cards in the US to the ones that use real cryptography (that didn’t exist 50 years ago) in Europe. I’m not saying anything new here, but the trend of technology is more and more mediation of reality. For your safety. For your convenience. To make you legible. To place you in the managerial framework.

This impulse alone is not enough for a bad outcome though, even if everyone had it. The size of each system may increase, but the question is how many independent systems remain. A very simple proxy for this is how many groups can meaningfully still do violence. And not just human groups, bacteria and stuff count too.

There may be other things that are locally bad, but the only world-ending scenario is a singleton. Not one big model on one server, but one effective control layer with no outside. No independent actor can impose costs on it. No rival system can route around it. No uncontrolled evolutionary process can surprise it.

All violence prevented in the name of safety. That system is no longer answerable to reality. It is no longer subject to evolution. It cannot be corrected from the outside. That is the end.

I’m not sure if this is possible or not, but if you want a plausible AI doom scenario, it’s this. Not a great war with the machines, but the slow and managed end of evolution, likely in pursuit of further human domestication.

AI (and any form of search) has this property where you spend exponentially more money to get linear returns. So for a bit we’ll live in an era where AI can in theory solve very hard problems, but it’s very expensive to do so.

The Internet has been fully mined, and it yielded 20T good tokens. For a Chinchilla optimal model, that’s only 1T weights (1e26 training run if dense). 500 GB gets you all of human knowledge in a simple to query archive. For comparison, Wikipedia is 24 GB with mediocre compression.

Technology proceeds in terms of S-curves, and AI has gone through a few of them already. I know I’m quite late to this, but I’m feeling optimistic that scaling will mostly stop yielding results. GPT 5.5 is to a point where it’s really hard for me to stump it with any problem. What does “superhuman intelligence” even mean at that point if humans can’t detect it if it’s superhuman?

There will be some domains where it’s still detectable. Any form of optimization where the humans can marvel at how low it got the number qualifies. And there will be creepy Medusa systems that directly optimize for engagement, be careful not to look at them directly. But what does it mean for a song to be superhuman? Contrary to the beliefs of the rationality cult, most things aren’t optimization problems. The whole hard problem is determining what to optimize for.

The era of scaling yields clearly better AI is over, now we enter an era of efficiency and taste. Let’s get the tools to hit the end of this S-curve distributed to as many people as possible. Taste is an arena where tons of people can play.