It’s not so much that it’ll go rogue and decide to punish people or that it’s designed to be good for humanity.

The fact that it’s possible to build a machine that punishes people who didn’t help create it provides an incentive to deliberately try to create that machine in order to avoid its punishment.

Believers in the basilisk aren’t trying to make ai to benefit humanity, they’re explicitly trying to create an AI that deliberately punishes those who didn’t build it because they fear that, if they don’t, someone else will and it will come for them.

The part about it being a magical, omnipotent, time traveling, mind torture device is just framing to make the thought experiment clearer. It’s not required for the basilisk to be effective. Even if the punishment is just death and it can’t get to everyone, and only goes off limited information, that’s still motivation enough for some people.

It is a scary thing, because it’s only silly if everyone thinks it’s silly and all it takes is one mad billionaire (eg. Elon) to build a lesser basilisk that can cause real harm.

While this advice is true for all models, when it comes to agentic tasks (add this small feature/write this test harness/find bugs/suggest improvements), open source models are still way behind, vibe code or not.

Claude Fable or even Opus in an editor like Zed have a 1 million token context window and will “think” through the goals of the application, test their changes, work through debugging processes the way a programmer would, stop to ask for clarification, check diagnostic tools and linters, prompt to run test code, etc.

Llama, Gemma and Qwen etc. Do lack a lot of the world knowledge to get the goals of the application, but they also just don’t have the debugging skills, won’t test their code, don’t always tool call correctly, get confused as the context increases and nobody has enough vram to run on large context sizes locally.

They can do autocomplete on small functions but aren’t really there for more complex tasks.

On top of that, the biggest problem is that the best open source models are trained and released by the same giant tech conglomerates that have an interest in not competing with their own products. Qwen is Alibaba, Llama is Meta, gpt-oss is OpenAI. Even the more “independent” ones, kimi (Moonshot) and GLM (z.ai) are mostly funded by Alibaba and Tencent. They’re released for research and marketing purposes and to please their corporate backers with inflated stock. Almost nobody has the resources to train new models from scratch. People make lots of merges and fine tunes but AI is not democratised the way that traditional programming tools have been.

Maybe some day there will be enough cheap compute for open source communities to pool together resources to build competing models but they’re not really there yet :(

There are quite a few. The best ones are sustainable closed loop datacenters with on-site solar which is becoming pretty common across the world, especially for new builds. Often producing more power than they need and feeding it back to the grid (especially if the local government has an energy buy back scheme).

But most data centers are pretty tiny and just built into an office building with a bunch of server racks.

Depending on where you live, a quick web search for data centers in your local area will probably show up dozens of them of varying quality hosting people’s websites and business apps etc. They aren’t any scarier than anything else you find in a city. They’re critical infrastructure that helps make the internet a thing. In most cases, if it wasn’t a datacenter, it would be a car yard or a factory, etc.

But! There are also truly evil datacenters. Like this insane Utah monstrosity built for a shitty purpose and the size of a freaking city. An obscene monument to the US tech cesspool’s hubris.

I mean they’re not all for AI and they’re not all environmentally devastating.

This one very much is.

That’s fair. The nuance that people lose is more that people are often painting them all with the same brush. Protesting any datacenter regardless of impact.

It becomes something like: “datacenters are evil and are a symbol of techno fascist distopia! If they build a datacenter in my city, the taps will run dry and Elon Musk will use it to make ai porn of my children!” Even if it’s a small solar powered closed loop that provides VPS, storage and web hosting for nerds and small businesses.

I also do think there’s also a scale of evil there. Some environmental impacts are not immediately obvious and might not be known about during planning. Some were built a long time ago with older tech and are a bit shitty but have a plan to transition to be more sustainable, etc.

The world is full of “alright but a little bit shit.” It’s not all perfect angels and mustache twirling villains.

I don’t want to detract too much from the real villains though. Nobody needs a 9GW datacity for military ai.

Not all data centres are evil and the issue is nuanced. This one sounds pretty evil though.

9GW is totally insane and they’re building a gas plant for it instead of renewables (although there’s some solar too). It’s closed loop so the water use fears once it’s running are probably a bit overblown, but the construction itself is going to be ecologically insane. The thing is basically a data city, 162 square km is even larger than a lot of cities and involves building an entire power plant and new energy infrastructure. Building it is a full megaproject and even just noise pollution and the construction impacts will mess with bird migration etc. Obviously the whole thing isn’t going to be full of data centre, some of that space is empty but still.

It’s also going to have the US military as a major client so… Pretty high up there on the evil scale IMO.