Nothing says that the observer which collapses a waveform needs to be conscious.

I’m not afraid of a little math so I’ll give the book a try, thanks for the recommendation!

This is really interesting. Do you know of anywhere that I could read more about comparing neuronal function to transformer architecture?

I have a dryer. All clothes go in the dryer because outside is wet and inside is tiny. If clothes don’t like the dryer, they shrink and we can’t wear them, therefore all clothes are dryer-safe. After dryer, clothes are piled on couch. When clean clothes are needed, they’re on the couch.

It’s interesting that the one person who actually seems to know what they’re talking about is the one getting downvoted. AI is bad at many things for many reasons, but that doesn’t mean we should just assume that anything derived from AI is automatically slop.

All that stuff is coming. It’s taking longer than people would have expected, but it’ll happen sooner or later. You’re right that we seem to be stuck with the dystopia, though. We’ll fight over an ever dwindling pool of work until there’s nothing left, and then we’ll die or live at the sufferance of those in charge. The only hope is complete systemic change, but that seems further away now than it has ever been, mainly because not enough people even care. They’re pacified by their streaming content and drip fed carefully cultivated outrage-bait to keep them from even imagining that uniting is a possibility.

But why do we need to work to earn money? Because if those jobs didn’t get done we couldn’t survive. What OP is asking, I think, is if those essential jobs were automated, nobody would need to work, so why aren’t we talking about that?

You can train it “you want to help me” and it will or you can train it “you want to wage war” and it will.

This isn’t how current AI models work and it looks like they never will (at least not without a completely different paradigm), but to answer the hypothetical;

-automate all agriculture, resource extraction, logistics and production,

-abolish almost all financial systems,

-provide nutrition, shelter, education, healthcare, etc. to all humans,

-allow humans to spend their time as they see fit.

I don’t see any path to get us to this, but if we somehow managed it I would imagine most people would take up sports, art, science, or hobbies, and would choose to spend more time with family and friends.

Geopolitically, I suppose the countries with the highest ratio of natural resources to population would end up dominating, unless some kind of international framework was enforced which ensured resources were distributed fairly.

They need to make a TV series based on Pussy Riot.

This is why Musk and co are trying to speedrun robot development.

I have no doubt that they would push for any regulation to advantage them, and they have the resources to achieve that. We still need to try. And the fear of an AI apocalypse isn’t something they just dreamed up recently to make their product look ‘good’ (or powerful, or whatever), nor was it just fished from sci fi. The worries about this scenario were why Altman, Amodei and Musk (among others) got into this business in the first place. They see ASI as inevitable, they sincerely believe it will be powerful enough to destroy humanity, and they each have a messiah-complex which makes them think that only they are intelligent and incorruptible enough to be in charge when it happens.

Of those three premises, I very much doubt any billionaire CEO is incorruptible, but the inevitability of ASI and the power that it would have if it was created are harder to dismiss. If there is even a minuscule chance that they are correct, then AI development should be stopped, until they can prove that it is safe and beneficial to everyone.

Edit: and I take it as a compliment that you only give me a 10% probability of being a bot. :-)

Context matters. In science, if you are proposing something which will upend the established consensus, the burden of proof is on the claimant to show that some detail has been missed, or that some other theory fits the evidence better. What Coxon and others are claiming is not a scientific theory, however. How could anyone provide evidence to prove that building an AGI would likely result in the extinction of the human race? What kind of evidence would be acceptable? And if we continue building bigger and smarter models and before we can gather the evidence it turns out the premise is correct, what then? ’Well, we’re all dead, but at least we stuck to our guns regarding that old axiom.’

I think that, given the potential stakes, the burden of proof should be on the AI companies to show that their technology is not harmful, and until they do they should be restricted from pursuing it. By that I mean that they need to show that their existing models are safe and can be controlled, and if they can’t they need to roll back to the Last Safe Model in their roster. I’d even go further, putting the onus on the companies to show that they are not on net damaging our environment, that their models cannot be compromised to provide terrorists information on building bioweapons, etc., that they will support and help strengthen the vulnerable and never (again) encourage people to commit suicide. And that’s just off the top of my head.

The harms already caused by this technology are staggering, and we need laws to put the brakes on development so that we can collectively adapt, and if the leaders of these companies are offering us a way to do that, let’s grab that opportunity.

That’s why propaganda is so insidious, it provides an alternate reality for people to inhabit so that they don’t need to believe that they’re on the wrong side. For example, if you are presented with two news stories, each true, with one extolling the benefits of what you do while the other points out the drawbacks, it is very hard to take the latter seriously. We automatically go through some very complex mental gymnastics to prove to ourselves that we are good, that what we are doing is good, and that anyone who doesn’t see that is either misguided or corrupt.

People in the big AI labs can be aware of the environmental harm their products are causing, along with the potential social harms, and even that AI might, at some point in the future, kill off humanity, and still convince themself that their decision to work on it is correct, as long as there is any uncertainty about the problems, and any possible benefits from their work. To be human is to be biased, and no amount of intelligence will inoculate you from it.

Why do so many people continue to work with regimes that they fundamentally disagree with? There are lots of reasons, and people are complex so one person can hold several, even contradictory, beliefs at once. Some possible answers;

-It’s easier to go along with it and think of myself as being insignificant to the whole.

-Everyone else is doing it, so it can’t be as bad as I think, right?

-Its too big for any one person to fight/it’s inevitable, so I may as well get something out of it, that way at least some good comes from it.

-If I just focus on my job and what I’m doing right now, I don’t need to worry about the horrible things going on elsewhere or the potential future consequences of what I’m doing.

“I’ve had recurring nightmares, That I was loved for who I am, And missed the opportunity, To be a better man”

-Muse, Hoodoo

Whoever makes it to the final, and whoever wins, I hope both sets of fans can put their differences aside and unite in song.

’Who’s the pedo, who’s the pedo, who’s the pedo in the files? Who’s the pedo in the files?’

They would use radiator panels which automatically swivel so they’re edge-on to the sun.

I think the bigger problems are;

1. The costs (monetary and environmental) of launching so many new satellites,
2. Large-scale computing technology is untested in that kind of environment and will likely encounter a number of issues and unforeseen problems (so more launches until they get it right),
3. Additional radiation will increase errors, so they will require a more robust design with more redundancy than Earth-based systems,
4. If they’re in a low orbit similar to Starlink satellites (which have an expected lifetime of 5 to 7 years) they will need to be constantly replaced.

Sounds like you’re early on in the game, just take your time with it and enjoy it. In real life there are a lot of things you can do to simplify your life or connect with nature, from just finding local hiking trails all the way up to living completely off grid. Everything is a compromise though, so just take things one step at a time and see where you are most comfortable.

Ministry for the Future by Kim Stanley Robinson might fit the kind of tone you’re going for. The whole book is about restructuring society to combat climate change, but there’s no actual apocalypse in the traditional sense. The first chapter is one of the most harrowing things I’ve ever read, which does set the scene for what’s to come and shows why the changes they make are necessary.

If the UK Government is so eager for sovereign AI capability, why are they relying on a US company to design, build, and presumably run it?