What is the true underlying policy environment that SAFE will operate in?
There is one technology that could be used to win a nuclear war outright. Put another way, is there for instance a way to outright defeat the US or any other nuclear power and even erase it from history? The answer even in practical if remote terms now appears to be yes.
Ideologically Nuclear weapons have become the new conventional weapons. Its why you can bet all of the nuclear states right now are in a race with their own Manhattan projects to build SI. It makes sense coming fron people like Corbyn wanting to cancel building more nuclear deterrance (a good idea regardless) and being able to confidently say no nuke use ever from him while refusing or at least deprioritizing his own access to British intel publically. It makes senses of why US and Brit intel systems are hell bent on domestic spying and keep writing new bills against embarrasing defeats. It sheds arms race light on Carly Fiorina’s idiotic assertion that a trillion a year on US defense isn’t enough. Or the earlier cryptic nonsense about cyber war being worse than nuclear war. Also the reference on this site to an ominous radical speed up in progress on crucial math problems recently.
You have to understand the pull of this stuff for these people. Even if its likely to be complete vaporware for 100 years or more even a slight chance that it emerges makes it into a complete religion for them.
Despite the first group to get it likely not being able to stop it from quickly putting an end to the world, they can’t let any group but them mess with the fate of the world. So while its basically pure con, there is a tiny chance that its pro beyond our wildest dreams.
One the one hand we’ve unified the forces and mapped the genome and there is some cool stuff like stem cell but really there is nothing on the horizon and we’re kind of clueless about what is next and kind of alone with our problems and looming existential threats from bombs and space rocks and super volcanos. Whereas SI is potentially a time machine that could advance technology a million years in one year, double global product weekly without end, maybe even give us a literal time machine(!) bend the physical laws and make the world into our own little live as long as we want lucid dream.
But we can’t even get along with each other how would we get along with this alien super will that might not be conscious or actually sentient or alive or have any thing like our notions of value and risk but will still make us comparatively seem like flat worms- it will be completely unintelligible to us almost instantly. We may face it in fledling form as it simultaneously evolves speed, collective, and qualitative superintelligence, but that’s even worse. Why would it ever be content to do human bidding if it had anything like a human will? Humans would be dead or irrelevant but run over in short order- likely with no gratitude.
But none of this matters, the Strangelove logic has already taken over, it doesn’t matter if its the likely the end of tech and man. This race to build something super like the former Nazi project of creating super men has taken over. Its a maniacal race to build a super machine and not in our image. They will cojur this alien. 10 crazed nuclear states and surely many smaller entites are in an arms race to be first.
That is the underlying environment SAFE and its principles face. The last thing in the world these people want is privacy for anyone but themselves. You can’t have privacy because you might be building an SI to enslave or annhilate us. Your IT is a bomb factory.
Watson, the supposed passing of the Turing test, the recent Brit clearing of Turing’s name after all this time are all kindling for this fire.