I have long held in question the idea that humans ‘are intelligent’. Especially at the collective registers. Clearly, the capacity to assemble complex objects is not the qualifying feature we’ve long thought it to be. Human intelligence is a possibility space, not a status.

In recent interviews, Eric Weinstein suggested something I find compelling, that, rather than our recent AI developments demonstrating that machines have minds, they have had quite the opposite effect: the demonstration that, at least, much of the time, we do not!

Methods demonstrate that merely improving on existing determinations ( by a complex vector-based weighting system ) of ‘which word comes next’ accounts for much if not most of the LLM’s behavior. Especially catastrophic mistakes and delusions.

Perhaps, much of the time, our own ‘intelligence’ is doing something quite similar. One possible exception space is that of metacognition ( one of my favored haunts ).

It should be clear that a »species that is not intelligent enough to know »what must not be done, is not yet intelligent. If ‘science’ as practiced by moderns lights the fuse of a multi-polar apocalypse that leads to vast extinctions or the end of life on Earth, it will invalidate »the entire history of human conception, invention, insight and discovery. This fact should reign over »all human thought and endeavor… and the fact that »it does not, should »force us to ask »extremely sober questions about what does, how it become dominant, and how we can defuse and replace it. Now. Not later.

In every waking moment and action.

I have long felt that any scientist possessed of even a modicum of moral or ethical concern, would conceal »all of their research, and »never publish — until and unless the species could demonstrate the establishment and maintenance of societies actively capable of preventing the weaponization of findings. For without such a society, as Frederic Brown noted in his story ‘The Weapon’, science is effectively the act of giving a loaded gun to a child who doesn’t understand harm… and pointing it directly at the history and future … not merely of humans, but of life on Earth.

No sane or intelligent mind would »ever do that.

Right?