I found myself using this as an opportunity to write a commentary on commentaries on Roko’s Basilisk at large, summarising some thoughts that I’ve had for years about how people read it. I was surprised that I found myself even wanting to! So ahead of my disagreements below, thank you for the essay.

What most people don’t understand is that Roko’s Basilisk was such an effective argument because Yudkowsky had strenuously argued for cryonics in the Sequences on the grounds that if you were revived after death, any repetition of your brain’s underlying quantum pattern would be consciously continuous with you

It isn’t thrown into Roko’s Basilisk at random, nor is it thrown into LessWrong theology at random, it’s a metaphysical cornerstone of their ethics and practical philosophy

The reason that it’s incorrect is also boringly philosophical: the argument relies for one of its premises on denying continuity of consciousness; therefore, the conclusion is inconsistent with its own premises and the argument is invalid.

It COULD be rescued pragmatically by rendering the NEW commitment to continuity of consciousness in different terms as, for example, an issue of subjective probabilities (like Pascal’s Wager, “between definitely dying and maybe achieving continuity, what do I have to lose?”) and I suspect that this what a lot of people who notice the problem do, wittingly or not. This solution is also built into Yudkowsky’s practical philosophy, which is replete with wagers of this kind.

In any case, large enough numbers of people evidently buy large enough chunks of LessWrong metaphysics to also buy this aspect of Roko’s Basilisk, to the point of framing it as a genuine infohazard (and then pretending not to have done so later on).

So I think Galileo’s Basilisk is off the mark, and the way that it’s off the mark is illuminating about rationalist philosophy.

2a.

There is a stroke of real genius to the way that Roko constructed his Basilisk out of some of the metaphysical toys he had just lying around near to hand, and most crucially shared in common with other LessWrongers. Indeed, the most glaring logical problem (which I articulated above) is off-loaded onto Yudkowsky, and the rest of the logic is basically acceptable to LWers and extremely simple to follow.

Galileo’s Basilisk, however, asks us to add EXTRA premises to this simple formula. And this works if we think that the continuity of consciousness idea the original relies on is just unmotivated woo (it is woo, but it isn’t unmotivated), the way that G’s interlocutor introduces an unmotivated auxiliary hypothesis to save the old theory of the spheres, permitting G to satirically repeat the same move. But from the perspective of rationalist metaphysics, it is GALILEO who first introduces an unmotivated auxiliary hypothesis, because we do not know how a superintelligence would emotionally handle its intelligence.

Meanwhile, the inferential logic of Roko’s Basilisk is comparatively bounded and secure.

The same problem emerges with Comrade Basilisk: we have to take several inferential steps along the way, through speculations on the value of experience, before we EVENTUALLY get to the classical wager ‘or you’ll burn in hell for eternity’. Rather than playing on beliefs that are ALREADY THERE, it burdens that wager with supporting those inferences (“believe or burn in hell”), whereas in the original the wager leaps naturally out of existing premises (“ACT on your beliefs, or burn in hell”). More on this in 2b.

(As an aside, I think the emphasis on WORKS over FAITH plays an important role in triggering the guilt reflex).

2b.

All of this finally triggers an important distinction to be made with respect to Rationalist vs Christian eschatology, and your treatment of Pascal’s Wager. In effect, LessWrongism logically constrains the kind of God implied by Roko’s theory, so there’s nothing arbitrary going on in deciding what God.

Rationalist sociology is essentially Mandevillian (or Clintonian, for that matter). Their essential model of social justice is that greed (along with other low motives) either produces good outcomes by itself or can/should be leveraged to produce good outcomes. Secondary to this (and emerging chronologically later within the movement) is altruism, which unlike greed has to be properly channeled from the very start, lest any charity be misplaced - a line of thought which progresses rapidly to Building God as the highest ideal in its own way. (by contrast, greed tends to at least create wealth even when unchanneled)

From this point of view, Roko’s Basilisk comes up trumps once again for sheer simplicity. The first human instinct is self-preservation. The second human instinct is altruism.

Contrasted with Comrade Basilisks’s burdening problem, the logic is crystalline. I already believe that my life could be almost infinitely extended by simulations in a superintelligence, all it takes now is for somebody to point out that that superintelligence has an arithmetically plausible reason to throw me in helljail if I don’t put in the work while I’m mortally living. This particular God springs fully formed from rationalist metaphysics plus the Mandevillian see-saw between self-preservation and altruism which insists that the greatest good comes from the leveraging of that instinct for self-preservation.

Whereas Comrade Basilisk does the same on one level (helljail), it still faces the burdening problem, of which some more detail:

Part of the genius of Roko’s Basilisk is that the work the sociology does is loaded on the world BEFORE heaven: we live in an imperfect world whose imperfections we can EXPLOIT (leverage) to get to heaven.

Comrade Basilisk’s eschatology frames the imperfections of our world AS the problem. For Roko, you CAN get to heaven through the eye of a needle, in fact it probably helps to be very rich to build the superintelligence. There is no such easy route in Comrade Basilisk’s world, since part of the very PROBLEM is the great hoarding of wealth.

This is also what distinguishes Roko’s / America’s Jesus (who wants you to be rich, fat, and happy) from (Classical) Christian Jesus (a messianic Jew arguing the virtues of poverty).

Plausibly, there ARE parodic versions of Roko’s Basilisk with more competitive claims on Ockham’s Razor, but I haven’t seen any.

This is all fine, by the way. I think it’s actually good to miss the point of Roko’s Basilisk and just laugh at it. However, it’s also useful to have a deeper theory.

I think that this ultimately points to a more complex engagement with political reality when it comes to the “what do we do” question. LessWrongians have a theory of political economy, which I described above at the three levels of greed, greed leveraged, and altruism, in descending levels of importance. The crucial and most deeply appealing feature of this vision is that it is virtue minimalist, and at the base layer essentially indulgent and even encouraging of a wide array of baser impulses.

Rival visions demand a bit more. They demand, for example, giving up your free time in comradeship and some of your secular ambitions in a recognition that the private accumulation of wealth is bad for the wider public and the world. For the fanatics, of course, Roko’s requirements are a bit stronger, but that is - of course - why they are the elite, and for the great majority of people the Basilisk is nothing but an infohazard.

Maybe you’re beginning to see where I’m coming from, so I’ll just say that it would take a whole other essay to outline that the problems with Roko’s Baslisk are less problems with LessWrong than they are the with political economy of liberalism. That’s an absolute clanger to finish on but I really don’t have any more space or time. A next line of thought would be to outline how any of this ties to the dream of immortality.

For what it’s worth, my personal preference is for a virtue maximal anarchist solution. Clean your soul!

If somebody talks very bluntly without caveats about a wide range of intellectually demanding subjects, and has a tendency to explain themselves with links to wikipedia articles (whether that’s about Mythbusters, psychoacoustics, or “High Modernism”*) I tend to get about as suspicious of that person and their motives as I do of other types of people who do similar stuff, from Eliezer Yudkowsky to Christian Revivalists, even if those motives boil down to nothing more threatening than an unchecked need to show off

*Come on man, you really should have read that one to check that it wasn’t a creepy screed that should have been deleted a long time ago

deleted by creator

i don’t know why people think your comment is good, because it doesn’t actually add anything about the numbers, even though you pretend to link to mistakes (actually the wikipedia pages for mythbusters and spiders georg)

i’ve never heard of benn jordan before, i’m just suspicious of your self-confidence in general

the last time i replied to something you said it was because you were astonishingly wrong about 20th century marxist (and post-marxist) theory, and you had the same air of breezy knowledgeability back then as well, even though you had evidently not read a single word of the names you dropped

https://awful.systems/post/6533847/9681534

i was much nicer than i should have been about it at the time, but The Work of Art in the Age of Mechanical Reproduction is an EXTREMELY short work. and you think it’s about “groups of artists”? Are you SURE?

deleted by creator

as Soyweiser points out above: noted harsh critic of rationalism and AI (the guy who makes) SMBC is a “bold choice”

surprised that CCP Grey guy is still around - the asshole who refuses to post corrections to factual errors in his infotainment videos? can you imagine a more perfect fit?

re: The Last Psychiatrist, is he a postrationalist? The impression I have always had is that rationalists read HIM, not the other way around. His odious Hunter Thompson impression is more appealing to them than is possible to generate from within the movement, I think

By all means try to use this blog post to hold onto whatever community you happen to find yourself in. It is a toolkit for strangling a scene to death and feeling good about it. The gamergate handbook for smearing people (not Shermer, just women) you happen to want out of your company.

Just restating what Chapman says is not a correction? Perhaps I wouldn’t come out swinging like this so hard except that perhaps nothing makes me more angry than being told I just don’t understand something blindingly straightforward, and perhaps if I did I would agree with you. Do you think other people just aren’t as smart as you are? Do you talk like that to people in daily life? Who the hell do you think you are?

Chapman’s fanatics are (eg.) the people who organize the independent film fest and talk endlessly about independent films but don’t direct or act in their own films, or the people who reply to posts but don’t initiate their own threads.

Yes, I am not confused, and this is precisely what makes his terminology useless. The people who reply to posts and talk endlessly about stuff are frequently deadbeats, but this is not captured by his pseudo-technical terminology. Chapman has defined “fanatics” self-servingly as the people who do the productive work and defined them against “mops” as people who are less interested and do not do productive work: it is a false and a very stupid distinction for the reasons I have outlined

Personally I don’t think I have met anyone fitting this description: “They love to be a part of something but they’re too insecure to let other people love it too, and they lose their ability to meaningfully contribute because they’re so busy policing the boundaries of the space.”

Yes, this is another problem with the essay. Since it only serves to flatter the prejudices of a sympathetic reader, the dynamics it claims to point out will only be recognisable to somebody willing to indulge those prejudices, whereas to the unsympathetic reader it just doesn’t have anything to say at all.*

*slight correction: it doesn’t have anything interesting to say at all. It does perform adequate speech-acts viz. draws the boundaries of what Chapman considers acceptable participation in a scene according to his “judgement” about who is and isn’t a fruitful and productive member of his little society. But that makes it an effective piece of political propaganda, a worthy role but not the one he would like to intend for it.

Hall published a noncommittal review of a dodgy-sounding book

I don’t even know who Harriet Hall is, but this undersells it. Just in the wikipedia link that we can all see, she says that the (fake) issues raised by Abigail Shriner in Irreversible Damage, “urgently need to be looked into”. That book, far from a mere “dodgy-sounding book”, was an enormously influential, best-selling, and intensely transphobic avalanche of lies and self-serving distortions.

Benjamin’s The Work of Art in the Age of Mechanical Reproduction is about how groups of artists establish symbols

That work is not primarily about symbols or groups of artists at all, and insofar as it is about symbols and groups of artists it is about how mechanical reproduction diminishes the ability of artists to control the symbolic structure of their artworks. With that established, that work is primarily about art in a technological society, and about how technics shape (a) art, (b) politics, © the mediating interface between the two (which is itself technology in “the age of mechanical reproduction)

Debord’s The Society of the Spectacle is about how consumerist states cultivate mass consciousness through mass media

For Debord, the state is merely part of the Spectacle. The book is about how capitalism itself mediates culture through mass media. The idea of “mass consciousness” is too vague to get much of a look-in here, although the book could be plausibly explained as developing the vague idea of “mass consciousness” into something more concrete.

I have been sufficiently tempted to point out one of the ways in which this is horribly stupid viz. it is written from the perspective of somebody who is a liability to whatever scene he claims to have been a part of

Fanatics want to share their obsession, and mops initially validate it for them too. However, as mop numbers grow, they become a headache. Fanatics do all the organizational work, initially just on behalf of geeks: out of generosity, and to enjoy a geeky subsociety. They put on events, build websites, tape up publicity fliers, and deal with accountants. Mops just passively soak up the good stuff.4 You may even have to push them around the floor; they have to be led to the drink. At best you can charge them admission or a subscription fee, but they’ll inevitably argue that this is wrong because capitalism is evil, and also because they forgot their wallet.

Everybody with half a brain knows that actually fanatics are frequently fucking deadbeats, who are therefore incapable of materially contributing beyond their physical presence. That’s fine, but it doesn’t lend itself to the financial stability of the collective enterprise, especially if they expect to get free stuff out of the bargain, which they frequently do. Of course, this isn’t the case for everybody, but that just proves the point that this is unbelievably fucking stupid

Mops relate to each other in “normal” ways, like people do on TV, which the fanatics find repellent. During intermission, geeks want to talk about the New Thing, but mops blather about sportsball and celebrities. Also, the mops also seem increasingly entitled, treating the fanatics as service workers.

“Fanatics” like this treat whatever community has sprung up around its artists as a vending machine for personal connection and social clout. They love to be a part of something but they’re too insecure to let other people love it too, and they lose their ability to meaningfully contribute because they’re so busy policing the boundaries of the space. This isn’t a “fanatic” actually, because again, it’s just a(n extreme and highly idealised) type of guy, but again that is proving the point of the stupidity of this enterprise.

Mops also dilute the culture. The New Thing, although attractive, is more intense and weird and complicated than mops would prefer. Their favorite songs are the ones that are least the New Thing, and more like other, popular things. Some creators oblige with less radical, friendlier, simpler creations.

cf. Bob Dylan, “Judas”

Even reading just a paragraph of the article you get the sense that Chapman is an insufferable dork who feels somehow burned about something in his past

The strangest thing about it is that the complaints are played so much to such an intense stereotype it’s as if Chapman is the only person in his model who actually embodies what would otherwise be a ludicrously idealised and caricatured type, and it turns out to be the horrible deadbeat superfan who ruins everything trying to make it his personal possession

Chapman is a fucking moron, and not rationalist curious but deeply embedded in rationalism. “Post-rationalism”, when it was new, was nothing more than a way of being into (at the shallow end) Deeprak Chopra type shit for personal growth on “rational” grounds (“if it works it isn’t stupid” or whatever) without getting kicked out of the clubhouse. It’s harder to see those outlines now because mainline rationalists have effectively adopted that plus far more extreme attitudes in their day to day over the last 5+ years, so post-rationalism looks harder to understand and more interesting than it really is or was.

Chapman himself was trying to do rationalist existentialism (hence his title, “Meaningness”, the quality of being meaningful, with particular respect to “having meaning in one’s life”).

Of course he was naive, but he’s also just writing yet another completely oblivious ass-pulled blog pretending to do meaningful sociology with just whatever shit came off the top of his dome. It’s identical to everything else written in this regard within 15 miles of LessWrong and should therefore be ignored except insofar as its laughed out of the room.

I knew they were writing under fake names

read that sentence back in a mirror to me

This isn’t a joke either. Read it back in the mirror. To ME. What do you think you see when you look in the mirror? WRONG.

Dogshit writing as well, “we would never wish for a war to occur”, read that sentence back in a mirror to me

War is the greatest human tragedy, but defence is indispensable. With our commitment to rebuilding the United States’ defensive industrial base, we at Ares aim to ensure this country is prepared to halt any conflict rapidly, and save countless live.

It’s true though. They’re not nazis. They’re incapable of being fired by any fundamentally political or spiritual ideals, no matter how ultimately black and nihilistic, at all. Even if these people were full-throated card-carrying members of the American Nazi party marching through Times Square with a swastika flag throwing out copies of Der Sturmer from a Panzer tank they wouldn’t be nazis. The fact is that they’re just the purest distillation of 20th-21st century media culture yet: they’re so utterly saturated in media that the only choice they’ve made, the only choice available to them, was whether to lean into the goodie or the baddie vibe, and they plumped for “baddie” because it suited their contrarian aesthetic and then, without even leaving a ripple on the surface, they slipped into the role and inhabited it so thoroughly that it is, literally, indistinguishable from who they are.

These people are nothing less, and 100% nothing more, than your childish glee at getting to play the villain in an RPG.

Yeah, it’s a philosophical question, which means you need a philosophical answer. Spitballing won’t help you figure shit out a priori because it turns out that learning how to think a priori effectively takes years of hard graft and is called “studying philosophy”. You should be asking people like me what “know” means in this context and what distinguishes memory in human beings from “memory” in an LLM (a great deal, as it happens!)

but all of this stuff is still relatively new and I’m sure it’ll get better with time

What is the exact point of taking this attitude? Anybody who cares to look knows exactly what’s wrong with this stuff. It’s an astonishingly, and I mean “astonishing” as in “actually beyond ordinary human comprehension” as in “literally awe-inspiring”, wasteful means (whether your energy source is fossil fuels or solar!) of doing - at the absolute outside best - extraordinarily basic shit. Every single day the window of useful applications and potential improvements narrows incredibly rapidly, and the people who are fundamentally steering the whole programme are proven liars and scam artists, and proven beyond any shadow of a doubt at that?

Who cares if it’s relatively new, or if there’s room for mild-mannered optimism? What practical teeth does that argument have? What purpose does it actually serve beyond satisfying a basically shallow political impulse to moderate perceivedly heightened emotive responses to these incredibly stark facts?

The only actually reasonable response to this farrago is full-throated opposition to every element of the whole show which is either a lie or covering for a lie, which is virtually every single element. If all that you’re left with is “hey, transformers are pretty cool, and I look forward to seeing how they contribute in their own partial way to our collective technical means of saving the planet, and incidentally anti-trust legislation should put people like Altman behind bars for the rest of their lives” then so be it! That’s a far more even-handed and fundamentally sensible response than blithely insisting that the occasional trinket has room for improvement - in fact if you’re liberal-minded it’s the essential output of any sensible thoughts on how to maintain a democratic society.

Honestly? Improvement

It appears that many of you have been hiding full blown hardout sneers from SneerClub, and I am baffled as to why