

Or conversely, Eliezer is vaguely aware it is a factor in cults, but figures as long as he only does it occasionally against the sneerers that obviously really deserve it he is safe from cult dynamics (or at least the downsides of cult dynamics).


Or conversely, Eliezer is vaguely aware it is a factor in cults, but figures as long as he only does it occasionally against the sneerers that obviously really deserve it he is safe from cult dynamics (or at least the downsides of cult dynamics).


One way or another they will find some rationalization to explain away inaccuracy of both the doomers and boosters’ predictions. Whether that is blaming the Democrats or sneerers or continually tacking on a few more years to the predicted doomsday date I can’t guess for certain.


There is loads of stuff they didn’t get right. The benefit of the narrative format is that any specific details they get blatantly wrong they can spin as just being part of the fictional narrative and not a deliberate prediction. They can also shrug off discredit by claiming that only their technical predictions were accurate, not their political predictions. In fact, as we’ve speculated recently on sneerclub, one move the boosters may make is blaming the bubble pop (and subsequent failure to produce AGI) on the Democrats winning, because obviously if the LLM companies had kept up their momentum just a few months longer surely they would have achieved recursive self improvement (and all the wondrous things they’ve promised that would follow).


I’m actually enjoying how relevant AI 2027 is staying, it’ll make it harder for the boosters to downplay it and easier for us to sneer at it over the next few years as it becomes increasingly blatant how wrong it was.


They are balancing the needs of getting bagholders to buy in in the first place (which means keeping the IPO valuation lower) and continuing to project confidence and hype.


means to an end.”
The end is “get enough money for just one more training run bro” both because they hope to reach recursive self improvement magic AGI God in the next training run and because their entire value proposition is centered around releasing “better” models.


Well obviously they wouldn’t bother calling themselves, that never accomplishes anything. They will instead raise money for a new lobbyist think-tank.


“Since you [Ms. Congresswoman] can’t manage an international total ban on all AI progress, the only alternative is to bail out my company so we can maybe have a small chance at inventing a friendly AI before the Chiense invent skynet. Obviously the Chinese are behind us and have only gotten as far as they have by distillation attacks to steal our models’ training (that we trained with honestly-and-totally-fair-use acquired scrapes of the entire internet), BUT if you just make us stop, the Chinese will catch up in 6 months and then we will have Chinese skynet, or worse yet, an AI aligned to communism and the CPP!”


Well, they’ve also reached model sizes where scrapes of the entire internet aren’t sufficiently large enough to train the model, so they also need more data. But yeah, they compute cost of actually training the models are prohibitively large.


or they are the ultimate politics understanders.
A lot of them fail to grasp how rightwing they (rationalists) actually are. Others have ideas that are extremely simplistic, like there are lots of lesswrong post proposing one cool trick to get trump on their side or get normies to understand AI doomerism.
The political reaction was not “please help the AI companies figure out AI safety,” but “shut that shit down right now.”
Yeah, they have all these big brain plans for research programs into aligning AI, not getting that if they actually get the public or any major political figures to really believe their doomer ideas “shut that shit down” with maybe a side of “throw some people in jail” is the common sense response.


They are capable of ignoring Eliezer’s instructions, but only when it means making an even worse decision.
Okay, actually, Eliezer used GenAI to slop out the picture for the visual novel we mocked recently, so I guess they just don’t care about trying to make an individual stand?


This has a lot of “are we the baddies” energy! I’m almost surprised there aren’t more tech workers that quit like this person apparently did? I say almost, because it doesn’t seem that hard for the existing pattern of jaded cynicism and/or greed to cover LLM companies.


On the upside, they probably won’t actually succeed at anything novel. On the downside, having workers breed anthrax (or something along those lines were the information is out there, if not necessarily publicly well known) under a chatbot’s instructions will still be bad.


What happened to the mythical cybersecurity abilities of Claude Mythos?
No, see, AI is only good for asymmetrically advantaged at generating headlines and marketing copy finding non-issues and known issues offensive cyber operations.


https://xkcd.com/808/ . I think the XKCD undersells the ability of capitalists to chase dumb ideas, especially in the short term. Like Theranos was only a step or two up from Homeopathy in terms of bad science?


VR did get a preemptive attempt to enshittify it with the Meta-verse, and the tech isn’t quite there, but it is coming along. 3d printers are nice, haven’t been fucked up yet as far as I known.


It was something stupider and sillier yeah. The link explains, but to tldr; exp(some complicated term) where the “some complicated term” has a division by a number that eventually goes to 0. I think it was… them trying to represent “horizon time” (as in that one stupid METR graph that gets repeated everywhere) increases happening faster and faster?


Actually it was literally an asymptote in the first version of their models they posted to github. They had a term that they were dividing by that ends up going to zero, so the entire equation breaks! https://www.lesswrong.com/posts/PAYfmG2aRbdb74mEp/a-deep-critique-of-ai-2027-s-bad-timeline-models
So, now we can answer the question from before: what time horizon does the red curve above predict for 2030?
Well, if you plug it into wolfram alpha, you get a time horizon of “-2542 - 11372i”
Yes, that is negative, and that is an imaginary number in there.
What’s actually happening is that when the term in the brackets above hit zero, we end up dividing by zero, and hit infinity. Beyond that, we get a negative number to the power of a non-integer, which gives nonsensical complex numbered answers.
The infinity point for this equation happens at a time of t = To/alpha, and will always occur at some point, no matter what initial parameters you use.
In fact, this infinity point is completely independent of both the time horizon and the SC threshold. If you keep the same alpha and doubling time, you could start with a time horizon of a nanosecond, and have a superhuman coding threshold of 1 trillion years, and the curve will still claim that superhuman coding will arrive before the end of 2029.
And indeed, reddit user mambo-12345 tried modifying the simulation parameters to drop the initial time horizon to 15 nanoseconds instead of 15 minutes, and the resulting curve still had a peak for the estimated superhuman coder arrival at 2026.5 years and a median at 2035. Credit to them for inspiring me to take a closer look here:


Judging by the way LLMs and generative AI has been used, even when we do get something vaguely like a realistic version of sci-fi fantasies, capitalism and people who can’t distinguish the limits of the real thing from the fantasy will manage to misuse it.
He did lie to them and tell different people different stories to try to manipulate them. But you are right among the things he lied about was AI “safety” related stuff, and (as samvines describes) he lied because he wasn’t a true believer like they were. (And thus tried to bypass their oversight in releasing ChatGPT.)
Lukewarm take, considering that LLM models could use a lot more thought and care in their development and release (although obviously not for the reasons the doomers want), the EA cultists were actually the (relatively) good guys in that whole saga of Sam Altman’s firing and rehiring.