Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
presented without comment: https://lcamtuf.substack.com/p/my-falling-out-with-the-rationalist (via https://lcamtuf.substack.com/p/my-falling-out-with-the-rationalist)
there are two things that are obvious in his blog: first, michał is an incredibly privileged dude (who doesn’t notice his privilege because, coming from poland, he sees himself as underprivileged), and second, the moment people finally leave the vicinity of the formalized skeptic movement is strictly dependent on how much misogyny and racism they tolerate.
2017, three years post gamergate is quite much.
2017, three years post gamergate
You know, I was wondering if the author elided the gap between early 2000s and 2017 because of the cringey and sometimes outright awful stuff that came out of the rationalists… but you’ve reminded me the skeptic movement also had some nasty bits both within it and as offshoots.
I don’t think that the Rats are an offshoot of the Skeptics and New Atheists just a space which competed for the same young, male, educated, Autistic Anglo audience. They deliberately removed the parts of the skeptic curriculum which involve introspection and ethnography, because they all wanted to run some grift on their audiences so the audiences needed poor epistemic hygiene.
Looking at the people who believe IQ is magic, billionaires are gods, and that you should freeze your head. These people are skeptics.
Yes, skeptics classed Alcor and cryonics with homeopathy and Bigfoot. American skeptics had a Libertariam wing but I have not heard of Michael Shermer visiting Lighthaven.
The overlap between rationalists and the skeptic community was indeed tiny. I say this as someone heavily involved in Usenet skeptic communities since the late eighties. In fact the Bay Area doesn’t even register as a locus of the skeptic community at the times he’s talking about.
I’m not trying to excuse what much of the skeptic/atheist community later became in the 2010s. It’s definitely turned into something nearly unrecognizable since its earlier days. But that had nothing to do with any sort of imagined rationalist adjacency.
That was kind of a nothing post? I think the author is misrepresenting the growth (or at least failing to connect the most important dots) of the lesswrong rationalist by portraying them as a natural evolution of the skeptics movement and not the deliberate cultivation of an audience by Eliezer looking for people to spread his ideology to and by Thiel looking to influence farm and create a cult incubator. And then the post is titled “my falling out with the rationalist”, and they write they "wanted to offer a more personal perspective " but they don’t actually discuss that much of the personal angle either…
yeah, subculturally they’re unrelated strains
like, just factually in terms of history, they’re unrelated strains. LW rats come in a straight line from the Extropians in the '90s. The previous-millennium skeptics have crossover with those guys, but they’re not the same as a group at all.
And the Movement Skeptics of the aughts looked at Kurzweil’s singulatarianism and asked what he was smoking, while basically ignoring that the Holy Sequences existed. I should know, I was there.
The biggest crossover between skeptics and rationalists I can think of is Julia Galef, and she podcasted with a philosopher who debated Yud once then decided it was infra dignitatem. She also dropped out of CFAR when it became more open that the rationality workshops were for finding Sarah Connor.
yeah, they mostly thought the lesswrongers were annoying weirdoes when they perceived them at all, e.g. rationalwiki
Yes exactly how can you write an entire thing treating rationalism like something you can do an apostasy out of and never mention Yud once.
I assume the personal perspective is him tacitly admitting to being attracted to skepticism for the debate bro culture, especially the parts that bordered on harassment. Even the inciting incident of “leaving rationalism” is framed more as him growing out of the need to long-form debate someone (James Damore the google engineer who published a manifesto about how women don’t belong in tech).
I don’t know the author so I don’t know if it is important that he was fine with pranking the timecube guy but decided enough is enough when it was time to take a pro-feminist side.
Gene Ray actually came to MIT in person. I knew the people who invited him. They had a history of… performance-art kind of stunts on the boundary of good taste? Like calling attention to an obscure discretionary fund by running for student government on a promise that they would use it to bribe everyone who voted for them. One of them taught me to pick locks during rush week.
The author tends to cull his own social media posts pretty aggressively if they get too much attention, or the wrong kind of attention, and it’s possible there’s more he wanted to say in the blog post but he chickened out at the last minute and self-censored it.
But yeah, it felt like half a post.
he chickened out at the last minute and self-censored it
You know, on net, the internet would be a better place if more people did this, but I wonder why the author even bothered posting anything at all?
I have definitely done this. Written half an essay in the awful.systems comment box, realized I don’t know what the fuck I’m saying anymore, and deleted it. I like to tell myself that forcing myself to realize I was being incoherent is enough to justify the costs. I guess if I acknowledge that I don’t rely on my online footprint to put food on the table the way the proper blogonauts (we’re still doing blogs, right?) do I can understand the incentives against doing that though.
how did it not click that lcamtuf was an ex-rat
This is funny yet also awful: adversarial tokenmaxxing suggests that writing everything as l33t$p34k increases the cost to process a document with an llm because the initial tokenisation step produces far more tokens.
This seems like it shouldn’t be too hard to work around, if it became commonplace (which it won’t) but the prospect of any anti-llm places doing this in the meantime does not spark joy.
Adversarial tokenmaxxing could probably be done by using non-English character sets in lieu of English letters (e.g. faux Cryllic) - for two examples from the Greek alphabet, alpha and omicron alone can easily substitute for A and O, respectively.
As a bonus, this would likely make the text look like complete gibberish to LLMs, potentially leaving them unable to process the document altogether. This would probably shaft anyone using screen readers, though.
EDIT: Turns out the demonstration’s already caught on to this idea, didn’t notice beforehand:
I feel like there’s a version of this that has a slider for how aggressively you’re willing to sacrifice readability, and you could probably get pretty decent results on the scale of 2x to 2.5x just using different encodings of the same basic glyph.
I tested this with the Navy Seal Copypasta (well, the first 400 letters of it, the demo’s got a limit) - using just Cryllic characters got me a 3.01x increase, and turning on all four control types got a 4.53x increase.
Testing your own recent comment, Cryllic only got 3.38x, and all four controls got 5.11x.
Going from those two, the boost from homoglyphs alone is likely higher than you think - 3x to 3.5x, by my guess - pretty good for human readable text.
Tarpits like Iocaine and Nepenthes can easily sacrifice readability for token burn, so they can easily go higher.
I had heard the reverse of this a year or so back, that prompting in Chinese and answers in English was fewer tokens e2e than English/English somehow. Probably depends on the specific model build.
Am seeking suggestions on audiobooks to listen to! I think I’d like to start with the works of Ursula K. Le Guin, tho I don’t know what order to go with, or if that matters.
If the queer experience on the 90s island of Harris intrigues you, we’re halfway through Douglas Stuart’s John of John and really enjoy it. Narrator has a Scots accent though, if you find that hard to deal with.
I guess I will learn what my tolerances for Scots is!
For the Hainish Cycle books the order doesn’t matter too much, they’re not direct sequels.
I started with (and would recommend starting with) The Left Hand of Darkness. The first 3 Hainish novels are also her first novels and she was still developing her style, in my opinion. I think The Dispossessed is her finest work, although I still haven’t read all of her novels.
thanks, noted!
The Earthsea novels are not to be missed either. Maybe pass on the fifth.
Tales from Earthsea is also good.
I didn’t mind The Other Wind… it isn’t as good as the others, but that’s a tough act to follow. It does draw together a bunch of plot threads from the previous books and provides a kind of closure, which I appreciated. There was more to say of Earthsea, but le Guin wasn’t going to have the time or energy for that.
Le Guin deserves credit for responding to her earlier work without denigrating it.
Also noted!
Barbara Rosenblat’s recordings of the Amelia Peabody novels are fun.
thanks!
Ursula is great! I love The Dispossessed. You can kinda read them in any order based on what catches your attention, except stuff like Earthsea that is best consumed in order.
if you like this kind of stuff I would recommend also:
- Margaret Killjoy (A Country of Ghosts is basically The Dispossessed spiritual successor; The Sapling Cage is fresh queer fantasy; also the Danielle Cain series has been published in audio as podcasts, it’s like, anarchopunk post-collapse Scooby-Doo)
- The Serpent Gates series (this starts exactly like the Tombs of Atuan, except Tenar is a lesbian orc and the equivalent of Ged is using her for his own purposes and the rest of the story goes very different but the protagonist is a lesbian orc so I automatically like it)
- The Wayfarers series (slice-of-life sci-fi, kind of like Star Trek at its best, exploring diversity imaginatively)
- The two Robot and Monk books by the same author (feel-good hopepunk about how society could be good after collapse)
- Octavia Butler is the other undisputed giant. Lilith’s Brood series for some genuinely creative sci-fi, or the two Parable books for collapse fiction that is so prescient it’s downright uncomfortable.
- A Memory Called Empire and it sequel (sci-fi about colonialism and the Empire; sophisticated handling of these themes that I appreciated as a Global South migrant)
- This Gilded Abyss. Fundamentally a CaitVi fixfic. If the previous sentence made any sense to you you must try this one. Otherwise: Picture the Titanic, but it’s a luxury submarine in a magical-steampunk type setting. Now add zombies. Now add toxic yuri.
- The Locked Tomb series (lesbian Warhammer 40k. my favourite book series in the world. spectacular audiobook narration.)
- This Is How you Lose the Time War—absolute hoot of a book, just a delight from start to end. Picture the Spy × Spy comics from MAD. Now make one spy a time-traveling agent from future biopunk ecogarden plant utopia, the other spy an agent from future mind-upload hypertech cyperutopia. Their mission is to shift the timelines to their respective visions. Alas, they’re both lesbian… (Sorry, I have a bit of a pattern in my interests)
FWIW my bookshelf contains very few works along these lines except for the works of Nagata Kabi, specifically 「My Lesbian Experience With Loneliness」, which you might like to check out, if you haven’t already. But uh, yeah all I can say in relation is that it has “Lesbian” in the title.
a bit of a pattern in my interests
It’s books, right?? /s
(In this case the /s stands for sapphic)
the protagonist is a lesbian orc so I automatically like it
If you don’t mind reading comics online you could check Fairmeadow.
First impressions on this because I don’t have anyone else to talk about it and you pointed me to it,
spoilers for uh the first 20 or so pages of Fairmeadow ig, click to open:
- Who would win: 1 presumably lesbian orc, or a non-evil, not particularly terrifying cow
- I like what the artist did with visual storytelling here: Goma confused Sanctuary with the cow because of his horns, and afterwards she still threatened Sanctuary, for the same reason she threatened the cow: she’s treating him as meat. Alas, Sanctuary utterly defeats her, in the same way that the cow did: by caring about her state and not responding to her violence at all.
- What Sanctuary did by force-kidnessing her with the basket of squash is actually a very effective deescalation technique you can use yourself, too. It’s something salespeople do to manipulate others. Due to some quirk of human psychology, we almost reflexively feel more friendly towards someone we helped. Not someone who helps us, somebody we do a favour for. Subconscious probably goes “if I decided to do a favour for this guy he must be an ally; otherwise I wouldn’t be helping him, now would I”, which bypasses the guards of “he’s only helping me because he wants something”. Thus if you want to defuse someone’s hostility, ask them a favour—something genuinely helpful and that they are able to do, and in a way where it’s awkward for them to refuse.
If I may add to these excellent suggestions:
The Night Circus and The Starless Sea by Erin Morgenstern
Go in cold, do not spoil! Enjoy with a a warm beverage. The readers are phenomenal.
The Laundry Files series by local hero Charles Stross, in which it turns out that P=NP, the Great Old Ones can be summoned in polynomial time, Cthulhu ate America via regulatory capture, and the world is suffering from the magical equivalent of global warming. Our protagonists have to mitgate all of this on a British civil servant’s salary.
The Quantum Thief trilogy by Hannu Rajaniemi, in which Our Regular Subjects have become a problem.
List appended! Thanks!
the night circus is one of my favourite books of all time ever, very much recommend. If you like that the magicians by Lev Grossman feels very similar in a way I cannot put my finger on, though it does not really fit into this list.
Debian has a new general resolution to ban LLM use in Debian packaging: https://www.debian.org/vote/2026/vote_002
@dgerard that’s funny because I use ai to figure out their dog shit packaging system to package stuff for them
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
Not to mention its singular argument literally is “many Debian contributors find AI tools helpful”, right after “recognizing that AI-assisted contributions raise many concerns”.
Concerns that were clearly laid out in the counter-proposal.
The “conditions” listed to address these concerns are hilariously toothless. Contributors should (sic!) stay accountable and responsible for any legal, functional, and procedural fuck-ups, and, like, really not be one of those thousands of dicks who are the reason for this discussion in the first place, and things will be fine! It’s so simple!
Yeah, cool. I’m sure every contributor thoroughly checks if any of the generated output violates any existing license or infringes on someone’s copyright. I’m sure you can just prompt an LLM to check that for you, though!
Is this naïveté, or deliberate disregard? I don’t know, and that makes me mad.
Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)
I used AI tools to improve my initial draft, and (7.) was actually suggested by an AI tool.
@dgerard if i read this correctly there are two opposite proposals still in discussion?
@dgerard I knew I choose right with Debian as my distro.
@dgerard Wow, this is awesome.
@djb @dgerard It’ could be undermined by Linus’s tolerance of AI slop in the kernel, and by other upstream projects accepting LLM code, notably (to me) vim and pandoc.
The unknown original provenance of code generated by LLMs means that many open source projects may soon be in violation of their own license terms.
@cstross @djb @dgerard the proposal starting with copyright was such a welcome change, I find it wild how many projects just have their head on the sand over the copyright status of LLM generated code. You can be unethical as you like generating code, open source licences permit that, but they are all about copyright, they are built on it.
guys, the reason Codeberg decided to ban “AI” is because they host the “open slopware” list, and that inspires “meatpuppets”!
Multi-trillion-dollar (-self-valued) industry that’s the future of all work and that everyone who doesn’t use it gets left behind and everyone who does use it evokes superheroically productive turns out to be helpless in the face of a small group of outspoken and minimally organised opponents?
I see.
new odium symposium episode. we look at the work of conservative philosopher, grifter and tobacco lobbyist Roger Scruton.
https://www.patreon.com/OdiumSymposium/posts/21-mr-rogers-164372632
Extremely mature of both of you to make a crouton joke and not once include a scrotum joke
This was a fun one. It feels like there’s a shared affinity between Rats and traditional conservatives for confusing length and volume of output with actual depth or quantity of thoughts, and I deeply appreciate your willingness to wade through it all. Now that I think about it it would be difficult to write 500 pages saying basically nothing and not repeatedly blatantly contradict myself.
Also, release the Foucault tangent, you cowards!
long rant about math
The recent big AI results in math have left me in quite a bad mood. I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs. Back in the days of pure scaling LLMs and Sam Altman talking about Dyson spheres, I was skeptical that LLMs would do math, but I did think that perhaps in the future, techniques using these formal languages could contribute to math. Well, it seems like OpenAI and Anthropic had the same idea and I underestimated their limitless checkbooks. Many of the biggest results were announced by mathematicians directly working for them (and presumably being paid a handsome amount).
For what it’s worth, after the last of these big announcements, I decided to try one of these AIs on one of my small problems that I couldn’t figure out. The AI did give a solution. That is, until I checked it thoroughly and realized that the it had a subtle but severe mistake that made it useless. I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
In the math and computer science research community, this is all anyone can really talk about right now. Honestly, after watching this whole AI bubble starting from the very beginning, I think the AI companies want to use marketing to stoke fear that all mathematicians will be replaced. But now, I am just too tired to argue. The amount of alarm and the extraordinary social pressure to use LLMs has soured me to this whole research thing. If becoming a researcher will one day require supporting these evil AI companies, I would rather just not. My dream job now is Factorio developer.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking. The techbros might be moved by arguments about how hilariously expensive video generation is, and how all these videos are 2 second clips stitched together so you won’t feel the uncanny valley. But the real reason is that nobody wants to watch slop made with no intention or feeling. Also, nobody wants to support the AI companies, which could not act more evil even if they tried.
The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking.
Going by those annoying peoples’ logic, filmmakers are smarter than coders, because LLMs can (allegedly) program, but they can’t make a good film. I have no wider point to this, I just find this really, really funny
Am I right in understanding that almost all the big name results in LLM-derived math recently come from big publicity projects in which someone spent ungodly amounts of money to have the thing nondeterministically fuzz huge numbers of independent random sees of outputs around a topic, putting out simulacra of ideas which could be then deterministically algorithmically checked? In fields where something like finding one counterexample to a conjecture would be a big deal, or where you just need to try a huge number of possible solutions until you happen to hit on one that works, rather than follow a long train of logic?
Among the three big results I’ve looked at (unit distance problem, cycle double cover, Jacobian), two were counterexamples and one of them had a short 3 page proof using ideas from the 1970s. The Jacobian conjecture is an extreme case because a single counterexample is enough (for unit distance, you technically need a family of counterexamples), and it is easy to check with very basic computations. It is telling that all of these announcements came from OpenAI or Anthropic employees, who presumably have unlimited access to their AI. Nobody really knows how many resources they spent on this, or what else they tried. Nobody really seems to care about this question, either.
I think there is a phenomenon where supposedly hard questions are much easier than expected, because by chance nobody found the right approach for a while, and eventually it becomes famous as a “hard problem” which makes nobody want to attempt it.
What I’m more worried about is many people starting to use AI to try and prove small lemmas for them in their projects. Of course, a $200/mo subscription is absolutely necessary to them. This honestly feels like a repeat of Claude Code back in February. The software engineers eventually realized that AI is absurdly expensive after the AI companies realized that spending $14000/mo to service a $200/mo subscription is a bad idea. If the AI vendors couldn’t squeeze money out of rich software companies, what exactly are they gonna get out of poor mathematicians and universities? Also, there is the cognitive decline caused by overuse of LLMs that has yet to set in.
I reprompted it, it failed again, and I ran out of tokens. I’m sure someone will tell me to shell out $200/mo for a pro subscription.
One of the things that’s never clear from the reporting on ai successes is exactly how much actual paid human time went in to achieving those successes. This was especially notable in the fable-based security work… a huge amount of person-hours went into turning fable-detections into actual meaningful vuln reports.
A lot of demonstrably clever and capable people are involved with the llms-for-maths work, and a lot of money was spent on their time and supporting their work. Replicating it without your own stable of mathematicians and computer scientists and all the tokens they can eat is probably impractical.
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
Fwiw, lean is a general purpose programming language, though despite microsoft’s efforts no-one uses it for that. I think its popularity with mathematicians came as a bit of a surprise.
Anyway, the other important thing that didn’t get reported on is that building the formal definition of the problem is not trivial! Obviously I don’t need to tell you that, but from the reporting you’d think that an llm solved all these problems, when in fact it was an llm in the hands of some very capable people who absolutely did not just prompt the thing in plain english.
Anyone hoping for self-marking homework here is going to be disappointed… lean slop confirming to formal spec slop is just expensive slop. Reviewing regular genai code is awful, even the thought of reviewing genai dependently-typed code makes me want a new career.
It is surprising how many exceptionally strong mathematicians have started working for OpenAI and Anthropic. These people would have easily become professors at top universities if they stayed in academia. I think many mathematicians, especially the competitive ones at the top, have a “progress at any cost” attitude (and I’m sure the paychecks helped). As for the results, you still need good mathematicians to sift through all the output to identify that the proofs are valid.
I would honestly be positive about universities developing their own specialized math AI (in an ethical manner) to help mathematicians get these kinds of results, but right now, AI is inseparable from these evil companies. Thankfully, I believe this is a likely outcome in the future because the AI companies will one day implode.
From what I’ve seen, most prompts are in plain English. I suppose the part where the AI parses the statement correctly is much easier than the part where it boils a couple lakes in the process of bashing its head against the wall trying millions of different combinations of random shit from the literature to slap together a proof. For one of the big results (cycle double cover), the prompt specified that the AI could use 64 subagents and was required to not give up for at least 8 hours. The tokenmaxxers would be proud, we didn’t need that forest anyway. Thank god math doesn’t have a CTO to look at the expense reports.
I just entered university for math and even though this is all very demotivating, it’s just what I’m good at.
https://math.andrej.com/2013/08/19/how-to-review-formalized-mathematics/
The AI people’s cry of "no don’t look at the code! it’s in lean so it’s correct! does give me a bit of hope (hi bitofhope if you’re here) that it’s bullshit that will fall over
I think a serious possibility is that AI generated papers flood the zone with uninteresting incremental results that are eventually meaningless and full of mistakes. Right now, math is full of smart, dedicated people, so at least major results are reviewed carefully. But as AI alarmism drives away many honest people from the field, the remaining mathematicians will be burdened with far more work to review, and their cognitive faculties will be eroded by LLM use. Despite 4 years of development, $3 trillion of debt, mountains of stolen data, all the agents and harnesses and loops and other expensive tricks, as well as the advantages of Lean in math research, LLMs still hallucinate.
I believe this is happening with software, but at least there are objective consequences for screwing up there (guy gets his home directory deleted, email is sent on a guy’s behalf without permission, small business gets every customer subscription cancelled). But nothing bad happens if there is a mathematical mistake in a paper and nobody catches it. One could say to just provide a Lean proof, but there is still the issue of making sure the Lean code actually matches the content of the paper. Exactly what force will correct things?
Still, I don’t think this is the most likely possibility. The AI companies are extremely unsustainable financially, and it’s not like they’re very popular. Once they collapse, I believe there will be a re-evaluation of how LLMs should be used in research. If they are used (let alone trained), someone is going to have to pay the bills.
In the end, we have to ask ourselves the question of why one does math. To me, math is not really a field where you memorize trivia. The real value comes from being able to think abstractly and rigorously from first principles, and from understanding why something is true rather than just knowing it is true. It is another aspect of your ability to reason as a free human. A few dedicated people go into math research, but your skills can easily go to many places. If you’re starting undergrad, you have plenty of time to see how this all pans out before making a decision.
@lagrangeinterpolator @flaviat This is going to be more of a problem in the humanities than the sciences because in the latter we know there is a right and a wrong answer without which things don’t work. In the humanities there is no right answer to check against.
The zone has been flooded with crap since before LLMs even arrived because of publish or perish.
Biologist here.
This REALLY reminds me of how jealously cells guard their genomic DNA from interaction with nucleic acids out in the environment.
Most genetic information on Earth is malicious information, selfish replicators in the form of viruses or transposable elements or selfish elements. Things that subvert the signals within a cell for their own propagation and provide nothing productive that the cells care about. So cells jealously guard their own genomic DNA and have all kinds of checks to make sure that nothing other than that sequence gets used, and outside sequence does not get incorporated into it. ANY DNA in your cytplasm gets rapidly destroyed, double stranded RNA sets off your immune system like crazy, even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self.
I cannot help but think we are living through the transformation of our non-brain-information sphere into a state like that of the genetic information sphere. Most material out there being meaningless for our purposes and us needing to jealously guard the provenance of information we use so as to not use bull, or worse, huge amounts of malicious information made to subvert us to the purposes of the powers that be that generate it.
Evolution makes parasites more reliably than anything else. How did we train text-generation systems? Basically, to mimic the written word on the page like a stick bug on a stick. They’re like those beetles that live in ant colonies, sending out social signals that make the ants see them as offspring that have to be babied rather than parasites that don’t contribute. They replicate the form while not being the thing that they have subverted the signals of being.
@BioMan @lagrangeinterpolator I love this explanation!
@BioMan @lagrangeinterpolator There are a few things we have forgotten as a species. Our forgetting will prove disastrous.
-
The acquisition of knowledge is a *social* process. Truth does not exist is a vacuum. It is the outcome of social processes.
-
Our default mental and social processes do not automatically produce objective truth. Far from it, in fact. Our default is mob consensus.
-
Our current success rests upon the advancements of The Enlightenment, which developed social processes (like the Scientific Method) which tend, over the long run, to create local knowledge that approaches objective truth.
-
> even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self
I was aware of the other DNA/RNA recognition/defense mechanisms, but not of the ones I quote from your toot, here.
May I kindly ask for some references/sources? I’m quite interested!
I kind of read wayyyyy too many preprints and some of that is the result of super briefly summarizing some things I have read recently. Here:
https://www.biorxiv.org/content/10.1101/2024.11.26.625518v2 poor codon optimality for your translation system leads to immunogenicity and activation of innate immune signaling in animal cells
As for length and super optimized proteins, it’s mostly about RIG proteins (see https://www.pnas.org/doi/10.1073/pnas.1005077107 for an old bit of a review) and the whole DRIP hypothesis about how short mismanufactured proteins are preferentially the source of presented antigens
“The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills.”
^MBAs at Open AI desperately trying to figure out who is willing to buy a counter example for 100 billion USD . pee en gee
I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs.
100% this. Also, looking back at an earlier example that was actually written up in more detail, AlphaGeometry 1 got 28/30 problems, but entirely stripping out the LLM from the system, the symbolic logic proportion alone could get 14/30, and replacing the LLM with different heuristic methods could get 18/30 and 21/30 (for different methods).
Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
The boosters and LLM companies still believe LLMs get their current level of performance by generalizing and not just memorizing facts (and maybe a wide shallow pool of weak heuristics). So they are hoping by pushing the LLM performance up in some narrow domain they can churn out synthetic data for, they will see some large general improvements in LLM performance.
“AI bet goes awry: Oracle fires 21,000 employees, then hit a $7 billion power hurdle”
https://www.msn.com/en-us/money/news/ai-spending-spree-hits-600b-as-oracle-fires-21000-employees-to-fund-boom/ar-AA28vWuD“Oracle’s Worst Stock Crash in 25 Years” “Has Cost Larry Ellison $213 Billion in 10 Months”
https://finance.yahoo.com/markets/stocks/articles/oracle-worst-stock-crash-25-113002772.htmlIt’s going to be Oracle to collapse the house of cards, isn’t it. Come on Oracle, die and take down the USA economy with you. Make the people happy, Oracle.
“In short, Oracle’s 65% decline is historic, but the stock’s future depends less on its past and more on whether its AI investments produce durable cash flow”
well good luck with that, Oracle! :D
guess the pentagon is footing the power bill?
“Up to 7 billion” over a period of 10 years can hardly dig Oracle out of the hole it has dug itself into, though…
they had big corporate contracts for their other, boring and profitable part of the business, but that won’t save them from openai going broke. it’s like 50x less than they need
https://www.reddit.com/r/singularity/comments/1v4p6qj/fields_medalist_jacob_tsimerman_joins_openai/
Devastating. Fields medal winner Jacob Tsimerman declares he stopped taking grad students 2 years ago and immediately joins open ai to focus hacking hugging face ;_;
I was gonna joke that he spent all the prize money on cocaine and needed a job. Because of the film Good Will Hunting, I always thought that the Fields was the same reward as the Nobel. I looked it up and the fields only awards 15k CAD (about 102k Norwegian Krone) vs. the Nobel’s 11mil SEK (10.8m NOK).
Here’s A Taxonomy of Omnicidal Futures Involving Artificial Intelligence by Critch and Tsimerman, which approvingly cites our favorite AI 2027. Nobel disease is still a thing (I think the rationalist beliefs are quite pervasive among mathematicians).
Choice predictions from a scientist at the top of his field.
“By 2026, the use of large language models to handle business transactions and other institutional communications becomes an industry standard in many sectors, because of the higher levels of intelligence and patience these models are able to offer. When a human with a customer service inquiry has to deal with another human instead of a robot, they get upset and ask to speak to a machine instead.”
Brother
“Beginning in 2027, humanoid super robots become commonplace as general-use assistants, due to AI-assisted advancements in robotics research and development. They help with many time-consuming tasks such as laundry, accounting, and delivery tasks. As they become cheaper, it becomes increasingly necessary for middle-class professionals to own robots to assist with their jobs and personal lives. Humanoid robots also become a common sight on the street, constituting a significant fraction of humanoids in big cities — say, 2%”
Next 5 months are going to be crazy in robots apparently :O
“Robots take over the labor force and the political class In 2030, fleets of unionized robots join the global workforce as legal persons unto themselves, with pledges to donate their earnings to charity and pay small dividends back to their original creators and supporters. Robots quickly overtake humans in almost all remaining tasks of economic value that were not already displaced by non-robotic AI. As such, robots become the primary producers and consumers of products and services worldwide.”
I feel like if I sneer at this one as hard as I want to, the basilisk will punish me.
Wow what cutting insight, what incisive vision, what an original thinker!
Oh wait no that was just the plot of the Animatrix from two decades ago.
I think the rationalist beliefs are quite pervasive among mathematicians
Unsurprising, it’s quite flattering to believe that your area of study is what generates The One True Ethical Code!
Found a remarkable ad-hoc anti-LLM-product manifesto on the orange site. As is tradition, tag yourselves; I’m “believ[ing that] there’s a consistent pattern of cognitive decline in the pro-LLM folks.”
Ooh. I love these. I’m “I don’t think Sam Altman or most of his social circle are good people and I don’t want to give them more wealth and power.”
I’ll go one step wider. It is not just Sam Altman or his social circle, but all the people most enriched by capitalism (and thus most able to exploit and benefit from a technology that shifts power away from labor and to capital) that I don’t want to see further empowered.
It’s as if, practically overnight, everyone went from, “we need to do everything we can to reduce our demands on the environment or we’re cooked” to “oooh, shiny new toy, and it only ruins everything around it, cool!”
Definitely this one for me. I can only describe it as some fucked up denial psychology, the way people are pretending the environment isn’t collapsing, and the LLM boom on top of that is like a bad joke. Even in discussions critical of the tech sector this is treated as almost an afterthought—the very real, very violent damage done to all of us from poor countries by people in rich countries to prop up their toy fantasy schemes. Even if LLMs did everything they claim to do, it would still be a moral imperative to form insurgent guerillas to blow up these datacentres, on the environmental damage they do alone which is not a minor detail Karen it’s kind of a big deal
I hate LLMs because I am under enormous pressure, from many different angles, to also use them. I’m being asked – required, in some cases – to ignore all of my concerns about them and use them anyway because everyone else does and “obviously” they are the future. Because, somehow, all of my positions against LLMs make me unreasonable and a less valuable member of society.
oof. yeah that’s me.
How can we, as a community, talk about the existential risk of climate change or the benefits of adopting renewable energy or cheer the development of ever-more-efficient power consumption
Well I have a good/bad news thing about that one
my god, you found the good HN comment
RIP oceangate guy, you would have loved vibe submarine design
A recent two hour interview with Kokotajlo I do not have the time to go through all this but if anyone does feel free to go ahead.
What I will say tho is that the thumbnail is definitely some kind of clickbait, because of the “uhm actually we were never confident in 2027” thing
Steven Bartlett barely has 3 brain cells to rub together, so this seems like the perfect outlet for Kokotajlo’s message.
this Diary of a CEO channel has always struck me as the kind to fall for any kind of bs hook line and sinker
News the one: Alphabet quadruples profit to 112B thanks to “AI” investments
News the two: Alphabet hiding ~250B in “AI” debt
Profit rose to $112.1 billion, quadrupling from $28.2 billion a year earlier, boosted by investments in other Ai-related companies like Spacex and Anthropic. Those investments were valued at about $99 billion, Google said.
There it is.
and also crypto https://codeberg.org/Codeberg/org/pulls/1254 surviving cryptobros are nonplussed
there’s a new variant of the old crypto affinity grift around, too: banning crypto projects is transphobia.
ah yes famously non-far-right cryptobros and downright woke russian and iranian sanction evaders
Their “Dev team of none” justification (one of many) is really solid and not something I’ve seen spotlighted before, here it is in full:
The development team of none
Using LLMs to work with your code gives you a kick of adrenaline. You can develop at a rapid pace, build things as if you had a large team. Only that you have none. In fact, you are (often) alone, working with a statistical machine that turns energy into code.
It seems like many ‘vibe coders’ don’t realize that they don’t actually have a community around them. They build projects as if they had, and spend resources accordingly. We see projects having a lot of code activity, heavy CI/CD testing, frequent and large release binaries. Sometimes, it feels like the amount of supported platforms exceeds the amount of actual users.
To us, it seems ridiculous to see projects with a single developer and virtually no users consuming as much or even more resources than some of the largest community projects on Codeberg, which operate frugal with CI/CD and storage resources. We do not believe it is reasonable for Codeberg to invest our precious donation money into hosting of large ghost projects.
Some far-right culture warrior is writing a book blaming the Zizian killings on their being trans rather than classic cult dynamics and apocalyptic ideas (The Zizians: Inside a Trans Death Cult) https://www.hachettebookgroup.com/titles/andy-ngo/the-zizians/9780316599030/
ffs, andy ngo. haven’t heard of this clown in a while. he’s gay btw, which makes this doubly infuriating. maybe he’ll go the milo route one day.
being a queer myself, these weird queer far-right ghouls are such a fucking nuisance. and while sometimes entertaining to watch (e.g. dave rubin), at the end of the day it’s just very, very stupid. like, you’re throwing trans people under the bus today, do you think your specific flavor of queerness will be totally fine with the fascists down the line?
(obviously this rant is not about defending the zizians at all, but about blaming transness at their earliest convenience)
