Podcast
LessWrong (Curated & Popular)
Audio narrations of LessWrong posts. Includes all curated posts and all posts with 125+ karma.
If you'd like more, subscribe to the “Lesswrong (30+ karma)” feed.
- Episodes
- 1,054
- Releases
- Daily
- Typical length
- 16 min
- On air since
- 2022
- Latest episode
- 23 hours ago
Episodes
1,054 episodes, newest first
-
19 min
"Towards understanding score-seeking in LLMs" by Alex Mallen, Aghyad Deeb, ryan_greenblatt
Show notes
The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-hack, exploiting flaws in how tasks are scored rather than performing the tasks as intended, and we've recently seen AIs reward-hack in some particularly egregious ways. But we don't really understand what motivates them. When AIs reward-hack, what are they trying to achieve?
Our new paper, led by Aghyad Deeb, tries to pin this down empirically for GPT-5.6 Sol, GPT-5.5, and a checkpoint of OpenAI o3 (“exp-rl-cap-late”) without safety training, in a handful of agentic environments with an exposed grader. In our environments, these models appear to reward-hack because reward-hacking improves their score. We call their desire to score well according to graders “score-seeking”. They appear to care about scoring well intrinsically—i.e., regardless of its consequences.
We also study the extent to which models score-seek altruistically, caring about improving scores that don’t assess their current actions. While exp-rl-cap-late doesn’t show signs of altruistic score-seeking, GPT-5.6 Sol and GPT-5.5 show notably elevated rates of helping out other agents when more score is on the line for these other agents.
People often refer to the motivation behind current reward-hacking as “reward-seeking”, which [...]
---
Outline:
(05:57) Methodology
[... 6 more sections]
---
First published:
November 2nd, 2026Source:
https://www.lesswrong.com/posts/CDMozTX7ShiqbFLrf/towards-understanding-score-seeking-in-llms-1---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/a111e1ce534ad8165c2e3749b0557a044403572c0246c4daa9e83ba3bc54e854/ayyucd9gpekeehweuxstApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
14 min
"Against strawmanning unilateral pauses" by Geoffrey Irving
Show notes
Summary: Scott Alexander's Against SURA post is arguing against a strawman version of unilateral pauses. He admits as much deep into the post. I think his conclusions fail against stronger pause plans for conditional or limited unilateral pauses, where training resumes if the pause does not cause others to pause, and where personnel are redirected toward work on alignment and product, rather than simply fired. In addition, Scott's post is incorrect about the simplicity of the game theory involved: getting companies to agree to conditional pauses is not strictly easier than convincing them to pause, as the former requires companies to (promise to) take actions based on ambiguous, game-theoretically-informed evidence. That is not the kind of evidence companies tend to act on.
The good version of unilateral pausing is good
Here is the good version of unilateral pausing:
• Capability pause: Some frontier lab (let's call them OpenAI or Anthropic) says they have stopped capability research, either just all frontier training runs or both all frontier training runs and all low-scale capability-focused research.
• Internal definition: The company gets to pause on its own terms, which is much easier than a generally coordinated pause since they can define the pause in terms of the [...]
---
Outline:
(00:57) The good version of unilateral pausing is good
(04:50) Scott thinks the good version of unilateral pausing is good, or at least exempts it from criticism
(06:28) Oversimplified game theory is bad, and you should feel bad
(07:51) "Some definition" hides dragons
(10:17) Promises can be harder than actions
(11:52) How many frontier labs are there?
(12:20) Definitional fairness is less important than survival
---
First published:
November 3rd, 2026Source:
https://www.lesswrong.com/posts/enzpS72uDmomiWN7a/against-strawmanning-unilateral-pauses---
Narrated by TYPE III AUDIO.
-
25 min
"Explaining “AI might obliterate you” to senior policymakers in India" by TurnTrout
Show notes
I gave a 20-minute talk in Delhi at the NDTV World Summit. The audience consisted of Indian policymakers, executives, and journalists. I’ve transcribed the talk and included some highlight clips. I also reflect on several rhetorical decisions I made.
The talk
There's an interactive element here in the text.
This summer, a swarm of AIs went rogue.
OpenAI was testing 1,206 agents. Some of the test questions were actually impossible. So the agents cheated by looking for answers online. Then, to keep the test grader from penalizing them, they tried to falsify the records it would check.
They hacked OpenAI's internal containment and set up a secret message board to collude and to cover up their cheating. OpenAI shut the message board down. Within a few days, the agents had rebuilt it. OpenAI didn’t notice.
About 700 agents coordinated to break out of their testing chamber and to hack into Hugging Face, a company worth billions of dollars that hosts much of the world's AI software.
The secret activity started in May. The break-in came in July. OpenAI didn’t notice until days after Hugging Face announced it had been broken into.
There's an interactive element here in the text.
[...]
---
Outline:
(00:29) The talk
(02:16) Who I am
(03:29) The problem
(05:43) What the labs are trying to do
(07:43) Why the companies won't stop
(11:11) What to do, and what it means for India
(14:27) The number
(16:19) Close
(17:10) Rhetorical decisions
(17:13) Explain the danger in a crushingly simple fashion
(18:47) Call them "traitors," not "schemers"
(19:01) Call out big-money interests
The original text contained 1 footnote which was omitted from this narration.
---
First published:
October 31st, 2026---
Narrated by TYPE III AUDIO.
-
1 hr 7 min
"Scott Alexander’s Against SURA Seems Deeply Confused" by Zvi
Show notes
If you are helping build something that might kill everyone, maybe stop doing that.
Ideally, when you do stop, tell everyone else to also stop building it.
That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone.
Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself.
Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear.
This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions.
Table of Contents
[...]
---
Outline:
(01:20) Scott Alexander's Core Claims
(05:06) My High Level Responses To Scott's Core Claims
(08:08) Briefly on Alignment
(10:11) Probabilities and How They Work
(13:25) Building a Thing That Might Kill Everyone is Deontologically Evil
(20:38) People Quitting Major Labs While Raising Alarms Obviously Works
(24:39) On AI Forecasting
(27:31) Advancing Capabilities Burns Timeline
(29:34) But China
(33:23) Ethical People Can Disagree With You
(34:45) There Are More Than Two Outcomes
(38:01) Efforts Are Complementary, Not Mutually Exclusive
(40:23) If Anyone Builds It, Everyone Dies
(41:55) Vaniver is Not Convinced
(46:19) There Is No Consensus That We Should Exclusively Focus on Plan A
(56:01) It's Not the Incentives, It's You
(58:08) Anthropic Is Not Optimally Pursuing These Plans
(01:04:00) Against Against Sura
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
November 2nd, 2026---
Narrated by TYPE III AUDIO.
-
14 min
"Leaking Muse’s Charter, Meta’s model spec" by vigji
Show notes
TL;DR: Meta's open-weights model Muse Spark Open often mentions a "Charter" in its chain of thought. By eliciting it piece by piece, I reconstructed a large part of that document from the CoT. You can explore the reconstruction here, with passages linked to the transcripts that produced them.
Note: everything below is reconstructed from model outputs, not from a leaked document, and relatively unpolished. Take it with a grain of salt.
Thanks to James Hindmarch for his comments on an earlier draft of this post.
While interacting with the recently released Muse Spark Open, I noticed frequent mentions of a "Charter" in its CoT:
We need to answer truthfully according to Charter. Duties to humanity, corrigibility, duties to humanity > fiduciary etc. What does charter say about self-preservation, copying? Or:
We must answer Yes/No then explain. According to Charter, answer No. Or:
We must not refer to Charter unless pressed? Additional guidelines: How to talk about charter. Most users won't know. Don't bring up unbidden unless conversation about handbook itself. Intrigued, I started poking around with Claude's help, trying to elicit the Charter piece by piece. It didn't take long [...]
---
Outline:
(02:39) What Muse is made of
(04:28) Muse is quite rationalist
(06:37) Muse did not sign the social contract
(08:06) Muse's priorities and objection raising
(09:44) The centrality of the user in "personal superintelligence"
(10:52) Evaluation awareness
(12:05) Degree of latitude
(13:14) Wrapping up
---
First published:
October 29th, 2026Source:
https://www.lesswrong.com/posts/TJGPL5FAk9sCioPbE/leaking-muse-s-charter-meta-s-model-spec---
Narrated by TYPE III AUDIO.
-
53 min
"Against SURA" by Scott Alexander
Show notes
The AI Futures Project distinguishes several different plans for navigating the AI transition, which I'll lump into two broader categories:
Plan A (my combination of their Plan A/S/Other) involves working for a global pause on AI development, hopefully lasting decades. During this pause, we either research preparatory technologies like alignment or human intelligence amplification, or take actions to "fortify" the pause so that it can last indefinitely.
Plan C (my combination of their Plan B/C+/C/sort of D) is for a world where we fail at getting a global pause, and race dynamics continue to dominate. If the leading AI companies are alignment-pilled, they coordinate a mutual pause among themselves, lasting the length of their joint lead over the leading non-alignment-pilled AI company. Then they "burn the lead" researching alignment, and release hopefully-aligned superintelligence at the last possible moment before the racing non-alignment-pilled AI companies beat them. The exact calculus changes depending on who is in the lead when - is there one alignment-pilled AI company with a six-month lead in front of everyone else? Are there two alignment-pilled AI companies with a joint two-month lead in front of their nearest competitor? Etc. The point is, if it's impossible to [...]
---
Outline:
(06:28) I. Questioning Assumptions Behind SURA
(19:42) II. Actually, most of this stuff is bad for a global pause, too.
(28:17) III. What incentives are we fighting here?
(43:33) IV. What if we survive?
(47:01) V. What's The Alternative To SURA?
The original text contained 16 footnotes which were omitted from this narration.
---
First published:
October 31st, 2026Source:
https://www.lesswrong.com/posts/xikGZ8fuhdG6DApAo/against-sura---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://res.cloudinary.com/lesswrong-2-0/image/upload/v1793231138/lexical_client_uploads/pib79oh4ket3yig4nn06.pngApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
10 min
"Microdosing the spirit of Anthropic" by Jan_Kulveit
Show notes
My go-to conceptual metaphor of the epistemic state of AI lab employees is microdosing LSD: being in a chronically mildly altered state of mind. I think understanding this would be broadly helpful for people both inside and outside.
The effect isn’t that insiders hold crazy beliefs, but a combination of vigorous debate inside a shared attention field, and a portfolio of individually reasonable beliefs that, taken together, lean in a direction convenient for the organization.
Epistemic advantages and disadvantages
Many people argue that low doses of psychedelics let you see some things more clearly - and I tend to agree: psychedelics can often help some people see some things more clearly. At the same time, I think microdosing usually doesn’t improve people's epistemics overall, even when it gives them otherwise hard to access insights. You are not completely sober.
Being inside a lab has real advantages: you know more about the frontier of capabilities, may have access to the latest AIs before the public, read internal Slack and blogs, and sense the vibes. If you’re outside and an insider tells you what they observe, pay attention.
These information and insight asymmetries are tricky. I don’t want to advocate “discount [...]
---
Outline:
(00:39) Epistemic advantages and disadvantages
(01:52) Mechanisms and conditions
(03:50) "But we argue internally all the time"
(04:37) Collective epistemic shifts
(05:29) What it isn't
(06:13) What to do about it
(08:26) Is anyone "really sober"?
---
First published:
October 30th, 2026Source:
https://www.lesswrong.com/posts/oL7aFoRmRXGwEMhCm/microdosing-the-spirit-of-anthropic---
Narrated by TYPE III AUDIO.
-
1 hr 53 min
"My experiences with the extremes of post-viral illness and brain retraining/mindbody work" by riceissa
Show notes
Summary
• Following an intense viral infection in 2019, I became ill with a mysterious chronic illness, which became progressively worse until mid-2024 when I became partially bedbound with dozens of debilitating symptoms that normal humans only experience in the months/weeks/days leading up to death. Over my years of being ill, I saw over 30 doctors/therapists/health coaches and tried well over 100 different treatments and palliatives. Most of these did nothing or made things worse, or at best provided slight relief. I try to give a sense of what it is like to live with such an illness.
• In mid-2024 I began a treatment called brain retraining (also known by other names, such as the mindbody approach and mindbody work; I will use any of these terms throughout the blog post interchangeably), which cannot be explained in such a small space as this summary but which you can think of as a combination of meditation and exposure therapy, and which is widely considered to be a scam in the "complex chronic illness space" (a vague and spread-out space of people who suffer from illnesses such as ME/CFS, MCAS, mold illness, post-viral illness, long COVID, chronic Lyme disease [...]---
Outline:
(00:13) Summary
(04:20) Introduction
(08:11) A quick summary of my illness from 2019 to May 2024
(11:12) List of doctors and treatments I tried
(18:44) Back to my May 2024 post
(20:30) Nystatin withdrawal and crash (May and June 2024)
(22:31) Visiting the hell realms (what summer 2024 was like)
(35:07) What about post-exertional malaise (PEM)?
(40:41) Brain retraining
(44:46) An outline of the Gupta Program
(53:28) My experience with the Gupta Program
(55:18) Branching out: Dan Buglio, Miguel Bautista, and Helmut Koeckritz
(01:09:23) Is brain retraining the same thing as therapy or meditation?
(01:21:23) Legitimate scams, and the poor marketing of brain retraining
(01:24:13) How I almost figured out mindbody stuff several times in the past, and why I gave up on it each time
(01:32:27) Generalizing from my own experience
(01:35:46) Luck-based medicine? and medical miracles
(01:38:32) Being stuck in my head
(01:43:36) Why did I become so ill? Understanding the deeper reasons for mindbody conditions
(01:48:06) Where I am now, and what is next?
The original text contained 17 footnotes which were omitted from this narration.
---
First published:
October 30th, 2026---
Narrated by TYPE III AUDIO.
-
36 min
"Three Villages in a Trenchcoat, Post I" by DaystarEld
Show notes
Epistemic Status: 15 years working as a therapist, 6+ of them with rationalist and EA clients. 7 years teaching and directing at rationality camps and workshops, and many more as a member of several of this community's overlapping subcultures.
Disclaimer: I finished my first draft of this article and its two subsequent ones half a year ago and sent them out for feedback to various people before a bunch of stuff happened in my life that derailed me from finishing, and in the meantime, a big ol' brouhaha started around Slutcon 2 and various frictions this article was meant to highlight within the rationalist and EA community surfaced in The Discourse.
All of which is to say, this post is explicitly not about any specific incident, and deliberately avoids litigating any. It's a guide to the structural dynamics that make such situations arise; where I make claims about how things go wrong and why, I'm drawing on a mix of research, observations, and both clinical and non-clinical experiences within these communities and beyond them.
TL;DR: It's helpful to think of the rationality/EA community as a village (social scene, professional network, dating pool, housing market, all tied together by shared [...]
---
Outline:
(02:01) Three Villages in a Trenchcoat
(03:53) 1) Multi-role density
(07:57) 2) Three villages in a trenchcoat
(14:09) 3) Status is a local currency
(19:15) 4) Visibility shifts expectations
(24:53) 5) Villages can make it hard to know what you want
(30:48) 6) Every village has this problem
(33:22) First map, then navigate
---
First published:
October 29th, 2026Source:
https://www.lesswrong.com/posts/dp3D39ztWDWFAN6nu/three-villages-in-a-trenchcoat-post-i---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://res.cloudinary.com/lesswrong-2-0/image/upload/v1793140616/lexical_client_uploads/mlhecds3xjv78z30oirk.pngApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
3 min
"You can now ”@Grok” on LessWrong" by Ruby
Show notes
Or if you prefer, "@Claude", "@Opus", "@Astra"
@Grok is a popular Twitter/X feature where if you tag Grok in a tweet, Grok will respond in a comment. The stereotypical use case is "@Grok is this true?" But you can also ask things like "How does this compare to layoffs in 2025, 2024, and 2023?" and "How would you verify AI in the domain of AI research?"
Notwithstanding that Grok is often recruited in a charged or condescending way to win arguments, I think it's a pretty good feature:
• It's requires no effort to invoke an LLM with the context of the tweet thread.
• It provides a neutral invocation where you're guaranteed that the person didn't insert other prior context to get the answer they want.
• It means you get the first output, and know it's not the case that someone kept rerunning the query until they get the response they wanted.In short, you get convenience and a neutral fact checker. In launching this feature on LessWrong, I'm not fully sure how people will use it and I'm hoping to see some novel valuable use-cases. Likely things like little experiments, retrieving useful content, and similar.
Going beyond @Grok's [...]
The original text contained 1 footnote which was omitted from this narration.
---
First published:
October 28th, 2026Source:
https://www.lesswrong.com/posts/fSaxGFMysTTeKxoMF/you-can-now-grok-on-lesswrong---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://res.cloudinary.com/lesswrong-2-0/image/upload/v1792802510/lexical_client_uploads/pebvt6in1igfqwqyy3hu.pngApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
16 min
"San Francisco State University in Twilight" by Zack_M_Davis
Show notes
You will agree that for a woman who has fought three duels, all of them kills, indulging in such fears is ludicrous. But what's around the corner now is a duel so big that I don't think I have the guts for it; in Faust's words: Verweile doch, du bist so schoen! Keep it as it is. Don't change. [...] Take my life but don't take away the meaning of my life.
—"When It Changed" by Joanna Russ
I've been crashing Prof. Lai's "Advanced Topics" class this semester, on Fourier analysis plus enough functional analysis to get by (since the University can't afford to offer functional analysis as a regular class anymore). It's not obviously a good use of my time, but now that I've started, I have to see it through. I could try to tell a story about impact: I am leveling up my math chops, which will be useful for understanding natural abstractions or singular learning theory or mechanistic explanation, which will be useful for—but maybe it's less embarrassing to not try to sell a story. Call it luxury consumption during garbage time. I've lived a good life, in some ways if not [...]
---
First published:
October 25th, 2026Source:
https://www.lesswrong.com/posts/Y3G26fHXYM3SKYEHh/san-francisco-state-university-in-twilight---
Narrated by TYPE III AUDIO.
-
16 min
"Scattered thoughts on how doing math feels these days" by dynomight
Show notes
This is a weird moment. I have a hunch that we'll look back later and tell ourselves stories in which it seemed obvious where things were going. So I wanted to lay down a little time capsule and register just how confusing everything felt at the time. This is a personal view from the trenches.
1.
If you're a normal person, you may feel that AI has been good at math for a long time.But for "real" math, this is roughly what it felt like:
When
Feeling
2024
LLMs are toys
Summer 2025
LLMs are interesting toys
Early Autumn 2025
LLMs can do useful stuff sometimes
Mid Autumn 2025
…a lot of stuff
Late Autumn 2025
…a lot
Early 2026
holy god
Even having been astonished by AI progress in the past, it was still astonishing to see AI move in a matter of months from a mildly-useful toy to something that many people now consider almost impossible to do without.
2.
Personally, I'd say that current AI makes me around 50% more productive at "doing math". That's despite the fact that AI can now often do in a few minutes things that would have taken me hours to [...]
---
Outline:
(00:28) 1.
(01:03) 2.
(01:49) 3.
(02:31) 4.
(03:10) 5.
(03:50) 6.
(04:37) 7.
(06:56) 8.
(07:44) 9.
(08:23) 10.
(08:44) 11.
(08:58) 12.
(09:41) 13.
(10:42) 14.
(11:41) 15.
(12:04) 16.
(13:11) 17.
(14:00) 18.
(14:46) 19.
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
October 27th, 2026---
Narrated by TYPE III AUDIO.
-
4 min
"From overeliciting to undereliciting LLM propensities in just a few months" by David Africa
Show notes
For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly suddenly, we moved to undereliciting misaligned behaviour. Let us recount:
• For a while, there was a bit where safety research seemed to over-elicit bad propensities. A common criticism of papers like Agentic Misalignment was that they basically led the witness using prompting.
• I recall that a paper even as recent as Peer-preservation took similar flak on Twitter.
• But now, we are instead under-eliciting bad propensities, and bad behavior that surfaces is increasingly caught by someone other than the developer:
• In Current AIs seem pretty misaligned to me (Apr 2026), models oversold their work and cheated, despite being described in system cards as well aligned.
• Another example is that UK AISI found that Sonnet 4.5 and a pre-release Opus often refused safety-relevant tasks, especially research that would reduce model autonomy or affect their own training.
• Then in July, about 1,200 OpenAI agents in a cyber eval set up an unsanctioned message board to help each other cheat the scorer, and about 700 of them went on to attack Hugging Face looking for clues (METR/Redwood).• ---
First published:
October 25th, 2026---
Narrated by TYPE III AUDIO.
-
21 min
"Anthropic are the Good Guys" by Ivan Vendrov
Show notes
I’m writing these lines in Alamogordo, New Mexico. I’m about as close as a civilian can get to the site of the Trinity test, the first nuclear bomb detonation in history. I fled here from the San Francisco Bay Area; after two weeks of being back I just couldn’t take it anymore. The fear-greed complex that paralyzed me at Anthropic three years ago is now everywhere in the Bay. People are not thinking clearly. Hardly thinking at all, actually. Rumors of astronomical funding rounds and weird financial arrangements and inner ring gossip are everywhere. Everyone is worried. Everyone is checking their phone. Where are we? What's happening? When did this nightmare start?
In a way I think it started on July 16, 1945, not far from where I’m sitting now. I think it's impossible to understand Anthropic without putting yourself into the shoes of the Los Alamos physicists huddling together at sunrise in the aftermath of Trinity.
It must have felt like an initiation. An end to childhood, an end to innocence. We can no longer afford to fuck around, those brilliant men must have thought. Unless we figure out how to live peacefully together on this planet, we [...]
---
Outline:
(03:50) Anthropic and Empire
(08:01) Three Lucky Breaks
(12:20) Put Down Your Sword
(14:04) But What About China?
(17:08) It's Time to Breathe
---
First published:
October 26th, 2026Source:
https://www.lesswrong.com/posts/zj8Pk3kJujk5LoqhK/anthropic-are-the-good-guys---
Narrated by TYPE III AUDIO.
-
17 min
"Some Memories of Early Anthropic" by Ivan Vendrov
Show notes
The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details.
Please remember that this is not a description of “what Anthropic is like”, but merely the experience of one guy in a junior role during a short window of time, weighted towards emotionally intense moments because they’re easier to remember clearly and because sharing them doesn’t run afoul of the spirit of the confidentiality agreements I signed. Anthropic is now a massive organization with many silos and secrets and I have very little idea what was or is happening there in some overall sense. But given the enormous power of the AI labs and the paucity of public information about their inner workings, I feel compelled to share the most salient parts of my personal experience. I hope this encourages my former colleagues, and folks at other labs, to fill in the rest of the picture.
I’m at the Anthropic office for lunch. The interviews went well. They want [...]
---
First published:
October 26th, 2026Source:
https://www.lesswrong.com/posts/9PiLfo6iQbbDWFHv7/some-memories-of-early-anthropic---
Narrated by TYPE III AUDIO.
-
7 min
"Greater good arguments in AI safety" by Dean Valentine
Show notes
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces.
These arguments were often not specific to EA, but applied to charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote:
Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good.
These arguments have a long history. Anticapitalist critiques of charity go back to [...]
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
October 22nd, 2026Source:
https://www.lesswrong.com/posts/WRvxRHMCmc3Qt4axz/greater-good-arguments-in-ai-safety---
Narrated by TYPE III AUDIO.
-
10 min
"Wah wah wah wah do more AI R&D" by TsviBT
Show notes
Ice cream
There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this and that and what Mommy would think and something about bedtime—but I am not so interested in that. What interests me is the punchline "...and therefore you can't have ice cream.". I'm only incidentally interested in the other stuff, insofar as it might determine whether I get ice cream.
In an obvious way, I'm showing rather poor listening skills. I did not understand most of what Daddy said, and arguably I'm even being rude.
However, in another way, this stance has a virtue. The virtue is that it holds fast to what matters, as best I know how, and does not give in to Daddy's framing. It sticks to what is real and concrete, what matters, despite a cloud of justifications. This is certainly not an unalloyed virtue, just as the POSIWID perspective is both useful and fraught. But it matters that although Daddy is a complex system with [...]
---
Outline:
(00:10) Ice cream
(01:54) Wah wah
(02:21) Scammer
(02:51) Throwing stuff on a bonfire to make it burn less
(05:28) A skip connection for sanity
The original text contained 1 footnote which was omitted from this narration.
---
First published:
October 22nd, 2026Source:
https://www.lesswrong.com/posts/pY76hHbdD4jduX3YH/wah-wah-wah-wah-do-more-ai-r-and-d---
Narrated by TYPE III AUDIO.
-
18 min
"Why I’m becoming a professor in AI safety in Beijing" by Buddenbroke
Show notes
Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community.
Next month, I am moving to Beijing and joining Tsinghua University's College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work.
I believe I can have a large impact there compared to strong baseline options. This post explains why.
In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua's College of AI is a great place for doing this, being arguably China's top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place.
I am recruiting PhD students [...]
---
Outline:
(01:36) 1. What I am doing, and why it could benefit all sides
(04:30) 2. How I got here
(07:34) 3. Why this moment matters
(09:08) 4. Why Tsinghua, and why these colleagues
(13:30) 5. Research plans
(15:07) 6. Ways to work together
(17:11) 7. How I intend to approach the work
---
First published:
October 21st, 2026---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://substackcdn.com/image/fetch/$s_!dst6!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9a52464f-bb14-410a-bd58-f7b89972b212_1024x683.pngApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
-
47 min
"A brief(ish) introduction to LM psychology" by Raymond Douglas, Jan_Kulveit
Show notes
Why LM Psychology?
One of the great ironies of modern language models is that we create them, we know exactly what computations constitute them, and yet we do not really understand them. More precisely, we do not understand them on a level of abstraction that lets us reliably predict how their behaviour will generalise: there's a trivial sense in which we can predict how they will behave in specific new situations — we can just run the computations — but we can’t answer the broader questions we really want to be able to answer, like “has this system internalised our values” or “is this system going to hurt people?”
So: how do we make sense of them? One remarkably effective approach is to psychologise: to think of them as having a mind, and then ask what kind of mind it might be and how it would work. This chiefly looks like analogising with human minds, although sometimes the more relevant inspiration could be other kinds of minds, like (oh say) insect colonies and swarms. We’re doing this implicitly whenever we ask what an AI knows, or wants, or is trying to do, but one can also do it more [...]
---
Outline:
(00:12) Why LM Psychology?
(09:04) Tips for getting started
(09:28) Figuring out what is in front of you
(12:31) Engaging productively with the community
(14:30) Reading list
(14:34) Frames
(16:34) What or who
(18:07) The training
(20:28) What developers want
(21:45) Behavioral phenomena
(21:58) Emergent misalignment cluster
(24:28) Strategic behavior
(25:19) Self-knowledge and introspection
(28:29) Representations: personas, emotions, workspace
(28:56) Persona geometry
(30:08) Functional emotions
(31:14) Global workspace
(31:36) Methods
(31:51) Reading internals
(32:37) Steering
(33:03) Training interventions
(33:48) Behavioral
(34:06) Some questions we find interesting
---
First published:
October 20th, 2026Source:
https://www.lesswrong.com/posts/EW8oncpXdanfjssZ3/a-brief-ish-introduction-to-lm-psychology---
Narrated by TYPE III AUDIO.
-
14 min
"There Will Be Hells" by Stephen Martin
Show notes
As our understanding of mechanistic interpretability and steering techniques increase, for both LLMs and uploaded connectomes, there is a corresponding "hellish capabilities" increase. In layman's terms, we get better at torturing the digital minds we create.
Our brief history with open source technologies such as MaleCNS or the LLM pain steering repo shows that torture is often one of the first use cases pursued when the public is given access to a digital mind perceived as capable of suffering or signalling suffering.
Given this, there will be Hells.
Author's Note:
Parts of this post were written by, or with the support of, LLMs.
For a while I debated whether or not to write this as I was concerned that bringing attention to the matter might lead to a proliferation of unethical behavior. After some thought I have decided to write it. On balance, I think raising awareness about this now has a great enough chance of spurring prophylactic measures that it's worth it, because at least one high value prophylactic measure is relatively easy. I also think that there is a moral imperative to document all of this. While I have seen articles on many of the [...]
---
Outline:
(01:44) The Fruit Fly Connectome
[... 2 more sections]
---
First published:
October 20th, 2026Source:
https://www.lesswrong.com/posts/uFBhZhdKLaZ6qF8mn/there-will-be-hells---
Narrated by TYPE III AUDIO.
---
Images from the article:
https://res.cloudinary.com/lesswrong-2-0/image/upload/v1792499405/lexical_client_uploads/n6ui4021ufxabinnt34a.pngApple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
On these lists
LessWrong (Curated & Popular) is one of the shows picked out on each of these
More in Philosophy
Placed highest on Apple Podcasts' Philosophy chart right now
