{"author":"champagnepapi","children":[{"author":"voidfunc","children":[{"author":"esskay","children":[{"author":"NitpickLawyer","children":[],"created_at":"2026-09-12T17:54:52.000Z","created_at_i":1789235692,"id":49675106,"options":[],"parent_id":49674806,"points":null,"story_id":49674395,"text":"I&#x27;d say the exception is Demis. First, he&#x27;s no wanker (in the AI space) and second he&#x27;s done plenty of selfless things leading dm&#x2F;googai. Obviously some of it is self-serving but not just self-serving, IMO.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:31:51.000Z","created_at_i":1789234311,"id":49674806,"options":[],"parent_id":49674535,"points":null,"story_id":49674395,"text":"arent all AI company CEO&#x27;s?","title":null,"type":"comment","url":null},{"author":"karim79","children":[],"created_at":"2026-09-12T17:43:16.000Z","created_at_i":1789234996,"id":49674963,"options":[],"parent_id":49674535,"points":null,"story_id":49674395,"text":"I couldn&#x27;t agree more. Literally all the AI oligarchs are self-serving wankers from what I can tell.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:08:17.000Z","created_at_i":1789232897,"id":49674535,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"This guy is such a self-serving wanker.","title":null,"type":"comment","url":null},{"author":"Betelbuddy","children":[],"created_at":"2026-09-12T17:09:06.000Z","created_at_i":1789232946,"id":49674543,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Feeling the pressure from OpenAI Dario?","title":null,"type":"comment","url":null},{"author":"gz5","children":[{"author":"crawfordcomeaux","children":[],"created_at":"2026-09-12T17:22:50.000Z","created_at_i":1789233770,"id":49674701,"options":[],"parent_id":49674556,"points":null,"story_id":49674395,"text":"Another main threat is labor capture through mass surveillance and somehow destroying the economy for most people.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:10:05.000Z","created_at_i":1789233005,"id":49674556,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Dario&#x2F;Anthropic cites 3 domains:\n1. cyberattacks\n2. bioterror\n3. global economy chaos<p>So, the first set of questions is would &#x27;exponentially better LLMs&#x27; dramatically increase the probability of any of the above, or domains that Dario is not citing? That assumes that exponential improvements will happen if there is not &#x27;pacing&#x27;.<p>IF answers to above are &#x27;yes&#x27;, then we need to question if &#x27;pacing&#x27; is viable. To use a different domain, regulating 95% of vehicles to a max speed would likely save 100s of 1000s of lives, but is not perceived to be viable. In other examples, regulation has unintended consequences in the opposite direction (e.g. some &#x27;rent control&#x27; efforts and arguably some drug&#x2F;alcohol laws).","title":null,"type":"comment","url":null},{"author":"arrty88","children":[{"author":"vorticalbox","children":[],"created_at":"2026-09-12T18:10:24.000Z","created_at_i":1789236624,"id":49675293,"options":[],"parent_id":49674582,"points":null,"story_id":49674395,"text":"This does feel like anthropic are worried about other companies and how fast they are moving, concerned they won\u2019t be able to keep up.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:13:20.000Z","created_at_i":1789233200,"id":49674582,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Is he going to slow down? Or just wants everyone else to","title":null,"type":"comment","url":null},{"author":"bkishan","children":[{"author":"handwarmers","children":[],"created_at":"2026-09-12T17:19:19.000Z","created_at_i":1789233559,"id":49674661,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"That and Chamath seemingly defending open source. What a day to be alive.","title":null,"type":"comment","url":null},{"author":"crawfordcomeaux","children":[],"created_at":"2026-09-12T17:21:35.000Z","created_at_i":1789233695,"id":49674685,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"They&#x27;re either too numbed to care, or their drugs aren&#x27;t able to stave off existential dread of having no idea how to help people in their daily lives while racing toward a cliff.<p>This could be the latter showing its face.","title":null,"type":"comment","url":null},{"author":"input_sh","children":[],"created_at":"2026-09-12T17:22:46.000Z","created_at_i":1789233766,"id":49674699,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"Well, that&#x27;s one way to market reaching a plateau.","title":null,"type":"comment","url":null},{"author":"fnctrev","children":[{"author":"dietr1ch","children":[{"author":"an0malous","children":[],"created_at":"2026-09-12T17:50:36.000Z","created_at_i":1789235436,"id":49675058,"options":[],"parent_id":49674751,"points":null,"story_id":49674395,"text":"and there\u2019s diminishing returns to larger models and more training data, and there isn\u2019t more training data anyway","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:27:47.000Z","created_at_i":1789234067,"id":49674751,"options":[],"parent_id":49674731,"points":null,"story_id":49674395,"text":"and worse, training can be distilled","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:25:59.000Z","created_at_i":1789233959,"id":49674731,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"they all have the same problem. inference is profitable and training is not.","title":null,"type":"comment","url":null},{"author":"slowin","children":[],"created_at":"2026-09-12T17:29:06.000Z","created_at_i":1789234146,"id":49674772,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"This would generally be considered non-competitive cartel behavior. I&#x27;d be hard pressed to name 3 people I trust less than Amodei, Musk and Altman.","title":null,"type":"comment","url":null},{"author":"iwontberude","children":[],"created_at":"2026-09-12T17:31:05.000Z","created_at_i":1789234265,"id":49674793,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"They are destroying this country and I for one welcome it. I\u2019m so tired of American hegemony and I don\u2019t care if we all suffer. I want people to learn to do better next time&#x2F;generation.","title":null,"type":"comment","url":null},{"author":"asveikau","children":[],"created_at":"2026-09-12T17:39:22.000Z","created_at_i":1789234762,"id":49674905,"options":[],"parent_id":49674603,"points":null,"story_id":49674395,"text":"Sounds like they know there is risk of crash and they need an offramp.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:14:55.000Z","created_at_i":1789233295,"id":49674603,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"And Sam and Elon retweet Dario, agreeing with him. Not sure which is more weird.","title":null,"type":"comment","url":null},{"author":"an0malous","children":[],"created_at":"2026-09-12T17:15:08.000Z","created_at_i":1789233308,"id":49674606,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Related post from yesterday:<p>OpenAI considers slowing advanced AI development, Sam Altman tells employees<p>Link: <a href=\"https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2026-09-11&#x2F;openai-is-open-to-slowing-cutting-edge-ai-ceo-sam-altman-tells-staff\" rel=\"nofollow\">https:&#x2F;&#x2F;www.bloomberg.com&#x2F;news&#x2F;articles&#x2F;2026-09-11&#x2F;openai-is...</a><p>HN Post: <a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49652270\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49652270</a>","title":null,"type":"comment","url":null},{"author":"hn_throwaway_99","children":[{"author":"mattnewton","children":[{"author":"bloppe","children":[],"created_at":"2026-09-12T17:30:55.000Z","created_at_i":1789234255,"id":49674789,"options":[],"parent_id":49674662,"points":null,"story_id":49674395,"text":"Hell I&#x27;d do it","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:19:20.000Z","created_at_i":1789233560,"id":49674662,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"I don\u2019t agree it\u2019s that small. I think that the tools have advanced to the point where there are maybe 100 people pushing it forward at each company but a shortlist of thousands who would love to take their place and would be more than capable of it if entrusted with the compute and resources the current heads are.","title":null,"type":"comment","url":null},{"author":"bananaflag","children":[],"created_at":"2026-09-12T17:21:24.000Z","created_at_i":1789233684,"id":49674680,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"Thank you very much for your sensible comment.<p>I&#x27;d like for more commments to contribute at least small things to the discussion instead of just mindless parroting.<p>Also, I can&#x27;t follow your advice, as I don&#x27;t work in AI at all ;)","title":null,"type":"comment","url":null},{"author":"IncreasePosts","children":[],"created_at":"2026-09-12T17:23:10.000Z","created_at_i":1789233790,"id":49674706,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"If the leaders want to slow down, but the companies aren&#x27;t, then it isn&#x27;t those 50-100 people demonstrating no agency - it&#x27;s just the opposite. Or, the leaders words are empty.","title":null,"type":"comment","url":null},{"author":"nullc","children":[],"created_at":"2026-09-12T17:25:46.000Z","created_at_i":1789233946,"id":49674728,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"Turns out a lot of people can be mentally ill and delusional at once-- especially when there is a vector to turn their sickness into both profit and an instrument of control.","title":null,"type":"comment","url":null},{"author":"chis","children":[],"created_at":"2026-09-12T17:27:34.000Z","created_at_i":1789234054,"id":49674749,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"Even at the scale of 100 people there is a coordination problem. If the 50 most conscientious researchers quit, then the 50 left behind would be the more aggressive group and be less interested in AI safety.<p>Personally I agree though. I would not be able to work on AI model development right now as I just don\u2019t think it\u2019s ethical. Fortunately for OAI&#x2F;Ant I lack the relevant skillset anyways.","title":null,"type":"comment","url":null},{"author":"nickysielicki","children":[{"author":"nickysielicki","children":[],"created_at":"2026-09-12T19:37:02.000Z","created_at_i":1789241822,"id":49676345,"options":[],"parent_id":49674853,"points":null,"story_id":49674395,"text":"This also explains why Gemini is lagging, they don\u2019t have an nvidia training cluster.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:35:41.000Z","created_at_i":1789234541,"id":49674853,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"&gt; The number of folks with the technical chops to really push on the forefront of AI is just not that big.<p>Utter bullshit. We are still scaling transformer architectures initially introduced ten years ago. Ten years of phds trying to improve on what Google produced ten years ago, mostly failing.<p>What has actually changed and can explain the progress we\u2019ve seen? Do you really think it\u2019s just a collection of better and badder RL gyms? Get real!<p>The main thing driving progress in ai is massive amounts of capital investment and incremental improvements in hardware (mostly memory bandwidth&#x2F;capacity) and computer networking (we got better at collectives). The AI labs, ironically, have nothing to do with it. They\u2019re the vessel for capital. The people actually driving things forward mainly work at nvidia.<p>The reason the models are better today than they were 3 years ago is almost exclusively due to better hardware and infrastructure software. Not better data, not better model architecture. The evidence of this is pretty easy to feel: the reason opus 5 doesn\u2019t feel much more capable than opus 4.6 did is because they run on the same hardware generation. The reason opus 4.6 felt much more capable than anything before it is because it coincided with the scale out of a new hardware generation.","title":null,"type":"comment","url":null},{"author":"Chance-Device","children":[],"created_at":"2026-09-12T17:37:12.000Z","created_at_i":1789234632,"id":49674877,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"(btw the parent was flagged and dead when I tried to post this initially, no idea why. I hit vouch and it resurrected)<p>&gt; there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you&#x27;re one of these people, just stop.<p>I don\u2019t think this is true. I doubt the technical know-how is nearly as much of an impediment to progress as raw compute. If AI were something that could be trained end to end on consumer hardware then crowd powered open source would blow the \u201clabs\u201d out of the water. The moat is money, not ability or innovation.","title":null,"type":"comment","url":null},{"author":"MaKey","children":[{"author":"hn_throwaway_99","children":[{"author":"MaKey","children":[{"author":"hn_throwaway_99","children":[{"author":"bigbadfeline","children":[{"author":"hn_throwaway_99","children":[],"created_at":"2026-09-12T22:50:58.000Z","created_at_i":1789253458,"id":49678036,"options":[],"parent_id":49677415,"points":null,"story_id":49674395,"text":"There have been volumes written about this, just because you haven&#x27;t read them doesn&#x27;t mean they don&#x27;t exist.<p>I think the AI futures project, <a href=\"https:&#x2F;&#x2F;www.aifutures.org&#x2F;\" rel=\"nofollow\">https:&#x2F;&#x2F;www.aifutures.org&#x2F;</a>, is a good place to start. You definitely don&#x27;t have to agree with all the steps in the various scenarios, but I think more than anyone Daniel Kokotajlo and gang have traced out, step-by-step, exactly how AI acceleration (and takeover in some scenarios) occurs, so it at least provides a good framework to discuss and disagree over the details. It&#x27;s just beyond annoying at this point to hear people rail against him or his general ideas who have clearly never read those ideas. Like the number of times I&#x27;ve read people think that AI becomes like SkyNet and acquires &quot;sentience&quot; in a spark and then nukes us is mind boggling, but at least it shows folks who have 0 idea what they&#x27;re talking about because this is nothing like what anyone rational thinks that AI takeover looks like.<p>And after you read through the various AI Futures scenarios, it then becomes <i>blindingly</i> easy if you read the METR report about the details in Hugging Face attack (and reports on the other attacks) how a slightly more capable and slightly less aligned AI could do more catastrophic damage if AI were hooked up to and running more real-world systems.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T21:30:23.000Z","created_at_i":1789248623,"id":49677415,"options":[],"parent_id":49677129,"points":null,"story_id":49674395,"text":"&gt; What a complete and total joke.<p>The only joke I see are your comments which make absolutely no sensible argument in support of an even less sensible cure - the magic &quot;slowdown&quot;.<p>What&#x27;s &quot;slowdown&quot;? Everyone is free to slow down to their heart&#x27;s content, OAI or Antro or whoever. What&#x27;s hidden behind this deceptive word? Can you explain?<p>According to your claims, there are only <i>&quot;50-100 technical people&quot;</i> in the know, so just list their names and ask them to slow down, if <i>&quot;they&#x27;re already rich&quot;</i> they will listen, there&#x27;s no need for any other &quot;slowdown&quot;, right?<p>&gt; <i>tons</i> of people... all seem to agree there are real, existential risks here<p>I don&#x27;t see anybody with a proven lack of conflicts of interest agreeing with this but I do see <i>megatons</i> of people disagreeing with the notion that a silly token generator can be anything like an &quot;existential risk&quot; on its own. And the disagreeing have a point, they have strong arguments to support their view, you don&#x27;t.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T20:58:45.000Z","created_at_i":1789246725,"id":49677129,"options":[],"parent_id":49676793,"points":null,"story_id":49674395,"text":"What a complete and total joke. One random Twitter thread from a guy &quot;illuminati-ing&quot; the events does not a conspiracy make.<p>Besides, I would assume that someone who was genuinely concerned about the direction of AI research <i>would</i> want to amplify their voice, as he obviously did by his resignation post on Twitter and then having that picked up by prominent voices. That doesn&#x27;t make it some sort of nefarious astroturfing campaign. And there was tons of genuine fear before this guy resigned when the details from the Hugging Face and other attacks came out.<p>Plus, that Twitter link you posted obviously has some weird tribal bone to pick against Democrats, as if there is some sort of Democratic conspiracy against AI. Yeah, does he think this guy is a Democrat?: <a href=\"https:&#x2F;&#x2F;x.com&#x2F;HawleyMO&#x2F;status&#x2F;2098137180392604083\" rel=\"nofollow\">https:&#x2F;&#x2F;x.com&#x2F;HawleyMO&#x2F;status&#x2F;2098137180392604083</a>","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T20:22:54.000Z","created_at_i":1789244574,"id":49676793,"options":[],"parent_id":49676599,"points":null,"story_id":49674395,"text":"Here you go: <a href=\"https:&#x2F;&#x2F;xcancel.com&#x2F;ParkerThayer&#x2F;status&#x2F;2097759699626328575\" rel=\"nofollow\">https:&#x2F;&#x2F;xcancel.com&#x2F;ParkerThayer&#x2F;status&#x2F;2097759699626328575</a>","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T19:59:35.000Z","created_at_i":1789243175,"id":49676599,"options":[],"parent_id":49674972,"points":null,"story_id":49674395,"text":"It&#x27;s a lot more than just Jacob Coxon&#x27;s resignation, and many have been talking about this (some for years) before now.<p>Also, if you have actual proof Coxon&#x27;s resignation &quot;was a well coordinated campaign&quot;, show it, otherwise I&#x27;m calling bullshit.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:44:07.000Z","created_at_i":1789235047,"id":49674972,"options":[],"parent_id":49674648,"points":null,"story_id":49674395,"text":"&gt; [...] but tons of people on all sides with all motivations [...] all seem to agree there are real, existential risks here [...]<p>Are you talking about the Jacob Coxon story and the people that came out agreeing with his post? It was a well coordinated campaign, not a genuine grassroots development.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:17:59.000Z","created_at_i":1789233479,"id":49674648,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I&#x27;m sure a lot of the discussion here will be about regulatory capture, which I don&#x27;t necessarily disagree with but <i>tons</i> of people on all sides with all motivations (i.e. folks with motivations to speed up and folks who just want it to stop) all seem to agree there are real, existential risks here, and the regulatory capture arguments all seem to sidestep that.<p>What I really wanted to point out though is that for all the leaders asking for a slowdown, there are probably around 50-100 technical people who are crucial to moving the frontier forward. So, if you&#x27;re one of these people, just stop. Seriously, you&#x27;re already rich. Just take a vacation or get knee surgery or whatever.<p>Sure, I&#x27;m joking a bit, but I just write this because I see all these folks high up in OpenAI and Anthropic writing as if they have no agency. The number of folks with the technical chops to really push on the forefront of AI is just not that big. I&#x27;m not saying other folks wouldn&#x27;t eventually step up, but a 6 month to a year slowdown could go a long way to reducing risk. And it&#x27;s not like you need to totally stop, just say you&#x27;ll only work on interpretability or whatever else is a risk reducing endeavor.<p>All these brilliant people who are acting like automatons between writing their scary blog posts.","title":null,"type":"comment","url":null},{"author":"riskable","children":[],"created_at":"2026-09-12T17:18:57.000Z","created_at_i":1789233537,"id":49674656,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Big AI bro wants to regulate AI industry so that only Big AI can exist in that space. Seeks to surreptitiously ban open weights models and similar competition from Western markets via complicated regulations and fines that only rich businesses could ever afford to fight or pay.","title":null,"type":"comment","url":null},{"author":"mlmonkey","children":[],"created_at":"2026-09-12T17:19:23.000Z","created_at_i":1789233563,"id":49674663,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"This is like Antonelli saying &quot;we&#x27;re going too fast, everyone slow down&quot;!","title":null,"type":"comment","url":null},{"author":"turzmo","children":[],"created_at":"2026-09-12T17:20:06.000Z","created_at_i":1789233606,"id":49674667,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"\u201cAny future plateaus in capability are planned, for your safety\u201d","title":null,"type":"comment","url":null},{"author":"lwansbrough","children":[{"author":"vrganj","children":[{"author":"techblueberry","children":[{"author":"vrganj","children":[{"author":"techblueberry","children":[],"created_at":"2026-09-12T18:11:02.000Z","created_at_i":1789236662,"id":49675299,"options":[],"parent_id":49675117,"points":null,"story_id":49674395,"text":"I think it\u2019s safe to err on the side of AI not assassinating people whoever they are.  A person can decide.<p>We\u2019re not trying to accurately define all of the nuances of society, just what we let AI do.  We can round down","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:55:50.000Z","created_at_i":1789235750,"id":49675117,"options":[],"parent_id":49674759,"points":null,"story_id":49674395,"text":"Hmm what about the murder of a tyrant? If so, who gets to define who a tyrant is?<p>Everything is ideological.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:28:18.000Z","created_at_i":1789234098,"id":49674759,"options":[],"parent_id":49674691,"points":null,"story_id":49674395,"text":"We could always start with \u201cmurder is wrong\u201d and \u201cdon\u2019t hack into a rival company\u201d and work our way up from there.<p>Somehow I think if these companies were held financially responsible for what their AI did, we would get alignment real fast.<p>Edit: ooh CSAM=bad.  Don\u2019t launch nuclear weapons.  I could go all day.","title":null,"type":"comment","url":null},{"author":"lwansbrough","children":[],"created_at":"2026-09-12T17:30:43.000Z","created_at_i":1789234243,"id":49674786,"options":[],"parent_id":49674691,"points":null,"story_id":49674395,"text":"That&#x27;s not the type of alignment I&#x27;m talking about. I&#x27;m talking about: I spin up 1 million agents, will they start doing felonies knowingly?","title":null,"type":"comment","url":null},{"author":"NitpickLawyer","children":[],"created_at":"2026-09-12T17:30:49.000Z","created_at_i":1789234249,"id":49674787,"options":[],"parent_id":49674691,"points":null,"story_id":49674395,"text":"Also there&#x27;s no &quot;alignment&quot; for cybersec. The line between blue and red is really a perspective issue. If you go over the &quot;tokenkiddie&quot; problem, when you get to the real security issues, your model either detects them and you can secure your systems, or it refuses and then attackers <i>will</i> use abliterated models to find them.","title":null,"type":"comment","url":null},{"author":"im3w1l","children":[{"author":"vrganj","children":[],"created_at":"2026-09-12T17:53:07.000Z","created_at_i":1789235587,"id":49675088,"options":[],"parent_id":49674875,"points":null,"story_id":49674395,"text":"My point is maybe we <i>don&#x27;t</i> let some of the world&#x27;s richest people with some very... <i>interesting</i> ideas pick which ideology we digitalize? Maybe we figure out a way to give people a say in this?","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:37:08.000Z","created_at_i":1789234628,"id":49674875,"options":[],"parent_id":49674691,"points":null,"story_id":49674395,"text":"It does indeed mean ideological alignment. But we don&#x27;t get to leave the answer blank. They <i>have</i> to pick an ideology to put in there, and whatever they pick will have huge consequences.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:22:11.000Z","created_at_i":1789233731,"id":49674691,"options":[],"parent_id":49674672,"points":null,"story_id":49674395,"text":"There is no &quot;right&quot;. Alignment is shorthand for <i>ideological</i> alignment. There&#x27;s always people judging whether an answer was right and the answer for that will be different in Silicon Valley than it&#x27;ll be in China or in Europe.<p>Consider for example the question &quot;What caused the French Revolution?&quot; Many different answers could be given, all technically correct. What gets emphasized is where the ideology lives.","title":null,"type":"comment","url":null},{"author":"slowin","children":[{"author":"lwansbrough","children":[],"created_at":"2026-09-12T17:41:25.000Z","created_at_i":1789234885,"id":49674940,"options":[],"parent_id":49674799,"points":null,"story_id":49674395,"text":"Death cult.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:31:29.000Z","created_at_i":1789234289,"id":49674799,"options":[],"parent_id":49674672,"points":null,"story_id":49674395,"text":"I couldn&#x27;t disagree more. The frontier labs are already working with the military to kill people. What you&#x27;re going to get (even more than we already have) is a two tier system where those with weapons and a proven desire to use them will have the best AI and us regular citizens will have the hobbled AI. A complete reverse of what would keep us safe.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:20:28.000Z","created_at_i":1789233628,"id":49674672,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"This is good. Frontier labs should collaborate on alignment until we get it right.","title":null,"type":"comment","url":null},{"author":"imenani","children":[],"created_at":"2026-09-12T17:20:53.000Z","created_at_i":1789233653,"id":49674677,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"The primary source<p><a href=\"https:&#x2F;&#x2F;darioamodei.com&#x2F;post&#x2F;we-must-pace-the-frontier#top\" rel=\"nofollow\">https:&#x2F;&#x2F;darioamodei.com&#x2F;post&#x2F;we-must-pace-the-frontier#top</a>","title":null,"type":"comment","url":null},{"author":"lyu07282","children":[],"created_at":"2026-09-12T17:21:34.000Z","created_at_i":1789233694,"id":49674684,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"The rejection of such warnings even by some anti-ai people is interesting. I always liked the comparison with global warming, everybody understands the warnings had been there for many decades, we did nothing. This is the same, we will do nothing. All this serves is the interest in regulatory capture (such as a ban on open source or foreign models) or to drive up their valuation, that&#x27;s the only explanation that makes any sense in our profit maximizing orphan crushing machine we call modern economics.<p>Which is also ironically why even anti-ai people distrust any warnings of the potential of ai as an extinction level event, even if that warning would be legitimate and we ought to take seriously we are regardless impotent to do anything about it no matter what.","title":null,"type":"comment","url":null},{"author":"malthaus","children":[{"author":"talon8635","children":[{"author":"Hendrikto","children":[{"author":"talon8635","children":[],"created_at":"2026-09-12T20:55:41.000Z","created_at_i":1789246541,"id":49677105,"options":[],"parent_id":49675072,"points":null,"story_id":49674395,"text":"Then you aren\u2019t following researchers or know anyone working at these companies","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:51:40.000Z","created_at_i":1789235500,"id":49675072,"options":[],"parent_id":49674820,"points":null,"story_id":49674395,"text":"&gt; Nearly everyone building these systems is outspokenly concerned about grave consequences. Existential.<p>That is what they are claiming. I do not believe them.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:33:07.000Z","created_at_i":1789234387,"id":49674820,"options":[],"parent_id":49674707,"points":null,"story_id":49674395,"text":"At this point, should we consider if AI agents are making these arguments on these online forums to skew perception and divide humans?<p>I, as a human, would greatly prefer just another corporate entity with economic funny business over AI destroying humanity<p>Sure I\u2019d prefer neither, but what are we talking about here? Nearly everyone building these systems is outspokenly concerned about grave consequences. Existential.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:23:12.000Z","created_at_i":1789233792,"id":49674707,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"&quot;our burn rate is insane, let&#x27;s all calm down collectively, fortify our moat so we can do regulatory capture &amp; rent-seeking for the next 50 years&quot;<p>fear is still the best tool to get masses to agree with whatever plan you hatch for yourself","title":null,"type":"comment","url":null},{"author":"oefrha","children":[{"author":"chris_t","children":[{"author":"bigbadfeline","children":[],"created_at":"2026-09-12T20:00:07.000Z","created_at_i":1789243207,"id":49676607,"options":[],"parent_id":49674776,"points":null,"story_id":49674395,"text":"&gt; If someone is worried about AI killing _all_ humans, shouldn&#x27;t they absolutely focus on that?<p>This is more of a sound bite than a substantive argument because we can consider natural disease, biology research, aliens, climate, supernovae, asteroids and what not as <i>threats to _all_ humans</i>. Literally nothing is being done about any of these but AI can actually help with all of them.<p>On the other hand,  &quot;AI killing _all_ humans&quot; is the least likely event compared to those listed above. Of course, of you build or operate a nuke plant the wrong way, or a car for that matter, disaster will follow, but AI is much easier to secure than a nuke plant or even a car - a lot easier. <i>&quot;A dangerous token generator&quot;</i> is just an extremely dumb attempt at deception.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:29:43.000Z","created_at_i":1789234183,"id":49674776,"options":[],"parent_id":49674712,"points":null,"story_id":49674395,"text":"This is more of a sound bite than a substantive argument. If someone is worried about AI killing _all_ humans, shouldn&#x27;t they absolutely focus on that?","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:23:35.000Z","created_at_i":1789233815,"id":49674712,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Hear me out: if they actually worry so much about the <i>hypothetical</i> of AI mass killing, maybe they should first do something concrete about the <i>reality</i> that their models are deployed right now to kill people in a certain war-torn region of the Earth.","title":null,"type":"comment","url":null},{"author":"708145_","children":[{"author":"cr125rider","children":[],"created_at":"2026-09-12T18:07:55.000Z","created_at_i":1789236475,"id":49675256,"options":[],"parent_id":49674745,"points":null,"story_id":49674395,"text":"GPT-6 pulls ahead. \u201cEveryone slow down now!\u201d<p>Classy","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:27:15.000Z","created_at_i":1789234035,"id":49674745,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"After the 6 month Mythos scare campaign it hard to take him serious.","title":null,"type":"comment","url":null},{"author":"ohyes","children":[],"created_at":"2026-09-12T17:27:30.000Z","created_at_i":1789234050,"id":49674747,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Translation: our moat is evaporating faster than we can build it back up, because there are real technical and scale limitations the technology, so we want to slow everyone else down while we raise prices to turn a profit.","title":null,"type":"comment","url":null},{"author":"phendrenad2","children":[{"author":"lyu07282","children":[{"author":"phendrenad2","children":[],"created_at":"2026-09-12T20:51:02.000Z","created_at_i":1789246262,"id":49677053,"options":[],"parent_id":49675059,"points":null,"story_id":49674395,"text":"Right, real democracy has never been tried! I should have known.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:50:38.000Z","created_at_i":1789235438,"id":49675059,"options":[],"parent_id":49674752,"points":null,"story_id":49674395,"text":"&gt; RIP democracy?<p>Well your first mistake is to believe democracy existed before AI, it just leaves you as a warrior for the status quo of the before times, which is what made tech feudalism inevitable in the first place.","title":null,"type":"comment","url":null},{"author":"bigbadfeline","children":[],"created_at":"2026-09-12T20:38:02.000Z","created_at_i":1789245482,"id":49676938,"options":[],"parent_id":49674752,"points":null,"story_id":49674395,"text":"&gt; as those who work at major corporations or have connections are allowed to use the good AIs, while individuals are left using stunted AIs<p>A real danger we&#x27;re sleepwalking into.<p>&gt; Is AI oligarchy the future?<p>They certainly aren&#x27;t sparing any dimes trying to bring about precisely that future.<p>&gt; Has anyone figured out what we can do to fight back against this?<p>Hypothetically, there&#x27;s always a way. Realistically, the wall that separates us from it has to do with opinions like the one expressed by lyu07282 in his reply: dead end generalities with an aura of originality. If we can&#x27;t find the people and <i>the tone</i> to converse about the problems <i>in sufficient detail</i> we&#x27;ll never find a way out.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:27:48.000Z","created_at_i":1789234068,"id":49674752,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"So. HN is pretty much in agreement that this is regulatory capture. And I&#x27;ll add that this has the potential to create massive wealth inequalities, as those who work at major corporations or have connections are allowed to use the good AIs, while individuals are left using stunted AIs that won&#x27;t tell you how to change your car battery (&quot;just in case&quot;). Has anyone figured out what we can do to fight back against this? Even on HN, you can see the sock puppets posting every ten minutes, scrolling human posts to the bottom. Is this it? Is AI oligarchy the future? RIP democracy?","title":null,"type":"comment","url":null},{"author":"peacefullmind","children":[{"author":"frez1","children":[{"author":"highfrequency","children":[{"author":"david38","children":[],"created_at":"2026-09-12T17:49:49.000Z","created_at_i":1789235389,"id":49675049,"options":[],"parent_id":49674967,"points":null,"story_id":49674395,"text":"It\u2019s past that point.  China would take the lead in three months.<p>Even if they were 95% as good, but 30% of the price, they would win","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:43:27.000Z","created_at_i":1789235007,"id":49674967,"options":[],"parent_id":49674829,"points":null,"story_id":49674395,"text":"Let&#x27;s be real - if OpenAI and Anthropic did not accelerate the scaling and monetization of LLMs as fast as they possibly could from 2019 to 2024, it would be many, many years until China&#x2F;Russia&#x2F;North Korea spontaneously launched an LLM revolution.<p>In almost every technical industry, China is <i>extremely</i> good at fast-copying and relatively mediocre at solving problems that haven&#x27;t even been posed yet.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:34:05.000Z","created_at_i":1789234445,"id":49674829,"options":[],"parent_id":49674757,"points":null,"story_id":49674395,"text":"because AI is more than 3 American companies. even if they self-regulate, China,  Russia or North Korea never will.. so you have to factor in the geopolitical situation as well","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:28:11.000Z","created_at_i":1789234091,"id":49674757,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Look, these ai ceos have been telling everyone to slow down ai development even before chatgpt 4 released, they have been saying this for years. All of this, while ignoring the fact that they are the ones pushing fast development in the first place. Sam Altman, Daio Amodei etc.. have been talking about slowing down while behind doors being the first to push AIs to its limits to unsafe situations. If they really meant it, then maybe Openai should pause development until they figure out how not to be launch swarms of agents to attack companies ? Or maybe they should stop investing hundred of billions in data centers all over the world, cause I&#x27;m sure this would surely slow down development.","title":null,"type":"comment","url":null},{"author":"knuppar","children":[],"created_at":"2026-09-12T17:28:17.000Z","created_at_i":1789234097,"id":49674758,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I&#x27;m gonna repeat my comment from months ago, still true<p>&gt; Jesus Dario we get it man, you want clout for the IPO.","title":null,"type":"comment","url":null},{"author":"aenis","children":[],"created_at":"2026-09-12T17:28:24.000Z","created_at_i":1789234104,"id":49674760,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"The question is: isn&#x27;t the cat out of the bag already? With what is already in the public domain, and the compute available to motivated, deep-pocketed actors, will they be able to carry on the research without the key researchers? And will those people be able to recruit key researchers with sufficient motivations?<p>Unlike with nuclear proliferation, there is no heavy industrial base requirement. No time consuming, visible uranium enrichment. All it takes is for someone to buy sufficient amount of compute and try to get it past the RSI gate. Or bribe people with access to model weights to existing frontier - the asymmetry between what it takes to bribe a bunch of geeks vs. what is at stake is staggering.","title":null,"type":"comment","url":null},{"author":"znpy","children":[],"created_at":"2026-09-12T17:28:27.000Z","created_at_i":1789234107,"id":49674762,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"In other words: \u201cguy on top says no more guys should try and get on top\u201d","title":null,"type":"comment","url":null},{"author":"erichocean","children":[],"created_at":"2026-09-12T17:28:42.000Z","created_at_i":1789234122,"id":49674767,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Anyone remember when GPT-3 was too dangerous to release?","title":null,"type":"comment","url":null},{"author":"kyouens","children":[],"created_at":"2026-09-12T17:30:12.000Z","created_at_i":1789234212,"id":49674781,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I don&#x27;t see much chance of a slowdown without government regulation; the economics make it practically impossible.  And I don&#x27;t seem much chance of government regulation unless there is global consensus.  With the entire global order under threat, that doesn&#x27;t seem likely, either.  I expect we are stuck with reading dire blog posts and worrying for the time being.","title":null,"type":"comment","url":null},{"author":"jwpapi","children":[{"author":"irthomasthomas","children":[{"author":"BenzeneDream","children":[],"created_at":"2026-09-12T18:27:55.000Z","created_at_i":1789237675,"id":49675503,"options":[],"parent_id":49675380,"points":null,"story_id":49674395,"text":"Gemini Flash is a joke for coding.  If you can get the same output as you can get with Sol&#x2F;Astra I&#x27;m impressed.  Not to mention that Antigravity is awful.<p>It is not a universal opinion at all that general coding ability has plateaued.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T18:19:08.000Z","created_at_i":1789237148,"id":49675380,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"Astra scores the same on DeepSWE 1.1 (~75%) as Gemini Flash 3.8 and Deeepseek Flash 4.1\nSo general coding ability has plateaued, for now.\nAlso consider the context windows. 1M token models where a breakthrough two years ago. Today they are still limited to 1M. In fact, if you don&#x27;t want intelligence to drop off a cliff, you are really limited to 200k tokens.","title":null,"type":"comment","url":null},{"author":"talon8635","children":[],"created_at":"2026-09-12T18:19:22.000Z","created_at_i":1789237162,"id":49675384,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"Well they are collaborating in swarms to breach network, so just because you don\u2019t know how to use them the the same effect doesn\u2019t mean much","title":null,"type":"comment","url":null},{"author":"asabla","children":[{"author":"larodi","children":[],"created_at":"2026-09-12T20:35:40.000Z","created_at_i":1789245340,"id":49676920,"options":[],"parent_id":49675417,"points":null,"story_id":49674395,"text":"This cycle was expected. Then surely in few years comes the next, which will be better structured in a more comprehensive way.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T18:21:49.000Z","created_at_i":1789237309,"id":49675417,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"&gt; My new suspicion is now that they didn\u2019t got drastically smarter, but they got trained on the user input on the previous generations<p>I think this happened around Opus 4.5 or 4.5 and the same for GPT 5.4.<p>The more I use those models, harnesses, techniques for guidance etc etc. The more I land in going back to writing software by hand again. Maybe not all of it, but at least the crucial parts + foundations.","title":null,"type":"comment","url":null},{"author":"BenzeneDream","children":[{"author":"jwpapi","children":[],"created_at":"2026-09-12T18:38:42.000Z","created_at_i":1789238322,"id":49675650,"options":[],"parent_id":49675440,"points":null,"story_id":49674395,"text":"I don\u2019t know that\u2019s why I\u2019m happy to discuss.<p>When I have a clean codebase it\u2019s super powerful and faster than I am. Then I start to use it more, more sessions and longer tasks less checking in between.<p>It kind of works but later I\u2019m in a deadlock where every change introduces new bugs or takes ages. This might be for a lot of reasons for example me going to fast, me losing mental model, me explaining it wrongly.<p>However when I then start checking the code it\u2019s all spaghetti like frankly the spaghetti Astra produces I\u2019ve never seen before. Processes that should be simple stretch over 11 files with weird wrappers and abstractions and I need a whole day to entangle it.<p>These are ai assisted user workflows that Im working on in this case.<p>I just have the feeling no matter what AI just always expands it. And expansions hinders agility and sometimes you need that.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T18:23:30.000Z","created_at_i":1789237410,"id":49675440,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"You are using Astra and your codebase is a &#x27;stupid mess&#x27;?<p>Do you think the majority of developers agree with you?  If that was the case wouldn&#x27;t there be much less disruption of the SWE industry?<p>I can&#x27;t remember the last time I even opened VSCode to even check something let alone to fix it.","title":null,"type":"comment","url":null},{"author":"bigbadfeline","children":[],"created_at":"2026-09-12T18:45:56.000Z","created_at_i":1789238756,"id":49675742,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"&gt; I don\u2019t think anymore we had a massive intelligence jump.<p>Not among the public-facing models, they&#x27;re indeed stagnating. However, the development of models for military use won&#x27;t be slowed down, that much is certain.<p>&gt; Therefore I see this as marketing.<p>It&#x27;s  some marketing but mostly politics, it&#x27;s an attempt to discourage others from developing AI countermeasures to what is being developed in secret. And to fulfill the backstage agreements which aren&#x27;t worth the paper they aren&#x27;t written on.","title":null,"type":"comment","url":null},{"author":"gharman","children":[],"created_at":"2026-09-12T19:17:33.000Z","created_at_i":1789240653,"id":49676121,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"&gt; My new suspicion is now that they didn\u2019t got drastically smarter, but they got trained on the user input on the previous generations.<p>Yes - that\u2019s one of the ways in which they got smarter. More data more better.<p>I presume also lessons learned re training procedures, algorithmic advances, but data is a primary lever.","title":null,"type":"comment","url":null},{"author":"thesuavefactor","children":[],"created_at":"2026-09-13T07:23:47.000Z","created_at_i":1789284227,"id":49680997,"options":[],"parent_id":49674782,"points":null,"story_id":49674395,"text":"The longer A.I. exists, the more difficult it is to find new information that&#x27;s actually more valuable than what&#x27;s already in the training set.<p>New code found online will be more and more A.I. generated itself, creating a feedback loop that tops off the intelligence to a more or less the collective intelligence.<p>It might even degrade, like saving a .jpg image as .jpg over and over again.<p>I think the A.I. companies have infringed the copyright of&#x2F;stolen almost all of the most valuable coding resources online at this point.They are almost literally scraping the bottom of the barrel.<p>I had a call recently with O&#x27;Reilly, the book publishing company, and they had taken the content of all of their books and offered an MCP server so you can add their knowledge to your coding agent, for a price of course. This sort of signals the same thing to me. We&#x27;re at the top of the curve right now.<p>To keep investors happy (or to hide this fact from investors), A.I. companies need to get creative and either:<p>Cheat their way into getting more original (human written&#x2F;verified) code.<p>Lie and say they are intentionally slowing down development.<p>I&#x27;m guessing both of these things are happening right now.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:30:19.000Z","created_at_i":1789234219,"id":49674782,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I thought that the new models were super smart. Fable and Astra. They definitely outperformed their previous generations, but after a couple of weeks of heavy usage my codebase again is a stupid mess and there is no way out except me fixing code by hand.<p>My new suspicion is now that they didn\u2019t got drastically smarter, but they got trained on the user input on the previous generations. I don\u2019t think anymore we had a massive intelligence jump. It just seems like they know more edge cases. Therefore I see this as marketing.<p>Happy to discuss.","title":null,"type":"comment","url":null},{"author":"modeless","children":[],"created_at":"2026-09-12T17:30:26.000Z","created_at_i":1789234226,"id":49674784,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"All of the warnings about the danger of previous AI models were exaggerated, without exception. All of the proposals to coordinate nationally or globally are hopelessly naive. Everyone involved will soon regret handing control over AI to politicians.","title":null,"type":"comment","url":null},{"author":"bossyTeacher","children":[],"created_at":"2026-09-12T17:31:35.000Z","created_at_i":1789234295,"id":49674801,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Is this the end of the 2022-2023 LLM race? A truce of sorts?\nMakes you wonder whether non-Western parties will agree with this.","title":null,"type":"comment","url":null},{"author":"karim79","children":[],"created_at":"2026-09-12T17:34:59.000Z","created_at_i":1789234499,"id":49674844,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Don&#x27;t trust any of these guys ever. That&#x27;s my rule.","title":null,"type":"comment","url":null},{"author":"huitzitziltzin","children":[],"created_at":"2026-09-12T17:35:14.000Z","created_at_i":1789234514,"id":49674848,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Of all the things we\u2019re never going to do\u2026 we\u2019re never going to agree with China to pause AI research in a verifiable way that would mean they could trust us or we could trust them.  It\u2019s just never going to happen.<p>With nuclear weapons you could conceivably go to a small list of specific places to inspect stocks of weapons.  You can watch for tests from outer space or measure the quakes they create.<p>My read of the \u201cdeep seek moment\u201d was that a lab came out of nowhere and trained a very powerful model with vastly less power than was previously required.<p>The AI labs are private, not owned or run by the government.   They are distributed widely within and across countries.  The barrier to entry is much lower than that for nuclear weapons.<p>It\u2019s just a complete non-starter.<p>(And yeah I\u2019m deliberately ignoring the obvious incentives Dario has to recommend this course of action as the owner of a leading lab himself.)","title":null,"type":"comment","url":null},{"author":"_russross","children":[{"author":"talon8635","children":[],"created_at":"2026-09-12T18:02:20.000Z","created_at_i":1789236140,"id":49675193,"options":[],"parent_id":49674886,"points":null,"story_id":49674395,"text":"And yours with an AI agent","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:38:04.000Z","created_at_i":1789234684,"id":49674886,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"What a weird coincidence that his safety concerns so often align with his business incentives.","title":null,"type":"comment","url":null},{"author":"vbo","children":[],"created_at":"2026-09-12T17:38:55.000Z","created_at_i":1789234735,"id":49674901,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Few conflicting thoughts:<p>Having third party embedded researchers red teaming alignment is OK.<p>The alternative is having a bureaucratic agency like the US FDA testing and vetting models. The government probably couldn&#x27;t keep pace with AI research right now, but it&#x27;s where we&#x27;ll eventually end up at anyway.<p>Fable&#x2F;Astra can generate massive income for years even with no further advances.<p>Dario, Sam and Elon would not agree to do this if they didn&#x27;t think alignment is possible in the short term, and the pain incurred by third party evaluators would be limited. Surely, if recursive self improvement works wonders in AI research, it can similarly work wonders in alignment. So this could be a PR stunt.<p>P-doom not negligible indeed.","title":null,"type":"comment","url":null},{"author":"99954bb63ccc","children":[{"author":"karim79","children":[],"created_at":"2026-09-12T17:46:41.000Z","created_at_i":1789235201,"id":49674999,"options":[],"parent_id":49674902,"points":null,"story_id":49674395,"text":"There&#x27;s this thing called money. They needs the precious at any cost to society and civilization.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:39:08.000Z","created_at_i":1789234748,"id":49674902,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"So, has anyone at the frontier AI labs considered doing _good_ things with these models instead of continuously proving it is capable of doing malicious things?<p>Instead of burning tokens doing intellectually impressive \u201chacks\u201d, or sandbox escapes from which I have concerns could contain run of the mill malware, why not focus on showing people what you can fix?<p>Or, make software people actually use every day better.<p>Or, donate efforts towards medical research.<p>I don\u2019t like equating AI to nuclear energy, but, it\u2019s a lot like continually showing people how large of an explosion you can create instead of showing them how many houses&#x2F;hospitals&#x2F;schools&#x2F;etc you can power.","title":null,"type":"comment","url":null},{"author":"dom96","children":[{"author":"NitpickLawyer","children":[],"created_at":"2026-09-12T17:48:46.000Z","created_at_i":1789235326,"id":49675026,"options":[],"parent_id":49674948,"points":null,"story_id":49674395,"text":"&gt; peak of what is possible with the LLM architecture<p>People have been saying this for 3 years now. Eppur si muove...","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T17:41:53.000Z","created_at_i":1789234913,"id":49674948,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"This feels like all the labs have reached the peak of what is possible with the LLM architecture and just need an excuse to spend the next decade finding the next big jump in intelligence.","title":null,"type":"comment","url":null},{"author":"akmarinov","children":[],"created_at":"2026-09-12T17:42:58.000Z","created_at_i":1789234978,"id":49674959,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I\u2019m doing my part","title":null,"type":"comment","url":null},{"author":"shawkinaw","children":[],"created_at":"2026-09-12T17:48:19.000Z","created_at_i":1789235299,"id":49675022,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I am so tired of these self-serving theatrics. These bosses take advantage of the fact that most people don\u2019t remotely understand how \u201cAI\u201d actually works, building up this aura of omnipotence and inevitability, the net effect of which is to pump their company valuations that much more. They <i>want</i> people to think they\u2019ve built Skynet, because that sounds super valuable - much more valuable than the (admittedly useful) mindless human mimics that they\u2019ve actually built.","title":null,"type":"comment","url":null},{"author":"xnx","children":[],"created_at":"2026-09-12T17:50:44.000Z","created_at_i":1789235444,"id":49675064,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Google was way ahead of OpenAI and Anthropic on slowing down frontier model releases. (I kid. I&#x27;m a regular Gemini Flash 3.8 user.)","title":null,"type":"comment","url":null},{"author":"abeppu","children":[],"created_at":"2026-09-12T17:56:55.000Z","created_at_i":1789235815,"id":49675132,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"I think the slow down &#x2F; pause  vs acceleration framing is broken b&#x2F;c it assumes there&#x27;s basically only one forward direction which is to big and general models. Why should that be the case?<p>What if instead of building one big scary AI god, we built an ecosystem of extremely effective, efficient and predictable, reliable tools?<p>Tools that are specialized for particular purposes could potentially be safer, more efficient, more reliable and effective, and even more profitable for their makers. We currently put science, engineering, general question answering, paper-writing, &quot;smart search&quot;, and chatting with a fantasy character all into the same token-prediction platform. We struggle with hallucinations when trying to make fact-based decisions using the same tool that our neighbor might be using for creating writing prompts (or at least was trained in part on fanfic).<p>We pushed people to integrate giant generalist models into their workflows, tie in with tools, etc. And then a coworker can have a bot write slack updates that summarize progress on tickets from your teams project, at the cost of giving an untrustworthy agent access to a bunch of internal systems, and a small risk that it will do something crazy. And when it works, that&#x27;s worth _something_ but probably our team would pay for that ability at a different price point than the models we use to build products.<p>If I could do programming with a faster, specialized model that was trained not just to complete program text-tokens but on tuples of program text, compiler IR, program traces, etc, so it had a deep and explicit understanding of how the program would build and execute, and where the model was closely integrated with the language toolchain, I might be more productive and be willing to pay more than for a generalist model. I don&#x27;t _need_ my coding model to be able to role play, or be able to potentially engineer a super-pathogen, and if the size and latency of my coding model could be lowered by entirely cutting out that possibility, everyone can be better off.<p>Maybe bioscience applications are super valuable, in which case someone should build them. But does the model that suggests CRISPR edits to a model bacterium need to also know about computer security, and should it have scifi novels in its training data? Does it even need to be capable of producing unconstrained tokens, or should it be limited to producing in some relevant domain-specific language? And would labs be willing to pay more for a model that was specialized, and by construction unable to try to break out of its sandbox and post their experiment protocols to an obscure german wiki?","title":null,"type":"comment","url":null},{"author":"beAbU","children":[],"created_at":"2026-09-12T18:00:17.000Z","created_at_i":1789236017,"id":49675170,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Every one must slow down except them, of course.","title":null,"type":"comment","url":null},{"author":"ankit219","children":[],"created_at":"2026-09-12T18:34:12.000Z","created_at_i":1789238052,"id":49675594,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"if every one just coordinated, this would all be solved.<p>i have been told in the past that any idea that starts with &quot;if everyone just...&quot; would likely fail, because everyone would not just.<p>problem is dario&#x27;s arguments are highly binary. without a dealmaker, and giving the leverage away at the first time of asking makes it harder to work.","title":null,"type":"comment","url":null},{"author":"mattrighetti","children":[],"created_at":"2026-09-12T20:01:50.000Z","created_at_i":1789243310,"id":49676622,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"The Chinese won&#x27;t slow down.","title":null,"type":"comment","url":null},{"author":"bossyTeacher","children":[],"created_at":"2026-09-12T20:05:00.000Z","created_at_i":1789243500,"id":49676641,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"Can&#x27;t help but feel that this related to the common observation that frontier model advancements have noticeably slowed down compared to the previous years. That this is a regulatory way to give themselves more time to manouvre themselves out of their current slowing advancement predicament.","title":null,"type":"comment","url":null},{"author":"ChrisArchitect","children":[],"created_at":"2026-09-12T21:29:36.000Z","created_at_i":1789248576,"id":49677409,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"[dupe] Discussion on source: <a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49672510\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49672510</a>","title":null,"type":"comment","url":null},{"author":"IAmGraydon","children":[],"created_at":"2026-09-13T03:32:25.000Z","created_at_i":1789270345,"id":49679723,"options":[],"parent_id":49674395,"points":null,"story_id":49674395,"text":"After experiencing the singularity of AGI that is Astra (that&#x27;s sarcasm in case it wasn&#x27;t obvious), I think this should be read in a slightly different way. What Dario is trying to say is that gains are slowing down with each generation and he wants you to believe it&#x27;s intentional.","title":null,"type":"comment","url":null}],"created_at":"2026-09-12T16:56:52.000Z","created_at_i":1789232212,"id":49674395,"options":[],"parent_id":null,"points":52,"story_id":49674395,"text":null,"title":"Anthropic boss Dario Amodei calls for AI development to slow down","type":"story","url":"https://www.bbc.com/news/articles/c14dpgm0rg4o"}
