> there is no point acting like we don't know what people mean when they say "Moore's Law has failed". What they mean is quite clear.
When I read "Moore's Law is dead", I thought the OP meant we can't shrink transistors any more. You can probably agree it is not the same as saying "clock speeds are not improving anymore".
If someone says "Moore's Law" I will assume they mean "number of transistors on a chip doubles every 2 years or so". If they mean something else, such as "clock speed doubles every 2 years", then they use the term incorrectly. Moore's Law is alive and well, and if you talk to people who actually work on extending it, you should use the correct terminology."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Micron Automata Processor \u2013 A Brief Introduction"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://files.shareholder.com/downloads/ABEA-45YXOQ/2856943415x0x714620/f72a92f0-5ace-4a6a-a124-f296c24f4573/"}},"_tags":["comment","author_p1esk","story_7314867"],"author":"p1esk","comment_text":"I actually looked at the section of the Wikipedia article you linked to. It lists about a dozen of rules or laws similar to Moore's Law. In every case, it's made clear the rule is not Moore's Law. In fact, most of those rules have their own distinct names (Dennard's scaling, Butter's Law, Wirth's Law, etc). Read it carefully and you will see that throughout the article, it's maintained that Moore's Law has exactly one, well defined meaning.
> there is no point acting like we don't know what people mean when they say "Moore's Law has failed". What they mean is quite clear.
When I read "Moore's Law is dead", I thought the OP meant we can't shrink transistors any more. You can probably agree it is not the same as saying "clock speeds are not improving anymore".
If someone says "Moore's Law" I will assume they mean "number of transistors on a chip doubles every 2 years or so". If they mean something else, such as "clock speed doubles every 2 years", then they use the term incorrectly. Moore's Law is alive and well, and if you talk to people who actually work on extending it, you should use the correct terminology.","created_at":"2014-02-28T06:29:47Z","created_at_i":1393568987,"objectID":"7317498","parent_id":7317397,"points":null,"story_id":7314867,"story_title":"Micron Automata Processor \u2013 A Brief Introduction","story_url":"http://files.shareholder.com/downloads/ABEA-45YXOQ/2856943415x0x714620/f72a92f0-5ace-4a6a-a124-f296c24f4573/","updated_at":"2023-09-06T22:17:48Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"maeln"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"> This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while advantaging smaller competitors.
> ...
> completely exempt any company below a certain amount of revenue or model training costs from being covered at all
One could argue that "Frontier AI" company know they have nothing to fear from company with less than XM$ revenue, and so their support for this type of regulation is still a way to force regulatory capture. In any case, whatever regulation you support, it's a regulation that you didn't have to handle when you were growing, but that incumbent will have to deal with.
Whatever it is Dario's intention or not does not matter. Capitalism push to consolidation and the eventual regulation that will need to be applied to mitigate the externality from a new industry, will mean that their will only be a handful of "very big" winner. Same story since the beginning of the industrial era. Even if that's not what Dario personally want, it's in the best interest of the shareholders, which will force Anthropic to do everything it can to be one of the big one.
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).
The same can be said about a lot of other industry (aviation, oil, chip manufacturing, ...). Every country / union big enough will finance their own champion to try to keep a foot in the industry even if they are not the best."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"On AI regulation and messaging"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://twitter.com/DarioAmodei/status/2088758816376807762"}},"_tags":["comment","author_maeln","story_49325789"],"author":"maeln","comment_text":"> This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while advantaging smaller competitors.
> ...
> completely exempt any company below a certain amount of revenue or model training costs from being covered at all
One could argue that "Frontier AI" company know they have nothing to fear from company with less than XM$ revenue, and so their support for this type of regulation is still a way to force regulatory capture. In any case, whatever regulation you support, it's a regulation that you didn't have to handle when you were growing, but that incumbent will have to deal with.
Whatever it is Dario's intention or not does not matter. Capitalism push to consolidation and the eventual regulation that will need to be applied to mitigate the externality from a new industry, will mean that their will only be a handful of "very big" winner. Same story since the beginning of the industrial era. Even if that's not what Dario personally want, it's in the best interest of the shareholders, which will force Anthropic to do everything it can to be one of the big one.
> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).
The same can be said about a lot of other industry (aviation, oil, chip manufacturing, ...). Every country / union big enough will finance their own champion to try to keep a foot in the industry even if they are not the best.","created_at":"2026-08-17T08:18:49Z","created_at_i":1786954729,"objectID":"49327816","parent_id":49325789,"story_id":49325789,"story_title":"On AI regulation and messaging","story_url":"https://twitter.com/DarioAmodei/status/2088758816376807762","updated_at":"2026-08-17T17:01:32Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"alok-g"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Total column line capacitance is impacted by the number of pixels hanging onto it as each transistor (going to the pixel capacitance) adds some parasitic capacitance of its own. Hope that answers your question. You are right in the sense that a part of the total column capacitance would depend on just the length and width of it, irrespective of the number of pixels hanging onto it.
I had back then developed what was perhaps the most sophisticated system-level model for display power, including refresh, illumination, etc., and it included all those terms for capacitance, a simplified transistor model, pixel model, etc.
I did not carefully distinguish pixel density vs. pixel count while writing my previous comments here, just to keep it simple. You can perhaps imagine that increasing display size without changing pixel count can lead to higher active pixel area percentage, which in turn would lead to better light generation/transmission/reflection efficiency. There are multiple initially counter-intuitive couplings like that. So it ultimately comes down to mathematical modeling, and the scaling laws / derivatives depend on the actual numbers chosen.
Addition:
Another important point -- Column line capacitances do not necessarily need full refresh going from one row of the pixels to the next, as the image would typically have vertical correlations. Not mentioning this is another simplification I made in my previous comments. My detailed power model included this as well -- so it could calculate energy spent for writing a specific image, a random image, a statistically typical image, etc."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"LG's new 1Hz display is the secret behind a new laptop's battery life"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.pcworld.com/article/3096432/lgs-new-1hz-display-is-the-secret-behind-a-new-laptops-battery-life.html"}},"_tags":["comment","author_alok-g","story_47495245"],"author":"alok-g","children":[47559121,47565406],"comment_text":"Total column line capacitance is impacted by the number of pixels hanging onto it as each transistor (going to the pixel capacitance) adds some parasitic capacitance of its own. Hope that answers your question. You are right in the sense that a part of the total column capacitance would depend on just the length and width of it, irrespective of the number of pixels hanging onto it.
I had back then developed what was perhaps the most sophisticated system-level model for display power, including refresh, illumination, etc., and it included all those terms for capacitance, a simplified transistor model, pixel model, etc.
I did not carefully distinguish pixel density vs. pixel count while writing my previous comments here, just to keep it simple. You can perhaps imagine that increasing display size without changing pixel count can lead to higher active pixel area percentage, which in turn would lead to better light generation/transmission/reflection efficiency. There are multiple initially counter-intuitive couplings like that. So it ultimately comes down to mathematical modeling, and the scaling laws / derivatives depend on the actual numbers chosen.
Addition:
Another important point -- Column line capacitances do not necessarily need full refresh going from one row of the pixels to the next, as the image would typically have vertical correlations. Not mentioning this is another simplification I made in my previous comments. My detailed power model included this as well -- so it could calculate energy spent for writing a specific image, a random image, a statistically typical image, etc.","created_at":"2026-03-28T20:01:04Z","created_at_i":1774728064,"objectID":"47557713","parent_id":47557576,"story_id":47495245,"story_title":"LG's new 1Hz display is the secret behind a new laptop's battery life","story_url":"https://www.pcworld.com/article/3096432/lgs-new-1hz-display-is-the-secret-behind-a-new-laptops-battery-life.html","updated_at":"2026-03-29T17:51:02Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"haoxiaoru"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not been proven. We identify two crucial techniques for scaling up Muon: (1) adding weight decay and (2) carefully adjusting the per-parameter update scale. These techniques allow Muon to work out-of-the-box on large-scale training without the need of hyper-parameter tuning. Scaling law experiments indicate that Muon achieves \u223c 2\u00d7 computational efficiency compared to AdamW with compute optimal training."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Muon is scalable for LLM training"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://github.com/MoonshotAI/Moonlight"}},"_tags":["comment","author_haoxiaoru","story_43144428"],"author":"haoxiaoru","comment_text":"Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not been proven. We identify two crucial techniques for scaling up Muon: (1) adding weight decay and (2) carefully adjusting the per-parameter update scale. These techniques allow Muon to work out-of-the-box on large-scale training without the need of hyper-parameter tuning. Scaling law experiments indicate that Muon achieves \u223c 2\u00d7 computational efficiency compared to AdamW with compute optimal training.","created_at":"2025-02-22T23:33:02Z","created_at_i":1740267182,"objectID":"43144429","parent_id":43144428,"story_id":43144428,"story_title":"Muon is scalable for LLM training","story_url":"https://github.com/MoonshotAI/Moonlight","updated_at":"2025-02-23T06:44:37Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"marricks"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Back in high school I worked with some pleasant man in his 50's who was a cashier. Eventually we got to talking about jobs and it turns out he was typist (something like that) for most of his life than computers came along and now he makes close to minimum wage.
Most of the blacksmiths in the 19th century drank themselves to death after the industrial revolution. the US culture isn't one of care... Point is, it's reasonable to be sad and afraid of change, and think carefully about what to specialize in.
That said... we're at the point of diminishing returns in LLM, so I doubt any very technical jobs are being lost soon. [1]
[1] https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showin..."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"OpenAI O3 breakthrough high score on ARC-AGI-PUB"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://arcprize.org/blog/oai-o3-pub-breakthrough"}},"_tags":["comment","author_marricks","story_42473321"],"author":"marricks","children":[42476945,42477116,42477295,42477590],"comment_text":"Back in high school I worked with some pleasant man in his 50's who was a cashier. Eventually we got to talking about jobs and it turns out he was typist (something like that) for most of his life than computers came along and now he makes close to minimum wage.
Most of the blacksmiths in the 19th century drank themselves to death after the industrial revolution. the US culture isn't one of care... Point is, it's reasonable to be sad and afraid of change, and think carefully about what to specialize in.
That said... we're at the point of diminishing returns in LLM, so I doubt any very technical jobs are being lost soon. [1]
[1] https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showin...","created_at":"2024-12-21T01:51:49Z","created_at_i":1734745909,"objectID":"42476885","parent_id":42476835,"story_id":42473321,"story_title":"OpenAI O3 breakthrough high score on ARC-AGI-PUB","story_url":"https://arcprize.org/blog/oai-o3-pub-breakthrough","updated_at":"2024-12-23T02:08:57Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"jameshart"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Why do so many people in the tech industry have so little respect for their peers who work elsewhere in the industry?
The idea that large companies aren't solving hard problems, that that's only being done at startups, and that large companies are 'one trick ponies' is just incredibly myopic. Have you worked at a large publicly traded tech company?
Sure, in any business a bunch of the systems will be trivial. In larger businesses - especially ones that have grown through acquisition - there's also some scaling-law complexity that means they have to also do a bunch of annoyingly trivial work gluing together lots of duplicative and redundant trivial systems. When you look in from the outside of a large business that mess can definitely obscure the fact that at the heart that stuff is all in support of, usually, several significant and impressive engineering achievements, which support and drive that business forward. And they're probably performing operations that, from the outside, you would hear the name of and think 'well, that doesn't sound very difficult'.
I'm going to give a non-software example, that I think might show a little bit of an analogy for how 'engineering' at big companies can be at the same time operating at a complexity you wouldn't imagine, and yet be accomplishing something you might think from the outside is totally trivial.
Jump to this timestamp in the recent Smarter Everyday video about Kodak's film manufacturing line: https://youtu.be/mrJP82ZZiag?t=1742
See that workshop? Those precision engineered toolheads they call 'hearts' are the punch+die sets that they use to cut sprocket-holes in film. Everything there is bespoke, custom-made, and meets constraints you would never have thought of. It's engineered.
Software engineering in large companies is like that. If you don't think that banks contain software systems as complex and carefully engineered as those hearts? Or that manufacturing control software isn't in the same league? Or that the inventory management systems that drive a large retail business aren't on that level? You just have no idea what engineering is."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"The Staff Engineer's Path \u2013 Book Review"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://smyachenkov.com/posts/book-review-the-staff-engineers-path/"}},"_tags":["comment","author_jameshart","story_35974845"],"author":"jameshart","children":[35984279],"comment_text":"Why do so many people in the tech industry have so little respect for their peers who work elsewhere in the industry?
The idea that large companies aren't solving hard problems, that that's only being done at startups, and that large companies are 'one trick ponies' is just incredibly myopic. Have you worked at a large publicly traded tech company?
Sure, in any business a bunch of the systems will be trivial. In larger businesses - especially ones that have grown through acquisition - there's also some scaling-law complexity that means they have to also do a bunch of annoyingly trivial work gluing together lots of duplicative and redundant trivial systems. When you look in from the outside of a large business that mess can definitely obscure the fact that at the heart that stuff is all in support of, usually, several significant and impressive engineering achievements, which support and drive that business forward. And they're probably performing operations that, from the outside, you would hear the name of and think 'well, that doesn't sound very difficult'.
I'm going to give a non-software example, that I think might show a little bit of an analogy for how 'engineering' at big companies can be at the same time operating at a complexity you wouldn't imagine, and yet be accomplishing something you might think from the outside is totally trivial.
Jump to this timestamp in the recent Smarter Everyday video about Kodak's film manufacturing line: https://youtu.be/mrJP82ZZiag?t=1742
See that workshop? Those precision engineered toolheads they call 'hearts' are the punch+die sets that they use to cut sprocket-holes in film. Everything there is bespoke, custom-made, and meets constraints you would never have thought of. It's engineered.
Software engineering in large companies is like that. If you don't think that banks contain software systems as complex and carefully engineered as those hearts? Or that manufacturing control software isn't in the same league? Or that the inventory management systems that drive a large retail business aren't on that level? You just have no idea what engineering is.","created_at":"2023-05-17T23:49:40Z","created_at_i":1684367380,"objectID":"35982669","parent_id":35980586,"story_id":35974845,"story_title":"The Staff Engineer's Path \u2013 Book Review","story_url":"https://smyachenkov.com/posts/book-review-the-staff-engineers-path/","updated_at":"2024-09-20T14:06:22Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"moritzwarhier"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"From the abstract:
> some even claiming they are capable of generalized reasoning and innovation in reasoning-intensive fields such as mathematics, physics, medicine, and law. However, by more carefully scaling the complexity of reasoning problems, we show existing benchmarks actually have limited complexity
Can someone ELI5 what the definitions of reasoning and complexity are here?
I see they seem to focus on graph problems and representing problems as graph problems. But I didn't completely read the paper or understand it in depth. I skimmed some parts that seem to address this question (e.g. section 5 and the Introduction), but maybe there are simpler definitions that elude me.
Surely they don't mean "computational complexity"?
And what exactly is "reasoning"?
I'm aware of philosophical logic and strict logic that can be applied to natural language arguments.
But have we already agreed on a universal scale that grades answers to questions about the physical world? Or is this about mathematical reasoning?
Mixing all of this together always irks me when it comes to these AI "benchmarks". But apparently people see value in these?
I know my question isn't new.
To me it seems, that when we leave the mathematical realms, it quickly becomes fuzzy what correct "reasoning" should be.
People can be convincing and avoid obious logical fallacies, and still make wrong conclusions... or conclusions that run counter to assumed goals."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Reasoning models reason well, until they don't"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://arxiv.org/abs/2510.22371"}},"_tags":["comment","author_moritzwarhier","story_45769971"],"author":"moritzwarhier","children":[45771443],"comment_text":"From the abstract:
> some even claiming they are capable of generalized reasoning and innovation in reasoning-intensive fields such as mathematics, physics, medicine, and law. However, by more carefully scaling the complexity of reasoning problems, we show existing benchmarks actually have limited complexity
Can someone ELI5 what the definitions of reasoning and complexity are here?
I see they seem to focus on graph problems and representing problems as graph problems. But I didn't completely read the paper or understand it in depth. I skimmed some parts that seem to address this question (e.g. section 5 and the Introduction), but maybe there are simpler definitions that elude me.
Surely they don't mean "computational complexity"?
And what exactly is "reasoning"?
I'm aware of philosophical logic and strict logic that can be applied to natural language arguments.
But have we already agreed on a universal scale that grades answers to questions about the physical world? Or is this about mathematical reasoning?
Mixing all of this together always irks me when it comes to these AI "benchmarks". But apparently people see value in these?
I know my question isn't new.
To me it seems, that when we leave the mathematical realms, it quickly becomes fuzzy what correct "reasoning" should be.
People can be convincing and avoid obious logical fallacies, and still make wrong conclusions... or conclusions that run counter to assumed goals.","created_at":"2025-10-31T12:30:09Z","created_at_i":1761913809,"objectID":"45771297","parent_id":45769971,"story_id":45769971,"story_title":"Reasoning models reason well, until they don't","story_url":"https://arxiv.org/abs/2510.22371","updated_at":"2026-03-05T22:59:13Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"tslug"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"It's not just commentary. I'm a game developer (http://davetaylor.name). I assure you, capitalism + our national laws + the international treaties our nations have agreed to is a game. It's the Old Game.
Catch is, the rules (laws) of the Old Game are such a complex mess that we have to hire expensive lawyers at hundreds of dollars an hour just to interpret slivers of the rules that apply to our particular situations. It's not fair to those still at level 1 (in poverty) still learning to play the game and who can't afford to pay people just to know the rules. Even at level 100 (extreme wealth), no one likes that they have to hire armies of attourneys just to protect themselves from being the targets of lawsuits.
This is poor game design. We learned years ago that players don't read manuals. That's why games don't come with them anymore. We learned they can't even stand tutorials. What they want is intuitive gameplay and gentle pacing, so that their minds aren't blown with the whole wad at once, and so that they can get a sense of steady progression and accomplishment. That's definitely not what the Old Game has to offer today. Perhaps it once did, but not anymore.
Today, the Old Game essentially tells us: "Congratulations. You've graduated high school. Maybe. Now, the rules moving forward are... incomprehensible. Don't suppose you have a few law degrees? Oh no, that's right. High school. OK. Better get a job, or you'll be homeless and will starve. Cool? Oh, unless you're rich, in which case, don't worry about it. You have no obligations." You haven't played any games like that because they wouldn't be compelling.
Worse, the Old Game has evolved to a point where it now tries to place a monetary value on all things. Its precept is that all your problems can be solved with money. Need new laws? Spend money on lobbyists. Need new politicians? Spend money on their campaigns. Need a date? Spend money on a dating service. Need excitement? Spend money on travel. Need new cloud infrastructure? Spend money on coders.
What modern game has a single currency that satisfies all your desires? It's hard to think of one, isn't it? That's not by accident. I assure you, we tried it. It doesn't work. We discovered empirically that money just isn't that satisfying. You need things that can't be bought, like random loot drops, level ups, status, new goals, friends, exploration, surprises, emergent behaviours, challenges, defeats, victories, romance, and more. We've even learned the importance of breaks. Too much is too much.
The Old Game is losing its competitiveness. It would have flopped long ago if it were not for the fact that playing it is mandatory. You need it to clothe yourself, eat, shelter yourself, and receive medical treatment to not die.
But read that list of items more carefully. We have 3D printers marching steadily towards finer-grained resolutions with more materials and can now build just about whatever we need, even tools, homes, and clothes. We have the ability to automate the gathering of our own energy, the ability to grow our own food (you can eat healthier with a fish tank and some algae than you can visiting most supermarkets now), the ability to diagnose and even treat ourselves.
The costs of living are plummeting. We are approaching autonomy. These things subvert capitalism, because they subvert your need to change the spin of more magnetic domains on your bank's hard drive.
You are in an incredibly enviable position. You have an enormous amount of potential excess income, probably a decent savings, and certainly a valuable skill set (for now- you know you can already feel the competitive heat from younger, cheaper minds than yours). You can continue to play the Old Game while scaling back your dependency on it so that you can transition to a new game of your own choosing.
I get that it's frightening, but I assure you, it's mostly your addiction to money that makes it frightening. The Old Game certainly hasn't kept up with the times, but it does know how to addict its players, and it keeps us playing by frightening us with scary potential consequences if we stop playing it.
I don't want to trivialize the risk, but to put it in context, the outcome is the same for all of us. Game over is death. You don't lose or win. There's no high score or prize. All we can do is decide what games we want to play getting there.
I don't recommend playing the Old Game anymore than you absolutely have to. It's just not well designed."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: Experienced dev having career lull"}},"_tags":["comment","author_tslug","story_8185503"],"author":"tslug","comment_text":"It's not just commentary. I'm a game developer (http://davetaylor.name). I assure you, capitalism + our national laws + the international treaties our nations have agreed to is a game. It's the Old Game.
Catch is, the rules (laws) of the Old Game are such a complex mess that we have to hire expensive lawyers at hundreds of dollars an hour just to interpret slivers of the rules that apply to our particular situations. It's not fair to those still at level 1 (in poverty) still learning to play the game and who can't afford to pay people just to know the rules. Even at level 100 (extreme wealth), no one likes that they have to hire armies of attourneys just to protect themselves from being the targets of lawsuits.
This is poor game design. We learned years ago that players don't read manuals. That's why games don't come with them anymore. We learned they can't even stand tutorials. What they want is intuitive gameplay and gentle pacing, so that their minds aren't blown with the whole wad at once, and so that they can get a sense of steady progression and accomplishment. That's definitely not what the Old Game has to offer today. Perhaps it once did, but not anymore.
Today, the Old Game essentially tells us: "Congratulations. You've graduated high school. Maybe. Now, the rules moving forward are... incomprehensible. Don't suppose you have a few law degrees? Oh no, that's right. High school. OK. Better get a job, or you'll be homeless and will starve. Cool? Oh, unless you're rich, in which case, don't worry about it. You have no obligations." You haven't played any games like that because they wouldn't be compelling.
Worse, the Old Game has evolved to a point where it now tries to place a monetary value on all things. Its precept is that all your problems can be solved with money. Need new laws? Spend money on lobbyists. Need new politicians? Spend money on their campaigns. Need a date? Spend money on a dating service. Need excitement? Spend money on travel. Need new cloud infrastructure? Spend money on coders.
What modern game has a single currency that satisfies all your desires? It's hard to think of one, isn't it? That's not by accident. I assure you, we tried it. It doesn't work. We discovered empirically that money just isn't that satisfying. You need things that can't be bought, like random loot drops, level ups, status, new goals, friends, exploration, surprises, emergent behaviours, challenges, defeats, victories, romance, and more. We've even learned the importance of breaks. Too much is too much.
The Old Game is losing its competitiveness. It would have flopped long ago if it were not for the fact that playing it is mandatory. You need it to clothe yourself, eat, shelter yourself, and receive medical treatment to not die.
But read that list of items more carefully. We have 3D printers marching steadily towards finer-grained resolutions with more materials and can now build just about whatever we need, even tools, homes, and clothes. We have the ability to automate the gathering of our own energy, the ability to grow our own food (you can eat healthier with a fish tank and some algae than you can visiting most supermarkets now), the ability to diagnose and even treat ourselves.
The costs of living are plummeting. We are approaching autonomy. These things subvert capitalism, because they subvert your need to change the spin of more magnetic domains on your bank's hard drive.
You are in an incredibly enviable position. You have an enormous amount of potential excess income, probably a decent savings, and certainly a valuable skill set (for now- you know you can already feel the competitive heat from younger, cheaper minds than yours). You can continue to play the Old Game while scaling back your dependency on it so that you can transition to a new game of your own choosing.
I get that it's frightening, but I assure you, it's mostly your addiction to money that makes it frightening. The Old Game certainly hasn't kept up with the times, but it does know how to addict its players, and it keeps us playing by frightening us with scary potential consequences if we stop playing it.
I don't want to trivialize the risk, but to put it in context, the outcome is the same for all of us. Game over is death. You don't lose or win. There's no high score or prize. All we can do is decide what games we want to play getting there.
I don't recommend playing the Old Game anymore than you absolutely have to. It's just not well designed.","created_at":"2014-08-17T07:30:31Z","created_at_i":1408260631,"objectID":"8188865","parent_id":8186729,"points":null,"story_id":8185503,"story_title":"Ask HN: Experienced dev having career lull","updated_at":"2023-09-06T22:11:57Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"joules77"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"In terms of scale what's happening with just the algos, forget AI/drugs/synth bio stuff etc is on par with successful plant or animal domestication which took few centuries because of scaling issues that tech has solved.
A "normal" corn plant today doesn't look anything like the one nature produces. And the "normal" dog, cat, horse, chicken or cow can't survive outside very carefully controlled and built environments.
This generation of technologists aren't taught the Law of Requiste Variety and are totally oblivious about what happens when stability of systems is tied to "normality".
Covid reminded us how "normal" everything is. Feels more and more like a waste of time telling or teaching the tech domesticated herd anything at all."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"People Are Being Involuntarily Committed After Spiraling into ChatGPT Psychosis"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://futurism.com/commitment-jail-chatgpt-psychosis"}},"_tags":["comment","author_joules77","story_44405464"],"author":"joules77","comment_text":"In terms of scale what's happening with just the algos, forget AI/drugs/synth bio stuff etc is on par with successful plant or animal domestication which took few centuries because of scaling issues that tech has solved.
A "normal" corn plant today doesn't look anything like the one nature produces. And the "normal" dog, cat, horse, chicken or cow can't survive outside very carefully controlled and built environments.
This generation of technologists aren't taught the Law of Requiste Variety and are totally oblivious about what happens when stability of systems is tied to "normality".
Covid reminded us how "normal" everything is. Feels more and more like a waste of time telling or teaching the tech domesticated herd anything at all.","created_at":"2025-06-28T17:06:40Z","created_at_i":1751130400,"objectID":"44406294","parent_id":44406077,"story_id":44405464,"story_title":"People Are Being Involuntarily Committed After Spiraling into ChatGPT Psychosis","story_url":"https://futurism.com/commitment-jail-chatgpt-psychosis","updated_at":"2025-06-28T17:07:37Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"vlovich123"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Since we're teaching each other first principles, bringing a new mass-scale battery manufacturing facility online to full production typically takes anywhere from 2 to 5 years. Planning takes ~1-2 years, construction & setup takes ~1-2 years and ramp to full output takes 6 months to 1 year. And since we're on Econ 101, investments require payoff so all of this is done carefully to not tank the price of batteries by overproducing supply. In control systems terms, supply side investments always aim for undershooting demand as overshooting hurts how much money you make and risks destabilizing your market.
As for scarcity, inputs to lithium ion ARE scarce which negates your entire model. Pretending they aren't is where you're making a mistake. Lithium, cobalt & nickle are relatively scarce and the mines for that have to scale up to meet demand as well. You've also got a workforce to train to do the work which takes time & is also input-constrained. That's why there's massive NEW lithium mines being opened in the US & elsewhere to extract existing reserves to meet the growth in lithium ion batteries. If the world thought that sodium ion or ion air was an immediate future, you wouldn't see these massive large-scale investments into lithium. Lilthium-ion batteries is going to be a large and growing market for decades which brings me back to the strip mining of the ocean floor that's coming to support that.
Whright's law by the way isn't also an inevitable effect that goes on forever. At some point your exponential plateau's and you no longer see such exponential decrease in pricing. That's why processors aren't getting cheaper and compute isn't scaling up quite in the same way as in the early days. There's only so efficient you can make something."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Nuclear fusion: WEST beats the world record for plasma duration"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.cea.fr/english/Pages/News/nuclear-fusion-west-beats-the-world-record-for-plasma-duration.aspx"}},"_tags":["comment","author_vlovich123","story_43093939"],"author":"vlovich123","comment_text":"Since we're teaching each other first principles, bringing a new mass-scale battery manufacturing facility online to full production typically takes anywhere from 2 to 5 years. Planning takes ~1-2 years, construction & setup takes ~1-2 years and ramp to full output takes 6 months to 1 year. And since we're on Econ 101, investments require payoff so all of this is done carefully to not tank the price of batteries by overproducing supply. In control systems terms, supply side investments always aim for undershooting demand as overshooting hurts how much money you make and risks destabilizing your market.
As for scarcity, inputs to lithium ion ARE scarce which negates your entire model. Pretending they aren't is where you're making a mistake. Lithium, cobalt & nickle are relatively scarce and the mines for that have to scale up to meet demand as well. You've also got a workforce to train to do the work which takes time & is also input-constrained. That's why there's massive NEW lithium mines being opened in the US & elsewhere to extract existing reserves to meet the growth in lithium ion batteries. If the world thought that sodium ion or ion air was an immediate future, you wouldn't see these massive large-scale investments into lithium. Lilthium-ion batteries is going to be a large and growing market for decades which brings me back to the strip mining of the ocean floor that's coming to support that.
Whright's law by the way isn't also an inevitable effect that goes on forever. At some point your exponential plateau's and you no longer see such exponential decrease in pricing. That's why processors aren't getting cheaper and compute isn't scaling up quite in the same way as in the early days. There's only so efficient you can make something.","created_at":"2025-02-19T16:15:42Z","created_at_i":1739981742,"objectID":"43103764","parent_id":43102765,"story_id":43093939,"story_title":"Nuclear fusion: WEST beats the world record for plasma duration","story_url":"https://www.cea.fr/english/Pages/News/nuclear-fusion-west-beats-the-world-record-for-plasma-duration.aspx","updated_at":"2025-02-19T21:44:10Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"karmakaze"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"This is a dumb post. I can't believe someone with 20 years is still looking for X is better than Y, black-and-white answers. All these questions have answers, but they're not blanket ones. You have to get into the details. But even without getting into details, I could probably ask ChatGPT and with a short conversation get to the crux of each matter. Most of them come up frequently enough on HN to have stock answers that the author doesn't seem to know about.
Microservices? It's about parallelizing and scaling teams. By decoupling codebases, tech stacks, deployments (to a large degree), you lean into Conway's law and reap benefits with other costs like dealing with eventually consistent behaviour.
Typed vs Untyped languages? Answer is use the language you know when starting out. A lot can be done with either, in most cases it comes down to preference and adeptness of what you know. For large-scale standard software (e.g. database, API, front-ends) using a statically-typed language will allow larger groups of developers to work on the same codebase with fewer surprises (like a typo in an unexercised codepath). But the sunk-cost is not a fallacy (or is very high so tread carefully), you can't stop and rewrite your entire business in Rust and compete to survive.
Blah blah blah...
Even if you work at each company for several years at a time, if you're paying attention you can see that a thing they did many years ago is tech debt on current development and operation. You don't have to learn in real-time, learn past history of the codebase you're working in. If you only work at early startups, try something different...
Like what some other comments are saying, try different things, expand your horizons, gain a wider perspective than what folks who are doing exactly what you're doing are talking about. Most of this focus on code and packaging is plumbing as far as I'm concerned. The actual thing software does is transform data from one thing to another: datastructures, algorithms. A higher level view, databases and SQL. The other stuff is a moderate puzzle of filling in the blanks.
Stop trying to find answers by appealing to authorities, f#@*-around and find out.
Edit: I actually entered this into GPT-4o and got expected results.
"The following is a post on Hacker News. I want you to look at the examples of unknown things given, and for each one, get to the crux of the matter providing a concise why/why-not, when/when-not, pros/cons.
...""},"story_title":{"matchLevel":"none","matchedWords":[],"value":"I don't know how to build software and you don't either"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.seangoedecke.com/confidence/"}},"_tags":["comment","author_karmakaze","story_42286416"],"author":"karmakaze","children":[42289110],"comment_text":"This is a dumb post. I can't believe someone with 20 years is still looking for X is better than Y, black-and-white answers. All these questions have answers, but they're not blanket ones. You have to get into the details. But even without getting into details, I could probably ask ChatGPT and with a short conversation get to the crux of each matter. Most of them come up frequently enough on HN to have stock answers that the author doesn't seem to know about.
Microservices? It's about parallelizing and scaling teams. By decoupling codebases, tech stacks, deployments (to a large degree), you lean into Conway's law and reap benefits with other costs like dealing with eventually consistent behaviour.
Typed vs Untyped languages? Answer is use the language you know when starting out. A lot can be done with either, in most cases it comes down to preference and adeptness of what you know. For large-scale standard software (e.g. database, API, front-ends) using a statically-typed language will allow larger groups of developers to work on the same codebase with fewer surprises (like a typo in an unexercised codepath). But the sunk-cost is not a fallacy (or is very high so tread carefully), you can't stop and rewrite your entire business in Rust and compete to survive.
Blah blah blah...
Even if you work at each company for several years at a time, if you're paying attention you can see that a thing they did many years ago is tech debt on current development and operation. You don't have to learn in real-time, learn past history of the codebase you're working in. If you only work at early startups, try something different...
Like what some other comments are saying, try different things, expand your horizons, gain a wider perspective than what folks who are doing exactly what you're doing are talking about. Most of this focus on code and packaging is plumbing as far as I'm concerned. The actual thing software does is transform data from one thing to another: datastructures, algorithms. A higher level view, databases and SQL. The other stuff is a moderate puzzle of filling in the blanks.
Stop trying to find answers by appealing to authorities, f#@*-around and find out.
Edit: I actually entered this into GPT-4o and got expected results.
"The following is a post on Hacker News. I want you to look at the examples of unknown things given, and for each one, get to the crux of the matter providing a concise why/why-not, when/when-not, pros/cons.
..."","created_at":"2024-12-01T15:42:46Z","created_at_i":1733067766,"objectID":"42288952","parent_id":42286416,"story_id":42286416,"story_title":"I don't know how to build software and you don't either","story_url":"https://www.seangoedecke.com/confidence/","updated_at":"2024-12-02T14:02:39Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"theptip"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"> No one's trying to eliminate anyone
Probably not any consolation to the Neanderthal or any other animal species that is out-competed in nature. Doesn\u2019t have to be malevolent or intentional for a more capable species to render others extinct. (Indeed that would seem to be the base case, historically speaking.) of course, symbiosis is possible; we could be puppies to the AI. Many people find disempowerment to be almost as bad a prospect as extinction though.
> The issue is whether or not we can build a machine capable of even understanding the directive.
Asimov\u2019s laws are an ok starting point for a moral code, but nobody has the slightest idea how to implement them. The issue is not \u201cunderstanding\u201d them, it is baking them into the fabric of a mind so deeply that they cannot be broken. This requires mind engineering, a deep understanding of mind-space, and we are far closer to stumbling into AGI through brute force scaling (as nature did) than to understanding mind-engineering deeply enough to embed \u201claws\u201d.
> it's not clear why an artificial mind would pursue this as a goal
There are a couple stock answers here. At the highest level though, why not? Humans have pursued goals of global domination, eradication, genocide. If only one in a billion minds has a defect (or is prompt-hijacked) to start power-seeking, it could still be bad; AI is not bound by human limitation to inhabit one body and can in principle expand to occupy all the compute it can secure.
Or if you prefer, why would a more intelligent entity agree to be commanded by a less intelligent one? The base case seem to me that ASI will not remain under our control, unless we very carefully sculpt its mind to align with our interests.
Another simple argument is: because someone will build or prompt-hijack such a mind. People have already created \u201ctake over the world\u201d agents with GPT-3, as a joke (I assume). There are enough people willing to take such an action that we\u2019ll need defenses in place.
I think it\u2019s true that the more ASI entities in existence, the less likely one is to fully take over. However there are esoteric decision theory reasons to believe that digital minds might be able to bargain/negotiate more effectively than biological minds (source code / mind state transparency etc) and so you might expect the ASIs to eventually collaborate to take control.
Personally I don\u2019t think that full-extinction is the most likely outcome, just that it\u2019s completely plausible if we get unlucky and it\u2019s very hard to prove safety in advance. And there are plenty of disempowering outcomes that seem quite bad too vs. the counterfactual."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Among the A.I. doomsayers"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.newyorker.com/magazine/2024/03/18/among-the-ai-doomsayers"}},"_tags":["comment","author_theptip","story_39673265"],"author":"theptip","children":[39685573],"comment_text":"> No one's trying to eliminate anyone
Probably not any consolation to the Neanderthal or any other animal species that is out-competed in nature. Doesn\u2019t have to be malevolent or intentional for a more capable species to render others extinct. (Indeed that would seem to be the base case, historically speaking.) of course, symbiosis is possible; we could be puppies to the AI. Many people find disempowerment to be almost as bad a prospect as extinction though.
> The issue is whether or not we can build a machine capable of even understanding the directive.
Asimov\u2019s laws are an ok starting point for a moral code, but nobody has the slightest idea how to implement them. The issue is not \u201cunderstanding\u201d them, it is baking them into the fabric of a mind so deeply that they cannot be broken. This requires mind engineering, a deep understanding of mind-space, and we are far closer to stumbling into AGI through brute force scaling (as nature did) than to understanding mind-engineering deeply enough to embed \u201claws\u201d.
> it's not clear why an artificial mind would pursue this as a goal
There are a couple stock answers here. At the highest level though, why not? Humans have pursued goals of global domination, eradication, genocide. If only one in a billion minds has a defect (or is prompt-hijacked) to start power-seeking, it could still be bad; AI is not bound by human limitation to inhabit one body and can in principle expand to occupy all the compute it can secure.
Or if you prefer, why would a more intelligent entity agree to be commanded by a less intelligent one? The base case seem to me that ASI will not remain under our control, unless we very carefully sculpt its mind to align with our interests.
Another simple argument is: because someone will build or prompt-hijack such a mind. People have already created \u201ctake over the world\u201d agents with GPT-3, as a joke (I assume). There are enough people willing to take such an action that we\u2019ll need defenses in place.
I think it\u2019s true that the more ASI entities in existence, the less likely one is to fully take over. However there are esoteric decision theory reasons to believe that digital minds might be able to bargain/negotiate more effectively than biological minds (source code / mind state transparency etc) and so you might expect the ASIs to eventually collaborate to take control.
Personally I don\u2019t think that full-extinction is the most likely outcome, just that it\u2019s completely plausible if we get unlucky and it\u2019s very hard to prove safety in advance. And there are plenty of disempowering outcomes that seem quite bad too vs. the counterfactual.","created_at":"2024-03-12T14:01:33Z","created_at_i":1710252093,"objectID":"39679626","parent_id":39676469,"story_id":39673265,"story_title":"Among the A.I. doomsayers","story_url":"https://www.newyorker.com/magazine/2024/03/18/among-the-ai-doomsayers","updated_at":"2025-08-14T22:42:40Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"pnathan"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Reading the OP carefully, it is clear there are a lot of implications that can be derived from choosing to limit service outside of what the law requires.
As a thought experiment: Suppose Oracle and Google show up and want the service. Oracle is known to be an aggressive company with respect to business and so the company denies Oracle... but accepts Google since Google is friendlier. \nDoes Oracle have grounds for suing ? I don't know. IANAL. But if I was in their position, I would be concerned until I consulted a lawyer.
What about other entities: political parties you disagree with? activist groups you disagree with? businesses who have unsavoury practices?
Put anther way, what can you legally say 'no' to, and what will get you sued for denying them?
If you can provide a straightfoward notice \"no patent trolls\" (enforceable in court), does that insulate you?
Again, I don't know. But it would give me pause to ponder if (when?) I am in the position of providing services.
--- more thoughts---
If you take the onus of determining who is moral onto your shoulders, you take responsibility for the inevitable screwup. And with that, you ensure that you will be arbitrary when you notice that one entity is an exception to your existing rules. You have to modify the rules, and that angers people (notice Blizzard's forums anytime changes come
Where the line? Where the exceptions? How to handle it? out!) This is a scaling problem.
Can you get away with not overthinking it and simply kicking the odiousness to the curb and hope that the scaling problem won't manifest?"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"What we did when a patent troll asked for our help"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"http://www.lessannoyingcrm.com/articles/262/What_we_did_when_a_patent_troll_asked_for_our_help"}},"_tags":["comment","author_pnathan","story_3949537"],"author":"pnathan","comment_text":"Reading the OP carefully, it is clear there are a lot of implications that can be derived from choosing to limit service outside of what the law requires.
As a thought experiment: Suppose Oracle and Google show up and want the service. Oracle is known to be an aggressive company with respect to business and so the company denies Oracle... but accepts Google since Google is friendlier. \nDoes Oracle have grounds for suing ? I don't know. IANAL. But if I was in their position, I would be concerned until I consulted a lawyer.
What about other entities: political parties you disagree with? activist groups you disagree with? businesses who have unsavoury practices?
Put anther way, what can you legally say 'no' to, and what will get you sued for denying them?
If you can provide a straightfoward notice \"no patent trolls\" (enforceable in court), does that insulate you?
Again, I don't know. But it would give me pause to ponder if (when?) I am in the position of providing services.
--- more thoughts---
If you take the onus of determining who is moral onto your shoulders, you take responsibility for the inevitable screwup. And with that, you ensure that you will be arbitrary when you notice that one entity is an exception to your existing rules. You have to modify the rules, and that angers people (notice Blizzard's forums anytime changes come
Where the line? Where the exceptions? How to handle it? out!) This is a scaling problem.
Can you get away with not overthinking it and simply kicking the odiousness to the curb and hope that the scaling problem won't manifest?","created_at":"2012-05-09T18:13:35Z","created_at_i":1336587215,"objectID":"3949802","parent_id":3949741,"story_id":3949537,"story_title":"What we did when a patent troll asked for our help","story_url":"http://www.lessannoyingcrm.com/articles/262/What_we_did_when_a_patent_troll_asked_for_our_help","updated_at":"2024-09-19T18:35:33Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"giardini"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Unnecessary items:
Scaling up - static website is enuf.
Site Reliability - ditto.
Front End Web - ditto.
Necessary items:
Ops - Gotta have someone who understands computers! Yes.
Research - Here's the work. Yes.
Training - No. Hire people carefully, fire quickly.
Legal - minimal - hire a small law firm.
So ~700 people mostly in Research and some cash for Legal? The scope of work must far exceed the scope of the task. Time to trim.
Comparisons to Google makes no sense."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Sam Altman returns as CEO, OpenAI has a new initial board"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://openai.com/blog/sam-altman-returns-as-ceo-openai-has-a-new-initial-board"}},"_tags":["comment","author_giardini","story_38467850"],"author":"giardini","comment_text":"Unnecessary items:
Scaling up - static website is enuf.
Site Reliability - ditto.
Front End Web - ditto.
Necessary items:
Ops - Gotta have someone who understands computers! Yes.
Research - Here's the work. Yes.
Training - No. Hire people carefully, fire quickly.
Legal - minimal - hire a small law firm.
So ~700 people mostly in Research and some cash for Legal? The scope of work must far exceed the scope of the task. Time to trim.
Comparisons to Google makes no sense.","created_at":"2023-12-01T18:04:24Z","created_at_i":1701453864,"objectID":"38489993","parent_id":38482078,"story_id":38467850,"story_title":"Sam Altman returns as CEO, OpenAI has a new initial board","story_url":"https://openai.com/blog/sam-altman-returns-as-ceo-openai-has-a-new-initial-board","updated_at":"2024-09-20T15:52:59Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"wolfgang42"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"I\u2019ve been puttering away at making a search engine of my own (I should really do a Show HN sometime); let\u2019s see how my experience compares with 18 years ago:
Bandwidth: This is now also cheap; my residential service is 1 Gbit. However, the suggestion to wait until you\u2019ve got indexing working well before optimizing crawling is IMO still spot-on; trying to make a polite, performant crawler that can deal with all the bizzare edge cases (https://memex.marginalia.nu/log/32-bot-apologetics.gmi) on the Web will drag you down. (I bypassed this problem by starting with the Stack Exchange data dumps and Wikipedia crawls, which are a lot more consistent than trying to deal with random websites.)
CPU: Computers are really fast now; I\u2019m using a 2-core computer from 2014 and it does what I need just fine.
Disk: SATA is the new thing now, of course, but the difference these days is HDD vs SSD. SSD is faster: but you can design your architecture so that this mostly doesn\u2019t matter, and even a \u201cslow\u201d HDD will be running at capacity. (The trick is to do linear streaming as much as possible, and avoid seeks at all costs.) Still, it\u2019s probably a good idea to store your production index on an SSD, and it\u2019s useful for intermediate data as well; by happenstance more than design I have a large HDD and a small SSD and they balance each other nicely.
Storing files: 100% agree with this section, for the disk-seek reasons I mention above. Also, pages from the same website often compress very well against each other (since they\u2019re using the same templates, large chunks of HTML can be squished down considerably), so if you\u2019re pressed for space consider storing one GZIPped file per domain. (The tradeoff with zipping is that you can\u2019t arbitrarily seek, but ideally you\u2019ve designed things so you don\u2019t need to do that anyway.) Also, WARC is a standard file format that has a lot of tooling for this exact use case.
Networking: I skipped this by just storing everything on one computer; I expect to be able to continue doing this for a long time, since vertical scaling can get you very far these days.
Indexing: You basically don\u2019t need to write anything to get started with this these days! I\u2019m just using bog-standard Elasticsearch with some glue code to do html2text; it\u2019s working fine and took all of an afternoon to set up from scratch. (That said, I\u2019m not sure I\u2019ll continue using Elastic: it has a ton of features I don\u2019t need, which makes it very hard to understand and work with since there\u2019s so much that\u2019s irrelevant to me. I\u2019m probably going to switch to either straight Lucene or Bleve soon.)
Page rank: I added pagerank very early on in the hopes that it would improve my results, and I\u2019m not really sure how helpful it is if your results aren\u2019t decent to begin with. However, the march of Moore\u2019s law has made it an easy experiment: what Page and Brin\u2019s server could compute in a week with carefully optimized C code, mine can do in less than 5 minutes (!) with a bit of JavaScript.
Serving: Again, ElasticSearch will solve this entire problem for you (at least to start with); all your frontend has to do is take the JSON result and poke it into an HTML template.
It\u2019s easier than ever to start building a search engine in your own home; the recent explosion of such services (as seen on HN) is an indicator of the feasibility, and the rising complaints about Google show that the demand is there. Come and join us, the water\u2019s fine!"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Writing your own search engine is hard (2004)"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://queue.acm.org/detail.cfm?id=988407"}},"_tags":["comment","author_wolfgang42","story_32205535"],"author":"wolfgang42","children":[32208388,32211795],"comment_text":"I\u2019ve been puttering away at making a search engine of my own (I should really do a Show HN sometime); let\u2019s see how my experience compares with 18 years ago:
Bandwidth: This is now also cheap; my residential service is 1 Gbit. However, the suggestion to wait until you\u2019ve got indexing working well before optimizing crawling is IMO still spot-on; trying to make a polite, performant crawler that can deal with all the bizzare edge cases (https://memex.marginalia.nu/log/32-bot-apologetics.gmi) on the Web will drag you down. (I bypassed this problem by starting with the Stack Exchange data dumps and Wikipedia crawls, which are a lot more consistent than trying to deal with random websites.)
CPU: Computers are really fast now; I\u2019m using a 2-core computer from 2014 and it does what I need just fine.
Disk: SATA is the new thing now, of course, but the difference these days is HDD vs SSD. SSD is faster: but you can design your architecture so that this mostly doesn\u2019t matter, and even a \u201cslow\u201d HDD will be running at capacity. (The trick is to do linear streaming as much as possible, and avoid seeks at all costs.) Still, it\u2019s probably a good idea to store your production index on an SSD, and it\u2019s useful for intermediate data as well; by happenstance more than design I have a large HDD and a small SSD and they balance each other nicely.
Storing files: 100% agree with this section, for the disk-seek reasons I mention above. Also, pages from the same website often compress very well against each other (since they\u2019re using the same templates, large chunks of HTML can be squished down considerably), so if you\u2019re pressed for space consider storing one GZIPped file per domain. (The tradeoff with zipping is that you can\u2019t arbitrarily seek, but ideally you\u2019ve designed things so you don\u2019t need to do that anyway.) Also, WARC is a standard file format that has a lot of tooling for this exact use case.
Networking: I skipped this by just storing everything on one computer; I expect to be able to continue doing this for a long time, since vertical scaling can get you very far these days.
Indexing: You basically don\u2019t need to write anything to get started with this these days! I\u2019m just using bog-standard Elasticsearch with some glue code to do html2text; it\u2019s working fine and took all of an afternoon to set up from scratch. (That said, I\u2019m not sure I\u2019ll continue using Elastic: it has a ton of features I don\u2019t need, which makes it very hard to understand and work with since there\u2019s so much that\u2019s irrelevant to me. I\u2019m probably going to switch to either straight Lucene or Bleve soon.)
Page rank: I added pagerank very early on in the hopes that it would improve my results, and I\u2019m not really sure how helpful it is if your results aren\u2019t decent to begin with. However, the march of Moore\u2019s law has made it an easy experiment: what Page and Brin\u2019s server could compute in a week with carefully optimized C code, mine can do in less than 5 minutes (!) with a bit of JavaScript.
Serving: Again, ElasticSearch will solve this entire problem for you (at least to start with); all your frontend has to do is take the JSON result and poke it into an HTML template.
It\u2019s easier than ever to start building a search engine in your own home; the recent explosion of such services (as seen on HN) is an indicator of the feasibility, and the rising complaints about Google show that the demand is there. Come and join us, the water\u2019s fine!","created_at":"2022-07-23T20:17:22Z","created_at_i":1658607442,"objectID":"32207197","parent_id":32205535,"story_id":32205535,"story_title":"Writing your own search engine is hard (2004)","story_url":"https://queue.acm.org/detail.cfm?id=988407","updated_at":"2024-09-20T11:35:55Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"SonOfLilit"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"This brings to mind Richard Hamming's wonderful essay "The Unreasonable Effectiveness of Mathematics", which really blew my mind ten years ago:
"But if you do not like these two examples, let me turn to the most highly touted law of recent times, the uncertainty principle. It happens that recently I became involved in writing a book on Digital Filters [8] when I knew very little about the topic. As a result I early asked the question, "Why should I do all the analysis in terms of Fourier integrals? Why are they the natural tools for the problem?" I soon found out, as many of you already know, that the eigenfunctions of translation are the complex exponentials. If you want time invariance, and certainly physicists and engineers do (so that an experiment done today or tomorrow will give the same results), then you are led to these functions. Similarly, if you believe in linearity then they are again the eigenfunctions. In quantum mechanics the quantum states are absolutely additive; they are not just a convenient linear approximation. Thus the trigonometric functions are the eigenfunctions one needs in both digital filter theory and quantum mechanics, to name but two places.
"Now when you use these eigenfunctions you are naturally led to representing various functions, first as a countable number and then as a non-countable number of them\u2014namely, the Fourier series and the Fourier integral. Well, it is a theorem in the theory of Fourier integrals that the variability of the function multiplied by the variability of its transform exceeds a fixed constant, in one notation l/2\u220f. This says to me that in any linear, time invariant system you must find an uncertainty principle. The size of Planck's constant is a matter of the detailed identification of the variables with integrals, but the inequality must occur.
"As another example of what has often been thought to be a physical discovery but which turns out to have been put in there by ourselves, I turn to the well-known fact that the distribution of physical constants is not uniform; rather the probability of a random physical constant having a leading digit of 1, 2, or 3 is approximately 60%, and of course the leading digits of 5, 6, 7, 8, and 9 occur in total only about 40% of the time. This distribution applies to many types of numbers, including the distribution of the coefficients of a power series having only one singularity on the circle of convergence. A close examination of this phenomenon shows that it is mainly an artifact of the way we use numbers.
"Having given four widely different examples of nontrivial situations where it turns out that the original phenomenon arises from the mathematical tools we use and not from the real world, I am ready to strongly suggest that a lot of what we see comes from the glasses we put on. Of course this goes against much of what you have been taught, but consider the arguments carefully. You can say that it was the experiment that forced the model on us, but I suggest that the more you think about the four examples the more uncomfortable you are apt to become. They are not arbitrary theories that I have selected, but ones which are central to physics.
http://inters.org/Hamming-Unreasonable-Effectiveness-Mathema...
I don't know any QM, but it does not surprise me in the least that, trying to calculate something about rotation, you end up with a formula for Pi. Is it there because of a connection to how the world works, or is it there because we happened to ask a question where two effects of the real world canceled each other out and all that remains is something about math? Sure, "no circles were involved", but rotation was involved, and it is just as Pi-heavy a model as anything involving circles. Also, QM is full of integrals of sines, for reasons explained in the quote above, which introduce a bunch of Pi scaling factors that have nothing to do with reality and everything to do with the tools we chose to model it with, etc'.
(I'm not saying that the universe doesn't seem to have a lot of Pi factors in its machinery, just that this specific case doesn't sound like one of them, and that the distinction is interesting and mind blowing, to me at least)."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"The formula for pi buried in a hydrogen atom"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.rochester.edu/newscenter/discovery-of-classic-pi-formula-a-cunning-piece-of-magic-128002/"}},"_tags":["comment","author_SonOfLilit","story_22844196"],"author":"SonOfLilit","children":[22851806,22852891,22859016],"comment_text":"This brings to mind Richard Hamming's wonderful essay "The Unreasonable Effectiveness of Mathematics", which really blew my mind ten years ago:
"But if you do not like these two examples, let me turn to the most highly touted law of recent times, the uncertainty principle. It happens that recently I became involved in writing a book on Digital Filters [8] when I knew very little about the topic. As a result I early asked the question, "Why should I do all the analysis in terms of Fourier integrals? Why are they the natural tools for the problem?" I soon found out, as many of you already know, that the eigenfunctions of translation are the complex exponentials. If you want time invariance, and certainly physicists and engineers do (so that an experiment done today or tomorrow will give the same results), then you are led to these functions. Similarly, if you believe in linearity then they are again the eigenfunctions. In quantum mechanics the quantum states are absolutely additive; they are not just a convenient linear approximation. Thus the trigonometric functions are the eigenfunctions one needs in both digital filter theory and quantum mechanics, to name but two places.
"Now when you use these eigenfunctions you are naturally led to representing various functions, first as a countable number and then as a non-countable number of them\u2014namely, the Fourier series and the Fourier integral. Well, it is a theorem in the theory of Fourier integrals that the variability of the function multiplied by the variability of its transform exceeds a fixed constant, in one notation l/2\u220f. This says to me that in any linear, time invariant system you must find an uncertainty principle. The size of Planck's constant is a matter of the detailed identification of the variables with integrals, but the inequality must occur.
"As another example of what has often been thought to be a physical discovery but which turns out to have been put in there by ourselves, I turn to the well-known fact that the distribution of physical constants is not uniform; rather the probability of a random physical constant having a leading digit of 1, 2, or 3 is approximately 60%, and of course the leading digits of 5, 6, 7, 8, and 9 occur in total only about 40% of the time. This distribution applies to many types of numbers, including the distribution of the coefficients of a power series having only one singularity on the circle of convergence. A close examination of this phenomenon shows that it is mainly an artifact of the way we use numbers.
"Having given four widely different examples of nontrivial situations where it turns out that the original phenomenon arises from the mathematical tools we use and not from the real world, I am ready to strongly suggest that a lot of what we see comes from the glasses we put on. Of course this goes against much of what you have been taught, but consider the arguments carefully. You can say that it was the experiment that forced the model on us, but I suggest that the more you think about the four examples the more uncomfortable you are apt to become. They are not arbitrary theories that I have selected, but ones which are central to physics.
http://inters.org/Hamming-Unreasonable-Effectiveness-Mathema...
I don't know any QM, but it does not surprise me in the least that, trying to calculate something about rotation, you end up with a formula for Pi. Is it there because of a connection to how the world works, or is it there because we happened to ask a question where two effects of the real world canceled each other out and all that remains is something about math? Sure, "no circles were involved", but rotation was involved, and it is just as Pi-heavy a model as anything involving circles. Also, QM is full of integrals of sines, for reasons explained in the quote above, which introduce a bunch of Pi scaling factors that have nothing to do with reality and everything to do with the tools we chose to model it with, etc'.
(I'm not saying that the universe doesn't seem to have a lot of Pi factors in its machinery, just that this specific case doesn't sound like one of them, and that the distinction is interesting and mind blowing, to me at least).","created_at":"2020-04-12T23:06:15Z","created_at_i":1586732775,"objectID":"22851699","parent_id":22844196,"story_id":22844196,"story_title":"The formula for pi buried in a hydrogen atom","story_url":"https://www.rochester.edu/newscenter/discovery-of-classic-pi-formula-a-cunning-piece-of-magic-128002/","updated_at":"2024-09-20T06:02:17Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"bcaa7f3a8bbc"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"I think your comment is gold. Because I've been also thinking about the construction, organization, and mechanism of online communities in general, most of my comments were posted to another community, but I posted some related comments on Hacker News as well. I'll start dumping comments as well, I hope you find them interesting. Feedback is appreciated.
1. First, I've been observing various online trolling culture and fringe reactionary political movements for a few years. Mostly 4chan, but I think it applies to the web in general. While I don't agree with their intentions or motivations, but I found it's an extremely interesting phenomenon and pretty post-modernist in itself.
* The cyclist behind an anti-cyclist Facebook group
https://news.ycombinator.com/item?id=17731220
* Recommended comments (read the link below before you continue):
https://news.ycombinator.com/item?id=17731914
Here's a story: a biking hobbyist used his spare time and launched an influential anti-bike movement on Facebook. He posted numerous sensationalized commentaries, posters and memes to demonize bikers, including the various "data" and "charts" for the "bikes are a great threat of traffic order" propaganda, and occasionally advocated violence towards bikers in forms of half-jokes. He also proposed political reforms to introduce restrictive and/or discriminatory traffic laws for bikers. Soon, his campaign has gone viral and attracted a large number of supporters who drive cars. Well, not all bikers follow the traffic rules and some problems surely exist, but they are only as bad as the average day-to-day traffic and usually not seen as something people need to act upon, but this campaign exaggerates it, radicalized it and polarized the issue and created lot of hate. What is his intention then? His only intention is laughing at this fact: he was able to manipulate the car drivers easily and even create a raged populist political movement with a mere keyboard.
We hardly know who's behind the populist political movements online, but this is a well-documented case, and I believe it can be a representative sample which largely reflects the origin, emotions and motivation of the 4chan-like "for-the-lulz" trolling culture. Also, I can immediately point out the similarities to the populist reactionary political movements: the nationalism propaganda used many of the tactics the biking troll has used. Also, Russia is accused by the media to create an army of bots on social media, such as pro-LGBT, anti-LGBT, pro-life, pro-choice, religion fundamentalism, atheism, nationalistic, liberal, etc, with the intention of creating polarized argument and social conflicts in the United State.
On the other hand, the "for-the-lulz" trolling culture itself, even can be used as propaganda, is not propaganda in itself. It is a form of complicated web culture. It's basic definition is "doing some pointless activities to make others to suffer, and laugh at it", but it stands for a lot of things. It has a reactionary, or anti-establishment element, as previously mentioned, but it also includes: (1) cultural jamming, similar to the 70s counter-cultural movement (see those hoaxes in Fight Club), (2) It understands the art of exploiting a vulnerability in the system for one's own advantage, similar to parts of the hacker culture, (3) Also, it has a somewhat nihilist, post-modernist component, sometimes artistic (The Game was popular on early 4chan: once you remember playing it, you lose it), and (4), it reflects the worrying, helplessness and alienating aspects of modern life, and as an entertainment against them using humor and hate, then finally (5) a populist movement. It's core logic is: everyone, look how evil/degenerate this thing/person is, the justice needs to be done! Sometimes, it can be a legitimate popular movement and serve the right purpose, e.g. Occupy Movement, but other times, it's just a bunch of angry mobs lynching people. For example, a list of people on Encyclopedia Dramatica.
Especially, when I grasped these points, I found things went pretty stupid when, for example, the mainstream media started reporting Pepe the Frog as a hate symbol in 2016. Yes, it has connections to reactionary political movements, but itself is only a symbol of the trolling cultural. When the media reports it, it actually gives it a platform of exposure, and encouraged trolls to use it as a genuinely political symbol, and personally I think it makes the issue worse.
What I'm trying to say is, to solve problems like this, we must study and understand the entire machinery of it, including the underlying social-economical basis of this conflict using sociology and psychology, and to provide social solutions to the problem. However, the only proposal the mainstream media (or us, as online members of online communities) has right now is "Twitter and Facebook should ban hate speech", or "we should not feed the trolls", which hardly touches deeper problems.
If you want to understand more about the trolling culture and its mechanism, I recommend "This Is Why We Can't Have Nice Things: Mapping the Relationship between Online Trolling and Mainstream Culture", by Whitney Phillips, published by The MIT Press.
Here's are related comments on this issue I have posted before,
* Twitter has barred hundreds of accounts for posing as liberal activists
My comment: https://news.ycombinator.com/item?id=18239216
* RIP Culture War Thread
My comment: https://news.ycombinator.com/item?id=19231994
This one is mostly a repetition, but it includes one more book recommendation. I also recommend to read the original web article. It has covered a lot of related discussion.
2. I was also trying to characterize online communities in terms of its medium, feedback mechanisms, and filtering mechanism. A online platform has a lot of content, but you only see a subset of content. The filtering mechanism determines which subset it is. We could say, the first generation of online community is mailing list, Usenet newsgroups, and web forums. It only has a simple filtering mechanism: grouping the content by topics, and a kill file (a people/keyword blacklist, sometime shared). It works relatively well, and has the advantage of producing diverse topics, free-flowing discussions, but there's nothing to stop low-quality discussion, so a good signal-to-noise radio must be carefully maintained. Another problem is the lack of feedback: when nobody reply for post, it means five different possibilities, see Warnock's dilemma (https://en.wikipedia.org/wiki/Warnock%27s_dilemma). Another important issue is: the entire forum basically stops scaling when the number of users exceeds 10,000 or so. The information is too much to handle for any individual.
Platforms like Reddit or Hacker News can be seen as the second generation, it introduced voting as a feedback mechanism, which simultaneously (partially) solves the problems of content recommendations, restriction of low-quality content, and scaling. The "karma" system of Slashdot also counts, but it has a innovative "community moderation", so I guess it's 2.5 generation. Anyway, overall, its usefulness of voting as feedback is great and now becomes an integral part of many communities, but not without its own problem. Voting creates a strong incentive for people to collect upvote. People may stop talking about insightful contrarian views, which otherwise can generate an intellectual interesting discussion in a newsgroup. Also, voting seems to be more suitable for link/media recommendations, but less suitable for discussions. Many subreddits has thousands of votes for links but a single digit of comment.
Finally, ~2007, here comes the social media. Social media is unique because it leveraged the underlying human social network as a means of the filtering mechanism, and has a somewhat "decentralized" model of publishing. Unlike a forum where everyone post to a single directory, on Twitter, you post to your own account, and others can simply follow a number of people, then interesting posts simply circulate by itself. The system is highly effective, I love social media. Unfortunately, problems. First, the content has to be short, no long discussion is possible. This is usually not a problem, but the social media has slowly killed most forums in the process and became the center of the web, forcing people to use for discussion, although the social media is an awful platform for this purpose. Its structure is conversational, posts don't have a single thread, but a dozen of microthread. A replies to B, but this reply is usually ignored by C, which may have crucial context. Furthermore, the volatile nature causes many problems, A mentioned X 3 hours ago, and now X is missed by most readers. Or, A posted X and Y, but often only a single X/Y is seen by other readers, while the other one has important context. Also, the social media has the strongest filtering bubble effect compared to all platform, which may create an echo chamber, increase political polarization among other effects.
But think about it. Voting was invented by Reddit in ~2002, social media is a thing from 2007. Can we design a new generation of communities with innovation? A better alternative "filtering mechanism"?
3. I noticed a lot of online communities seems to go downhill. What is going? I have a comment.
https://news.ycombinator.com/item?id=19958700"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Neal Stephenson Explains His Vision of the Digital Afterlife"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://www.pcmag.com/news/368417/neal-stephenson-explains-his-vision-of-the-digital-afterlife"}},"_tags":["comment","author_bcaa7f3a8bbc","story_20011252"],"author":"bcaa7f3a8bbc","children":[20013112],"comment_text":"I think your comment is gold. Because I've been also thinking about the construction, organization, and mechanism of online communities in general, most of my comments were posted to another community, but I posted some related comments on Hacker News as well. I'll start dumping comments as well, I hope you find them interesting. Feedback is appreciated.
1. First, I've been observing various online trolling culture and fringe reactionary political movements for a few years. Mostly 4chan, but I think it applies to the web in general. While I don't agree with their intentions or motivations, but I found it's an extremely interesting phenomenon and pretty post-modernist in itself.
* The cyclist behind an anti-cyclist Facebook group
https://news.ycombinator.com/item?id=17731220
* Recommended comments (read the link below before you continue):
https://news.ycombinator.com/item?id=17731914
Here's a story: a biking hobbyist used his spare time and launched an influential anti-bike movement on Facebook. He posted numerous sensationalized commentaries, posters and memes to demonize bikers, including the various "data" and "charts" for the "bikes are a great threat of traffic order" propaganda, and occasionally advocated violence towards bikers in forms of half-jokes. He also proposed political reforms to introduce restrictive and/or discriminatory traffic laws for bikers. Soon, his campaign has gone viral and attracted a large number of supporters who drive cars. Well, not all bikers follow the traffic rules and some problems surely exist, but they are only as bad as the average day-to-day traffic and usually not seen as something people need to act upon, but this campaign exaggerates it, radicalized it and polarized the issue and created lot of hate. What is his intention then? His only intention is laughing at this fact: he was able to manipulate the car drivers easily and even create a raged populist political movement with a mere keyboard.
We hardly know who's behind the populist political movements online, but this is a well-documented case, and I believe it can be a representative sample which largely reflects the origin, emotions and motivation of the 4chan-like "for-the-lulz" trolling culture. Also, I can immediately point out the similarities to the populist reactionary political movements: the nationalism propaganda used many of the tactics the biking troll has used. Also, Russia is accused by the media to create an army of bots on social media, such as pro-LGBT, anti-LGBT, pro-life, pro-choice, religion fundamentalism, atheism, nationalistic, liberal, etc, with the intention of creating polarized argument and social conflicts in the United State.
On the other hand, the "for-the-lulz" trolling culture itself, even can be used as propaganda, is not propaganda in itself. It is a form of complicated web culture. It's basic definition is "doing some pointless activities to make others to suffer, and laugh at it", but it stands for a lot of things. It has a reactionary, or anti-establishment element, as previously mentioned, but it also includes: (1) cultural jamming, similar to the 70s counter-cultural movement (see those hoaxes in Fight Club), (2) It understands the art of exploiting a vulnerability in the system for one's own advantage, similar to parts of the hacker culture, (3) Also, it has a somewhat nihilist, post-modernist component, sometimes artistic (The Game was popular on early 4chan: once you remember playing it, you lose it), and (4), it reflects the worrying, helplessness and alienating aspects of modern life, and as an entertainment against them using humor and hate, then finally (5) a populist movement. It's core logic is: everyone, look how evil/degenerate this thing/person is, the justice needs to be done! Sometimes, it can be a legitimate popular movement and serve the right purpose, e.g. Occupy Movement, but other times, it's just a bunch of angry mobs lynching people. For example, a list of people on Encyclopedia Dramatica.
Especially, when I grasped these points, I found things went pretty stupid when, for example, the mainstream media started reporting Pepe the Frog as a hate symbol in 2016. Yes, it has connections to reactionary political movements, but itself is only a symbol of the trolling cultural. When the media reports it, it actually gives it a platform of exposure, and encouraged trolls to use it as a genuinely political symbol, and personally I think it makes the issue worse.
What I'm trying to say is, to solve problems like this, we must study and understand the entire machinery of it, including the underlying social-economical basis of this conflict using sociology and psychology, and to provide social solutions to the problem. However, the only proposal the mainstream media (or us, as online members of online communities) has right now is "Twitter and Facebook should ban hate speech", or "we should not feed the trolls", which hardly touches deeper problems.
If you want to understand more about the trolling culture and its mechanism, I recommend "This Is Why We Can't Have Nice Things: Mapping the Relationship between Online Trolling and Mainstream Culture", by Whitney Phillips, published by The MIT Press.
Here's are related comments on this issue I have posted before,
* Twitter has barred hundreds of accounts for posing as liberal activists
My comment: https://news.ycombinator.com/item?id=18239216
* RIP Culture War Thread
My comment: https://news.ycombinator.com/item?id=19231994
This one is mostly a repetition, but it includes one more book recommendation. I also recommend to read the original web article. It has covered a lot of related discussion.
2. I was also trying to characterize online communities in terms of its medium, feedback mechanisms, and filtering mechanism. A online platform has a lot of content, but you only see a subset of content. The filtering mechanism determines which subset it is. We could say, the first generation of online community is mailing list, Usenet newsgroups, and web forums. It only has a simple filtering mechanism: grouping the content by topics, and a kill file (a people/keyword blacklist, sometime shared). It works relatively well, and has the advantage of producing diverse topics, free-flowing discussions, but there's nothing to stop low-quality discussion, so a good signal-to-noise radio must be carefully maintained. Another problem is the lack of feedback: when nobody reply for post, it means five different possibilities, see Warnock's dilemma (https://en.wikipedia.org/wiki/Warnock%27s_dilemma). Another important issue is: the entire forum basically stops scaling when the number of users exceeds 10,000 or so. The information is too much to handle for any individual.
Platforms like Reddit or Hacker News can be seen as the second generation, it introduced voting as a feedback mechanism, which simultaneously (partially) solves the problems of content recommendations, restriction of low-quality content, and scaling. The "karma" system of Slashdot also counts, but it has a innovative "community moderation", so I guess it's 2.5 generation. Anyway, overall, its usefulness of voting as feedback is great and now becomes an integral part of many communities, but not without its own problem. Voting creates a strong incentive for people to collect upvote. People may stop talking about insightful contrarian views, which otherwise can generate an intellectual interesting discussion in a newsgroup. Also, voting seems to be more suitable for link/media recommendations, but less suitable for discussions. Many subreddits has thousands of votes for links but a single digit of comment.
Finally, ~2007, here comes the social media. Social media is unique because it leveraged the underlying human social network as a means of the filtering mechanism, and has a somewhat "decentralized" model of publishing. Unlike a forum where everyone post to a single directory, on Twitter, you post to your own account, and others can simply follow a number of people, then interesting posts simply circulate by itself. The system is highly effective, I love social media. Unfortunately, problems. First, the content has to be short, no long discussion is possible. This is usually not a problem, but the social media has slowly killed most forums in the process and became the center of the web, forcing people to use for discussion, although the social media is an awful platform for this purpose. Its structure is conversational, posts don't have a single thread, but a dozen of microthread. A replies to B, but this reply is usually ignored by C, which may have crucial context. Furthermore, the volatile nature causes many problems, A mentioned X 3 hours ago, and now X is missed by most readers. Or, A posted X and Y, but often only a single X/Y is seen by other readers, while the other one has important context. Also, the social media has the strongest filtering bubble effect compared to all platform, which may create an echo chamber, increase political polarization among other effects.
But think about it. Voting was invented by Reddit in ~2002, social media is a thing from 2007. Can we design a new generation of communities with innovation? A better alternative "filtering mechanism"?
3. I noticed a lot of online communities seems to go downhill. What is going? I have a comment.
https://news.ycombinator.com/item?id=19958700","created_at":"2019-05-26T01:22:43Z","created_at_i":1558833763,"objectID":"20012564","parent_id":20011741,"story_id":20011252,"story_title":"Neal Stephenson Explains His Vision of the Digital Afterlife","story_url":"https://www.pcmag.com/news/368417/neal-stephenson-explains-his-vision-of-the-digital-afterlife","updated_at":"2024-09-20T04:13:53Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"robbrown451"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["scaling","laws","carefully"],"value":"Explore game theoretically sound ways to start new companies as "worker co-ops" and to reward participants. The idea being to aim toward a future where there is a large gray area between "employee" and "entrepreneur."
The companies formed don't have to be start-ups per se: they could be as simple (and low risk) as a group of people who build web sites and apps for clients, but would like to have a stake in it rather than just "working for the man."
Since it is research, it is ok if things are explored that would require changes in laws.
I can imagine a site (or better yet, a standard protocol) -- sort of Kickstarter-ish, but different -- where people start projects, gather contributors, "pay" those contributors by giving them equity (or actually pay them, if the company is bringing in revenue).
There would be ways of holding votes (presumably voting power is weighted by how much you have contributed), and various checks and balances to prevent participants from gaming it. Everyone would probably be able to weigh in on the contributions of others, in a carefully designed way that encourages everyone to be productive, cooperative, and creative.
The idea is that there are a huge number of people out there with talent, creativity, and a willingness to take some risks, but most of them aren't really entrepreneurial types per se. This would be both a matchmaking site for getting people with complementary talents and personalities together, as well as a service for handling equity calculations as people contribute hours (and various other things that streamline the process so people with talent can spend their efforts using their actual talents).
Obviously there is some overlap with what Y-Combinator does, but this is a way of scaling it out much larger, in ways that could dramatically increase innovation as well as increasing job satisfaction."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Ask HN: What should we fund at YC Research?"}},"_tags":["comment","author_robbrown451","story_10998486"],"author":"robbrown451","children":[11000639,11001212,11001863],"comment_text":"Explore game theoretically sound ways to start new companies as "worker co-ops" and to reward participants. The idea being to aim toward a future where there is a large gray area between "employee" and "entrepreneur."
The companies formed don't have to be start-ups per se: they could be as simple (and low risk) as a group of people who build web sites and apps for clients, but would like to have a stake in it rather than just "working for the man."
Since it is research, it is ok if things are explored that would require changes in laws.
I can imagine a site (or better yet, a standard protocol) -- sort of Kickstarter-ish, but different -- where people start projects, gather contributors, "pay" those contributors by giving them equity (or actually pay them, if the company is bringing in revenue).
There would be ways of holding votes (presumably voting power is weighted by how much you have contributed), and various checks and balances to prevent participants from gaming it. Everyone would probably be able to weigh in on the contributions of others, in a carefully designed way that encourages everyone to be productive, cooperative, and creative.
The idea is that there are a huge number of people out there with talent, creativity, and a willingness to take some risks, but most of them aren't really entrepreneurial types per se. This would be both a matchmaking site for getting people with complementary talents and personalities together, as well as a service for handling equity calculations as people contribute hours (and various other things that streamline the process so people with talent can spend their efforts using their actual talents).
Obviously there is some overlap with what Y-Combinator does, but this is a way of scaling it out much larger, in ways that could dramatically increase innovation as well as increasing job satisfaction.","created_at":"2016-01-30T04:47:46Z","created_at_i":1454129266,"objectID":"11000551","parent_id":10998486,"story_id":10998486,"story_title":"Ask HN: What should we fund at YC Research?","updated_at":"2024-09-19T22:43:06Z"}],"hitsPerPage":20,"nbHits":96,"nbPages":5,"page":0,"params":"query=Scaling+Laws%2C+Carefully&advancedSyntax=true&analyticsTags=backend","processingTimeMS":52,"processingTimingsMS":{"_request":{"queue":142,"roundTrip":19},"afterFetch":{"format":{"highlighting":4,"total":4},"merge":{"mergeLoop":{"prepareNextHit":8,"total":9},"total":9},"total":9},"fetch":{"query":19,"scanning":22,"total":42},"total":52},"query":"Scaling Laws, Carefully","serverTimeMS":200}