Berserker event is effectively an implementation detail of one of possible events in the Dark Forest.
Like Wikipedia says, Dark Forest is for people who are limited in tech and resources. Or have a weird sense of ethics, but ask Robin Hanson how long he thinks that would last.
It’s an interesting contrast to actual dark forest theory—-AI doesn’t want to destroy us, so we don’t need to hide for existential sake. But imagine walking outside and as soon as you do, innumerable copies of you spring up, and each action and sound you make is replicated and amplified. Like a weird Phantom Tollbooth meets Alice in Wonderland on DMT
That's honestly not a problem, unless someone believes they're entitled to 100% of value generated by their interactions with the world - but that's a level of greed way beyond Scrooge McDuck, and the kind of thinking that defines the dreaded "late stage capitalism".
It really is an option to publish something and not care about knowing exactly who read it, and what they're doing with it.
There's 8 billion people on the planet. Unless you're a celebrity, this is not a real problem for you at this point (and if you are, it's a business problem).
There's no way for a large model learning on the entirety of the Internet to somehow convert "copying likeness, mannerisms, writing style, etc." into damaging your reputation; doing that is something hyper-targeted, and at this point (and in conceivable future), there's no middle ground between "plausible deniability" and "someone targeting you specifically, which they could do just as well in pre-AI times".
You're also not as unique as you think. There are many people with same mannerism, many people with same writing style, etc. Nor are those things constant over time.
The flip side of not being a unique snowflake is also that anyone's contributions to the public Internet are, for purposes of AI training, worth approximately $0 on the margin, and impact the model just as much. It's in the volume of data that the patterns emerge that LLMs learn, volume too big for any person to be entitled to a meaningful part of it.
It is about the right to have my own value not diluted.
It is not about caring to know who read each one of my words.
It is about the fact that people can later say „xyz wrote that” instead of „oh well another AI generated crap comment” when in reality it was someone’s original thought.
It is not capitalism, but basic respect for other human being.
"We attribute a thought to a person through a chain of custody. Without it, the thought could be attributed to anyone, or to a generative language model. Not that it matters anyway." -- Abraham Lincoln, ~1862.
Plus pragmatically speaking this sounds suspiciously like something that's "not a problem" until it is a problem. If a poorly written LLM regurgitates my name and lines from a blog post (or god forbid hallucinates a blog post by me as a citation) in some defense of Pol Pot or something that's going to be come a problem for me very quickly.
EDIT: Fridge thought: quite frankly if someone's making money off my interactions in this economy you can bet your last dollar I want my fair share of it. It's not greedy to gain value from your interactions with the world: you already do by interacting since that's a bi-lateral relationship ie it affects each party equally. It IS greedy for a third party who did not participate in the interaction to expect value from it.
The “tech bros” controlling AI don’t want to destroy us either but they’re not against if it means $$, they’re indifferent.
And the ad money only keeps flowing harder than it did last quarter if they do destroy us, so it’s a pretty easy choice to destroy society for an easy buck.
of course this is a simplified version of real life that doesn’t take foreign influence and stuff like that into account.
To the degree that we've collectively built a dark forest, it's been in part due to this lack of multi-disciplinarianism.
Humans are amazingly adept at rationalization.
If you suspect that you're being lied to by the consensus, feel free to read alternatives by heterodox scholars. I suspected that they're crank.
It's not like I particularly agree with the way things work now, only that we shouldn't make things worse by making obviously stupid moves in the wrong direction.
Because Skinner boxes exist. There are entire multi-billion dollar industries built around them (casinos, gacha games/loot boxes), just to pick a super low-hanging-fruit example.
And very little of psychology covers what is used in ads.
For example, history is frustrating because you are going to end up in unique circumstances and you're not going to be able to see into the future, only the present. Yet, we must at least have an understanding of the world around us and critical thinking skills that can only come from with doing history.
But depressingly few of my peers seem to be true general "artisan scientists" as I see it, understanding engineering to be not only a job but a craft, a discipline, an art form, a power and a responsibility.
So much is possible today. Individuals in developed countries, and even many in underdeveloped countries with access to tech, have more velocity at their fingertips than almost any individual in history.
We've built critical systems which rapidly create value with the slightest gesture. But few really take the time to sit down and connect enough dots to see enough of a bigger picture to make meaningful and considerate modifications to the status quo. Few push back against what they're told, few take the time to deeply appreciate even a fraction of the complex systems which create modern society.
We have to do better about educating ourselves, our peers, and our offspring. We need to encourage generalists, facilitate access to multi-disciplinarian education, and hold ourselves and others to higher ethical standards which are themselves informed by a broad perspective.
This certainly comes at the cost of one's personal time. However, I think a majority of us are overentertained and could be better about conditioning ourselves to routinely seek out and learn new things.
But yeah, people should be curious and willing to try things, even things that doesn't seem to be the thing for them.
I would love to choose to be an artisan, a craftsman, a changer of the world, a multi-discipline Renaissance Man, (or just an over-entertained couch potato) but I need to make my mortgage payment next month.
For the most part, our station in life is something we constantly wage war against from the moment of birth. Coming from a very poor and abusive background, I was homeless by 16 and left with a choice of fully committing to boring technical work, or taking advantage of already being at rock bottom and surgically improving my skills over years until I've become as well-rounded as I would like. It's a lifelong journey, though.
I do wish you luck and I hope you do find time to accomplish some of your desires. You will definitely have to create time and space for it though, free time likely won't magically appear within your current routine.
I'm sorry to rebutt your very first assertion but we had been in a cultural and scientific Renaissance and the last 15 years have been the slow unwinding of that. We got lucky that the internet explosion overlapped with the tail end of publicly supported cultural and scientific production.
I hung out with physics majors in college, all of them smarter than us compsci chuds, and uniformly they are absolutely struggling to survive as post docs or in industry. One of my college buddies has worked at nasa for 4 different firms and has had to move to Texas, Kansas, and Maryland for these gigs and has once again landed on a project where the funding got cut and is looking for a new job. Another works in nano scale semiconducting and had to move to Finland to get project funding from the EU since the US has made basic research funding so scarce. And after several post doc roles he is leaving the field after his last grant wasn't renewed, with not an ounce of negative feedback. Just, sorry we don't have any money anymore. He's now going to go into failure analysis for a mobile phone manufacturer to pay the bills.
The woes of cultural production have also been well documented.
We are in a cultural and scientific collapse
https://academia.stackexchange.com/questions/25910/what-does...
There are plenty of engineers for hire, and sociologists can build their own vision.
This isn't a new concept, but the compartmentalization of subjects in school is one of the factors at play along with a hierarchy of subjects and students. You can observe how a lot of computer science folks hold mathematics and physics in high regard, and biologists, and sociologists in low regard. It's no coincidence that this maps to relative possession of the habitus valued by the ideological hegemony.
Again, it works to reinforce social statuses rather than dismantle them, often in ways that are detrimental to those actively, albeit unknowingly, contributing to the same systems.
Engineers due tend to be pretty critical of things that can’t be accurately and consistently measured (not meant as a slight. It’s true, but it doesn’t make those soft sciences worthless), but they aren’t the only ones making these systems.
Most of the time, engineers are given requirements and build systems to satisfy them.
You wouldn’t blame a construction worker for gentrification, right?
While I agree that disdain is not the right reaction, neither is misdescribing what these disciplines are. And the most important fact about them is: they are not sciences.
We do not have a science of how people work, how groups of people work, how collective action works. Dealing with people is an art. It requires applying non-repeatable human judgments to non-repeatable human situations, and doing so constantly, day after day, year after year, realizing that there is never going to be a stable state of things, there is never going to be a time when you have dealt with all the conflicts and solved all the problems and can sit back and relax.
Engineers are understandably uncomfortable with such a situation because it goes against everything we (I say "we" because I'm an engineer myself) are taught. We are taught to engineer solutions to problems, solutions that keep working, if not forever, at least for a long enough time that we aren't constantly having to manage them. A well built bridge does not require constant engineer attention once it is finished.
But there is no such thing when it comes to humans and human relationships. They do require constant attention in order to survive and flourish. But we have no science of how to do that, the way we have a science of how to build bridges. All we have is our fallible human judgment, and if we're lucky, some rules of thumb that, while they aren't science, are hopefully better than nothing. Unfortunately, I think what rules of thumb we have have been mostly ignored in the process of building systems like social media.
Cite something or don't @ me.
Yes, we do, but their predictive power, if it exists at all, is very poor. Science is about building models with good predictive power. If you don't have that, you don't have a science, at least not in any way that matters.
And there are social science hypotheses that can predict outcomes.
To that degree, studying enough sociology to be able to make the statement, "Compartmentalization and hierarchy of academic subjects contributes to software engineers being unwitting used to replicate social structures," is not dependent on sociology being a science. It is true without relying on an arbitrary epistemology.
Findings don't replicate, data is used to tell lies, policy built on scientific findings is mostly unrelated to the findings, focus is primarily guided by culture instead of discovery or curiosity. I read a lot of sociology, and the general level of critical thinking and concern for the truth in journal articles is much lower than biology, engineering, or even economics.
It's true that the most egregiously wrong prediction in the Federalist papers, that the US would not have political parties because it was too large, was the work of Madison, not Hamilton. But AFAIK none of Hamilton's predictions turned out very well either.
If you’re not thinking about that it just means you have no idea what stuff you are loading into your designs. It’s still there you’re just not aware of it. It’s like being the unwitting carrier of a disease or a parasite.
Makes me think of brain slugs in Futurama for some reason.
Now I say that stem is just a tool that does no good by itself. It's just as harmful as beneficial. An engineer doesn't like to think of themself as a tool, but tough shit. Without the humanities an engineer is exactly that, merely a tool that others wield, and others decide how to wield and to what ends.
Stem tells you how to build a bridge or a bomb. The humanities tells you whether/when/why to build a bridge or a bomb.
Many times these practitioners of these ”soft sciences” are cloistered away in the ivory towers of academia and away from regular people whom they actually look down on because they don’t match their theories.
At some point between, say, 2000 and 2012, everyone forgot that. The Web became an extension of the real world. Now, in 2024, the Web seems to have displaced the real world as the locus of public discourse.
> It names an experience of paranoia and anxiety that by the end of the 2010s was widespread among people with meaningful connections between their online personas and their ability to maintain their standard of living.
This kind of thing always struck me as a poor choice. The great thing about maintaining one's anonymity on the Web, or, to be more precise, adopting a collection of alternate identities, is that they can be discarded easily. A faux pas needn't follow you around, and can't follow you to where you make your living.
> The last and most most dangerous weakness of the Dark Internet Forest as a frame is that it positions the broad landscape of connection as something that “we” can simply do without—and without which we will indeed feel better and be more productive.
> On the level of the individual, this is true for certain values of “we”: for people who are, in any sense, established; people who already have the social status they required to succeed in their field; people whose work doesn’t depend on them needing to find (and re-find and re-find) readers or customers; people whose professional and personal networks are already strong enough to catch them if they slip; people with money.
> So what about everyone else? Should people without those forms of access and capital simply forgo all the benefits afforded by access to broad networks?
Through the twentieth century, as aristocracy gave way to meritocracy, people learnt how to climb the greasy pole without access to these broad networks. Social mobility is largely a consequence of the structure of the economy, and not necessarily a consequence of networking. We are often told that, 'It's not what you know, it's who you know,' but maybe the people who found other ways to the top simply aren't as effective at sharing their secrets as the shmoozers.
> This all stops being an individual problem and becomes a collective one when bad products of the social internet get worse, as when platform turmoil and manipulation helps remodel the offline world in the image of the most grotesque parts of the online one.
Conflict is coming. We've reached a point, I think, we're it's inevitable. I don't think it's clear exactly what the ideas underlying the conflict will be, or what factions will contest it, or in what spaces it will be contested, but there is a range to choose from. The winning move in this situation is to keep informed about events, but otherwise to keep one's head down. Don't challenge ideas you find objectionable, but agree in a mostly-disinterested way. Take it from someone with family members who still have the machete scars.
September.
You could take a screenshot, but not a photo, you couldn't have a real photo inside the computer without a scanner.
You could plug into the phone line if you had dial-up, but you didn't make calls over the Internet. If you did call someone, you had to both install Skype or something.
You could "watch TV" on the computer if you used a cable tuner, but there wasn't TV online.
Reality stayed in reality, and although the Internet has always supervened on reality, the Internet stayed inside the Internet.
Somewhere around "computer phone times" this stopped being the case.
Now politics happens on Twitter, you can order a pizza online and it's not a credit card scam, it actually arrives. You can place a call and with the right aeons it may even connect to a POTS telephone.
At some point it shifted from "The Internet is where a bunch of nerds who like The Internet talk about The Internet" to "The Internet is plugged into real life and is part of real life" to even "Real life is part of the Internet"
The TV reality. It's pretty peaceful when everyone agrees, even when what they agree upon is false.
I feel the "american dream" got that part right. Social mobility works best when you can start working locally to move up. That is, very low barrier to entrepreneurship. "Who you know" stops being a problem then, everyone can work their way into larger and larger networks.
For better or worse, I don't think we can get rid of "who you know" issue, because it's 100% natural. When you start a venture - whether it's a local shop or an Internet company or just remodeling your dog's kennel - you don't go to Global Employee Search Directory and look for potential co-founders; you just do it with a friend. Now, even the largest corporation, when you go all the way up to the top, is still a glorified version of neighbors building a common tool shack. So is almost every subgraph in the corporate ladder, if you zoom in on it.
So the problem, as I see it, is with an economy where most people have to effectively beg strangers for a job - i.e. employees (myself, I am such a person too). In this scenario, "who you know" at the lower levels becomes unfair to the larger whole.
I'm not sure what should be done about it, though.
(This is just some random thought I have, I haven't managed to sort them into a coherent whole just yet, sorry.)
Regular people almost never used those words. Only the media did.
The Dark Forest experience that's causing "mass migrations" on the Internet is that, if you venture out into social media carelessly, you're entering an aggressive and frequently unpleasant space which is close to the opposite of "fun" for the average person. Here follows a laundry list here of things which you might expect to encounter in an unpleasant online space, most of which have a kind of tabloid quality: scams, nonstop sales pitches, porn, extreme violence, celebrity worship, ideological extremism, racism, sexism, etc. And more (that list is not exhaustive).
So people find that signing up for the average generic platform that welcomes everybody is a bad experience, and migrate to more niche ones. That's been the path I've followed too, all the way here.
I think I must be confused.
So yes I think you're confused.