- cross-posted to:
- enoughmuskspam@lemmy.world
- cross-posted to:
- enoughmuskspam@lemmy.world
We will use Grok 3.5 (maybe we should call it 4), which has advanced reasoning, to rewrite the entire corpus of human knowledge, adding missing information and deleting errors.
Then retrain on that.
Far too much garbage in any foundation model trained on uncorrected data.
You must log in or register to comment.
I’m just seeing bakes in the lies.
Unironically Orwellian
I remember when I learned what corpus meant too
The thing that annoys me most is that there have been studies done on LLMs where, when trained on subsets of output, it produces increasingly noisier output.
Sources (unordered):
- What is model collapse?
- AI models collapse when trained on recursively generated data
- Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
- Collapse of Self-trained Language Models
Whatever nonsense Muskrat is spewing, it is factually incorrect. He won’t be able to successfully retrain any model on generated content. At least, not an LLM if he wants a successful product. If anything, he will be producing a model that is heavily trained on censored datasets.
It’s not so simple, there are papers on zero data ‘self play’ or other schemes for using other LLM’s output.
Distillation is probably the only one you’d want for a pretrain, specifically.
That’s a hell of way to say that he wants to rewrite history.
The plan to “rewrite the entire corpus of human knowledge” with AI sounds impressive until you realize LLMs are just pattern-matching systems that remix existing text. They can’t create genuinely new knowledge or identify “missing information” that wasn’t already in their training data.
Remember the “white genocide in South Africa” nonsense? That kind of rewriting of history.
It’s not the LLM doing that though. It’s the people feeding it information
Yes.
He wants to prompt grok to rewrite history according to his worldview, then retrain the model on that output.
Try rereading the whole tweet, it’s not very long. It’s specifically saying that they plan to “correct” the dataset using Grok, then retrain with that dataset.
It would be way too expensive to go through it by hand
Literally what Elon is talking about doing…
Generally, yes. However, there have been some incredible (borderline “magic”) emergent generalization capabilities that I don’t think anyone was expecting.
Modern AI is more than just “pattern matching” at this point. Yes at the lowest levels, sure that’s what it’s doing, but then you could also say human brains are just pattern matching at that same low level.
Nothing that has been demonstrated makes me think these chatbots should be allowed to rewrite human history what the fuck?!
That’s not what I said. It’s absolutely dystopian how Musk is trying to tailor his own reality.
What I did say (and I’ve been doing AI research since the AlexNet days…) is that LLMs aren’t old school ML systems, and we’re at the point that simply scaling up to insane levels has yielded results that no one expected, but it was the lowest hanging fruit at the time. Few shot learning -> novel space generalization is very hard, so the easiest method was just take what is currently done and make it bigger (a la ResNet back in the day).
Lemmy is almost as bad as reddit when it comes to hiveminds.
You literally called it borderline magic.
Don’t do that? They’re pattern recognition engines, they can produce some neat results and are good for niche tasks and interesting as toys, but they really aren’t that impressive. This “borderline magic” line is why they’re trying to shove these chatbots into literally everything, even though they aren’t good at most tasks.
Tech bros see zero value in humanity beyond how it can be commodified.
What he means is correct the model so all it models is racism and far-right nonsense.
To be fair, your brain is a pattern-matching system.
When you catch a ball, you’re not doing the physics calculations in your head- you’re making predictions based on an enormous quantity of input. Unless you’re being very deliberate, you’re not thinking before you speak every word- your brain’s predictive processing takes over and you often literally speak before you think.
Fuck LLMs- but I think it’s a bit wild to dismiss the power of a sufficiently advanced pattern-matching system.
I said literally this in my reply, and the lemmy hivemind downvoted me. Beware of sharing information here I guess.
But Grok 3.5/4 has Advanced Reasoning
Surprised he didn’t name it Giga Reasoning or some other dumb shit.
Gigachad Reasoning
Faek news!
What a dickbag. I’ll never forgive him for bastardizing one of my favorite works of fiction (Stranger in a Strange Land)
Humm…this doesn’t sound great
I wonder how many papers he’s read since ChatGPT released about how bad it is to train AI on AI output.
First error to correct:
We will use Grok 3.5 (maybe we should call it 4), which has advanced reasoning, to rewrite the entire corpus of human knowledge, adding missing
informationerrors and deletingerrorsinformation.We have never been at war with Eurasia. We have always been at war with East Asia
Elon should seriously see a medical professional.
He should be locked up in a mental institute. Indefinitely.
I’ll have you know he’s seeing a medical professional at least once a day. Sometimes multiple times!!!
(On an absolutely and completely unrelated note ketamine dealers are medical professionals, yeah?)
How high on ketamine is he?
3.5 (maybe we should call it 4)
I think calling it 3.5 might already be too optimistic
Whatever. The next generation will have to learn to trust whether the material is true or not by using sources like Wikipedia or books by well-regarded authors.
The other thing that he doesn’t understand (and most “AI” advocates don’t either) is that LLMs have nothing to do with facts or information. They’re just probabilistic models that pick the next word(s) based on context. Anyone trying to address the facts and information produced by these models is completely missing the point.
Wikipedia is not a trustworthy source of information for anything regarding contemporary politics or economics.
So what would you consider to be a trustworthy source?
Wikipedia is not a trustworthy source of information for anything regarding contemporary politics or economics.
Wikipedia presents the views of reliable sources on notable topics. The trick is what sources are considered “reliable” and what topics are “notable”, which is why it’s such a poor source of information for things like contemporary politics in particular.
Wikipedia presents the views of reliable sources on notable topics
Absolutely nowhere near. This is why America is fucked.
Again, read the rest of the comment. Wikipedia very much repeats the views of reliable sources on notable topics - most of the fuckery is in deciding what counts as “reliable” and “notable”.
And again. Read my reply. I refuted this idiotic. take.
You allowed yourselves to be dumbed down to this point.
You had started to make a point, now you are just being a dick.
No. You calling me a ‘dick’ negates any point you might have had. In fact you had none. This is a personal attack.
A bit more than fifteen years ago I was burned out in my very successful creative career, and decided to try and learn about how the world worked.
I noticed opposing headlines generated from the same studies (published in whichever academic journal) and realised I could only go to the source: the actual studies themselves. This is in the fields of climate change, global energy production, and biospheric degradation. The scientific method is much degraded but there is still some substance to it. Wikipedia no chance at all. Academic papers take a bit of getting used to but coping with them is a skill that most people can learn in fairly short order. Start with the abstract, then conclusion if the abstract is interesting. Don’t worry about the maths, plenty of people will look at that, and go from there.
I also read all of the major works on Western beliefs on economics, from the Physiocrats (Quesnay) to modern monetary theory. Read books, not websites/a website edited by who knows which government agencies and one guy who edited a third of it. It is simple: the cost of production still usually means more effort, so higher quality, provided you are somewhat discerning of the books you buy.
This should not even be up for debate. The fact it is does go some way to explain why the US is so fucked.
___Books are not immune to being written by LLMs spewing nonsense, lies, and hallucinations, which will only make more traditional issue of author/publisher biases worse. The asymmetry between how long it takes to create misinformation and how long it takes to verify it has never been this bad.
Media literacy will be very important going forward for new informational material and there will be increasing demand for pre-LLM materials.
Yes I know books are not immune to llm’s. The classics are all already written - I would suggest peple start with them.
Wikipedia gives lists of their sources, judge what you read based off of that. Or just skip to the sources and read them instead.
Just because Wikipedia offers a list of references doesn’t mean that those references reflect what knowledge is actually out there. Wikipedia is trying to be academically rigorous without any of the real work. A big part of doing academic research is reading articles and studies that are wrong or which prove the null hypothesis. That’s why we need experts and not just an AI to regurgitate information. Wikipedia is useful if people understand it’s limitations, I think a lot of people don’t though.
For sure, Wikipedia is for the most basic subjects to research, or the first step of doing any research (they could still offer helpful sources) . For basic stuff, or quick glances of something for conversation.
This very much depends on the subject, I suspect. For math or computer science, wikipedia is an excellent source, and the credentials of the editors maintaining those areas are formidable (to say the least). Their explanations of the underlaying mechanisms are in my experience a little variable in quality, but I haven’t found one that’s even close to outright wrong.
Yeah because 1. obviously this is what everybody does. And 2. Just because sources are provided does not mean they are in any way balanced.
The fact that you would consider this sort of response acceptable justification of wikipedia might indicate just how weak wikipedia is.
The other thing that he doesn’t understand (and most “AI” advocates don’t either) is that LLMs have nothing to do with facts or information. They’re just probabilistic models that pick the next word(s) based on context.
That’s a massive oversimplification, it’s like saying humans don’t remember things, we just have neurons that fire based on context
LLMs do actually “know” things. They work based on tokens and weights, which are the nodes and edges of a high dimensional graph. The llm traverses this graph as it processes inputs and generates new tokens
You can do brain surgery on an llm and change what it knows, we have a very good understanding of how this works. You can change a single link and the model will believe the Eiffel tower is in Rome, and it’ll describe how you have a great view of the colosseum from the top
The problem is that it’s very complicated and complex, researchers are currently developing new math to let us do this in a useful way
Thinking wikipedia or other unbiased sources will still be available in a decade or so is wishful thinking. Once the digital stranglehold kicks in, it’ll be mandatory sign-in with gov vetted identity provider and your sources will be limited to what that gov allows you to see. MMW.
Wikipedia is quite resilient - you can even put it on a USB drive. As long as you have a free operating system, there will always be ways to access it.
I keep a partial local copy of Wikipedia on my phone and backup device with an app called Kiwix. Great if you need access to certain items in remote areas with no access to the internet.
They may laugh now, but you’re gonna kick ass when you get isekai’d.
Yes. There will be no websites only AI and apps. You will be automatically logged in to the apps. Linux, Lemmy will be baned. We will be classed as hackers and criminals. We probably have to build our own mesh network for communication or access it from a secret location.
Spoiler: He’s gonna fix the “missing” information with MISinformation.
She sounds Hot
She’s unfortunately can’t see you because of financial difficulties. You gotta give her money like I do. One day, I will see her in person.
