You must log in or register to comment.
Self-hosted LLMs are neat and private, if you (already) have the hardware/budget and some cheap (solar PV) surplus juice to drive it.
Do you know anyone doing all of that? Running a local model on off-grid power?
Someone mentioned that, but it doesn’t have to be off-grid. My 2 kWp system is grid-tied and if I would start running a local LLM it would be an incentive to add another couple.
It may sound odd but I’ve heard people bring up the above point a few times. “AI isn’t bad - if you run it on a sustainable off-grid system it’s perfectly fine.”
But I have NOT encountered people actually doing this even once, let alone a few times. That’s why I was asking.
Anything that wastes energy can be recast as harmless if you put it on an imaginary sustainable setup but that just seems like fantasizing.
Renewable infra isn’t sustainable, since it can’t be built with renewable energy alone. As such it’s just a fossil multiplier. I make about 2/3rd of my own net power, since my electricity meter is old enough to run backwards. I can’t install more orelse my meter would read negative. I intend to run a Qwen model that fits my 8 GB HBM Radeon card as an experiment during the day, starting end of this month.
If this works I consider running something hotter, and double my PV capacity. Adding AC or an EV would be another option. None of this is sustainable, but not exactly a fantasy.
What would you like to know about it?
I glossed over it the other day here -
https://aussie.zone/comment/24336560
From go to woah (including the compute), total project cost is about $1400 USD / ~$2K AUD. I realise I have the privilege of claiming that’s not a lot, but for a hobby spend that makes me happy / does useful stuff (see TV example above), it’s not a lot.
It just sounded like the kind of thing that makes good sense on paper but I’m wondering if it actually works out well. Sounds like you are in the process of getting ready to try it.
I am indeed. With the amount of solar here - and the relatively low cost for this sort of mini DIY set up - it’s something I’ve been meaning to do for a while.
oh so yours is trained on ethical sources? you compensated the creators for their work and did so with their consent to machine training?
What the fuck are ethical sources? Where is compensation and consent when I unapologetically use pirate archives of books and papers? You people crack me up.
the fact that you have no idea what I’m talking about either means you’re full of shit, or cartoonishly stupid and evil.
Probably both
The fact that you immediately descend into ad hominems indicates that you’re a just another troll that should be posting from hexbear.
you’re simply used to llm sycophancy, not talking to actual people.
you really not understanding training on sources compensated for their content is hilariously stupid
You’re not understanding what standing on the shoulders of giants means. Locally hosted open weights models are free/libre and puts you in control of your data. Since we’re talking about hilariously stupid.
aw poor baby got schooled and cried to the mods lol
This is not a paywall, but it can be bypassed if it were a paywall.
Lots of folks asked for AI, but not like this!
Nobody has provided ‘AI’, at least if those characters are intended to be an initialisation of Artificial Intelligence.
Whenever anyone is impressed by the function or output of a LLM to the level they’re convinced it’s a representation of intelligence, their input and opinion of the targeted subject should be summarily dismissed, and if they persist, ridiculed.
Are you sure? The rich have always wanted slaves they could control. All of this is mainly to end the dependence in the masses to create society. I highly suspect a coordinated depopulation event if AI becomes sufficient enough.
I highly suspect a coordinated depopulation event if AI becomes sufficient enough
Inb4 someone mentions climate change is already well on its way…
The timing is suspiciously perfect that humanity is committing global suicide at the exact same moment it’s creating machine life not dependent on the environment.
Just look at what happened to the global horse population after the automobile was invented
I remember after the rise of social media, the world was always going towards decentralized finance, blockchain and many other things. Hell even Meta burnt massive cash in metaverse which supported cryptos. But after Covid, the whole AI episode started playing out of nowhere, and even now, the whole push around it just feels unnatural, almost as if staged. Today’s so called AI success is 30% based purely on merit and 70% based on coordinated media management and unsoliciated offerings of desperate investment.
The innovations in AI/ML were always part of the modular infrastructure, specific to the enterprise level products and not su much for the retail users.
I think the idea of decentralized finance was to normalize company scrip again.
Now that AI can be used to track and geofence workers, you don’t actually need to get rid of the USD. You just need to have an AI flag every transaction that happens 50 miles outside your registered address to be considered fraudulent and rejected. The anti-fraud infrastructure is already there.
I think the idea of decentralized finance was to normalize company scrip again.
It also makes regulatory scrutiny even harder, enabling many traditional methods of fuckery which have been made illegal in other contexts.
The whole idea behind defi isn’t just minimizing frauds; it’s literally to decentralize the monetory system and giving the control in the hands of people. The last thing people need is another layer of data collection on top of it.
AI has been around a lot longer than most people realize. Grouping all AI into whatever cloud garbage silicon valley tech bros are selling makes them look like Luddites.
Language is constantly evolving and the meaning of certain terms shifts between eras. Right now everyone understands “AI” to mean the LLM slop factories being forced on the world by a trillion dollars in investment cash. If you want to be upset about that, be upset at Silicon Valley for spending so much of that VC money on marketing to corrupt the public’s understanding.
Also, “Luddite” isn’t the insult you think it is. The luddites were a pro-labor movement, not an anti-technology movement, and many of the labor protections we have today (despite being under attack by big tech) are owed to their successes.
“A Luddite is someone who opposes new technology, computers, or automation. The word comes from 19th-century English textile workers who smashed factory machines that threatened their jobs, though today it is often used as a general label for anyone who dislikes or resists tech.”
Interesting lecture about meaning of words there bro.
Your definition of the Luddites is strong on who they were and what they did, but weak on their motivations.
While this definition isn’t necessarily wrong, it [purposefully] leaves out important context, namely that the so-called Luddites opposed technology and automation due to cuts to their incomes and rights, which were very obviously coming.
If industrial corporations guaranteed to keep all employees, to not lower their salaries, and to maintain the quality standards - the people we know today as Luddites would have never protested.
It was never about technology itself, despite what top Google results or AI summary will tell you. It was about people’s livelihoods, rights and freedoms, as well as quality of the craft.
Yeah, because the ownership class fought tooth-and-nail to corrupt the public’s understanding of the name, just like they’re doing now with ‘AI’. You can’t be mad at the bastardization of one term without being mad at the other when the same forces lead to both.
So: your point is the usage of AI in common vernacular should not have changed from its meaning 50 years ago, and using it in its modern sense as shorthand for LLMs and generative models is bad.
And your point is also that the modern usage of a Luddite is correct and you should ignore the original group and the conditions that led this group to smash the textiles because the usage of that word changing is fine and we should not look at original usage?
Got it. Words change. Sometimes. And other times no.
But… But… Will nobody think of the stakeholders?
In order for AI to have a market there would need to be a large scale demonstration of the uses and benefits of AI in many various industries, most of which do not involve image generation or writing a single line of computer code.
I’m so sick of asking people for demonstrations of AI use and it being 100% programmers going “oh it’s really good at writing functions”, now I have no idea if it is, or isn’t, any good at writing functions, but since I don’t write functions, that example is of no use to me.
I want to know how good it is at writing technical documentation for proprietary hardware solutions that the AI will have no experience of in its training set, because I suspect it sucks at it. How good is it at taking poorly written guides in Chinese, translated into English via Klingon and ancient Babylonian, how does it handle it if it comes across the sentence “absolutely do not ever warning” with no additional information as to what I absolutely should never not do. Because that’s when thinking is involved, and AI doesn’t think.
OK, here’s a real life use case, small time as it is, that has nothing to do with anything rarefied.
I have a TCL Android TV that, after some “unapproved measures”, runs without a Google account - FLauncher as the front end, telemetry blocked, home media streaming, YouTube ad-free. The yar har har works.
Two days ago the kids come in. TV’s not working.
Jellyfin: fine. Nova Player: fine. Stremio, YouTube, anything hitting the net: dead.
We’d had a brownout that day, so I figured the router was scrambled. Reset it, re-added the network… and Smarttube now launches, plays for 30 seconds, then hard-locks. Just weird JSON vomit and then dead.
At that point I’m thinking the TV is fucked. Options:
-
factory reset and reinstall the workarounds - 2-3hrs of pain-in-the-ass fuckery, but it wouldn’t be the first time.
-
crawl the net decoding cryptic error codes.
-
take it in for repair (good luck with that).
-
fall down a TV diagnostics rabbit hole.
Instead, I chose option 5)
I told my local LLM: "This TV is on this IP. This laptop has ADB access. I’ve given you MCP tools with ADB libraries. Here’s the issue, here’s what I tried. Check prior history of what I did for this project, figure out what’s wrong now and fix it, or tell me it’s broken.
If it looks like the brownout caused corruption, factory reset, stream-install the backup APKs directly, sign in my accounts, get it back up and running.
Failing that, if it’s looking like a hardware issue, push a diagnostic tool, test the TV and tell me which parts I might need, as well as likely costs using eBay or Amazon. I’m happy to spend $50 on this shitbox and not one cent more. If the fix looks like it’s more than $50, find me a TV of this size under $200. Prefer local shop X or Y.
Present everything as markdown in the usual folder, using house style. Ping me on my phone if you need me to authorise anything or when you’re done with the TL;DR."
The agent ADBs in, pokes around, tests, checks the WiFi, and figures out the brownout had corrupted the date and time.
Turns out this p.o.s doesn’t have a RTC back up battery.
Wrong clock = certs flagged as invalid = no net.
It syncs the date and time with my location. Boom, everything comes back.
Now, could I have twigged it myself?
Yes - eventually. But the agent already had the tools, knew the TV’s prior state, and had a record of what I’d already tried.
It could’ve reset the TV, sideloaded all my apps, re-signed into my accounts - the whole lot - if needed.
This is not the only time I’ve had real world use of my local, and it had very little to do with “writing functions”.
I had a project - jailbreak my tcl. Over time, I’d stored enough context of what I had done in the LLM and its tools, so that it could do what I’d do (and more), faster, based on the constraints I had put in place.
People that call these things “spicy autocorrect” are demonstrating either a skill issue or a lack of imagination.
You managed to set up that tech stack but needed an agent with unrestricted access to figure out the time was wrong 😂
There are skill issues and a lack of imagination here but it’s not from people dismissing the use of LLM…
Using a sledgehammer to crack a wallnut can be satisfying :)
Besides, the kids are always finding fun and inventive ways to break the TV, so I figured set it up once and then let the bot handle it.
Twelfth Doctor: This is Clara, not my assistant. She’s a… Some other word.
Clara: I’m his carer.
Twelfth Doctor: Yeah, my carer. She cares so I don’t have to.
I get doing things because you can, that doesn’t mean it’s not wholly unsuited to the task and massive over kill that you only use because either 1. It’s self hosted so you are already paying for it or 2. It’s super cheap. You likely wouldn’t entertain such a set up if you were paying the true price for training and running the models.
Actually, I’ve been toying around with doing just that.
https://github.com/karpathy/nanochat
Given the surplus of power from solar I have, my set up (discussed elsewhere), I can run the training process end to end over about 50 days. It could be fun to make something truly bespoke from scratch.
Returning to my prior example - it was only overkill for this task (and only because the problem turned out to be relatively simple), but I wanted to answer against the claim that people were using LLMs and agents for just programming.
The example I cited IS silly, but it happened - IRL - 3 days ago. I can cite several much more “serious” use cases if you’d like.
On the broader topic of costs: none of us can really comprehend the embedded costs in the things we use everyday.
However unlike (say) the recurring embedded costs of buying bottled water, a cup of coffee or streaming a movie, the costs of my set up are paid once and pay off multiple times.
Ultimately, I get why there’s a FuckAI sentiment… but really we should expand that to “Fuck Corpo AI”. My feeling is that it’s not AI that people are against per se, it’s the corpos and what they’re doing with them. Too much of that gets flattened and tribal.
The “true cost” argument only holds if you assume the only valid infrastructure is someone else’s. I don’t make that assumption, and I’d say that neither do others in the enthusiast community.
-
Basically the general AI chat interaction gives a pretty good indication of capabilities and limitations as it pertains to various applications. It sounds right, but frequently is not right, and that has predictable implications.
Particularly it is really good at generating credible sounding content. It does not have a model for ‘facts’ so correctness is a persistent issue. For coding, the strategies involve having some sort of absolute validation to say whether it is “correct” or not, whether that’s some human reading it, or in the die-hard scenarios, someone writing a whole system to let it try, get pass/fail, try again on fail, and spin until it is done or you stop it because it just spins and spins. The former can be selectively useful, the latter I find to be more obnoxious than just doing the work.
So broadly, scenarios in which the “truth” is objective and/or in the input directly is where it can provide useful results. The examples of translation are an area where it is more helpful, since it isn’t tasked with adding any ‘facts’ merely reworking the input into another language. “Absolute do not ever warning” type stuff is the result of the human trying to write in non-native, so they would feed their native work into LLM and probably get better result.
The other super depressing thing is slop fiction and deep fakes. Here “facts” don’t matter, as long as it seems consistent. Even as it adds elements that may be ‘wrong’, well, it’s fiction so wrong doesn’t matter as much so long as it seems narratively consistent. Enabling people lacking in drive and creativity to pump out as much volume as people with a lot of drive and creativity with more interesting things to say.
Of course, the bubble is predicated on it working to replace most white collar work, which it is not poised to do. There’s a lot it can do, but it’s penchant for generating incorrect output is a liability that can’t be afforded in a lot of contexts. A lot of places just put the onus on the victim of the AI support bot by saying up front “you can’t trust anything this bot is going to say, but you are going to be getting the bot anyway”. Even as human support might be asked to intervene, the human is still likely to be able to review the chat quickly and maybe take a minute to see what the user wanted, click some button in the system the customer needed clicked, and move on, instead of 10 minutes of back and forth to figure out what the user needs. So they could still lay off 75% of those support people and, thanks to AI, still maintain apparent responsiveness, even if the users are annoyed. Lots of orgs have shifted to have sales be human because a customer gets pissed, but support from AI chatbot, because, well, you got their money, screw em, they can deal with some annoyance.
It’s quite terrible at writing technical documentation of even systems of pure code it’s coded itself , it’s been all down hill since opus 4.6 for writing in any kind of coherent English to convey information .
The thing is so full of slogans, irrelevant counting , and meta commentary announcing what it’s going to say moronically, and weird AF corporate sleazeball speak, now it can rattle on for pages saying mere sentences worth of information .
I’ve avoided it entirely but I was trying to look for forum threads to help me get the right console commands to fix Skyrim glitches. At first I didn’t realize I was getting AI results, but at some point I realized not only was I using AI, it was a MASTER at fixing Skyrim glitches.
No, it’s just regurgitating stuff other people came up with… A game that came out recently won’t be half as fixable because people won’t have written so much about it… And AI will cause less useful information to be created about it.
It can only know what’s already been written about.
It can and will actually extrapolate from what it knows to create something new. In the case of a new game, it will give mixed result as what it gives is just something plausible, but will make it sound a fact.
It can only know what’s already been written about Just like a human, isn’t it?
I don’t even think anyone is clear on what they are supposedly working toward, which appears to be some kind of singularity AI god a la Hyperion’s TechnoCore… this shit is feeling more like Christiantiy every day.
Better take than their last one on AI
I would say a lot of people would like competent assistants to do stuff like “Show me every email that mentions a hard drive order.” But not the AI they are trying to foist on us now.
Or read disclosure for my members so I don’t have to. But that it fucking things are fucking incompetent. Work insurance people will get line pissed that got me instead of banking. They will say I told the AI I needed to order checks. Yet it sends them to insurance division instead of banking.
Sucks because most customers hate AI answering the phone I especially hate it. It fucking dumb and I just want to talk to person.
But even job hunting had two prospective jobs tried to use AI to conduct the interview. They fucking outsourcing that. I refused both interviews.
I just want to talk to person.
Careful what you are wishing for, person may be on other side of the world answering your call at 4am local time with most polite scripts which you could be reading yourself from website better.
I don’t know how people do that job, I did it for week and couldn’t stand it.
Talking to braindead drones suck, but at least someone is being employed. I’m dropping a doctor because their AI answering machine won’t let me talk to a human.
so people were asking for linux for decades
find "$path" -type f -print0 | xargs -0 grep -e "$what_to_look_for" /dev/nullYeah that, but with natural language input and output. To me the (mostly) natural language is the most impressive part of generative AI.
Though with the ability to not even have to think about synonyms and indirect references or foreign language or potentially if someone sent you a damn screenshot of a terminal output instead of the text…
What drives me up a fucking wall these days is how people will use AI to make a more “professional” message, then due to it’s verbosity it needs an AI to summarize on the receiving end.
I can tell when people use AI because it doesn’t sound professional. It’s not direct enough. It doesn’t get to the point and uses a lot of filler.
It sounds like how students think professional sounds.
Well, also some executives that I can’t stand that, if I didn’t know better, would swear were ChatGPT for decades…
My attitude is if you put no effort into creating something then I will put no effort into consuming it.
Yet when I tell that to our marketing department they just complain about how it would be expensive to pay someone to do it themselves. You get what you pay for, except in this instance where ai is expensive and still bad.
My AI will talk to your AI.
Especially since when you ask it those questions, it’s still massively limited.
I once tried Apple’s replacement AI Siri thing, and asked it what the date would be next Tuesday. I got “I can’t help with that”.
Regular old Siri handled that just fine.
Even then, neither worked with “When’s the last occurrence of this event on my calendar?”. They’d either go “I’m sorry, I don’t understand”, or happily tell me when the next event was.
Neither of them seem like very difficult questions.
compared to many LLMs, I’d prefer an application default to “I don’t know / can’t do that” than the absurdly frequent lying that’s prevalent.
nobody asked for automobiles
This likely refers to the famous fictitious quote from Heny Ford that wanna-be self-important “innovators” like to trot out to show how they are smarter than the customer when they put out some unasked for bullshit.
"If I had asked people what they wanted, they would have said faster horses.”
People like to imagine the car as some miraculous thing no one but a genius could have imagined wanting and the genius descending from on high and delivering. Truth was people could easily imagine and want it, the genius was in knowing how to make what people knew they wanted.
Instead, Henry Ford said: "If there is any one secret of success, it lies in the ability to get the other person’s point of view and see things from that person’s angle as well as from your own”
So essentially the exact opposite of “screw the customers, we know better!”
Yeah, nobody asked for fire, either.
your life is made immeasurably easier by both technologies
We assume that the AI boom is the result of market demand. In reality, it is the result of investment decisions made by concentrated capital, which is reshaping the economy on its own terms.
But that’s always been true under capitalism. It’s just that in the past generally large capital expenditures have gone towards things the wealthy think will get them more wealth. There have always been capitalists who spend their money directing the economy to do what they want, rather than what the “market” wants. They just haven’t had the majority of the capital being spent.
The last 50 years has seen an unprecedented concentration of wealth. The wealthy are literally a thousand times proportionately wealthier than poor people than they’ve been at any other point in human history. There’s not much wealth to squeeze out of everyone else, so the economy is even more at the vain whims of those with all the capital as they look for things to spend it on.
Tech news has been reporting for literal decades that Apple, Amazon, Google, and similar have been sitting on giant war chests of profit with nothing to spend it on. Well, they’ve found something they want to spend it on, so they’re emptying 20 years of profits in 5. They’ve now completely blown through those stores, and don’t want to slow down, so now they’re taking on unprecedented levels of debt. Something is going to give out and things are going to get bad within the next 5 years.
It’s still market demand, it’s just not the market you think it is. It’s not for some end product or service, it’s for speculative investment opportunities. It’s a few levels of abstraction above actually making or doing things.
they’ve found something they want to spend it on, so they’re emptying 20 years of profits in 5.
That’s what .com did as well.
reporting for literal decades that Apple, Amazon, Google, and similar have been sitting on giant war chests of profit with nothing to spend it on.
I remember when a “shockingly large” war chest was $30B - that’s like table stakes anymore.
