I use the Alpaca flatpak, it just lets you download a variety of models, manages them all inside a contained local environment.
Even has some tools support that is expanding, basic web searches, speech to text, text to speech… and if you can find a GGUF format model, supposedly Alpaca can run this manually, and there’s a good deal on huggingface.
Unfortunately, if you’re running Windows, I… have no clue how to set up an LLM there.
Also your tin foil hat thing isn’t even tin foil hat.
Like, various people in the AI space have outright stated that they want to see a paradigm where everyone just rents compute time from them because PCs are othereise too expensive, while acting like it just happens to be the new reality that everything is so expensive, for some reason.
Nvidia went from gaming GPUs being about 50% of its business to something more like 5%, in about 5 years.
Fortunately the AI bubble will be popping soon, as … everyone has run out of money to lend.
Unfrotunately this will destroy the economies of the West.
For the moment, I haven’t had the motivation to switch everything over to Linux, but it is coming down the line. To that end, I do know how to set up models and windows, and it’s not all that hard, but what is the specific model name? Is it just the Quen 8b?
Come to think of it, I might actually be able to install the flat pack into the Windows subsystem for Linux if it behaves the way I think it’s supposed to.
I use the Alpaca flatpak, it just lets you download a variety of models, manages them all inside a contained local environment.
Even has some tools support that is expanding, basic web searches, speech to text, text to speech… and if you can find a GGUF format model, supposedly Alpaca can run this manually, and there’s a good deal on huggingface.
https://github.com/Jeffser/Alpaca
Unfortunately, if you’re running Windows, I… have no clue how to set up an LLM there.
Also your tin foil hat thing isn’t even tin foil hat.
Like, various people in the AI space have outright stated that they want to see a paradigm where everyone just rents compute time from them because PCs are othereise too expensive, while acting like it just happens to be the new reality that everything is so expensive, for some reason.
Nvidia went from gaming GPUs being about 50% of its business to something more like 5%, in about 5 years.
Fortunately the AI bubble will be popping soon, as … everyone has run out of money to lend.
Unfrotunately this will destroy the economies of the West.
Yay capitalism!
For the moment, I haven’t had the motivation to switch everything over to Linux, but it is coming down the line. To that end, I do know how to set up models and windows, and it’s not all that hard, but what is the specific model name? Is it just the Quen 8b?
Come to think of it, I might actually be able to install the flat pack into the Windows subsystem for Linux if it behaves the way I think it’s supposed to.
Could be a very interesting experiment.