Also check out LLM Studio and GPT4all. Both of these let you run private ChatGPT alternatives from Hugging Face and run them off your ram and processor (can also offload to GPU).
I can’t find a way to run any of these on my homeserver and access it over http. It looks like it is possible but you need a gui to install it in the first place.
ssh -X
The x is “x11 forwarding” - don’t know about you but I don’t run an X on a headless server. Plus a GUI install is not exactly the best for reproducability which at least I aim for with my server infrastructure.
You don’t need to run an X server on the headless server. As long as the libraries are compiled in to the client software (the GUI app), it will work. No GUI would need to be installed on the headless server, and the libraries are present in any common Linux distro already (and support would be compiled into a GUI-only app unless it was Wayland-only).
I agree that a GUI-only installer is a bad thing, but the parent was saying they didn’t know how it could be done. “ssh -X” (or -Y) is how.
That’s a huge today-I-learned for me, thank you! I took ill throw xeyes on it just to use ssh - C for the first time in my life. I actually assumed wrong.
I’ll edit my post accordingly!
Something i am really missing is a breakdown of How good these models actually are compared to eachother.
A demo on hugging face couldnt tell me the boiling point of water while the authors own example prompt asked the boiling point for some chemical.
Are they as good as chatgpt?
Mistral is thought to be almost as good. I’ve used the latest version of mistral and found it more or less identical in quality of output.
It’s not as fast though as I am running it off of 16gb of ram and an old GTX 1060 card.
I’m sure that meth is for personal use right? Right?
Absolutely. Synthesizing hard drugs is time consuming and a lot of hard work. Only I get to enjoy it.
No one gets my mushrooms either ;)
brf tek says hi
I just buy my substrate online. I’m far less experimental than most. I just want it to work in a consistent way that yields an amount I can predict.
What I really want to grow is Peyote or San Pedro, but the slow growth and lack of sun in my location would make that difficult.
Whazzat? I’ve only met uncle Ben!
Can you provide links for those? I see a few and don’t trust search results
You can search inside LM studio for uncensored or roleplay. Select the size you want then it’s all good from there.
They’re the first results on all major search engines.
No.
Is it as good as chatgpt?
The question is quickly answered as none is currently that good, open or not.
Anyway it seems that this is just a manager. I see some competitors available that I have heard good things about, like mistral.
Local LLMs can beat GPT 3.5 now.
Open source good, together monkey strong 💪🏻
Build cool village with other frens, make new things, celebrate as village
