As you may know, ChatGPT collects a lot of data on the users for the improvement of their AI, but this poses risks in its own way. I was wondering whether there are privacy alternatives to ChatGPT. Perhaps on F-Droid or Aurora/PlayStore, or for Linux.
Are there any alternatives you know of? Or are there other ways to interact with ChatGPT without giving personal information, such as a privacy focussed front-end?
The best option is to run them models locally. You’ll need a good enough GPU - I have an RTX 3060 with 12 GB of VRAM, which is enough to do a lot of local AI work.
I use Ollama, and my favourite model to use with it is Mistral-7b-Instruct. It’s a 7 billion parameter model optimised for instruction following, but usable with 4 bit quantisation, so the model takes about 4 GB of storage.
You can run it from the command line rather than a web interface - run the container for the server, and then something like
docker exec -it ollama ollama run mistral, giving a command line interface. The model performs pretty well; not quite as well on some tasks as GPT-4, but also not brain-damaged from attempts to censor it.By default it keeps a local history, but you can turn that off.
Is that 12GB RAM just for the model or total?
I think the GPU has 12GB of physical ram.