The privacy mess is troubling because the chatbots actively encourage you to share details that are far more personal than in a typical app.
  • exu@feditown.comEnglish
    0·
    9 months ago

    You’ll want to use a quantised model on your GPU. You could also use the CPU and offload some parts to the GPU with llama.cpp (an option in oobabooga). Llama.cpp models are in the GGUF format.