Pro@programming.dev to Technology@lemmy.worldEnglish · 6 days agoGoogle quietly released an app that lets you download and run AI models locallygithub.comexternal-linkmessage-square44fedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkGoogle quietly released an app that lets you download and run AI models locallygithub.comPro@programming.dev to Technology@lemmy.worldEnglish · 6 days agomessage-square44fedilink
minus-squaregens@programming.devlinkfedilinkEnglisharrow-up0·5 days agoLlama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.
minus-squareDiplomjodler@lemmy.worldlinkfedilinkEnglisharrow-up0·5 days agoIs there any useful model you can run on a phone?
minus-squareGreg Clarke@lemmy.calinkfedilinkEnglisharrow-up0·6 days agoHas this actually been done? If so, I assume it would only be able to use the CPU
minus-squareEuphoma@lemmy.mllinkfedilinkEnglisharrow-up0·6 days agoYeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk
minus-squareAmbiguousProps@lemmy.todaylinkfedilinkEnglisharrow-up0·6 days agoThat’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.
minus-squareOhVenus_Baby@lemmy.mllinkfedilinkEnglisharrow-up0·5 days agoHow is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.
minus-squareGreg Clarke@lemmy.calinkfedilinkEnglisharrow-up0·6 days agoYes, that’s my setup. But this will be useful for cases where internet connection is not reliable
Ollama can’t run on Android
Llama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.
Try PocketPal instead
Is there any useful model you can run on a phone?
You can use it in termux
Has this actually been done? If so, I assume it would only be able to use the CPU
Yeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk
That’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.
How is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.
Yes, that’s my setup. But this will be useful for cases where internet connection is not reliable