Someday the AI will get good, and I’ll want to chat with it securely.
GPT4 is pretty good now. I’m not convinced it will be secure until we can run it locally on our own hardware.
As soon as we can run it locally, I plan to do so. Even if it means using a GPT4 quality LLM when far better exists if I use a cloud service.
The habit of sending tokens right as they generate is a dumb sales gimmick
Seems like it would be trivial to just place tokens in a buffer on the server and send output to the client in say 1KB chunks (a TCP packet can’t be much bigger than that anyway, and it needs a bit of space for routing metadata).
And if the entire output is less than 1KB… pad it out to that length. Pretty standard to do that anywhere you care about security… e.g. if you were to dump the password table databases… they’re all 256 bits. That’s obviously not the real length - most will be shorter, some will be longer. Whatever they are it’s cryptographically expanded (or shortened) to 256.
GPT4 is pretty good now. I’m not convinced it will be secure until we can run it locally on our own hardware.
As soon as we can run it locally, I plan to do so. Even if it means using a GPT4 quality LLM when far better exists if I use a cloud service.
Seems like it would be trivial to just place tokens in a buffer on the server and send output to the client in say 1KB chunks (a TCP packet can’t be much bigger than that anyway, and it needs a bit of space for routing metadata).
And if the entire output is less than 1KB… pad it out to that length. Pretty standard to do that anywhere you care about security… e.g. if you were to dump the password table databases… they’re all 256 bits. That’s obviously not the real length - most will be shorter, some will be longer. Whatever they are it’s cryptographically expanded (or shortened) to 256.