Well, maybe if they weren’t using AI as a hypeword and just called it adaptive or GPT.
Every company that has been trying to push their shiny, new AI feature (which definitely isn’t part of a rush to try and capitalize on the prevalence of AI), my instant response is: “Yeah, no, I’m finding a way to turn this shit off.”
My response is even harsher…“Yeah, no, I’m finding a way to never use this company’s services ever again.” Easier said than done, but I don’t even want to associate with places that shove this in my face.
LLMs: using statistics to generate reasonable-sounding wrong answers from bad data.
Often the answers are pretty good. But you never know if you got a good answer or a bad answer.
And the system doesn’t know either.
For me this is the major issue. A human is capable of saying “I don’t know”. LLMs don’t seem able to.
Accurate.
No matter what question you ask them, they have an answer. Even when you point out their answer was wrong, they just have a different answer. There’s no concept of not knowing the answer, because they don’t know anything in the first place.
The worst for me was a fairly simple programming question. The class it used didn’t exist.
“You are correct, that class was removed in OLD version. Try this updated code instead.”
Gave another made up class name.
Repeated with a newer version number.
It knows what answers smell like, and the same with excuses. Unfortunately there’s no way of knowing whether it’s actually bullshit until you take a whiff of it yourself.
So instead of Prompt Engineer, the more accurate term should be AI Taste Tester?
From what I’ve seen you’ll need an iron stomach.
With proper framework, decent assertions are possible.
- It must cite the source and provide the quote, not just a summary.
- An adversarial review must be conducted
If that is done, the work on the human is very low.
That said, it’s STILL imperfect, but this is leagues better than one shot question and answer
Except LLMs don’t store sources.
They don’t even store sentences.
It’s all a stack of massive N-dimensional probability spaces roughly encoding the probabilities of certain tokens (which are mostly but not always words) appearing after groups of tokens in a certain order.
And all of that to just figure out “what’s the most likely next token”, an output which is then added to the input and fed into it again to get the next word and so on, producing sentences one word at a time.
Now, if you feed it as input a long, very precise sentence taken from a unique piece, maybe you’re luck and it will output the correct next word, but if you already have all that you don’t really need an LLM to give you the rest.
Maybe the “framework” you seek - which is quite akin to a indexer with a natural language interface - can be made with AI, but it’s not something you can do with LLMs because their structure is entirely unsuited for it.
They really aren’t. Go ask about something in your area of expertise. At first glance, everything will look correct and in order, but the more you read the more it turns out to be complete bullshit. It’s good at getting broad strokes but the details are very often wrong.
Now imagine someone that doesn’t have your expertise reading that answer. They won’t recognize those details are wrong until it’s too late.
That is about the experience I have. I asked it for factual information in the field I work at. It didn’t gave correct answers. Or, it gave working protocols which were strange and would not be successful.
Sounds familiar. Citation please
deleted by creator
<greentext>
Be me
Early adopter of LLMs ever since a random tryout of Replika blew my mind and I set out to figure what the hell was generating its responses
Learn to fine-tune GPT-2 models and have a blast running 30+ subreddit parody bots on r/SubSimGPT2Interactive, including some that generate weird surreal imagery from post titles using VQGAN+CLIP
Have nagging concerns about the industry that produced these toys, start following Timnit Gebru
Begin to sense that something is going wrong when DALLE-2 comes out, clearly targeted at eliminating creative jobs in the bland corporate illustration market. Later, become more disturbed by Stable Diffusion making this, and many much worse things, possible, at massive scale
Try to do something about it by developing one of the first “AI Art” detection tools, intended for use by moderators of subreddits where such content is unwelcome. Get all of my accounts banned from Reddit immediately thereafter
Am dismayed by the viral release of ChatGPT, essentially the same thing as DALLE-2 but text
Grudgingly attempt to see what the fuss is about and install Github Copilot in VSCode. Waste hours of my time debugging code suggestions that turn out to be wrong in subtle, hard-to-spot ways. Switch to using Bing Copilot for “how-to” questions because at least it cites sources and lets me click through to the StackExchange post where the human provided the explanation I need. Admit the thing can be moderately useful and not just a fun dadaist shitposting machine. Have major FOMO about never capitalizing on my early adopter status in any money-making way
Get pissed off by Microsoft’s plans to shove Copilot into every nook and cranny of Windows and Office; casually turn on the Opympics and get bombarded by ads for Gemini and whatever the fuck it is Meta is selling
Start looking for an alternative to Edge despite it being the best-performing web browser by many metrics, as well as despite my history with “AI” and OK-ish experience with Copilot. Horrified to find that Mozilla and Brave are doing the exact same thing
Install Vivaldi, then realize that the Internet it provides access to is dead and enshittified anyway
Daydream about never touching a computer again despite my livelihood depending on it
</greentext>
I like my AI compartmentalized, I got a bookmark for chatGPT for when i want to ask a question, and then close it. I don’t need a different flavor of the same thing everywhere.
The irony is companies are being forced to implement it. Like our board has told us we must have “AI in our product.”. It’s literally a solution looking for a problem that doesn’t exist.
Who cares about consumers inject more venture capital funds pls.
There are even companies slapping AI labels onto old tech with timers to trick people into buying it.
That one DankPods video of the “AI Rice cooker” comes to mind
For what it’s worth, rice cookers have been touting “fuzzy logic” for like 30 years. The term “AI” is pretty much the same, it just wasn’t as buzzy back then.
Yeah that’s the one I saw
Cuz everyone knows it’s BS, or mostly BS with extra data mining
No shit Sherlock
In your own words, tell me why you’re calling today.
My medication is in the wrong dosage.
You need to refill your medication is that right?
No, my medication is in the wrong dosage, it’s supposed to be tens and it came as 20s.
You need to change the pharmacy where you’re picking up your medication?
I need to speak to a human please.
I understand that you want to speak to an agent, is that right?
Yes.
Chorus, 5x. (Please give me your group number, or dial it in at the keypad. For this letter press that number for that letter press this number. No I’m driving, just connect me with an agent so I can verify over the phone)
I’m sorry, I can’t verify your identity please collect all your paperwork and try calling again. Click
Why ever would we be mad?
Also just listening and reading what people say. We don’t want fucking AI anything. We understand what it might do. We don’t want it.
deleted by creator
Citation please?
You’re mad that someone investigates and elaborates on causes of why using llm marketing bullshit is a bad idea? Weird.
deleted by creator
To be honest, I lost all interest in the new AMD CPUs because they fucking named the thing “AI” (with zero real-world application).
I’m in the market for a new PC next month and I’m gonna get the 7800X3D for my VR gaming needs.
AI is garbage.
AI is just an excuse to lay off your employees for an objectively less reliable computer program, which somehow statistically beats us in logic.
I’ve used LLMs a lot over the post couple years. Pro tip. Use it a lot and learn the models. Then they look much more intelligent as you the user becomes better. Obviously if you prompt “Write me a shell script to calculate the meaning of life, make my coffee, and scratch my nuts before 9AM” it will be a grave disappointment.
If you first design a ball fondling/scratching robot, use multiple instances of LLMs to help you plan it out, etc. then you may be impressed.
I think one of the biggest problems is that most people interacting with llms forget they are running on computers and that they are digital and not like us. You can’t make assumptions like you can with humans. Usually even when you do that with us you just get stuff you didn’t want because you weren’t clear enough. We are horrible at instructions and this is something I hope AI will help us learn how to do better. Because ultimately bad instructions or incomplete information doesn’t lead to being able to determine anything real. Computers are logic machines. If you tell a computer to go ride a bike at best it’ll go out and do all the work to embody itself in a robot and buy a bike and ride it. Wait, you don’t even know it did it though because you never specified for it to record the ride…
A very few of us are pretty good at giving computers clear instructions some of the time. Also though, I have found just forcing models to reason in context is powerful. You have to know to tell it to “use a drill down tree style approach to problem solving. Use reflection and discussion to explore and find the optimal solution to reasoning through the problem.” Might still give you bad results. That is why you have to experiment. It is a lot of fun if you really just let your thoughts run wild. It takes a lot of creative thinking right now to really get the most out of these models. They should all be 110% open source and free for all. BTW Gemini 1.5 and Claude and Llama 3.1 are all great, nd Llama you can run locally or on a rented GPU VM. OpenAI I’m on the fence about but given who all is involved over there I wouldn’t say I would trust them. Especially since they want to do a regulatory capture.
Asking the chat models to have self-disccusion and use/simulate metacognition really seems to help. Play around with it. Often times I am deep in a chat and I learn from its mistakes, it kinda learns from my mistakes and feedback. It is all about working with and not against. Because at this time LLMs are just feed forward neural networks trained on supercomputer clusters. We really don’t even know what they are capable of fully because it is so hard to quantify, especially when you don’t really know what exactly has been learned.
Q-learning in language is also an interesting methodology I’ve been playing with. With an imagine generator for example though, you can just add (Q-learning quality) and you may get more interesting and quality results. Which itself is very interesting to me.
