There have been multiple things which have gone wrong with AI for me but these two pushed me over the brink. This is mainly about LLMs but other AI has also not been particularly helpful for me.
Case 1
I was trying to find the music video from where a screenshot was taken.
I provided o4 mini the image and asked it where it is from. It rejected it saying that it does not discuss private details. Fair enough. I told it that it is xyz artist. It then listed three of their popular music videos, neither of which was the correct answer to my question.
Then I started a new chat and described in detail what the screenshot was. It once again regurgitated similar things.
I gave up. I did a simple reverse image search and found the answer in 30 seconds.
Case 2
I wanted a way to create a spreadsheet for tracking investments which had xyz columns.
It did give me the correct columns and rows but the formulae for calculations were off. They were almost correct most of the time but almost correct is useless when working with money.
I gave up. I manually made the spreadsheet with all the required details.
Why are LLMs so wrong most of the time? Aren’t they processing high quality data from multiple sources? I just don’t understand the point of even making these softwares if all they can do is sound smart while being wrong.
If hallucinations cannot be eliminated, how are they decreasing them (allegedly)?
Actually according to studies, the most recent versions of all the major LLMbecile vendors are hallucinating more, not less.
by special casing a lot of things. Like expert systems, in the 80s
What do you mean?
the “guardrails” they mention. They are a bunch of if/then statements looking to work around methods that the developers have found to produce undesirable outputs. It doesn’t ever mean “the llm will not bo doing this again”. It means “the llm wont do this when it is asked in this particular way”, which always leaves the path open for “jailbreaking”. Because you will almost always be able to ask a differnt way that the devs (of the guardrails, they don’t have much control over the llm itself) did not anticipate.
Expert systems were kind of “if we keep adding if/then statements, we would eventually cover all the bases and get a smart, reliable system”. That didn’t work then. It won’t work now either
I have experienced this first hand. Asking LLMs explicit things leads to “I can’t help you with that” but if I ask it in a roundabout way, it gives a straight answer.