Of course not, the whole point of disinformation is that it sounds correct, that’s AI’s bread and butter!
Supposedly ChatGPT had an update in September, but it doesn’t agree that Trump was found guilty in may 34 times. When I give it sources it says ok, but it doesn’t upload correct learned information.
That’s because it doesn’t learn, it’s a snapshot of its training data frozen in time.
I like Perplexity (a lot) because instead of using its data to answer your question, it uses your data to craft web searches, gather content, and summarise it into a response. It’s like a student that uses their knowledge to look for the answer in the books, instead of trying to answer from memory whether they know the answer or not.
It is not perfect, it does hallucinate from time to time, but it’s rare enough that I use it way more than regular web searches at this point. I can throw quite obscure questions at it and it will dig the answer for me.
As someone with ADHD with a somewhat compulsive need to understand random facts (e.g. “I need to know right now how the motor speed in a coffee grinder affects the taste of the coffee”) this is an absolute godsend.
I’m not affiliated or anything, and if anything better comes my way I’ll be happy to ditch it. But for now I really enjoy it.
I love that someone even bothered with a study.
(Edit: To be clear, I am both amused, and also genuinely appreciate that the science is being done.)
Confirmation of anecdotes or gut feelings is still science. At some point you need data rather than experience to help people and organizations change their perception (see: most big tech companies lighting billions of dollars on fire on generative AI).
Not to mention based on the numbers in the article I imagine the AI might actually do better than an average human would do. It wasn’t as much of a “duh” as I thought it would be.
Agreed!
I don’t mean sarcasticly, honestly. As you said, it’s still valuable science.
That’s true. But still. Duh.
For many hundreds of years, blood-letting was an obvious thing to do. As was just giving people leeches for medical ailments. And ingesting mercury. We thought having sex with virgins would cure STDs. We thought doses of radiation was good for us. And tobacco. We thought it was obvious that the sun revolved around Earth.
It is enormously important to scientifically confirm things, even if they do seem obvious.
Think about it this way: remember those upside-down answer keys in the back of your grade school math textbook? Now imagine if those answer keys included just as many incorrect answers as correct ones. How would you know if you were right or wrong without asking your teacher? Until a LLM can guarantee a right answer, and back it up with real citations, it will continue to do more harm than good.
I think the next step in AI is learning how to control and direct the speech, rather than just make computers talk.
They are surprisingly good for being a mere statistical copycat of words on the internet. Whatever the second tier innovation is that jumps AI into true reasoning rather than pattern matching is going to be wild.
Shock pickachu
