I hear people saying things like “chatgpt is basically just a fancy predictive text”. I’m certainly not in the “it’s sentient!” camp, but it seems pretty obvious that a lot more is going on than just predicting the most likely next word.
Even if it’s predicting word by word within a bunch of constraints & structures inferred from the question / prompt, then that’s pretty interesting. Tbh, I’m more impressed by chatgpt’s ability to appearing to “understand” my prompts than I am by the quality of the output. Even though it’s writing is generally a mix of bland, obvious and inaccurate, it mostly does provide a plausible response to whatever I’ve asked / said.
Anyone feel like providing an ELI5 explanation of how it works? Or any good links to articles / videos?
Most of my job is predicting the next word I’m going to type.
I get a mail. I read it. Then I wrote the first word of my reply, the most likely word after the last word of the original mail. Then the next one. Then the next one.
Or in a meeting. Someone says something. Then I say the first word of my reply. Then the next one.
Predicting the next word well in a wide number of cases is what most of us do daily all the time. It’s a very difficult and versatile and complex skill.