They’re very good at predicting the next word, so their choice of “a” or “an” is likely to make sense in context. But you can absolutely ask a GPT to continue a sentence that appears to use the wrong word.

For instance, I just tried giving a GPT this to start with:

My favorite fruit grows on trees, is red, and can be made into pies. It is a

And the GPT finished it with:

delicious and versatile fruit called apples!

So as you can see, language is malleable enough to make sense of most inputs. Though occasionally, a GPT will get caught up in a nonsensical phrase due to this behavior.

I think when people say it’s only predicting the next word, it’s a bit of an oversimplification to explain that the AI is not actually intelligent. It’s more or less stringing words together in a way that seems plausible.