I think a part of it is the scale of the data used to train it means there wasn’t likely much curation of that data. So it might complete the text using a wikipedia article, a knowledge forum post, a forum post that derailed the original topic, a forum post written by someone confidently wrong, a troll post, or two people arguing about the answer. In that last case, you might be able to get it to hash out the entire argument by asking if it’s sure about that after each response.
Which is also probably how it can correctly respond to “are you sure?” follow ups in the first place, because it was going off some forum post that someone questioned and then there was a follow-up.
It’s more complicated than that because it’s likely not just rehashing any one single conversation in any response, but all of those were a part of its training, and its training is all it knows.
I think a part of it is the scale of the data used to train it means there wasn’t likely much curation of that data. So it might complete the text using a wikipedia article, a knowledge forum post, a forum post that derailed the original topic, a forum post written by someone confidently wrong, a troll post, or two people arguing about the answer. In that last case, you might be able to get it to hash out the entire argument by asking if it’s sure about that after each response.
Which is also probably how it can correctly respond to “are you sure?” follow ups in the first place, because it was going off some forum post that someone questioned and then there was a follow-up.
It’s more complicated than that because it’s likely not just rehashing any one single conversation in any response, but all of those were a part of its training, and its training is all it knows.