By Daniel Glass
Originally Posted on LinkedIn
Why do LLMs seem so knowledgeable? Don’t they just predict what word likely comes next in a sequence of words?
Simply put, LLMs learn language through context. That is, learning the meaning of words based on the way they are used, where they are in a sentence, the sentiment in which they are used, and so on.
The training process consists of taking a lot of example language (sentences, paragraphs, papers, etc.), deconstructing it, and rebuilding it. The same principles that we used for our warehouse task time prediction example are used here. Pass the input into the model, compare the model’s prediction to our target value, and change pieces of our model in order to get our prediction closer to the target. In this case, our target is the actual next word in a sequence and our prediction is what we expect it to be.
An interesting side effect of training, is that our model will come away with some knowledge in the process. If a sentence we are training on is something like “The capital of France is _______.” — predicting the next word means knowing what the capital of France is. With enough examples of ‘Paris’ used in our training data, we start to understand what ‘Paris’ means. But remember: context is everything. ‘Paris’ doesn’t always mean the capital of France. If we are talking about the Trojan War, ‘Paris’ would likely refer to the Trojan prince. This shows that just a definition of a word is not enough — the context in which it is used is needed to truly know what a word means.
What if we decided to come up with our own definition of ‘Paris’ right now, and used it in a sentence with ChatGPT? Should we expect ChatGPT to know the new meaning given the context that we used it in? In general, can we expect our machine learning models to know things outside of the scope of training? Let’s talk about this next time!
But in the meantime, check out how Cellaware has implemented real world applications for ML/AI in the warehousing and distribution industry. Schedule your free demo today!
Comments