• 0 Posts
  • 10 Comments
Joined 3 years ago
cake
Cake day: July 6th, 2021

help-circle
  • I use copilot on a daily basis for programming. It has made me much more productive and it’s a real pleasure to use it. Nothing overhyped about it.

    Curious to see what it will bring for other domains, e.g. for dealing with emails.

    I do agree that there’s a lot of filtering happening. Not a huge deal for more applications. Luckily you can run your own models that are not filtered. I can definitely see a future where you run your own models locally. Afaik Apple recently did some stuff around that.





  • Correct, it’s not just regurgitating words, it’s predicting which token comes next. A token is sometimes a whole word, but for longer ones it’s part of a word (and some other rules that define how tokenization works).

    How it knows which token comes next is why the current generation of LLMs is so impressive. It seems to have learned the rules the underpin our languages, to the point that it seems to even understand the content. It doesn’t just know the grammer rules (without anyone telling it, it just learned the patterns), it also knows which words belong to each other in which context.

    It’s your prompt + some preset other context (e.g. that it is an OpenAI LLM) that creates that context. So being able to predict a token correctly is one part, the other is having a good context. This is why prompt engineering quickly became a thing. This is also why supporting bigger contexts is another thing (but a larger context requires way more processing power, so there’s a trade-off there).

    It’s btw not just the trained model + context that gives you the output of ChatGPT. I’m pretty sure there are layers before and after, possibly using other ML models, that filter content or make it more fit for processing. This is why you can’t ask it how to make bombs, even though those recipes are in its training set and it very likely can create a recipe based on that.