How do we choose the next token from a transformer (LLM)? #explainer #programming #ml

The output of a transformer (LLM) is a score (logit) for every possible next token. We then use temperature, softmax, top-p and sampling to choose the next token, excerpt from my video "I Built an LLM From Scratch" Watch the full video here: https://www.youtube.com/watch?v=YmLp8qe87A0 ------------------------------------------------------------------------------ http://www.syntax.fm Brought to you by https://sentry.io/syntax #webdevelopment #webdeveloper #javascript #syntax #syntaxfm #webdev