Z2H Video 2, round 2, finished watching [Post #15, Day 32]
February 6, 2025•89 words
Just finished watching Video 2, now attempting the exercises.
E01: train a trigram language model, i.e. take two characters as an input to predict the 3rd one. Feel free to use either counting or a neural net. Evaluate the loss; Did it improve over a bigram model?
This is not an easy one! I started with trying the neural net framework, going to switch to the explicit counts method now.
I have also been watching a new video that Andrej just published yesterday: Deep Dive into LLMs like ChatGPT.