Neural Networks: Zero to Hero [Post #6, Day 7]

I have started Video 1 of 10 of Andrej Karpathy's Neural Networks: Zero to Hero playlist. Video 1 is titled The spelled-out intro to neural networks and backpropagation: building micrograd. I am about 30 minutes in so far and following along by coding along in a Jupyter Notebook. This appears to be pretty much the same MLP NN architecture with the backpropagation algorithm I built previously with Graham Ganssle's tutorial but it's a different coding approach, focused on object-oriented programming (OOP) style. I am still working on understanding this style, I don't quite get it yet. It is good that I have learned the NN concepts previously and now I am being taught them in a bit different way with a different coding style, I think that's probably good and will deepen my understanding.

I am having a go again at understanding OOP programming style and making it stick.

I have created a class called "Value" that defines a blueprint for creating objects. That specific first line is called the class definition. Each object within my Value class will be a Value object, an object of the Value class. My Value class encapsulates both data (numerical value and metadata like children, operation, and label) and behavior (how to combine Value objects using operations). In my application, each object is a node (neuron) that holds the numerical value (which I think is the weight value), remembers how it was created (by op and prev), and has a label for debugging.

I have some methods which are functions that belong to my Value class. The first parameter is always "self". My methods are special methods or "magic methods" because they have the double underscores around Python built-in functions.

The init double underscore magic method is the constructor which is called when I create a new Value object/instance, x = Value(2.0) calls this magic method (it calls the constructor). It is the magic method for object initialization, it is called when creating new objects/instances.

Encapsulation is another term. The single underscore prefix in front of variables is a way to say that variable is an internal or "protected" attribute, it is a Python convention. Said another way, they are internal implementation details.

Tasks to still do after making it through Video 1:

• I need to go through the manual backpropagation process again, I need to understand what we are actually doing, starts at 00:37:30 and ends at 01:09:10.