Deep learning specialization, course 2 [ Post #26, Day 53]
February 27, 2025•218 words
I am about to continue with the graded programming assignment for Course 2, Week 2. Week 2's material included optimization algorithms, for training neural nets more efficiently and faster. We covered mini-batch gradient descent first. Then focused on improving the optimization algorithm itself, learning about gradient descent with momentum (which incorporates a moving average over gradients), gradient descent with RMSprop, and the Adam optimization algorithm which incorporates both momentum and RMSprop. We then talked about learning rate decay.
Excerpt from the end of the Week 2 Jupyter Notebook:
Congratulations! You've made it to the end of the Optimization methods notebook. Here's a quick recap of everything you're now able to do:
- Apply three different optimization methods to your models
- Build mini-batches for your training set
- Use learning rate decay scheduling to speed up your training
Nice!
I also watched a video last night of Andrew Ng giving tips for succeeding in the AI/ML space. He said it's not as glamorous as some people think, you still have to do the dirty work like cleaning data sets, etc. His best tip for entering and advancing in the AI/ML space is to read research papers and reproduce them. Do this for not 1-2 papers, but for 30-50, and new ideas of your own will start to come.