The Blunt Guide to Mathematically Rigorous Machine Learning

I recently wrote a brief guide on the Math required for Machine Learning. People liked it, and asked me to write one on how to master ML at a mathematically rigorous, conceptual level. That is the focus of this guide, no bullshit, no easy routes, and real, fundamental understanding. I’ll be going through the later part of the curriculum myself.

A quick question to ask yourself: Why do I want to learn ML? The following material can be very difficult at times, and keeping discipline is often a matter of keeping your core motivation at heart. For example, I’m trying to validate a new brain inspired theoretical neural network architecture, and to be able to reason about it effectively, I need to have a deep intuition about current architectures and their underlying mathematics.

Prioritize Chapters 1–4 and Chapters 7–8. This covers supervised learning, linear regression, classification, Model Assessment and Inference. Its okay if you don’t understand it at first, absolutely nobody does. Keep reading it and learning whatever math you need to until you get it. If you want, knock the whole book out, you won’t regret it.

Both books focus on R, which is worth learning.

Once you’ve finished Elements, you’re in a great position to take Stanford’s ML course, taught by Andrew Ng. You can think about this like the mathematically rigorous version of his popular Coursera course. Going into this course, make sure to refresh your Multivariate Calculus and Linear Algebra skills, as well as some probability. They provide some handy refresher guides on the site page.

Do all the exercises and problem sets, and try doing the programming assignments in both R and Python. You’ll thank me later.

At this point, you’re starting to get formidable. You have a fundamental mathematical understanding of many popular, historic techniques in Machine Learning, and can choose to dive into any vertical you want. Of course, most people want to go into Deep Learning because of its significance in industry.

If you’ve made it this far, congratulations, you’re probably in an excellent place to make sense of the latest papers in field. Just go onto Arxiv and Google Scholar and look at both seminal papers and recently papers that are popular. Remember that ML is a fast moving field and the literature changes, so keep checking back in every few months.

If you’re feeling particularly bold or find something cool, try implementing it yourself. The learning process will be invaluable.

There are a lot of AI residency programs popping up at OpenAI, Google, Facebook, Uber, and a few other places. You are probably a pretty good candidate, give them a shot.

If you get this far, holy shit. Well done. The journey is never over, but you’re in an excellent place and you understand ML as well as many experts. I think.

Oh and those of you just starting, I’m right there with you. Race you to the end ;)

If you enjoyed it, please let me know by clapping or commenting! I’m working on some interesting stuff, including brain inspired neural networks that have adaptive topology. I’ll be updating this publication as I go along.

Related posts:

Constant upgrades in technology affect not only how we conduct business, but also how we live our lives. From the cars we drive, to using smartphones for mobile banking, industries are capitalizing…