Tomorrow I start a Machine Learning Class at Columbia University through their executive eductation program. The program is a partnership between Columbia and a company called Emeritus. There are lots of great, free online resources so you might ask why I chose to pay for this course. In my experience, the classes I’ve seen online have either been too rudimentary for me or way beyond what I can do. There’s a gap for people like me who’ve been practicing engineers for 2 decades who have a strong mathematical background.
I recently attended the 2020 HotChips conference. A number of talks focused on how to scale out ML models and the need for Message Passing Interfaces between different machines. One of the most interesting places for hardware is possibly doing sums in the network as part of the AllGather operation so the TPU/GPUs can continue multiplications. There are some interesting problems here that hardware can potentially solve so I’m taking it on myself to learn more about MPI.