Prerequisites and preparatory materials for NMA Deep Learning

Welcome to the Neuromatch Academy! We’re really excited to bring deep learning to such a wide and varied audience. We’re preparing an amazing set of lectures and tutorials for you!

Preparing yourself for the course

People are coming to this course from a wide range of disciplines and with varying levels of background, and we want to make sure everybody is able to follow and enjoy the school from day 1. This means you need to know the basics of programming in Python and some core math concepts. Below we provide more details.

Programming

This course will be run using Python. If you’ve never programmed in Python, now is a good time to start practicing! We expect students to be familiar with variables, lists, dicts, the numpy and scipy libraries as well as plotting in matplotlib. Practice a little bit every day and you’ll be in great shape by the time the class starts.

We have NMA Python workshop materials (W0D1 and W0D2 here). You should go through this NMA-made content at your own pace before the course. Note that it has a neuroscience focus in the examples used but the Python basics will be extremely useful for the deep learning course.

Besides these NMA materials, we recommend the Software carpentry 1-day Python tutorial or the free Edx course Using Python for Research. For a more in-depth intro, see the scipy lecture notes. Finally, you can follow the Python data science handbook, which also has a print edition.

If you’re coming from a Matlab background, you can quickly get up to speed with this cheatsheet. You may also enjoy this paperback on Neural Data Science with both Matlab and Python versions.

Math skills

Deep learning relies on linear algebra, probability, basic statistics, and calculus (derivates and ODEs).

We highly recommend going through our refreshers on linear algebra, calculus, and statistics (W0D3, W0D4, W0D5 here). You will be able to ask questions on discord before the course starts. Note that these have a neuroscience focus in the examples used but the math basics will be extremely useful for the deep learning course.

Linear algebra: You will need a good grasp of linear algebra to follow along, as linear algebra is crucial for almost anything quantitative involving more than one number at a time. You need to know vector and matrix addition and multiplication, rank, bases, determinants, inverses, and eigenvalue decomposition. In addition to our W0D3, we highly recommend this beautiful lecture series. Another great resource is Khan academy. Here is a series of exercises on linear algebra in Python.

Statistics: Understanding statistics is also important; you should be comfortable with means and variances, and the normal distribution. In addition to our W0D4, for a refresher, we recommend selective readings (i.e. chapters 6-7 from Russ Poldrack’s book “Statistical thinking of the 21st century”.

Calculus: Finally, basic calculus is crucial; you should know what integrals and derivatives are, and understand what a differential equation means. If you need to refresh your memory on differential and integral calculus, Gilbert Strang’s book is a good refreshment book. For differential equations, we recommend studying chapter 0-1 (including exercises!) of Jiri Lebl’s book “Differential equations for engineers”.

The Neuromatch Academy team.