Toggle navigation sidebar

Toggle in-page Table of Contents

Neuromatch Academy: Deep Learning (Experimental)

Introduction

Schedule

Technical Help
- Using jupyterbook
  - Using Google Colab
  - Using Kaggle
- Using Discord

Quick links and policies

Prerequisites and preparatory materials for NMA Deep Learning

Basics Module

Basics And Pytorch (W1D1)
- Tutorial 1: PyTorch
Linear Deep Learning (W1D2)
Multi Layer Perceptrons (W1D3)
- Tutorial 1: Biological vs. Artificial Neural Networks
- Tutorial 2: Deep MLPs

Fine Tuning

Optimization (W1D5)
- Tutorial 1: Optimization techniques
Regularization (W2D1)
- Tutorial 1: Regularization techniques part 1
- Tutorial 2: Regularization techniques part 2
Deep Learning: The Basics and Fine Tuning Wrap-up

ConvNets and Generative Models

Convnets And Dl Thinking (W2D2)
Modern Convnets (W2D3)
- Tutorial 1: Learn how to use modern convnets
- Bonus Tutorial: Facial recognition using modern convnets
Generative Models (W2D4)

Natural Language Processing

Attention And Transformers (W2D5)
- Tutorial 1: Learn how to work with Transformers
- Bonus Tutorial: Understanding Pre-training, Fine-tuning and Robustness of Transformers
Time Series And Natural Language Processing (W3D1)
Dl Thinking2 (W3D2)
- Tutorial 1: Deep Learning Thinking 2: Architectures and Multimodal DL thinking
Deep Learning: Convnets and NLP

Unsupervised and Reinforcement Learning

Unsupervised And Self Supervised Learning (W3D3)
- Tutorial 1: Un/Self-supervised learning methods
- Bonus Lecture: Melanie Mitchell
Basic Reinforcement Learning (W3D4)
- Tutorial 1: Basic Reinforcement Learning
- Bonus Lecture: Chealsea Finn
Reinforcement Learning For Games And Dl Thinking3 (W3D5)

Deploy Models on the Web

Deploy Models (Bonus)
- Bonus Tutorial: Deploying Neural Networks on the Web

Project Booklet

Introduction to projects
Daily guide for projects
Modeling Step-by-Step Guide
Project Templates
Models and Data sets

Powered by Jupyter Book

repository
open issue

.md

Attention And Transformers

Attention And Transformers¶

previous

Bonus Lecture: Geoffrey Hinton

next

Tutorial 1: Learn how to work with Transformers

By Neuromatch

The contents of this repository are shared under under a Creative Commons Attribution 4.0 International License. Software elements are additionally licensed under the BSD (3-Clause) License.