Catastrophic forgetting in Lifelong learning

An empirical analysis of the proposed solutions to the catastrophic forgetting problem. Experiments on using Experience Replay, Adapters, Meta-learning (MAML and OML) to train BERT for lifelong text classification.

Divergence in Deep Q-Learning: Tips and Tricks

Two tricks are used to train DQNs - Target Network which mitigates divergence and Experience Replay which facilitates convergence towards better policies.

Verbs as Nouns, Functions as Data

Grokking higher-order functions in Lisp is like breaking down the wall between nouns (data) and verbs (functions)

Workflow for Deep Learning Projects

Guidelines and best practices for developing deep learning systems

Reimplementing the PyTorch training loop in simple Python

The PyTorch training loop relies on the abstractions over data (Dataset, Dataloader) and abstractions over model updates (Parameters, Optimizers). Here we re-implement these abstractions in plain Python to understand them better.

Building the foundations of Deep Learning from scratch

We implement the foundations of deep learning systems: optimized matrix multiplications for the forward pass and reverse mode auto-differentiation for the backward pass.

Fun, Physics & Machine Learning: A Summer @ CERN Openlab

My summer internship at CERN Openlab 2018