Catastrophic forgetting in Lifelong learning
An empirical analysis of the proposed solutions to the catastrophic forgetting problem. Experiments on using Experience Replay, Adapters, Meta-learning (MAML and OML) to train BERT for lifelong text classification.
Divergence in Deep Q-Learning: Tips and Tricks
Two tricks are used to train DQNs - Target Network which mitigates divergence and Experience Replay which facilitates convergence towards better policies.
Verbs as Nouns, Functions as Data
Grokking higher-order functions in Lisp is like breaking down the wall between nouns (data) and verbs (functions)
Workflow for Deep Learning Projects
Guidelines and best practices for developing deep learning systems
Reimplementing the PyTorch training loop in simple Python
The PyTorch training loop relies on the abstractions over data (Dataset, Dataloader) and abstractions over model updates (Parameters, Optimizers). Here we re-implement these abstractions in plain Python to understand them better.
Building the foundations of Deep Learning from scratch
We implement the foundations of deep learning systems: optimized matrix multiplications for the forward pass and reverse mode auto-differentiation for the backward pass.
Fun, Physics & Machine Learning: A Summer @ CERN Openlab
My summer internship at CERN Openlab 2018
