What makes a neural network remember?