Pre-training neural networks

Pre-training

Pre-training

Pre-training. eg train on general pictures before specific stuff. means you fit many parameters for detecting edges etc firsts

Learning rate

Reduce the learning rate.

Resetting parameters

Replace last layer (softmax) for new problem.

Freeze layers

Freeze feature learning of early layers.

Catastrophic interference

When retraining model on new data, model may forget answers to old data.