AI coffe break: notes1
Contents
How to check if a neural network has learned a specific phenomenon?
Question:
How do we check if a neural network trained on task A has learned a phenomenon specific to task B?
To NLP, the task is called probing/diagnostic classifier/probing task.
How do we do probing?
- Freeze the parameters of Input layer and Hidden layers,
- Replace the last layers to Probing layers,
- Train the Probing layers on a small dataset
If task A is Masked word prediction and task B is Sentiment analysis, then:
