HPO with Hydra

• Your machine must have at least one GPU.

• We assume PyTorch version 1.8.1.

Summary

In this example, we’ll run hyperparameter optimization on a batches of models.

To run the fine-tuning script for 3 models, launch python3 three-task-lm.py. We also have a 12 model script which you can run with python3 twelve-task-lm.py.