• Learning curves will help isolate the problem.

  • If this is a high-variance issue (higher error in test than training), then we can use RF, remove features, use regularization, get more data, etc.

  • If this is a high-bias issue (training and testing error high), we can add features, gradient boosting, add complexity.

We can also try the following:

• Run optimization longer (more iterations) \(\rightarrow\) Fixes optimization algorithm

• Change optimization method \(\rightarrow\) Fixes optimization algorithm

• Hyperparameter tuning \(\rightarrow\) Fixes optimization objective

• Alternative models (e.g., non linear models) \(\rightarrow\) Fixes optimization objective

• Ablative analysis could also be useful if this was a multistep framework.