Analyze prediction error
Model interpretability
Gradient descent vs. stochastic gradient descent
Model interpretability
Gradient descent vs. stochastic gradient descent
Learning curves will help isolate the problem.
If this is a high-variance issue (higher error in test than training), then we can use RF, remove features, use regularization, get more data, etc.
If this is a high-bias issue (training and testing error high), we can add features, gradient boosting, add complexity.
We can also try the following:
Run optimization longer (more iterations) \(\rightarrow\) Fixes optimization algorithm
Change optimization method \(\rightarrow\) Fixes optimization algorithm
Hyperparameter tuning \(\rightarrow\) Fixes optimization objective
Alternative models (e.g., non linear models) \(\rightarrow\) Fixes optimization objective
Ablative analysis could also be useful if this was a multistep framework.
Prediction error, Bias-variance tradeoff, Diagnostics, Learning curves
- Bias-variance equation Easy (Bias-variance tradeoff, Formula derivation)
- Multiclass evaluation metrics Medium (Multiclass, Diagnostics)
- Example of high-bias and high variance Medium (Bias-variance tradeoff)
- Momentum Hard (Neural networks, Deep learning, Optimization)
- Bias-variance biased estimator Medium (Bias-variance tradeoff)
Thank you, we have received your feedback.