Main Challenges of Machine Learning
1. Insufficient Quantity of Training Data
Definition: A scenario where the dataset used to train a machine learning model is too small to learn patterns effectively.
Explanation: The model can't generalize well with limited data, leading to poor performance on unseen data.
Example: Training a facial recognition model with only 50 images results in poor accuracy in real-world scenarios.
2. Nonrepresentative Training Data
Definition: When the training data doesn’t reflect the variety and distribution of real-world inputs.
Explanation: The model may perform poorly on diverse or biased real-world examples it hasn’t seen before.
Example: A spam detector trained only on English emails may fail to detect spam written in other languages.
3. Poor Quality Data
Definition: Data that contains errors, noise, missing values, or outliers that negatively impact model training.
Explanation: The model might learn misleading patterns and yield inaccurate predictions.
Example: A dataset of temperature sensors with many missing or corrupted readings will degrade prediction accuracy.
4. Irrelevant Features
Definition: Features that do not help in predicting the target and may introduce noise.
Explanation: Including irrelevant features can lead to overfitting and increased computational complexity.
Example: Using the brand of a person’s phone to predict loan repayment might not be relevant.
5. Underfitting the Training Data
Definition: When the model is too simple to learn the underlying pattern of the data.
Explanation: It leads to high bias and poor performance on both training and test data.
Example: Using a straight line to model a curved relationship between two variables.
6. Overfitting the Training Data
Definition: When the model learns not just the pattern but also the noise in the training data.
Explanation: This results in great training accuracy but poor generalization to new data.
Example: A decision tree that memorizes outliers performs poorly on unseen data.
7. Regularization
Definition: A technique used to reduce overfitting by penalizing large model parameters.
Explanation: Regularization methods like L1 and L2 help simplify the model and improve generalization.
Example: Adding λ * sum(weights²) in linear regression helps control model complexity (Ridge Regression).
Common Regularization Techniques:
- L1 Regularization (Lasso)
- L2 Regularization (Ridge)
8. Testing and Validating
Definition: Methods to evaluate model performance. Validation is used during training; testing is done after training is complete.
Explanation: Helps tune hyperparameters and assess how the model will perform on unseen data.
Example: Splitting a dataset 70% training, 15% validation, 15% testing to evaluate and tune a classifier.
9. Hyperparameter Tuning
Definition: The process of choosing the best hyperparameters that define how the model learns.
Explanation: Techniques like grid search and random search help optimize model performance.
Example: Finding the best value for k in k-NN using cross-validation.
10. Model Selection
Definition: Choosing the best algorithm for a task based on performance metrics.
Explanation: Involves comparing models like decision trees, SVMs, and neural networks on the same task.
Example: Comparing several classifiers and selecting the one with the highest F1 score on validation data.