Solution

The correct answer is With training iterations error on training set decreases but test set increases

Key Points

• Overfitting model learns the training data too well, capturing noise or minor fluctuations.
• It performs well on the training data but poorly on unseen data because it has essentially 'memorized' the training set rather than understanding the underlying patterns. The option that describes overfitting is: 2) With training iterations, error on the training set decreases but the error on the test set increases.
• In overfitting, the model starts fitting the training data too closely, capturing noise and specific patterns in the training set that may not generalize well to new, unseen data.
• As a result, the model's performance on the test set deteriorates even though it continues to improve on the training set.

​        

Additional Information

• Underfitting: The model is too simple and fails to learn the underlying structure of the data. As a result, it performs poorly on both the training data and the new, unseen data because it doesn't capture the complexity required.
• The goal in machine learning is to find a balance between overfitting and underfitting for the model to generalize well to new data.