What is Loss Function?
Loss Function — A mathematical function that measures how far an AI model’s predictions deviate from the actual true values.
The loss function measures how wrong a model’s predictions are. During training, the goal is to minimize this function. Different tasks use different loss functions — cross-entropy for classification, mean squared error for regression, and custom losses for specialized tasks.
Frequently Asked Questions
How does the loss function affect training?
It defines what the model optimizes for. Choosing the wrong loss function means the model optimizes for the wrong objective, even if training appears successful by other metrics.
What is the most common loss function?
Cross-entropy loss for classification tasks and mean squared error for regression tasks. For LLM training, next-token prediction uses cross-entropy loss.
Can I create a custom loss function?
Yes. Custom losses let you encode business-specific priorities — for example, penalizing false negatives more than false positives in a medical diagnosis model.