api: YAML JSON TOON Database version: 1.0.0 format: json dataset: id: 14 slug: ml-evaluation-metrics title: Machine Learning Evaluation Metrics description: "Common ML evaluation metrics: Accuracy, Precision, Recall, F1 Score, ROC-AUC, MAE, MSE, and R-squared with formulas and best-use scenarios." summary: "Evaluation metrics for machine-learning models with type, formula, best use and range: accuracy, precision, recall, F1, ROC-AUC, MAE, MSE and R-squared, extended with specificity, log loss, RMSE, Matthews correlation, IoU, perplexity, BLEU, nDCG and Cohen's kappa." use_when: - You need the formula and interpretation of a metric. - You are choosing a metric for imbalanced classification, ranking or generation. caveats: - A single metric hides trade-offs; report several and choose by the cost of different errors. - Metrics such as BLEU correlate imperfectly with human judgment of quality. - ROC-AUC can range from 0 to 1; 0.5 is a random ranking, not the lower bound. Report prevalence and the decision threshold alongside threshold-specific metrics. related: - statistics-basics - ai-ethics-interpretability - regularization-techniques - llm-glossary - rag-retrieval-metrics - experimental-design-pitfalls category: AI & ML Terms category_slug: ai-ml-terms tags: ml,evaluation,metrics,data science view_count: 0 created_at: 1777673262 updated_at: 1791799373 sources: - title: scikit-learn model evaluation url: "https://scikit-learn.org/stable/modules/model_evaluation.html" reading: - evaluate-rag-retrieval-before-generation reviewed_at: "2026-10-12" provenance: Original editorial reference notes by YJTOON, informed by the linked primary documentation. External specifications retain their own licenses. entry_count: 21 fields: - section: metrics field: name type: string example: Accuracy - section: metrics field: type type: string example: Classification - section: metrics field: formula type: string example: (TP + TN) / (TP + TN + FP + FN) - section: metrics field: best_for type: string example: Balanced datasets - section: metrics field: range type: string example: "[0, 1], higher is better" data: metrics: - name: Accuracy type: Classification formula: (TP + TN) / (TP + TN + FP + FN) best_for: Balanced datasets range: "[0, 1], higher is better" - name: Precision type: Classification formula: TP / (TP + FP) best_for: Minimizing false positives range: "[0, 1], higher is better" - name: Recall type: Classification formula: TP / (TP + FN) best_for: Minimizing false negatives range: "[0, 1], higher is better" - name: F1 Score type: Classification formula: 2 * (Precision * Recall) / (Precision + Recall) best_for: Imbalanced datasets range: "[0, 1], higher is better" - name: ROC-AUC type: Classification formula: Area under ROC curve best_for: Ranking quality range: "[0, 1]; 0.5 is the random-ranking baseline, and lower values are possible" - name: MAE type: Regression formula: mean(|y_true - y_pred|) best_for: Average error magnitude range: "[0, inf), lower is better" - name: MSE type: Regression formula: mean((y_true - y_pred)^2) best_for: Penalizing large errors range: "[0, inf), lower is better" - name: R-squared type: Regression formula: 1 - (SS_res / SS_tot) best_for: Proportion of variance explained range: (-inf, 1], higher is better - name: Specificity type: Classification formula: TN / (TN + FP) best_for: Measuring how well negatives are identified range: "[0, 1], higher is better" - name: Log Loss type: Classification formula: "-(1/N) * sum(y log p + (1 - y) log(1 - p))" best_for: Evaluating predicted probabilities range: "[0, infinity), lower is better" - name: Matthews Correlation Coefficient type: Classification formula: (TP*TN - FP*FN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN)) best_for: Imbalanced binary classification range: "[-1, 1], higher is better" - name: RMSE type: Regression formula: sqrt(mean((y - y_hat)^2)) best_for: Regression where large errors should weigh more, in the units of the target range: "[0, infinity), lower is better" - name: Intersection over Union type: Detection and segmentation formula: "|A intersect B| / |A union B|" best_for: Overlap between predicted and true regions range: "[0, 1], higher is better" - name: Perplexity type: Language modeling formula: exp(-(1/N) * sum(log p(w_i | context))) best_for: Comparing language models on the same text range: "[1, infinity), lower is better" - name: BLEU type: Machine translation formula: Brevity penalty * geometric mean of n-gram precisions best_for: Comparing translations with references range: "[0, 1], higher is better" - name: nDCG type: Ranking formula: DCG / ideal DCG best_for: Search and recommendation quality with graded relevance range: "[0, 1], higher is better" - name: Cohen's Kappa type: Agreement formula: (p_o - p_e) / (1 - p_e) best_for: Agreement between annotators beyond chance range: "[-1, 1], higher is better" - name: Average Precision type: Ranking / binary classification formula: sum((recall_n - recall_previous) * precision_n) best_for: Summarizing the precision-recall tradeoff when positives are rare range: "[0, 1], higher is better; prevalence affects the baseline" - name: Brier Score (binary) type: Probability prediction formula: mean((p_positive - y_binary)^2) best_for: Evaluating probabilistic predictions alongside calibration plots range: "[0, 1], lower is better" - name: Balanced Accuracy type: Classification formula: mean(recall_per_class) best_for: Giving each class equal weight in the recall average range: "[0, 1], higher is better" - name: Pinball Loss type: Quantile regression formula: mean(max(q * error, (q - 1) * error)); error = y_true - y_pred best_for: Evaluating a specified conditional quantile range: "[0, inf), lower is better"