How to Choose a Machine Learning Decision Threshold

A machine learning model can produce an accurate probability, but that probability does not automatically tell you what action to take.

For example, a fraud detection model might predict that a transaction has a 72% probability of being fraudulent. Should the system block it? Review it manually? Or allow it to proceed?

The answer depends on the decision threshold.

A common default is to classify an observation as positive when its predicted probability is at least 0.50. However, 0.50 is not a universal rule. In many real-world machine learning applications, choosing the right threshold can be just as important as choosing the model itself.

In this guide, you will learn what a machine learning decision threshold is, why 0.50 is often inappropriate, how thresholds affect precision and recall, and how to choose a threshold using Python.

What Is a Machine Learning Decision Threshold?

A decision threshold is the probability cutoff used to convert a model’s predicted probability into a final class.

Suppose a binary classification model predicts whether a customer will cancel a subscription.

The model might produce probabilities like:

Customer	Predicted Probability	Prediction at 0.50
A	0.82	Churn
B	0.67	Churn
C	0.49	No churn
D	0.23	No churn
E	0.08	No churn

With a threshold of 0.50:

• Probability ≥ 0.50 → Positive
• Probability < 0.50 → Negative

Therefore, customers A and B would be classified as likely to churn.

The important point is that the model produces a score or probability, while the threshold determines how that score becomes a decision.

Why Is 0.50 Not Always the Best Threshold?

The 0.50 threshold is convenient, but it is not necessarily optimal.

Imagine a disease screening model. Missing a person who actually has the disease could have serious consequences.

In that situation, you may prefer a lower threshold such as 0.20.

That means the model becomes more willing to classify someone as positive.

You will usually identify more true positives, but you may also generate more false positives.

The opposite situation can occur with a system where false alarms are expensive.

For example, suppose a bank automatically blocks transactions classified as fraudulent. A very low threshold could result in many legitimate customers having their transactions blocked.

You may therefore choose a higher threshold to reduce false positives.

This is why the best threshold depends on the cost of different types of errors.

How a Decision Threshold Affects Classification

Consider the following model predictions:

0.91
0.84
0.76
0.63
0.57
0.48
0.41
0.29
0.16
0.05

At a threshold of 0.50, the first five observations are classified as positive.

If you lower the threshold to 0.40, seven observations become positive.

If you increase it to 0.70, only three observations become positive.

So changing the threshold changes the number of:

• True positives
• False positives
• True negatives
• False negatives

This directly affects classification metrics.

Decision Threshold and Precision

Precision measures how many observations predicted as positive were actually positive.

The formula is:

Precision = TP / (TP + FP)

where:

• TP = True Positives
• FP = False Positives

If you increase the decision threshold, the model becomes more selective about predicting the positive class.

This will often reduce false positives and increase precision.

For example, consider a fraud detection model.

At a threshold of 0.30:

• 200 transactions are flagged
• 100 are actually fraudulent

Precision:

100 / 200 = 0.50

At a threshold of 0.70:

• 100 transactions are flagged
• 80 are actually fraudulent

Precision becomes:

80 / 100 = 0.80

The higher threshold produced better precision.

However, this does not necessarily mean it is the better threshold.

Decision Threshold and Recall

Recall measures how many of the actual positive cases the model successfully identifies.

The formula is:

Recall = TP / (TP + FN)

where FN represents false negatives.

Lowering the decision threshold generally makes the model classify more observations as positive.

That can increase recall because the model is less likely to miss positive cases.

For example, a medical screening model might have:

Threshold = 0.70
Recall = 0.72

After lowering the threshold:

Threshold = 0.30
Recall = 0.94

The model now identifies more of the actual positive cases.

The tradeoff is that more healthy people may also be incorrectly classified as positive.

The Precision-Recall Tradeoff

Threshold selection is often about finding an acceptable balance between precision and recall.

A simplified pattern looks like this:

Threshold	Precision	Recall
0.20	0.42	0.96
0.30	0.51	0.92
0.40	0.61	0.86
0.50	0.70	0.78
0.60	0.78	0.67
0.70	0.86	0.54
0.80	0.92	0.39

These numbers are illustrative, but they demonstrate an important principle:

Lower thresholds usually favor recall, while higher thresholds usually favor precision.

There is no threshold that automatically wins in every situation.

Start With the Business Cost of Errors

Before calculating an “optimal” threshold, understand what happens when the model is wrong.

Consider a fraud detection system.

There are two important mistakes:

False Positive

A legitimate transaction is classified as fraudulent.

Potential consequences:

• Customer frustration
• Declined transactions
• Support costs
• Lost revenue

False Negative

A fraudulent transaction is classified as legitimate.

Potential consequences:

• Financial loss
• Chargebacks
• Security problems
• Customer complaints

If false negatives are much more expensive than false positives, you may want a lower threshold.

If false positives are more expensive, a higher threshold may make more sense.

This is often more useful than blindly maximizing a statistical metric.

Use a Validation Set to Choose the Threshold

One important rule is to avoid selecting the threshold using the same data used to train the model.

A typical workflow is:

Training data
     ↓
Train model
     ↓
Validation data
     ↓
Generate probabilities
     ↓
Test multiple thresholds
     ↓
Select threshold
     ↓
Evaluate once on test data

The validation set is used to make decisions such as:

• Which threshold should we use?
• Should we prioritize precision?
• Should we prioritize recall?
• What probability cutoff meets our business requirement?

The test set should remain untouched until the final evaluation.

How to Choose a Threshold Using Precision and Recall

Suppose your business requires recall of at least 90%.

You can test different thresholds and select the highest threshold that still achieves the required recall.

Why the highest threshold?

Because, among thresholds meeting the recall requirement, a higher threshold may provide better precision.

For example:

Threshold	Precision	Recall
0.20	0.45	0.97
0.30	0.55	0.94
0.40	0.65	0.91
0.50	0.72	0.86

If your minimum recall requirement is 90%, the 0.40 threshold is a reasonable candidate.

The 0.50 threshold fails the recall requirement.

Choosing a Threshold With F1 Score

If precision and recall are both important, you can use the F1 score.

The F1 score is the harmonic mean of precision and recall:

F1 = 2 × (Precision × Recall)
     / (Precision + Recall)

You can calculate the F1 score at different thresholds and select the threshold that produces the highest value.

However, maximizing F1 is not always the right business decision.

If missing a positive case is significantly worse than creating a false positive, you may prefer a threshold that prioritizes recall even if it produces a lower F1 score.

Choosing a Threshold With a Business Metric

Sometimes the best approach is to define a custom cost function.

Suppose:

Cost of False Positive = $5
Cost of False Negative = $100

A false negative is therefore much more expensive.

You could calculate the total cost at different thresholds:

Total Cost =
(FP × Cost of FP) +
(FN × Cost of FN)

Then select the threshold that minimizes the expected cost.

This approach can be much more meaningful than simply maximizing accuracy.

Example: Selecting a Threshold in Python

Let’s use a small binary classification example with scikit-learn.

import numpy as np
from sklearn.metrics import precision_score, recall_score, f1_score

y_true = np.array([
    0, 0, 1, 1, 1,
    0, 1, 0, 1, 0
])

y_probability = np.array([
    0.10, 0.25, 0.65, 0.80, 0.55,
    0.35, 0.70, 0.20, 0.90, 0.45
])

thresholds = np.arange(0.10, 1.00, 0.05)

results = []

for threshold in thresholds:
    y_pred = (y_probability >= threshold).astype(int)

    precision = precision_score(
        y_true,
        y_pred,
        zero_division=0
    )

    recall = recall_score(
        y_true,
        y_pred,
        zero_division=0
    )

    f1 = f1_score(
        y_true,
        y_pred,
        zero_division=0
    )

    results.append({
        "threshold": threshold,
        "precision": precision,
        "recall": recall,
        "f1": f1
    })

for result in results:
    print(result)

This code evaluates several possible thresholds rather than assuming that 0.50 is automatically correct.

You can then identify the threshold with the highest F1 score:

best_result = max(
    results,
    key=lambda x: x["f1"]
)

print(best_result)

Choosing a Threshold Based on Minimum Recall

Suppose your application requires recall of at least 90%.

You can filter the results:

valid_results = [
    result
    for result in results
    if result["recall"] >= 0.90
]

best_threshold = max(
    valid_results,
    key=lambda x: x["precision"]
)

print(best_threshold)

This approach says:

Find thresholds that satisfy the recall requirement, then choose the one with the best precision.

That is often more practical than simply selecting the threshold with the highest overall score.

Using a Precision-Recall Curve

Scikit-learn can also calculate precision and recall across different probability thresholds.

from sklearn.metrics import precision_recall_curve

precision, recall, thresholds = precision_recall_curve(
    y_true,
    y_probability
)

You can inspect how precision and recall change as the threshold changes.

For larger datasets, plotting the relationship can make threshold selection easier.

import matplotlib.pyplot as plt

plt.plot(thresholds, precision[:-1], label="Precision")
plt.plot(thresholds, recall[:-1], label="Recall")

plt.xlabel("Decision Threshold")
plt.ylabel("Score")
plt.title("Precision and Recall by Decision Threshold")
plt.legend()
plt.show()

The point is not simply to find a visually attractive threshold.

You should combine the curve with your actual business requirements.

What About Accuracy?

Accuracy is often a poor metric for threshold selection when classes are imbalanced.

Imagine a fraud dataset containing:

99,000 legitimate transactions
1,000 fraudulent transactions

A model that predicts every transaction as legitimate would achieve:

99,000 / 100,000 = 99% accuracy

That sounds impressive.

But the model detects zero fraudulent transactions.

This is why threshold selection should consider metrics such as:

• Precision
• Recall
• F1 score
• Specificity
• Balanced accuracy
• Expected business cost
• Expected revenue

The appropriate metric depends on the problem.

Threshold Selection for Imbalanced Classification

Decision thresholds become particularly important when the positive class is rare.

Examples include:

• Fraud detection
• Equipment failure prediction
• Disease screening
• Cybersecurity alerts
• Customer churn
• Loan default prediction

In these problems, the default 0.50 threshold may produce undesirable results.

For example, if only 1% of customers are likely to default, the probability distribution may be heavily skewed toward low values.

A threshold of 0.50 might be too conservative to identify enough high-risk customers.

Don’t Tune the Threshold on the Test Set

This is one of the most important mistakes to avoid.

Suppose you try:

Threshold 0.30 → F1 = 0.71
Threshold 0.40 → F1 = 0.74
Threshold 0.50 → F1 = 0.69
Threshold 0.60 → F1 = 0.66

If these numbers come from your test set and you choose 0.40 because it performs best, you have effectively used the test set to optimize your model.

Your final test score may therefore be overly optimistic.

Instead:

Train → Validation → Select threshold → Test

Keep the test set for the final unbiased evaluation.

Threshold Selection vs Model Training

Changing the decision threshold does not necessarily retrain the machine learning model.

Suppose a model outputs:

0.73
0.61
0.48
0.31

You can change the threshold from 0.50 to 0.40 without retraining the model.

The probabilities remain the same.

Only the final classification changes.

This distinction is important:

Model training learns the relationship between features and predictions.

Threshold selection determines how those predictions are converted into decisions.

What If the Model Probabilities Are Poorly Calibrated?

Threshold selection assumes that model scores or probabilities provide useful information about risk.

However, some classification models can produce poorly calibrated probabilities.

For example, if a model predicts:

Probability = 0.80

that does not automatically mean that approximately 80% of similar cases will be positive.

Calibration methods can help make predicted probabilities more meaningful.

Common approaches include:

• Platt scaling
• Isotonic regression
• Calibration curves

This can be particularly important when probabilities themselves are used for decision-making.

Common Mistakes When Choosing a Decision Threshold

1. Always using 0.50

The default threshold is convenient, but it may not match your application’s requirements.

2. Optimizing accuracy alone

Accuracy can hide serious problems with imbalanced datasets.

3. Ignoring false negatives

In healthcare, fraud, cybersecurity, and other applications, false negatives can be extremely costly.

4. Ignoring false positives

Too many false positives can overwhelm human reviewers or negatively affect customers.

5. Selecting the threshold on the test set

Use validation data for threshold selection and reserve the test set for final evaluation.

6. Choosing a threshold without understanding the business

A statistically optimal threshold may not be operationally optimal.

7. Assuming the threshold will remain optimal forever

Data distributions can change.

If customer behavior, fraud patterns, or other real-world conditions change, the threshold may need to be reviewed.

A Practical Decision Threshold Workflow

A reliable threshold-selection process can look like this:

Step 1: Train the model

Train your classifier using the training dataset.

Step 2: Generate probabilities

Use the validation set to obtain probability predictions.

probabilities = model.predict_proba(X_validation)[:, 1]

Step 3: Define the objective

Decide what matters most.

For example:

Minimum recall = 90%

or:

Minimize financial cost

or:

Maximize F1 score

Step 4: Test multiple thresholds

Evaluate thresholds such as:

0.10
0.15
0.20
...
0.90

Step 5: Select the threshold

Choose the threshold that satisfies your business and statistical requirements.

Step 6: Lock the threshold

Once selected, treat it as part of your model’s production configuration.

Step 7: Evaluate on the test set

Run the final model and selected threshold on previously unseen test data.

Step 8: Monitor performance

Track metrics after deployment and review the threshold when the underlying data or business costs change.

Choosing a machine learning decision threshold is not simply a matter of setting the cutoff to 0.50.

The right threshold depends on what your model is being used for and the consequences of making different types of mistakes.

A lower threshold generally allows the model to identify more positive cases, improving recall while potentially increasing false positives. A higher threshold usually makes the model more selective, which can improve precision while reducing recall.

For practical machine learning projects, start by understanding the cost of false positives and false negatives. Then use a validation dataset to test different thresholds and select one based on the metric or business objective that actually matters.

The most important lesson is simple:

Don’t ask, “What is the standard threshold?” Ask, “What decision does this threshold need to support?”

That question will usually lead to a much better machine learning system.

Frequently Asked Questions