How to Deploy a Machine Learning Model Using FastAPI

Have you ever trained a machine learning model, felt great about the accuracy, and then realized you had absolutely no idea how to let anyone else actually use it?

That moment is where a lot of data professionals hit a wall. Training a model in a Jupyter notebook is one skill. Making that model available to a web application, a dashboard, or a business tool is a completely different one. The gap between a trained model and something people can interact with is deployment, and for Python developers, FastAPI is one of the cleanest ways to close that gap.

FastAPI lets you wrap your trained model in a REST API with just a few dozen lines of code. Once it is running, any application that can send an HTTP request can get predictions from your model in real time. No more manual predictions in a notebook. No more emailing CSV files back and forth. Your model becomes a live service that responds in milliseconds.

This guide walks through the entire process from scratch, with working code at every step.

What Is FastAPI and Why Use It for ML Deployment?

FastAPI is a modern Python web framework built specifically for creating APIs quickly and cleanly. It is built on top of Starlette for the web layer and Pydantic for data validation, and it uses Python type hints to automatically generate request validation and interactive documentation.

Think of it this way. You bake a cake in your kitchen, but if you want to sell slices to customers you need a counter, a menu, and a way for people to place orders. FastAPI is the counter and the ordering system. Your trained model is the cake. The API is what lets customers order and receive their slice without ever stepping into your kitchen.

For machine learning specifically, FastAPI has three real advantages over alternatives like Flask. It is significantly faster at handling concurrent requests because it is built on async Python. It automatically validates incoming data against your schema and returns clear error messages when something does not match. And it generates interactive documentation at /docs the moment your server starts, so you can test your endpoints from a browser without writing any extra code.

Setting Up Your Environment

Before writing any code, install the required packages:

pip install fastapi uvicorn scikit-learn pandas numpy pydantic

FastAPI handles the API logic. Uvicorn is the ASGI server that actually runs your FastAPI application and listens for incoming requests. Scikit-learn is used here to train and save the example model, though this approach works identically with models trained using XGBoost, LightGBM, or any other library that can be serialized to a file.

Step by Step: Deploy Your First ML Model With FastAPI

Step 1: Train and Save Your Model

First, train a simple model and save it to disk using joblib. This example uses a logistic regression classifier on the Iris dataset, but the pattern is the same regardless of model type or dataset.

python

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
import joblib

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42
)

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

joblib.dump(model, 'iris_model.pkl')
print('Model saved.')

joblib.dump() serializes your trained model object into a file called iris_model.pkl. This file is what your FastAPI application will load when it starts up, so it does not retrain the model on every request. The model trains once and serves forever, or until you replace the file with a retrained version.

Step 2: Define Your Input Schema With Pydantic

Before building the API endpoint, define what a valid prediction request looks like. Pydantic BaseModel handles this. Every field gets a Python type annotation, and FastAPI uses that annotation to automatically validate incoming request data before it ever reaches your prediction code.

python

from pydantic import BaseModel

class IrisInput(BaseModel):
    sepal_length: float
    sepal_width: float
    petal_length: float
    petal_width: float

If someone sends a request with a string where a float is expected, or leaves a field out entirely, FastAPI returns a 422 Unprocessable Entity response with a clear explanation of what went wrong before your model ever sees the data. This is one of the biggest practical advantages over Flask, where you would have to write that validation logic yourself.

Step 3: Build the FastAPI Application

Now create the main application file. Call it main.py:

python

from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import numpy as np

app = FastAPI()

model = joblib.load('iris_model.pkl')

class IrisInput(BaseModel):
    sepal_length: float
    sepal_width: float
    petal_length: float
    petal_width: float

@app.get('/')
def root():
    return {'message': 'Iris classifier API is running'}

@app.post('/predict')
def predict(data: IrisInput):
    features = np.array([[
        data.sepal_length,
        data.sepal_width,
        data.petal_length,
        data.petal_width
    ]])
    prediction = model.predict(features)[0]
    species = ['setosa', 'versicolor', 'virginica']
    return {
        'prediction': int(prediction),
        'species': species[prediction]
    }

The model loads once when the application starts, not on every request. This matters for performance. Loading a model file from disk on every prediction request would make your API extremely slow. Loading it at startup means the model stays in memory and every prediction runs against the already-loaded object.

The /predict endpoint receives the validated input, converts it to a numpy array in the shape the model expects, calls model.predict(), and returns the result as JSON.

Step 4: Run the API Server

Start the server from your terminal:

uvicorn main:app --reload

main refers to the filename main.py. app is the FastAPI instance you created inside that file. –reload tells uvicorn to restart the server automatically whenever you save changes to your code, which is useful during development but should be removed in production.

Once the server starts, open a browser and go to http://127.0.0.1:8000/docs. FastAPI has automatically generated an interactive Swagger UI where you can test your /predict endpoint directly from the browser without writing a single line of client code.

Step 5: Send a Prediction Request

Send a request programmatically using Python:

python

import requests

payload = {
    'sepal_length': 5.1,
    'sepal_width': 3.5,
    'petal_length': 1.4,
    'petal_width': 0.2
}

response = requests.post('http://127.0.0.1:8000/predict', json=payload)
print(response.json())

The response comes back as:

{'prediction': 0, 'species': 'setosa'}

Your model is now a live API. Any application that can send a POST request with a JSON body can get predictions from it in real time.

How FastAPI ML Deployment Works Internally

When uvicorn starts, it loads your main.py file from top to bottom. The model loads into memory at that point and stays there for the lifetime of the server process. When a POST request arrives at /predict, FastAPI reads the request body, validates it against your IrisInput schema using Pydantic, and only calls your predict function if validation passes. Your function runs, numpy formats the features, model.predict() runs the inference, and FastAPI serializes the return value as a JSON response.

The entire round trip from request to response for a simple model like this takes under 10 milliseconds on a local machine.

Endpoint Types and When to Use Each

Endpoint Type	HTTP Method	Use Case	Example
Single prediction	POST /predict	One input, one result	Classify one customer
Batch prediction	POST /predict/batch	List of inputs, list of results	Score 1000 leads at once
Health check	GET /health	Verify server is running	Used by load balancers
Model info	GET /model/info	Return model version or metadata	Audit what model is live

Common Limitations

No built-in model versioning. FastAPI handles serving, not model lifecycle management. If you retrain and replace iris_model.pkl while the server is running, the old model stays loaded in memory until the server restarts. For production deployments where you update models frequently, you need a strategy for versioning model files and reloading them without downtime.

Single process by default. Running uvicorn with one worker means all prediction requests share one Python process. For high traffic you need either multiple workers with uvicorn main:app –workers 4 or a container orchestration system like Kubernetes to run multiple replicas.

No authentication out of the box. The /predict endpoint is open to anyone who can reach it. For anything beyond a local demo you should add API key authentication or OAuth2, both of which FastAPI supports natively through its security utilities.

Large models take time to load on startup. A model that is several hundred megabytes takes a few seconds to load from disk when the server starts. In serverless environments where servers spin up on demand this can add latency to the first request after a cold start.

Common Mistakes to Avoid

Loading the model inside the predict function. Calling joblib.load() inside the endpoint function reloads the model from disk on every single request. This is one of the most common beginner mistakes and it tanks performance immediately. Always load the model once at startup, outside any function definition.

Forgetting to convert input to the right numpy shape. model.predict() expects a 2D array where each row is one sample. Passing a 1D array of features causes a shape error. Always wrap your features in a list of lists: np.array([[f1, f2, f3, f4]]) produces shape (1, 4) which is correct.

Not handling prediction errors. If the model receives unexpected values or the input triggers an error, an unhandled exception returns a 500 Internal Server Error with no useful information. Wrap your prediction logic in a try/except block and return a structured error response.

Running with –reload in production. The –reload flag is for development only. In production it causes unnecessary file watching overhead. Remove it and manage server restarts through your deployment infrastructure.

FastAPI ML Deployment Cheat Sheet

Task	Code
Install dependencies	pip install fastapi uvicorn scikit-learn joblib
Save model	joblib.dump(model, ‘model.pkl’)
Load model at startup	model = joblib.load(‘model.pkl’)
Define input schema	class Input(BaseModel): field: float
Create prediction endpoint	@app.post(‘/predict’) def predict(data: Input)
Format features for predict	np.array([[data.f1, data.f2, data.f3]])
Run server	uvicorn main:app –reload
View auto-generated docs	http://127.0.0.1:8000/docs
Run with multiple workers	uvicorn main:app –workers 4
Test endpoint with Python	requests.post(url, json=payload)

Deploying a machine learning model with FastAPI takes your work from a notebook experiment to something people can actually use. The core pattern is always the same: save the trained model with joblib, load it once at startup, define your input schema with a Pydantic BaseModel, create a POST endpoint that formats the input and calls model.predict(), and run the server with uvicorn.

That pattern works for logistic regression and it works for gradient boosting models with hundreds of features. The complexity of your model does not change how you serve it. What changes is the input schema, the feature engineering you apply before calling predict, and how you format the output before returning it.

Start with the single prediction endpoint from this guide. Get it running locally and test it from the /docs page. Then add a health check endpoint, then a batch prediction endpoint, then authentication. Each addition is a small isolated change to a working foundation. Once the API works locally, containerizing it with Docker and deploying to Render, Railway, or AWS App Runner takes it from your laptop to a public URL that anyone can query.

FAQs