101 FastAPI Tips by The FastAPI Expert
This repository contains trips and tricks for FastAPI. If you have any tip that you believe is useful, feel free to open an issue or a pull request.
Consider sponsor me on GitHub to support my work. With your support, I will be able to create more content like this.
Tip
Remember to watch this repository to receive notifications about new tips.
By default, Uvicorn doesn't comes with uvloop and httptools which are faster than the default
asyncio event loop and HTTP parser. You can install them using the following command:
pip install uvloop httptoolsUvicorn will automatically use them if they are installed in your environment.
Warning
uvloop can't be installed on Windows. If you use Windows locally, but Linux on production, you can use
an environment marker to not install uvloop on Windows
e.g. uvloop; sys_platform != 'win32'.
There's a performance penalty when you use non-async functions in FastAPI. So, always prefer to use async functions.
The penalty comes from the fact that FastAPI will call run_in_threadpool, which will run the
function using a thread pool.
Note
Internally, run_in_threadpool will use anyio.to_thread.run_sync to run the
function in a thread pool.
Tip
There are only 40 threads available in the thread pool. If you use all of them, your application will be blocked.
To change the number of threads available, you can use the following code:
import anyio
from contextlib import asynccontextmanager
from typing import Iterator
from fastapi import FastAPI
@asynccontextmanager
async def lifespan(app: FastAPI) -> Iterator[None]:
limiter = anyio.to_thread.current_default_thread_limiter()
limiter.total_tokens = 100
yield
app = FastAPI(lifespan=lifespan)You can read more about it on AnyIO's documentation.
Most of the examples you will find on the internet use while True to read messages from the WebSocket.
I believe the uglier notation is used mainly because the Starlette documentation didn't show the async for notation for a long time.
Instead of using the while True:
from fastapi import FastAPI
from starlette.websockets import WebSocket
app = FastAPI()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket) -> None:
await websocket.accept()
while True:
data = await websocket.receive_text()
await websocket.send_text(f"Message text was: {data}")You can use the async for notation:
from fastapi import FastAPI
from starlette.websockets import WebSocket
app = FastAPI()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket) -> None:
await websocket.accept()
async for data in websocket.iter_text():
await websocket.send_text(f"Message text was: {data}")You can read more about it on the Starlette documentation.
If you are using the while True notation, you will need to catch the WebSocketDisconnect.
The async for notation will catch it for you.
from fastapi import FastAPI
from starlette.websockets import WebSocket, WebSocketDisconnect
app = FastAPI()
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket) -> None:
await websocket.accept()
try:
while True:
data = await websocket.receive_text()
await websocket.send_text(f"Message text was: {data}")
except WebSocketDisconnect:
passIf you need to release resources when the WebSocket is disconnected, you can use that exception to do it.
If you are using an older FastAPI version, only the receive methods will raise the WebSocketDisconnect exception.
The send methods will not raise it. In the latest versions, all methods will raise it.
In that case, you'll need to add the send methods inside the try block.
Since you are using async functions in your application, it will be easier to use HTTPX's AsyncClient
instead of Starlette's TestClient.
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def read_root():
return {"Hello": "World"}
# Using TestClient
from starlette.testclient import TestClient
client = TestClient(app)
response = client.get("/")
assert response.status_code == 200
assert response.json() == {"Hello": "World"}
# Using AsyncClient
import anyio
from httpx import AsyncClient, ASGITransport
async def main():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
response = await client.get("/")
assert response.status_code == 200
assert response.json() == {"Hello": "World"}
anyio.run(main)If you are using lifespan events (on_startup, on_shutdown or the lifespan parameter), you can use the
asgi-lifespan package to run those events.
from contextlib import asynccontextmanager
from typing import AsyncIterator
import anyio
from asgi_lifespan import LifespanManager
from httpx import AsyncClient, ASGITransport
from fastapi import FastAPI
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
print("Starting app")
yield
print("Stopping app")
app = FastAPI(lifespan=lifespan)
@app.get("/")
async def read_root():
return {"Hello": "World"}
async def main():
async with LifespanManager(app) as manager:
async with AsyncClient(transport=ASGITransport(app=manager.app)) as client:
response = await client.get("/")
assert response.status_code == 200
assert response.json() == {"Hello": "World"}
anyio.run(main)Note
Consider supporting the creator of asgi-lifespan Florimond Manca via GitHub Sponsors.
Since not long ago, FastAPI supports the lifespan state, which defines a standard way to manage objects that need to be created at startup, and need to be used in the request-response cycle.
The app.state is not recommended to be used anymore. You should use the lifespan state instead.
Using the app.state, you'd do something like this:
from contextlib import asynccontextmanager
from typing import AsyncIterator
from fastapi import FastAPI, Request
from httpx import AsyncClient
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
async with AsyncClient(app=app) as client:
app.state.client = client
yield
app = FastAPI(lifespan=lifespan)
@app.get("/")
async def read_root(request: Request):
client = request.app.state.client
response = await client.get("/")
return response.json()Using the lifespan state, you'd do something like this:
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Any, TypedDict, cast
from fastapi import FastAPI, Request
from httpx import AsyncClient
class State(TypedDict):
client: AsyncClient
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[State]:
async with AsyncClient(app=app) as client:
yield {"client": client}
app = FastAPI(lifespan=lifespan)
@app.get("/")
async def read_root(request: Request) -> dict[str, Any]:
client = cast(AsyncClient, request.state.client)
response = await client.get("/")
return response.json()If you want to find the endpoints that are blocking the event loop, you can enable the AsyncIO debug mode.
When you enable it, Python will print a warning message when a task takes more than 100ms to execute.
Run the following code with PYTHONASYNCIODEBUG=1 python main.py:
import os
import time
import uvicorn
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def read_root():
time.sleep(1) # Blocking call
return {"Hello": "World"}
if __name__ == "__main__":
uvicorn.run(app, loop="uvloop")If you call the endpoint, you will see the following message:
INFO: Started server process [19319]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: 127.0.0.1:50036 - "GET / HTTP/1.1" 200 OK
Executing <Task finished name='Task-3' coro=<RequestResponseCycle.run_asgi() done, defined at /uvicorn/uvicorn/protocols/http/httptools_impl.py:408> result=None created at /uvicorn/uvicorn/protocols/http/httptools_impl.py:291> took 1.009 secondsYou can read more about it on the official documentation.
The BaseHTTPMiddleware is the simplest way to create a middleware in FastAPI.
Note
The @app.middleware("http") decorator is a wrapper around the BaseHTTPMiddleware.
There were some issues with the BaseHTTPMiddleware, but most of the issues were fixed in the latest versions.
That said, there's still a performance penalty when using it.
To avoid the performance penalty, you can implement a Pure ASGI middleware. The downside is that it's more complex to implement.
Check the Starlette's documentation to learn how to implement a Pure ASGI middleware.
If the function is non-async and you use it as a dependency, it will run in a thread.
In the following example, the http_client function will run in a thread:
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from httpx import AsyncClient
from fastapi import FastAPI, Request, Depends
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[dict[str, AsyncClient]]:
async with AsyncClient() as client:
yield {"client": client}
app = FastAPI(lifespan=lifespan)
def http_client(request: Request) -> AsyncClient:
return request.state.client
@app.get("/")
async def read_root(client: AsyncClient = Depends(http_client)):
return await client.get("/")To run in the event loop, you need to make the function async:
# ...
async def http_client(request: Request) -> AsyncClient:
return request.state.client
# ...As an exercise for the reader, let's learn a bit more about how to check the running threads.
You can run the following with python main.py:
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
import anyio
from anyio.to_thread import current_default_thread_limiter
from httpx import AsyncClient
from fastapi import FastAPI, Request, Depends
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[dict[str, AsyncClient]]:
async with AsyncClient() as client:
yield {"client": client}
app = FastAPI(lifespan=lifespan)
# Change this function to be async, and rerun this application.
def http_client(request: Request) -> AsyncClient:
return request.state.client
@app.get("/")
async def read_root(client: AsyncClient = Depends(http_client)): ...
async def monitor_thread_limiter():
limiter = current_default_thread_limiter()
threads_in_use = limiter.borrowed_tokens
while True:
if threads_in_use != limiter.borrowed_tokens:
print(f"Threads in use: {limiter.borrowed_tokens}")
threads_in_use = limiter.borrowed_tokens
await anyio.sleep(0)
if __name__ == "__main__":
import uvicorn
config = uvicorn.Config(app="main:app")
server = uvicorn.Server(config)
async def main():
async with anyio.create_task_group() as tg:
tg.start_soon(monitor_thread_limiter)
await server.serve()
anyio.run(main)If you call the endpoint, you will see the following message:
❯ python main.py
INFO: Started server process [23966]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
Threads in use: 1
INFO: 127.0.0.1:57848 - "GET / HTTP/1.1" 200 OK
Threads in use: 0Replace the def http_client with async def http_client and rerun the application.
You will not see the message Threads in use: 1, because the function is running in the event loop.
Tip
You can use the FastAPI Dependency package that I've built to make it explicit when a dependency should run in a thread.
