Skip to content

iloveitaly/python-starter-template

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Python & React Router Project Template

This is an extremely opinionated web application template:

  1. Full-stack integration tests. This includes HTTPS and production-build JS on CI.
  2. Eliminate magic commands. Env vars, developer environments, infra, etc should all be documented in code.
  3. Containerized builds.
  4. Full-stack typing.
  5. Use boring core technology. No fancy databases, no novel languages, no obscure frameworks.

Tech Stack

Here's the stack:

  • Development lifecycle. Justfile + Direnv + Mise + Lefthook + Localias + 1Password for local development & secret configuration
  • Backend. Uv + Ruff + Python + FastAPI + ActiveModel + SQLModel + SQLAlchemy + Alembic + Celery + TypeId + Playwright + Mailers
  • Frontend. Pnpm + TypeScript + React + Vite + Vitest + React Router (in SPA mode) + ShadCN + Tailwind + ESLint + Prettier + HeyAPI
  • Services. Postgres + Redis + Mailpit
    • Docker Compose for running locally and on CI
    • Mailpit for local email testing
    • OrbStack is recommended for local docker development for nice automatic domains
  • Observability. Sentry + Clerk (user management) + PostHog
  • Build. Docker + nixpacks for containerization
  • CI/CD. GitHub Actions for CI/CD
  • Deployment. Up to you: anything that supports a container.

Cost of Complexity

This is not a simple application template.

There are many things I don't like about this setup. There's a complexity cost and I'm not sure if it's worth it. It's definitely not for the faint of heart and solves very specific problems I've experienced as codebases and teams grow.

Modern web development is all about tradeoffs. Here are the options as I see them:

  1. Use Rails, HotWire, etc.
    1. You lose React, all of the amazing ui libraries that come with it, the massive JS + Py labor market, great tooling (formatting, linting, etc), typing (Sorbet is not great), 1st class SDKs for many APIs (playwright, for example), and better trained LLMs.
    2. You get a battle-tested (Shopify! GitHub!) beautifully crafted batteries-included framework.
    3. You get a really beautiful language (I think Ruby is nicer than Python).
  2. Use full stack JavaScript/TypeScript.
    1. You have to work with JavaScript everyday. I've given up on backend JavaScript. The whole ecosystem is a mess and I don't enjoy working in it. Non-starter for me. A personal preference and hill I'll die on.
    2. You get access to a massive labor market, great tooling, typing, and well-trained LLMs.
  3. Use Python & React.
    1. You lose simplicity. You have to deal with two languages, which means more complex build systems and additional cognitive load.
    2. You lose a single beautifully crafted stack and instead have to stitch together a bunch of different tools (even if they are well-designed independently). Python's ecosystem is mature but not cohesive like Rails (no, Django is not even close).
    3. You get full-stack typing (if you do it right).
    4. You get access to the great tooling (static analysis and improved LLM performance) on both Python and JavaScript.
    5. You can move fast with React and all of the amazing UI libraries built on top of it, without having to deal with full stack JavaScript.
    6. You get access to massive JS + Py labor markets.

This template uses #3.

Getting Started

There are a couple of dependencies which are not managed by the project:

  • zsh. The stack has not been tested extensively on bash.
  • mise
  • docker (or preferably OrbStack)
  • Latest macOS
  • VS Code

You can use a different setup.

Copier is the easiest way to get started:

mkdir your-project
uv tool run --with jinja2_shell_extension copier copy https://github.com/iloveitaly/python-starter-template . --trust

The neat thing about copier is you pull updates from this template later on if you'd like:

uv tool run --with jinja2_shell_extension copier update --trust --skip-tasks --skip-answered

If you want to skip updates in a particular directory ('web' in this case):

uv tool run --with jinja2_shell_extension copier update --trust --skip-tasks --skip-answered --exclude web

Once you've copied the template and have the above dependencies installed, you can run:

mise install
just setup

That should do most of what you need. Here are some bits you'll need to handle manually:

  • Note that you'll probably want to manually install the shell hooks for a couple tools which you will need to do manually.
  • If you use 1p for secrets, you'll need to set up the 1Password CLI.

Simple Mode

If you despise dev productivity tooling (no judgement!) you can:

  • Avoid using direnv.
  • Avoid tying into your local 1password.

Ask a friend who has the system fully configured to run:

just direnv_bash_export

You can simply source the resulting file when you create a new shell session.

The primary downside to this approach is:

  • Any mutated API keys from 1Password will not be automatically updated
  • Updated ENV configuration will not be automatically used
  • You cannot easily

Advanced Mode

There are some extra goodies available if you are adventurous:

just requirements --extras

Shell Completions

Here are a couple additions to your shell environment that you'll probably want to make:

Production

Frontend

  • window.SENTRY_RELEASE has the commit sha of the build.
  • devDependencies should only contain dependencies that are required for local development. All dependencies required for building the frontend should be in dependencies.

Usage

Naming Recommendations

General recommendations for naming things throughout the system:

  • Longer names are better than shorter names.
  • All lowercase GitHub organizations and names. Some systems are case sensitive and some are not.
  • All 1password fields should be hyphen-separated and not use spaces.

Pytest

Migrations

  1. SQLModel provides a way to push a model definition into the DB. However, once columns and not tables are mutated you must drop the table and recreate it otherwise changes will not take effect. If you develop in this way, you'll need to be extra careful.
  2. If you've been playing with a model locally, you'll want to just db_reset and then generate a new migration with just db_generate_migration. If you don't do this, your migration may work off of some of the development state.
  3. The database must be fully migrated before generating a new migration just db_migrate otherwise you will get an error.

Environment Lifecycle

Across py, js, and db the following subcommands are supported:

  • clean. Wipe all temporary files related to the environment.
  • setup. Set up the environment.
  • nuke. Clean & setup.
  • play. Interactive playground.
  • lint. Run linting.
  • lint-fix. Run linting and fix all errors.
  • test. Run tests.
  • dev. Run the development server.

There are top-level commands for many of these (clean, setup, dev, etc) which run actions for all environments.

Linting

More linting tools are better, as long as they are well maintained. This project implements many linting tools (including DB SQL linting!). This could cause developer friction at some point, but we'll see how this scales as the codebase complexity grows.

Database Cleaning

Implemented by activemodel which is a package created specifically for this project.

Two methods are needed:

  • Transaction. Used whenever possible. If you create customize secondary engines outside the context of activemodel this will break.
  • Truncation. Transaction-based cleaning does not work is database mutations occur in a separate process.
    • This gets tricky because of platform differences between macOS and Linux, but the tldr is although it's possible to share a DB session handle between the test process and the uvicorn server running during integration tests, it's a bad idea with lots of footguns, so we opt for truncation.

Testing Architecture

Here's how this project thinks about tests:

  • The test environment should mirror production as closely as possible and for higher level tests (integration or smoke tests) we should put in extra engineering effort to get there. Accept additional complexity in integration tests to mirror the production environment more closely.
  • All core application workflows should be covered with an integration test. I think of an integration test as a browser-based test using production-built javascript/HTML.
  • Most common flows should be covered by a functional test. I think of a functional test as e2e tests on specific API routes or jobs. Primarily testing backend logic and not interaction with the UI.
  • Unit tests should be used for tricky code or to validate regressions.

JavaScript/UI/Remix Tests

screen.debug() // Logs the DOM structure

React Router Routes

Here's a more complex example of how to define nested routes + layouts in react router 7.

The routes.ts is meant to be a simple way to define routes configuration at a high level. Unlike previous react router versions, loaders and other options are not available

import type { RouteConfig } from "@react-router/dev/routes"
import { index, layout, prefix, route } from "@react-router/dev/routes"

export default [
  index("routes/index.tsx"),
  layout("layouts/authenticated.tsx", [
    ...prefix("intake", [
      layout("layouts/intake.tsx", [
        index("routes/intake/index.tsx"),
        route("/:id", "routes/intake/detail.tsx"),
      ]),
    ]),
    ...prefix("notes", [
      layout("layouts/notes.tsx", [
        index("routes/notes/index.tsx"),
        route(":id", "routes/notes/detail.tsx"),
      ]),
    ]),
    route("/settings", "routes/settings.tsx"),
  ]),
] satisfies RouteConfig

We have a special handler in fastapi to serve the JS files when any route not defined explicitly by FastAPI is requested. A "better" (and more complex) way to handle this is to deploy JS assets to a CDN and use a separate subdomain for the API server. Keeping everything in a single docker container and application is easier on deployment.

Debugging Javascript Vite Build

Here's how to use debugger statements within vite code:

VITE_BUILD_COMMIT=-dirty node inspect web/node_modules/@react-router/dev/bin.js build

Pytest Integration Tests

To debug integration tests, you'll want to see what's happening. Use --headed to do this:

pytd tests/integration/user_creation_test.py -k signup --headed

Note that --headed does change a lot of underlying Chrome parameters which can impact how the browser interacts with the web page.

If a test fails, a screenshot and trace is generated that you can use to replay the session and quickly visually inspect what went wrong.

page.pause()

TypeID by Default

From my time at Stripe, I became a big fan of IDs with metadata about the object they represent. There's a great project, TypeID, which has implemented this across a bunch of languages. I've adopted it here.

Secrets

The secret management is more complex that it seems like it should be. The primary reason behind this is to avoid secret drift at all costs. Here's why:

  • Different developers have slightly different secrets that create a "works on my machine" problem.
  • New secrets are not automatically integrated into local (or CI) environments causing the project to break when a new environment variable is introduced.
  • Cumbersome and undocumented CI & production secret management. I want a single command to set secrets on CI & prod without any thinking or manual intervention.

Solving these problems adds more complexity to the developer experience. We'll see if it's worth it.

Dependencies

Non-language dependencies are always tricky. Here's how it's handled:

  • brew is used to install tools required by development scripts
  • mise is used for runtimes, package managers (uv, pnpm), and runners (direnv + just). Mise should be used to install any tools required for running build* and test* Justfile recipes but not tools that are very popular (like jq, etc) which are better installed by the native os package manager.
    • Note that asdf could probably be used instead of mise, but I haven't tested it and there's many Justfile recipes that use mise.
  • apt is used to install some of the tools required to run CI scripts (like zsh), but most are omitted since they should never run in production.
  • Direct install scripts are used to install some more obscure packages (localias, nixpacks, etc) that are not well supported by the os package manager or mise.

DevProd

Here are the devprod principles this project adheres to:

  • There should be no secret scripts on a developers machine. Everything to setup and manage a environment should be within the repo. I've chosen a Justfile for this purpose.
  • The entire lifecycle of a project should be documented in code. Most of the time, this means Justfile recipes.

Deployment

  • Containers should be used instead of proprietary build systems (Heroku, etc).
  • Ability to build production-identical containers locally for debugging
  • Container building and registry storage should be handled on CI. This reduces vendor lock in.

Python Job Queue

RQ

I really liked the idea of RQ as a job queue. The configuration and usage seemed much more simple.

However, it didn't work for me.

At this point, I gave up trying and switched to celery. The big downside with celery is there's no way to run Flower inside an existing application (you can't bolt it on to a fastapi server) so you'll need an entirely separate container running the flower application.

Here's the Procfile command that worked for RQ:

worker: rq worker --with-scheduler -w rq.worker.SpawnWorker

I've left some of the configuration around in case you want to try it out.

Celery

Related

Other Templates

Great Open Source Projects

Great for grepping and investigating patterns.

Python

Javascript/Remix/React Router