python

How Can You Effortlessly Manage Multiple Databases in FastAPI?

Navigating the Multiverse of Databases with FastAPI: A Tale of Configuration and Connection

How Can You Effortlessly Manage Multiple Databases in FastAPI?

Let’s dive into the world of managing multiple databases in a FastAPI project. It may sound like a maze at first, but with a sprinkle of the right tools and steps, we can make it a breeze.

First things first, we need to get a few libraries on board. For this, you’ll want to grab FastAPI, SQLAlchemy, and alembic. These are our main players, helping us drive database migrations and object-relational mapping in Python.

pip install fastapi sqlalchemy alembic

If MySQL is your database of choice, make sure to add the mysql-connector-python library too.

pip install mysql-connector-python

Now, let’s get started with configuring our database connections. For multiple databases, we need to set up separate connection strings for each. We can keep this tidy in our configuration file.

from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

# Database configs
db1_config = {
    'username': 'username1',
    'password': 'password1',
    'ip': 'localhost',
    'name': 'db1',
    'port': '3306'
}

db2_config = {
    'username': 'username2',
    'password': 'password2',
    'ip': 'localhost',
    'name': 'db2',
    'port': '3306'
}

# Create engines for each database
engine1 = create_engine(f'mysql+mysqlconnector://{db1_config["username"]}:{db1_config["password"]}@{db1_config["ip"]}:{db1_config["port"]}/{db1_config["name"]}')
engine2 = create_engine(f'mysql+mysqlconnector://{db2_config["username"]}:{db2_config["password"]}@{db2_config["ip"]}:{db2_config["port"]}/{db2_config["name"]}')

# Session makers
Session1 = sessionmaker(bind=engine1)
Session2 = sessionmaker(bind=engine2)

# Base model classes
Base1 = declarative_base()
Base2 = declarative_base()

Once the connection setup is done, we move on to defining our database models. Each model should inherit from the Base class we defined for each database.

from sqlalchemy import Column, Integer, String

# Models for Database 1
class User1(Base1):
    __tablename__ = "users1"
    id = Column(Integer, primary_key=True, index=True)
    name = Column(String, index=True)
    email = Column(String, unique=True, index=True)


# Models for Database 2
class User2(Base2):
    __tablename__ = "users2"
    id = Column(Integer, primary_key=True, index=True)
    name = Column(String, index=True)
    email = Column(String, unique=True, index=True)

One of the key features of FastAPI is its dependency injection system, which simplifies the management of database sessions. This lets us define functions that yield database sessions on a per-request basis.

def get_db1():
    db = Session1()
    try:
        yield db
    finally:
        db.close()

def get_db2():
    db = Session2()
    try:
        yield db
    finally:
        db.close()

Now we can use these dependency functions in our FastAPI routes to interact with the corresponding databases.

from fastapi import FastAPI, Depends
from sqlalchemy.orm import Session

app = FastAPI()

@app.get("/users1/")
def read_users1(db: Session = Depends(get_db1)):
    users = db.query(User1).all()
    return users

@app.get("/users2/")
def read_users2(db: Session = Depends(get_db2)):
    users = db.query(User2).all()
    return users

Sometimes, you might need to dynamically choose which database to interact with based on the request details. Middleware can save the day here. It can inspect requests and set the right database connection accordingly.

from fastapi import FastAPI, Request
from fastapi.middleware.base import BaseHTTPMiddleware

class DynamicDBMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        if "api1" in str(request.url):
            request.state.db = get_db1()
        else:
            request.state.db = get_db2()
        response = await call_next(request)
        return response

app.add_middleware(DynamicDBMiddleware)

When it comes to database migrations, Alembic is the tool of choice. Initialize Alembic in your project to get started:

alembic init alembic

Then, configure Alembic to be aware of your multiple databases in its env.py file.

from sqlalchemy import engine_from_config, pool
from alembic import context
from config import db1_config, db2_config

config = context.config
fileConfig(config.config_file_name)

target_metadata = [Base1.metadata, Base2.metadata]

def run_migrations(ctx, engines):
    for engine in engines:
        with engine.connect() as connection:
            ctx.configure(connection=connection, target_metadata=target_metadata)
            ctx.run_migrations()

def run_migrations_offline():
    run_migrations(context, [engine_from_config(config.get_section('db1'), prefix='sqlalchemy.'), engine_from_config(config.get_section('db2'), prefix='sqlalchemy.')])

def run_migrations_online():
    engines = [
        engine_from_config(db1_config, prefix='sqlalchemy.'),
        engine_from_config(db2_config, prefix='sqlalchemy.')
    ]
    run_migrations(context, engines)

if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()

And there you go. Managing multiple databases in a FastAPI project doesn’t have to be daunting. By setting up clear configurations and using FastAPI’s dependency injection system and middleware, the complexity reduces significantly. Tools like Alembic help maintain database schema consistency across your different databases, making your application scalable and maintainable. This structured approach not only makes database management more efficient but also ensures your application can smoothly handle different scenarios and environments.

Keywords: FastAPI, SQLAlchemy, alembic, multiple databases, MySQL, dependency injection, database migration, dynamic database, middleware, database configuration



Similar Posts
Blog Image
Going Beyond Decorators: Creating a Custom Python Annotation System

Custom annotations in Python enhance code functionality, adding metadata and behavior. They enable input validation, performance monitoring, and code organization, acting like superpowers for your functions and classes.

Blog Image
Ready to Build APIs Faster than The Flash?

Harness Speed and Scalability with FastAPI and PostgreSQL: The API Dream Team

Blog Image
Ready to Supercharge Your API Game with FastAPI and GraphQL?

Harnessing FastAPI and GraphQL for High-Performance, Flexible Web APIs Using Strawberry

Blog Image
Is Your FastAPI App Missing the Magic of CI/CD with GitHub Actions?

FastAPI Deployment: From GitHub Actions to Traefik Magic

Blog Image
Can Python Really Tame an Elephant-Sized Dataset?

Navigating Gargantuan Data in Python Without Going Bonkers

Blog Image
Mastering Python's Abstract Base Classes: Supercharge Your Code with Flexible Inheritance

Python's abstract base classes (ABCs) define interfaces and behaviors for derived classes. They ensure consistency while allowing flexibility in object-oriented design. ABCs can't be instantiated directly but serve as blueprints. They support virtual subclasses, custom subclass checks, and abstract properties. ABCs are useful for large systems, libraries, and testing, but should be balanced with Python's duck typing philosophy.