SQLite DB Migrations with PRAGMA user_version
This blog used a simple homegrown blogging engine that I wrote backed by a SQLite database. I have a function in the flask app that performs database migrations. My current approach has been to keep a folder full of migrations and run them sequentially whenever the app starts.
This works well for the case of adding and removing tables since SQLite has the handy IF NOT EXISTS
option. However, when you are altering an existing table, this entire model falls apart since IF NOT EXISTS
no longer works.
Practically, this means that outside of a fresh install my database migrations are useless.
I am still being stubborn and not using a well written solution like Alembic (which I would highly recommend for a “serious” project) for this blog. Instead, I discovered that SQLite comes with a built in mechanism to keep track of the user schema. This is the pragmastatement, and specifically user_version.
Using PRAGMA user_data for DB Migrations
My migrations folder structure looks like this:. ├── blog.db ├── blog.py ├── __init__.py ├── migrations │ ├── 0001_initial_schema.sql │ ├── 0002_add_unique_index_to_posts_tags.sql │ ├── 0003_add_fts.sql │ ├── 0004_add_column_to_post.sql │ ├── 0005_add_comments_table.sql │ └── 0006_add_admin_flag_to_comments.sql
000N_migration_description.sql
. Each migration file has the following statement in it:
PRAGMA user_version=N; (where N is the 000"N" part of the file name)
user_version
to be equal to the current version as defined by the file name.
The code to do stuff with the database is shown below:
def connect_db():
"""Connects to Database."""
rv = sqlite3.connect(
app.config['DATABASE'],
detect_types=sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES)
rv.row_factory = sqlite3.Row
return rv
def get_db():
"""Opens new db connection if there is not an
existing one for the current app ctx.
"""
if not hasattr(g, 'sqlite_db'):
g.sqlite_db = connect_db()
return g.sqlite_db
def migrate_db():
"""Run database migrations."""
def get_script_version(path):
return int(path.split('_')[0].split('/')[1])
db = get_db()
current_version = db.cursor().execute('pragma user_version').fetchone()[0]
directory = os.path.dirname(__file__)
migrations_path = os.path.join(directory, 'migrations/')
migration_files = list(os.listdir(migrations_path))
for migration in sorted(migration_files):
path = "migrations/{0}".format(migration)
migration_version = get_script_version(path)
if migration_version > current_version:
print("applying migration {0}".format(migration_version))
with app.open_resource(path, mode='r') as f:
db.cursor().executescript(f.read())
print("database now at version {0}".format(migration_version))
else:
print("migration {0} already applied".format(migration_version))
The relevant part to this blog post is the migrate_db()
function. Two things are happening.
- The
get_script_version()
helper function extracts the integer from the migration name. current_version
gets the current value ofuser_version
of your database.- We iterate over each migration file in the
migrations
folder and perform a simple check. If the migration version is larger than the current_version we run the migration, otherwise it gets skipped.
Thank you for reading! Share your thoughts with me on bluesky, mastodon, or via email.
Check out some more stuff to read down below.
Most popular posts this month
- 2024
- Reinstalling Windows at 1am
- SQLite DB Migrations with PRAGMA user_version
- My Custom Miniflux CSS Theme
- How to Disable Wayland in Debian Testing
Recent Favorite Blog Posts
This is a collection of the last 8 posts that I bookmarked.
- Give Your Spouse the Gift of a Couple's Email Domain from mtlynch.io
- Skip the Next iPhone from Articles on Jose M.
- Have smart glasses finally hit an inflection point? from The Torment Nexus
- The McPhee method from the jsomers.net blog
- Pluralistic: LLMs are slot-machines (16 Aug 2025) from Pluralistic: Daily links from Cory Doctorow
- Pluralistic: Bluesky creates the world's weirdest, hardest-to-understand binding arbitration clause (15 Aug 2025) from Pluralistic: Daily links from Cory Doctorow
- Just a Little More Context Bro, I Promise, and It’ll Fix Everything from Jim Nielsen’s Blog
- The Futzing Fraction from Deciphering Glyph
Articles from blogs I follow around the net
Futureverse announces restructuring two years after raising $54 million
In 2023, there was no shortage of buzzy press coverage for Futureverse, which promised to build a metaverse and gaming-focused blockchain. They partnered with Ready Player One author Ernest Cline to build the "Readyverse". Th…
via Web3 is Going Just Great September 30, 2025New site, kinda
If you’re reading this blog using RSS or via email (when I remember to send the content via email), you likely didn’t notice it. And if you’re reading my blog in the browser but are not a sharp observer, chances are, you also didn’t notice it. A n…
via Manuel Moreale — Everything Feed September 30, 2025Pluralistic: Announcing the Enshittification tour (30 Sep 2025)
Today's links Announcing the Enshittification tour: Come say hi, why dontcha? Hey look at this: Delights to delectate. Object permanence: HP defeat device; It Gets Better; OPM hack endangers CIA in Beijing; Self-driving cars crash. Upcoming appearance…
via Pluralistic: Daily links from Cory Doctorow September 30, 2025Generated by openring