[Devops] Streamlined branching strategy for schema

[lowlydba]

♻️ This top comment is evergreen, and will evolve to match the current state of this effort.

---

Branching strategy V1

V1 - superseded by v2

> [!NOTE] > Branch names used below were chosen for clarity, but can be almost anything. Assume the requisite branch protections/settings will be configured to make all of this feasible.

This is a draft suggestion of a simpler, more automatable branching flow for the schema repository. Consider this as a starting point for changes, please call out concerns/questions.

Two branches

We maintain two long-lived branches:

• main, the integration branch. This is where all normal work lands. It is always in a green, releasable state, but it is not a snapshot of the last release - it tracks what the next release will look like.
• breaking, a parallel branch where breaking changes accumulate between releases. It stays continuously in sync with main so it never drifts. This is opposed to the current approach main_2026_02_12, where its a long-lived breaking branch per release.

graph LR
    A[feature/*] -->|squash merge\n nonbreaking PR| B[main]
    A -->|squash merge \nbreaking PR | C[breaking]
    B -->|CI auto-rebase\n on every merge| C
    C -->|merge at release\n + tag v1.x.x| B

Loading

How normal work flows

All regular work follows the same pattern:

1. Branch off main
2. Open a PR back to main
3. CI runs automatically and checks your changes are compatible with the breaking branch
4. If all checks passes and you are up-to-date with main, your PR is squash-merged — your branch history is yours to work however you like, but a single clean commit lands on main
5. Once merged, breaking is automatically rebased onto the new main

You never need to touch breaking directly.

flowchart TD
    A[PR opens to main] --> B[CI: dry-run check\nsimulate squash onto main\ntest breaking rebase]
    B --> C{conflict?}
    C -->|no| D[Squash merge to main]
    C -->|yes| E[Author rebases feature branch\nonto breaking,\npushes to own branch to pre-emptively resolve]
    E -->|re-run check| B
    D --> F[CI: auto-rebase breaking onto main force-push]
    F --> G{failsafe check: rebase fails?}
    G -->|yes| H[Open GitHub issue\nassign PR author]
    G -->|no| I[Done]

Loading

If your PR fails the compatibility check

Occasionally a change conflicts with work that's staged in breaking. If this happens, CI will post a comment on your PR explaining exactly what to do. The short version: you rebase your own feature branch onto breaking to resolve the conflict, push your branch, and the check re-runs automatically.

You are only ever asked to fix your own branch.

If your PR fails compatibility check post-merge

While unlikely, this failsafe check in CI will automagically open a GitHub issue and assign it to you. Until resolved, the breaking branch will be out of sync with main.

Breaking changes

If your work contains breaking changes, target breaking instead of main when you open your PR. Everything else works the same way — squash merge, CI checks, and so on.

Monthly release

Once a month, the breaking branch is merged into main and the result is tagged as the release. That's it. There's no separate release branch — the tag on main is the release.

By shifting merge conflicts left in our process, this should always result in a clean, simple merge (making it easy to automate more of the core release steps in the future).

flowchart LR
    A[breaking\n] -->|merge into main| B[main\nnow includes breaking changes]
    B -->|tag v1.x.0| C[release\ntagged commit on main]

Loading

What this means for contributors

• Always branch from main
• Target main for normal changes, breaking for breaking changes
• Use whatever commit style you like on your branch — only the final squashed commit lands on shared branches
• If CI flags a conflict, follow the instructions in the comment — it will tell you exactly what to run
• Never push directly to main or breaking

Branching Strategy V2

After discourse/feedback, see the comment below with a V2 iteration

Child issues on this ticket will implement phased changesets to bring us towards the desired state, allocating time and sequencing for still TBD decisions (i.e. around individual package versioning).

Overview

Streamlined branching strategy and release automation for the schema repository. Replaces the current dev → staging → main manual flow and per-release long-lived branches (e.g. main_2026_02_12) with a two-branch model, automated CI safety nets, and a single-action release trigger.

Two-branch model (end state)

• main - integration branch. All normal work lands here. Always releasable.
• vnext - accumulates major changes between releases. Stays continuously rebased onto main. Merges to main at release time.

Phases

Phase	Title	Status
Phase 0	Branch infrastructure	Ready to start
Phase 1	CI safety net	Ready to start after P0
Phase 2	Versioning & release-trigger logic	⚠️ Decisions required before starting
Phase 3	Package publish workflows	Starts after P2
Phase 4	Documentation polish	Starts after P3

Phase 0

Pure infrastructure, no code risk. Migrates dev → main as default branch, retargets open PRs, creates vnext, configures branch protections.

Phase 1

Adds automated PR compatibility checks (dry-run rebase of vnext), post-merge auto-rebase of vnext onto main, and an advisory PR target check.

Phase 2 — Versioning & release-trigger logic ⚠️

Defines the per-package version scheme (<major>.<minor> git-controlled, <patch> CI-computed from CodeArtifact) and the vnext→main release process. Requires team decisions before implementation:

• Baseline <major>.<minor> versions for all 12 packages
• Changelog approach (per-PR fragments, auto-generated from PR metadata, or manual)

Phase 3

Implements build + publish automation to CodeArtifact (dev + release repos) and public PyPI.

Phase 4

Editorial pass on docs/branching-strategy.md + docs/versioning.md. Adds diagrams, walkthroughs, and FAQ. No new procedures.

Open questions

• ❔ Do we need to resolve below Phase 2 questions before starting on Phase 0 & 1?
• ❓ Phase 2 - Baseline <major>.<minor> versions for all 12 packages in packages/* — what version scheme & how to define versions (see Phase 2 issue for initial suggestion)
• ❔ Phase 2 - Changelog approach

[lowlydba]

Victor Schappert (@vcschapp) Let me know your thoughts, especially as we transition to stable versions for schema, having a good flow for the branches/changes will make life easier.

[lowlydba]

We can discuss @ tomorrow's (4/8) schema meeting

[vcschapp]

Thanks for this John McCall (@lowlydba)! The explanation and diagramming is super crisp.

I think there's three areas where we probably need to add some detail and/or question some assumptions.

Primary areas

These are my primary areas for discussion.

1. We should explain how central data pipeline ("CDP") gets access to the schema in general.
2. We should explain how CDP gets early access to schema changes.
3. We should explain how public PyPI gets access to the schema, and what the relationship is between public PyPI's versions and CDP's internal early access versions.
4. While we might still land on needing monthly or monthly-ish releases, I'd like to at least question that assumption.

On item 2, early access for CDP, one option we could consider is that every commit merged on the "breaking" branch results in auto-publishing an internal version of (the affected package(s)|all packages) to the internal CodeArtifact repo. We could do something like take the existing base version number, e.g., v1.2.3; auto-bump up the patch version, e.g., v1.2.4 to generate a version number that is strictly higher; then add an -unstable or -internal suffix; and if that conflicts with a previously auto-published breaking version just overwrite the previous one, as in v1.2.4-internal. Or add an auto-bumped suffix so you get v1.2.4-internal.1, v1.2.4-internal.2, etc. Central data pipeline could then consume these by taking from the internal CodeArtifact and setting prerelease = "allow".

Secondary areas

1. Small terminology point. We're trying to use the terms "major" and "minor" (which themselves might be wrong) rather than "breaking" and "non-breaking" because pretty much any change at the schema level seems to break someone.
2. We are planning to version each Python package independently. It would be good to touch on that plan in the proposal and address whether it affects anything.

Tertiary areas

With the Python version of the schema and a good rigorous definition of "major" and "minor" we can probably write a CI/CD step that auto-classifies the theme-level packages based on breaking-ness.

[lowlydba]

Victor Schappert (@vcschapp) Thanks so much for the detailed feedback. This is getting a bit long now, let me know if this being just an Issue becomes obtuse (we could switch to a doc, RFD, etc.). I'll be attending next week's schema wg meeting as well.

Here's a "v2" of my initial context above. What I am excited about is that it is drawing on other stabilizing / standardizing work done recently, so this can also help bring schema into broader parity with how we develop across other repos and groups. An implicit goal here for me will be to ensure this is all well documented at both the schema and broader OMF levels.

Summary of changes:

• Using vnext instead of breaking for branch name, otherwise using major/minor terms instead of breaking/non-breaking
• Added publishing & versioning sections (🆕)
• Release cadence description is more vague intentionally (I believe planning for release capacity instead of specific cadences makes for better processes, so I like that you called this out)

---

V2 Branching & Release Strategy

Branching strategy

[!NOTE]
Branch names used below were chosen for clarity, but can be almost anything. Assume the requisite branch protections/settings will be configured to make all of this feasible.

This is a draft suggestion of a simpler, more automatable branching flow for the schema repository. Consider this as a starting point for changes, please call out concerns/questions.

• Branching strategy
  • Two branches
  • How normal work flows
  • Release cadence 🆕
  • Dev versioning for internal consumers 🆕
  • Public PyPI releases 🆕
  • CDP access to schema 🆕
  • What this means for contributors

Two branches

We maintain two long-lived branches:

• main, the integration branch. This is where all normal work lands. It is always in a green, releasable state, but it is not a snapshot of the last release - it tracks what the next release will look like.
• vnext, a parallel branch where major changes accumulate between releases. It stays continuously in sync with main so it never drifts. This is opposed to the current approach main_2026_02_12, where its a long-lived major branch per release.

graph LR
    A[feature/*] -->|squash merge<br/>minor PR| B[main]
    A -->|squash merge<br/>major PR | C[vnext]
    B -->|CI auto-rebase<br/>on every merge| C
    C -->|merge at release<br/>+ tag v1.x.x| B

Loading

How normal work flows

All regular work follows the same pattern:

1. Branch off main
2. Open a PR back to main
3. CI runs automatically and checks your changes are compatible with the vnext branch
4. If all checks passes and you are up-to-date with main, your PR is squash-merged — your branch history is yours to work however you like, but a single clean commit lands on main
5. Once merged, vnext is automatically rebased onto the new main

You never need to touch vnext directly.

flowchart TD
    A[PR opens to main] --> B[CI: dry-run check<br/>simulate squash onto main<br/>test vnext rebase]
    B --> C{conflict?}
    C -->|no| D[Squash merge to main]
    C -->|yes| E[Author rebases feature branch<br/>onto vnext,<br/>pushes to own branch to pre-emptively resolve]
    E -->|re-run check| B
    D --> F[CI: auto-rebase vnext onto main force-push]
    F --> G{failsafe check: rebase fails?}
    G -->|yes| H[Open GitHub issue<br/>assign PR author]
    G -->|no| I[Done]

Loading

If your PR fails the compatibility check

Occasionally a change conflicts with work that's staged in vnext. If this happens, CI will post a comment on your PR explaining exactly what to do. The short version: you rebase your own feature branch onto vnext to resolve the conflict, push your branch, and the check re-runs automatically.

You are only ever asked to fix your own branch.

If your PR fails compatibility check post-merge

While unlikely, this failsafe check in CI will automagically open a GitHub issue and assign it to you. Until resolved, the vnext branch will be out of sync with main.

Major changes

If your work contains major changes, target vnext instead of main when you open your PR. Everything else works the same way — squash merge, CI checks, and so on.

Release cadence

Releases are made as needed — there is no fixed schedule. The branching model is designed to support any cadence, from continuous releases to periodic batches.

When a release is ready, the vnext branch is merged into main and a GitHub Release is created. That's it. There's no separate release branch — the tag on main is the release.

flowchart LR
    A[vnext] -->|merge into main| B[main<br/>now includes major changes]
    B -->|create GitHub Release<br/>+ tag vX.Y.Z| C[release<br/>tagged commit on main]

Loading

Dev versioning for internal consumers

Every merge to vnext triggers CI to auto-publish a development version of all affected packages to the internal CodeArtifact dev repository. This gives internal consumers (e.g. CDP) early access to major changes without waiting for a monthly release.

Version scheme

Dev builds use a PEP 440 local version identifier: +dev.N, where N is a monotonically increasing build number (e.g. the CI run number).

The base version is always the last published release version. For example, if the last release was v1.2.0, dev builds on vnext are published as:

v1.2.0+dev.1
v1.2.0+dev.2
v1.2.0+dev.3
...

At release time, vnext is merged to main and tagged as v1.3.0 — the +dev.N versions are never promoted and never reach public PyPI (PyPI rejects local version identifiers by design).

CodeArtifact topology

The dev repository sits at the innermost position in the upstream chain:

CDP consumer → CA dev repo → CA release repo → PyPI

Dev builds are published exclusively to the CA dev repo and are never copied downstream. This means:

• No --pre flag or prerelease = "allow" is needed — local versions install like normal releases
• Third-party packages resolved through the PyPI upstream are never affected
• The release repo and PyPI remain clean

Consuming dev builds

To track the latest dev build of a package, pin with a >= lower bound against the current base version:

# pyproject.toml / uv
dependencies = [
  "overture-schema-places>=1.2.0+dev.0",
]

pip and uv will resolve to the highest available 1.2.0+dev.N in the CA dev repo. Once v1.3.0 is released to the CA release repo, the same consumer will naturally upgrade to v1.3.0 on the next resolution without any pin change.

flowchart LR
    A[merge to vnext] -->|CI publishes<br/>v1.2.0+dev.N| B[CA dev repo]
    B -->|upstream fallthrough| C[CA release repo]
    C -->|upstream fallthrough| D[PyPI]
    E[CDP consumer] -->|resolves from| B

Loading

Public PyPI releases

When a GitHub Release is created (i.e. the release tag is published via the GitHub UI or API), CI automatically builds and publishes all packages to public PyPI. The release trigger and attested publish workflow follows the same pattern as overturemaps-py — creating the GitHub Release is the single action that kicks off the full publish pipeline.

Packages are published with Trusted Publishing and PyPI attestations, providing a verifiable link from the published package back to the specific CI run and commit that produced it. No long-lived PyPI API tokens are required.

The relationship between the internal dev versions and the public release versions is straightforward:

Version	Where	Who consumes
v1.2.0+dev.1, v1.2.0+dev.2, ...	CA dev repo only	CDP and other internal consumers
v1.3.0	public PyPI (available in CA via upstream)	Everyone

The +dev.N versions are never promoted to the CA release repo or PyPI — they exist only as a forward-looking preview of the next major release. Once v1.3.0 is published, internal consumers naturally resolve to it without any pin changes.

flowchart LR
    A[GitHub Release<br/>created] -->|triggers CI| B[build + publish<br/>with attestations]
    B -->|publish| C[public PyPI<br/>v1.3.0]
    C -->|upstream fallthrough| D[CA release repo]

Loading

CDP access to schema

[!NOTE]
This section needs more input from the CDP team. The assumption here is that CDP treats the schema packages like any other versioned Python dependency — consuming from the CA dev repo for early access and bumping to the public release version as part of their normal dependency update process. Please flag if there are additional access patterns or constraints we should account for.

Under this model, CDP does not require any special integration with the schema repository's branching or release process:

• For early access to major changes: CDP points at the CA dev repo and pins >=<last-release>+dev.0 to track dev builds as they land on vnext
• For stable consumption: CDP bumps their schema package versions to the new release after it publishes to PyPI, the same as any other dependency upgrade

What this means for contributors

• Always branch from main
• Target main for minor changes, vnext for major changes
• Use whatever commit style you like on your branch — only the final squashed commit lands on shared branches
• If CI flags a conflict, follow the instructions in the comment — it will tell you exactly what to run
• Never push directly to main or vnext

[vcschapp]

One possible rough spot we may have to through is around release cadence:

Releases are made as needed — there is no fixed schedule. The branching model is designed to support any cadence, from continuous releases to periodic batches.

This appeals to me (per my comment above) but I'm also realizing that if CDP is taking straight from vnext we're going to have to find a way to generate official releases with those changes in time for the release so public PyPI has the schema version in time for the release.

[vcschapp]

One other quick thought on the public releases.

The ideal process Seth Fitzsimmons (@mojodna) and I talked through some months back was to have public releases all driven from a single version-controlled action, basically the committing of an updated version number, and have that drive the PyPI publish and the GH release. The mechanism to do that is kind of half-baked in there today, but it would need to also find a release notes file in the commit and auto-generate the GH release to be done. Can we build that?

[vcschapp]

https://lf-overturemaps.atlassian.net/wiki/spaces/SCHEM/pages/14286874/Schema+versioning+and+stability

[lowlydba]

One other quick thought on the public releases.

The ideal process Seth Fitzsimmons (@mojodna) and I talked through some months back was to have public releases all driven from a single version-controlled action, basically the committing of an updated version number, and have that drive the PyPI publish and the GH release. The mechanism to do that is kind of half-baked in there today, but it would need to also find a release notes file in the commit and auto-generate the GH release to be done. Can we build that?

Victor Schappert (@vcschapp) All of this is relatively low-effort and very flexible, so totally doable. Feel free to drop any more specifics around this here, and/or I'll loop back to discuss in more detail when I start to make changes in this area 👍🏻

I've got no strong feelings on the release notes content, but I personally really enjoy using approach Ansible uses with their CLI antsibull-changelog that uses changelog fragments per-PR to generate a consistent Changelog whenever the next release is - a bit more upfront work, but you never have to deal with "remembering" what all the changes you're about to release are, and the releases become that much more automated.