(maintainer-information)=

# Maintainer information

## Making a release

For branch organization we use a variation of the [GitHub
Flow](https://guides.github.com/introduction/flow/) with
the latest release branch named `stable` (due to ReadTheDocs constraints).

### Making a major release

Assume for concreteness that we are releasing version 0.20.0.

#### Preparation

1. Make a tracking issue with a title like "0.20.0 release planning". Add the
   checklist:
   - [ ] Update packages
   - [ ] Look for open PRs to add to the release milestone
   - [ ] Make sure all PRs in the release milestone are merged
   - [ ] Write release notes
   - [ ] Tidy changelog

2. Make a release notes blog post at pyodide-blog:
   https://github.com/pyodide/pyodide-blog

3. Generate the list of contributors for the release at the end of the release
   notes blog post with:

   ```sh
   git shortlog -s 0.19.0.. | cut -f2- | grep -v '\[bot\]' | sort --ignore-case | tr '\n' ';' | sed 's/;/, /g;s/, $//' | fold -s
   ```

   where `0.19.0` is the tag for the last major release.

4. Read the changelog and tidy it up by adding subsections, organizing, and proof
   reading it. Make a pull request with these changes titled "Rearrange changelog
   for 0.20.0 release" and merge it.

5. Make sure all the PRs that we want to release are merged and that the release notes are ready.

#### Releasing

1. Switch to the main branch

2. From the root of the repository run:
   ```
   ./tools/bump_version.py 0.20.0 --tag
   ```
   This makes a release commit and tags it.

3. Push the release commit and tag to upstream. This triggers the release CI.
   ```
   git push upstream main 0.20.0
   ```

4. Wait for CI to pass and release to be created.

5. Rename the `stable` branch to a release branch for the previous major
   version. For instance if last release was, `0.20.0`, the corresponding
   release branch would be `0.20.X`:

   ```sh
   git fetch upstream stable:stable
   git branch 0.20.X stable
   git push -u upstream 0.20.X
   ```

6. Create a new `stable` branch:

   ```sh
   git switch main
   git switch -C stable
   git push upstream stable --force
   ```

7. Set the version back to next development version with:
   ```sh
   git switch main
   ./tools/bump_version.py 0.21.0 --dev
   git push upstream main
   ```

### Making a minor release

Assume for concreteness that we are releasing version 0.27.2.

#### Preparation

1. Go through the commits on the main branch since the last release, find ones
   you want to backport and add the "needs backport" label to the pull requests.
   You can do this manually in the web interface on the GitHub PR or you can use

   ```sh
   ./tools/backport.py add-pr <pr-number>
   ```

2. List out the `needs backport` PRs that are missing changelog entries with

   ```sh
   ./tools/backport.py missing-changelogs
   ```

   and double check that every PR that should have a changelog does have one.

3. Read the changelog and tidy it up by adding subsections, organizing, and
   proof reading it. Make a pull request with these changes titled e.g.,
   "Rearrange changelog for 0.27.2 release" and merge it.

4. Make the backport branch (on top of stable):

   ```
   ./tools/backport.py backport-branch
   ```

5. Make the update-changelog branch (on top of main) with:

   ```
   ./tools/backport.py changelog-branch
   ```

6. Open PRs for these two branches with:

   ```
   ./tools/backport.py open-release-prs
   ```

7. Use the backport branch PR as the release tracker.

8. Make sure that the CI passes on the backports branch and it is approved. When
   it does pass, set the date for the release in the changelog with:
   ```
   ./tools/backport.py set-date
   git switch backports-for-0.27.2
   git push -f
   git switch changelog-for-0.27.2
   git push -f
   ```
   Then merge the two PRs.
9. Run
   ```
   ./tools/backport.py clear-prs
   ```
   to clear all the "needs backport" labels.

#### Releasing

1. Switch to the stable branch and `git pull`.
2. From the root of the repository run:
   ```
   ./tools/backport.py bump-version --tag
   ```
   This makes a release commit and tags it.
3. Push the release commit and tag to `upstream/stable`. This triggers the release
   CI.
   ```
   git push upstream stable 0.27.2
   ```
4. Wait for CI to pass and the release to be created.

### Making an alpha release

Assume for concreteness that we are releasing 0.28.0a1.

#### Preparation

Any single maintainer can decide on their own to make an alpha release, it is
not required to discuss it with other maintainers.

Name the first alpha release `x.x.xa1` and in subsequent alphas increment the
final number. No preparation is necessary. Do not make any changes to the
changelog.

#### Release instructions

1. Switch to the main branch and `git pull`.
2. From the root of the repository run:
   ```
   ./tools/bump_version.py 0.28.0a1 --tag
   ```
   This makes a release commit and tags it.
3. Push the release commit and tag to `upstream/main`. This triggers the release
   CI.
   ```
   git push upstream main 0.28.0a1
   ```
4. Put the version back with:
   ```
   git revert 0.28.0a1 -n && git commit -m "Back to development version"
   git push upstream main
   ```
5. Wait for CI to pass and the release to be created.

## Fixing documentation for a released version

Cherry pick the corresponding documentation commits to the `stable` branch. Use
`git commit --amend` to add `[skip ci]` to the commit message.

## Updating the Docker image

Anyone with an account on hub.docker.com can follow the following steps:

1. Make whatever changes are needed to the Dockerfile.
2. Build the docker image with `docker build .` in the Pyodide root directory.
   If the build succeeds, docker will give you a hash for the built image.
3. Use `python ./tools/docker_image_tag.py` to find out what the new image tag
   should be. Tag the image with:
   ```sh
   docker image tag <image-hash> <your-docker-username>/pyodide-env:<image-tag>
   ```
4. Push the image with:
   ```sh
   docker image push <your-docker-username>/pyodide-env:<image-tag>
   ```
5. Replace the image in `.circleci/config.yml` with your newly created image.
   Open a pull request with your changes to `Dockerfile` and `.circleci/config.yml`.
6. When the tests pass and the pull request is approved, a maintainer must copy
   the new image into the `pyodide` dockerhub account.
7. Then replace the image tag in `.circleci/config.yml`,
   `.devcontainer/devcontainer.json`, and `run_docker` with the new image under
   the `pyodide` dockerhub account.

It's also possible to update the docker image by pushing your changes to the
`Dockerfile` to a branch in the `pyodide/pyodide` repo (not on a fork) and
clicking `Run workflow` on
https://github.com/pyodide/pyodide/actions/workflows/docker_image.yml.

(updating-packages)=

## Updating packages

Before updating the Python version and before making a major Pyodide release, we
try to update all packages that are not too much trouble. Run

```sh
make -C packages update-all
```

to update all packages and make a pull request with these changes. There will be
build/test failures, revert the packages that fail the build or tests and make a
note to update them independently.

## Updating pyodide-build

To change the version of pyodide-build, change the commit of the pyodide-build submodule.

```bash
cd pyodide-build
git checkout "<COMMIT HASH>"
```

To test with a fork of pyodide-build, change the `.gitmodules` file to point to your fork and update the commit hash

```ini
# .gitmodules
[submodule "pyodide-build"]
	path = pyodide-build
	url = https://github.com/<yourfork>/pyodide-build
```

```bash
git submodule sync
cd pyodide-build
git checkout "<COMMIT HASH>"
```

## Upgrading the ABI

Each year, we aim to release a new Pyodide ABI version with a new CPython and
Emscripten version.

Similar to the CPython alpha release, we stabilize the ABI early before
releasing the new Pyodide version so that package maintainers can update their
packages to use the new ABI ([#5580](https://github.com/pyodide/pyodide/issues/5580)).

### 0. Stabilize and freeze the ABI

Discuss with the maintainers to decide what ABI changes to make before the
release.

Once the ABI changes are decided, create a `20XX_0` branch (targeting year) from
the `main` branch. All PRs that change the ABI should be merged to the `20XX_0`
branch.

### 1. Disable `USE_PREBUILT_PACKAGES`

Pyodide CI uses prebuilt packages built with the ABI of the last release. To
upgrade the ABI, you need to disable them, otherwise the CI will fail.

- Find the `USE_PREBUILT_PACKAGES` environment variable in
  [.circleci/config.yml](https://github.com/ryanking13/pyodide/blob/unvendor-recipes/.circleci/config.yml)
  and set it to `false`.

### 2. Upgrade Emscripten and CPython version

Update the Emscripten and CPython versions respectively.

Updating the Emscripten version is not mandatory, but it is recommended to
benefit from the latest features and bug fixes.

Updating Emscripten and CPython separately is often easier than updating them
together.

#### 2.1. Upgrade the Emscripten version

Updating Emscripten requires the following three steps:

1. Rebase the patches in `emsdk/patches` onto the new Emscripten version.
2. Update the Emscripten version in `Makefile.envs`.
3. Update the `struct_info` JSON file in `src/js/` to match the version of the
   file in Emscripten.

All three steps are automated by `tools/update_emscripten.py`. To update, run:

```sh
./tools/update_emscripten.py new_version
```

If there are rebase conflicts, you will have to manually finish the rebase. Once
the rebase is completed, rerun `update_emscripten.py`. It will start over the
rebase from scratch but reuse your conflict resolutions using the git rerere
feature.

After this is done, commit all the changes and open a PR. There are frequently
complicated CI failures.

#### 2.2. Upgrade CPython version

##### Prerequisites

The desired version of CPython must be available at:

1. The "specific release" section of https://www.python.org/downloads
2. https://hub.docker.com/_/python
3. https://github.com/actions/python-versions/releases

##### Steps

1. Follow the steps in "Updating the Docker image" to create a Docker image for
   the new Python version.

2. Make sure you are in a Python virtual environment with the new version of
   Python and with `requirements.txt` installed. (It is also possible to work in
   the Docker image as an alternative.)

3. Update the Python version in `Makefile.envs`.

4. Update the Python version in the following locations:
   - `.github/workflows/main.yml`
   - `docs/conf.py`
   - `docs/development/contributing.md`
   - `docs/development/building-and-testing-packages.md`
   - `environment.yml`
   - `.pre-commit-config.yaml`
   - `pyproject.toml`

   (TODO: make this list shorter.)

5. Check [python/cpython-source-deps](https://github.com/python/cpython-source-deps) for
   updated versions of bundled C libraries. Pyodide bundles `zstd` from this repository
   (see `ZSTDTARBALL` in `cpython/Makefile`). Update the tag and URL if a newer version
   is available, and verify the build still works.

6. Rebase the patches:

   - Clone CPython and cd into it. Checkout the Python version you are upgrading
     from. For instance, if the old version is 3.13.2, use `git checkout v3.13.2`
     (Python tags have a leading `v`). Run:

     ```sh
     git am ~/path/to/pyodide/cpython/patches/*
     ```

   - Rebase the patches onto the new version of Python. For instance, if updating
     from Python v3.13.2 to Python 3.14.1:

     ```sh
     git rebase v3.13.2 --onto v3.14.1
     ```

   - Resolve conflicts and drop patches that have been upstreamed. If you have
     conflicts, make sure you are using diff3:

     ```sh
     git config --global merge.conflictstyle diff3
     ```

   - Generate the new patches:

     ```sh
     rm ~/path/to/pyodide/cpython/patches/*
     git format-patch v3.14.1 -o ~/path/to/pyodide/cpython/patches/
     ```

7. Try to build Python with `make -C cpython`. Fix any build errors. If you
   modify the Python source in-tree after a failed build, it may be useful to
   run `make rebuild`.

8. Try to finish the build with a top-level `make`. Fix compile errors in
   `src/core` and any link errors. It may be useful to apply
   [`upgrade_pythoncapi.py --no-compat`](https://github.com/python/pythoncapi-compat/blob/main/upgrade_pythoncapi.py)
   to the C extension in `src/core`.

   The file most tightly coupled to the CPython version is
   `src/core/stack_switching/pystate.c`. Consult the
   [greenlet TPythonState.cpp](https://github.com/python-greenlet/greenlet/blob/master/src/greenlet/TPythonState.cpp)
   to figure out how to fix it.

9. Run:

   ```sh
   python tools/make_test_list.py
   ```

   Then run the core tests with `pytest src/tests/test_core_python.py` and
   either fix the failures or update `src/tests/python_tests.yaml` to skip or
   xfail them.

10. Try to build packages with:

    ```sh
    pyodide build-recipes '*'
    ```

11. Fix failing package tests.

12. Update standard library stubs in `src/templates`. We currently have
    `webbrowser.py` and `ssl.py` that we implement ourselves. If you are
    updating the Python version, you may need to update these stubs. Review the
    Python docs for the standard library modules and check if the APIs have
    changed. Search for "in version 3.XX" to find the relevant changes.

### Old major Python upgrades

| version | pr         |
| ------- | ---------- |
| 3.13    | {pr}`5498` |
| 3.12    | {pr}`4435` |
| 3.11    | {pr}`3252` |
| 3.10    | {pr}`2225` |
| 3.9     | {pr}`1637` |
| 3.8     | {pr}`712`  |
| 3.7     | {pr}`77`   |

### 3. Release a new Pyodide alpha version

After all the changes are merged, release a new Pyodide alpha version so that
people can build packages with the new ABI.

### 4. Update packages and release new package set

TODO: Update the process after [PEP 783](https://peps.python.org/pep-0783/) is accepted.

Go to [pyodide/pyodide-recipes](https://github.com/pyodide/pyodide-recipes) and
follow the guidelines in
[MAINTAINERS.md](https://github.com/pyodide/pyodide-recipes/blob/main/docs/MAINTAINERS.md).
You'll need to create a tag for the existing package sets and update
`default_cross_build_env_url` in the main branch.

If you updated the Python version, also update the `python` version used in the
CI. When opening the PR, include the `full build` label to trigger a full build
of all packages; otherwise, only a small set of packages will be built.

Most likely, some packages will fail to build. You can disable them by adding
`_disabled: true` to the recipes of the failing packages. You don't need to
handle all the build failures in the same PR—you can open multiple PRs to fix
the build failures or let the package maintainers handle them.

Create a new tag for the new package set when the build is successful. This will
create a new release with the new package set.

### 5. Re-enable `USE_PREBUILT_PACKAGES` and upgrade the ABI in the main branch

Go back to `pyodide/pyodide` and update the `PYODIDE_PREBUILT_PACKAGES_BASE`
variable in `Makefile.envs` to the new package set you created in the previous
step.

Then re-enable `USE_PREBUILT_PACKAGES` in `.circleci/config.yml` and open a PR
to merge it into the main branch.

### 6. Release a new Pyodide version

After enabling most of the packages that were available in the previous ABI
version, we are ready to release a new Pyodide version with the new ABI.

Update {ref}`pyodide-platform-abi` to match the new ABI.

Merge the `20XX_0` branch into `main` and open a PR, then release a new Pyodide
version with the new ABI.