Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

PyO3 使用者指南

歡迎閱讀 PyO3 使用者指南！本書是 PyO3 API 文件 的輔助說明，包含範例與文件，詳細解釋 PyO3 的各種使用情境。

本指南內容的大致順序如下：

  1. 開始使用
  2. 封裝 Rust 程式碼供 Python 使用
  3. 如何從 Rust 使用 Python 程式碼
  4. 其餘深入進階概念的主題

請從左側章節選擇主題，或繼續往下閱讀 PyO3 的 README。

PyO3

actions status benchmark codecov crates.io minimum rustc 1.83 discord server contributing notes

提供 Rust 對 Python 的綁定，包含建立原生 Python 擴充模組的工具；也支援在 Rust 二進位程式中執行並與 Python 程式碼互動。

使用方式

需要 Rust 1.83 或更高版本。

PyO3 支援以下 Python 散布版：

  • CPython 3.7 或更高版本
  • PyPy 7.3（Python 3.11+）
  • GraalPy 25.0 或更高版本（Python 3.12+）

你可以使用 PyO3 以 Rust 撰寫原生 Python 模組，或在 Rust 二進位程式中嵌入 Python。以下各節將依序說明。

從 Python 使用 Rust

PyO3 可用來產生原生 Python 模組。第一次嘗試最簡單的方法是使用 maturin。maturin 是用來以最少設定建置並發布以 Rust 為基礎的 Python 軟體包的工具。以下步驟會安裝 maturin、用它產生並建置新的 Python 軟體包，接著啟動 Python 匯入並執行軟體包中的函式。

首先，依照以下命令建立一個包含 Python virtualenv 的新目錄，並使用 Python 軟體包管理工具 pip 將 maturin 安裝到該 virtualenv：

#（將 string_sum 替換為想要的軟體包名稱）
$ mkdir string_sum
$ cd string_sum
$ python -m venv .env
$ source .env/bin/activate
$ pip install maturin

仍在這個 string_sum 目錄中時，執行 maturin init。這會產生新的軟體包來源碼。當詢問要使用哪種綁定時，選擇 pyo3 綁定：

$ maturin init
✔ 🤷 What kind of bindings to use? · pyo3
  ✨ Done! New project created string_sum

此命令產生的最重要檔案是 Cargo.toml 與 lib.rs，大致如下：

Cargo.toml

[package]
name = "string_sum"
version = "0.1.0"
edition = "2021"

[lib]
# The name of the native library. This is the name which will be used in Python to import the
# library (i.e. `import string_sum`). If you change this, you must also change the name of the
# `#[pymodule]` in `src/lib.rs`.
name = "string_sum"
# "cdylib" is necessary to produce a shared library for Python to import from.
#
# Downstream Rust code (including code in `bin/`, `examples/`, and `tests/`) will not be able
# to `use string_sum;` unless the "rlib" or "lib" crate type is also included, e.g.:
# crate-type = ["cdylib", "rlib"]
crate-type = ["cdylib"]

[dependencies]
pyo3 = "0.28.2"

src/lib.rs

/// 以 Rust 實作的 Python 模組。此模組名稱必須與 `Cargo.toml` 中的
/// `lib.name` 設定一致，否則 Python 將無法
/// 匯入此模組。
#[pyo3::pymodule]
mod string_sum {
  use pyo3::prelude::*;

  /// 將兩個數字的總和格式化為字串。
  #[pyfunction]
  fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
    Ok((a + b).to_string())
  }
}

最後，執行 maturin develop。這會建置軟體包並安裝到先前建立並啟用的 Python virtualenv。接著即可在 python 中使用該軟體包：

$ maturin develop
# maturin 執行編譯時會輸出大量進度訊息...
$ python
>>> import string_sum
>>> string_sum.sum_as_string(5, 20)
'25'

要修改軟體包時，只需編輯 Rust 來源碼，然後重新執行 maturin develop 以重新編譯。

若要一次複製貼上完成全部步驟，請使用下列 bash 腳本（將第一個命令中的 string_sum 替換為想要的軟體包名稱）：

mkdir string_sum && cd "$_"
python -m venv .env
source .env/bin/activate
pip install maturin
maturin init --bindings pyo3
maturin develop

如果你想要執行 cargo test 或在 Cargo workspace 中使用此專案，但遇到連結器問題，可參考 FAQ 的替代做法。

除了 maturin 之外，也可以使用 setuptools-rust 或手動建置。兩者都比 maturin 更有彈性，但起步時需要更多設定。

從 Rust 使用 Python

若要在 Rust 二進位程式中嵌入 Python，你需要確保 Python 安裝包含共享程式庫。以下步驟示範如何（在 Ubuntu 上）完成此設定，並提供一段執行嵌入式 Python 直譯器的範例程式碼。

在 Ubuntu 上安裝 Python 共享程式庫：

sudo apt install python3-dev

在基於 RPM 的散佈版（如 Fedora、Red Hat、SuSE）上，請安裝 python3-devel 軟體包以取得 Python 共享程式庫。

使用 cargo new 建立新專案，並在 Cargo.toml 中加入 pyo3：

[dependencies.pyo3]
version = "0.28.2"
# Enabling this cargo feature will cause PyO3 to start a Python interpreter on first call to `Python::attach`
features = ["auto-initialize"]

顯示 sys.version 與目前使用者名稱的範例程式：

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let sys = py.import("sys")?;
        let version: String = sys.getattr("version")?.extract()?;

        let locals = [("os", py.import("os")?)].into_py_dict(py)?;
        let code = c"os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
        let user: String = py.eval(code, None, Some(&locals))?.extract()?;

        println!("Hello {}, I'm Python {}", user, version);
        Ok(())
    })
}

指南中有專章提供大量相關範例。

工具與程式庫

  • maturin 使用 pyo3、rust-cpython 或 cffi 綁定建置並發布 crate，並將 Rust 二進位程式打包為 Python 軟體包
  • setuptools-rust 為 Rust 提供支援的 Setuptools 外掛。
  • pyo3-built 透過簡單的巨集將 built crate 取得的中繼資料以 PyDict 形式暴露
  • rust-numpy NumPy C-API 的 Rust 綁定
  • dict-derive Derive FromPyObject 以自動將 Python dict 轉為 Rust struct
  • pyo3-log 將 Rust 連接到 Python 日誌系統的橋接
  • pythonize 用於將 Rust 物件轉成相容 JSON 的 Python 物件的 Serde 序列化工具
  • pyo3-async-runtimes 與 Python 的 Asyncio 程式庫與 Rust 的 async runtime 互通的工具
  • rustimport 可在 Python 直接匯入 Rust 檔案或 crate，無需手動編譯；預設提供 pyo3 整合並自動產生 pyo3 綁定程式碼
  • pyo3-arrow 為 pyo3 提供的輕量 Apache Arrow 整合
  • pyo3-bytes bytes 與 pyo3 的整合
  • pyo3-object_store object_store 與 pyo3 的整合

範例

  • anise A modern, high-performance toolkit for spacecraft mission design, notably used to help softly land Firefly Blue Ghost on the Moon on 02 Feb 2025.
  • arro3 連結 Rust arrow crate 的最小化 Apache Arrow Python 函式庫
  • bed-reader 簡潔且高效地讀寫 PLINK BED 格式
    • 展示 Rayon/ndarray::parallel（包含錯誤擷取與控制執行緒數）、Python 型別到 Rust 泛型，以及 GitHub Actions
  • blake3-py BLAKE3 密碼雜湊函式的 Python 綁定
    • 在 GitHub Actions 上針對 macOS、Linux、Windows 進行平行化建置，包含 free-threaded 3.13t 的 wheel。
  • cellular_raza 以細胞代理為基礎的模擬框架，用於從零構建複雜模型
  • connector-x 在 Rust 與 Python 中將資料從資料庫載入到 DataFrame 的高速函式庫
  • cryptography 部分功能以 Rust 實作的 Python 密碼學函式庫
  • css-inline 以 Rust 實作的 Python CSS 內嵌工具
  • datafusion-python 綁定 Apache Arrow 記憶體內查詢引擎 DataFusion 的 Python 函式庫
  • deltalake-python 基於 delta-rs 並整合 Pandas 的原生 Delta Lake Python 綁定
  • fastbloom 以 Rust 實作的快速 bloom filter | counting bloom filter，同時支援 Rust 與 Python
  • fastuuid Rust UUID 函式庫的 Python 綁定
  • fast-paseto 高效能 PASETO（Platform-Agnostic Security Tokens）實作，提供 Python 綁定
  • feos 以 Rust 進行極速熱力學建模，並提供完善的 Python 介面
  • finalytics Rust | Python 投資分析函式庫
  • forust 以 Rust 撰寫的輕量級梯度提升決策樹函式庫
  • geo-index 提供緊湊、不可變、零拷貝空間索引的 Rust crate 與 Python 函式庫
  • granian 供 Python 應用使用的 Rust HTTP 伺服器
  • haem 用於生物資訊問題的 Python 函式庫
  • hifitime A high fidelity time management library for engineering and scientific applications where general relativity and time dilation matter.
  • html2text-rs 將 HTML 轉為標記或純文字的 Python 函式庫
  • html-py-ever 透過 kuchiki 使用 html5ever 加速 HTML 解析與 CSS 選取
  • hudi-rs Apache Hudi 的原生 Rust 實作，並提供 C++ 與 Python API 綁定
  • inline-python 在 Rust 程式碼中直接內嵌 Python 程式碼
  • johnnycanencrypt 支援 Yubikey 的 OpenPGP 函式庫。
  • jsonschema 高效能的 Python JSON Schema 驗證器
  • mocpy 天文 Python 函式庫，提供用於描述單位球面上任意覆蓋區域的資料結構
  • obstore 以 Rust 驅動、最簡潔且高吞吐量的 Python 介面，可存取 Amazon S3、Google Cloud Storage、Azure Storage 與其他相容 S3 的 API
  • opendal 資料存取層，可讓使用者以統一方式輕鬆且高效地從各種儲存服務取回資料
  • orjson 快速的 Python JSON 函式庫
  • ormsgpack 快速的 Python msgpack 函式庫
  • polars Rust | Python | Node.js 的高速多執行緒 DataFrame 函式庫
  • pycrdt Rust CRDT 實作 Yrs 的 Python 綁定
  • pydantic-core 以 Rust 撰寫的 pydantic 核心驗證邏輯
  • primp 可透過模擬標頭與 TLS/JA3/JA4/HTTP2 指紋來偽裝瀏覽器的最快 Python HTTP 用戶端
  • radiate：用於遺傳程式設計與演化演算法的高效能演化引擎
  • rateslib 使用 Rust 擴充的 Python 固定收益函式庫
  • river Python 的線上機器學習，計算量大的統計演算法以 Rust 實作
  • robyn 具備 Rust runtime 的超高速非同步 Python Web 框架。
  • rust-python-coverage 含 Rust 與 Python 自動化測試覆蓋的 PyO3 專案範例
  • rnet 帶有黑魔法的非同步 Python HTTP 用戶端
  • sail 在相容 Apache Spark 的前提下統一串流、批次與 AI 工作負載
  • tiktoken 可搭配 OpenAI 模型使用的快速 BPE 分詞器
  • tokenizers Rust 撰寫的 Hugging Face tokenizers（NLP）的 Python 綁定
  • tzfpy 快速將經緯度轉為時區名稱的軟體包
  • toml-rs 以 Rust 撰寫的 Python 高效能 TOML v1.0.0 與 v1.1.0 解析器
  • utiles 快速的 Python 網路地圖磚工具

文章及其他媒體

參與貢獻

歡迎大家參與 PyO3！你可以用許多方式支持專案，例如：

  • 在 GitHub 與 Discord 協助 PyO3 使用者解決問題
  • 改善文件
  • 撰寫功能與修復錯誤
  • 發表如何使用 PyO3 的部落格與範例

若你願意投入時間參與 PyO3，但不確定從哪裡開始，可參考我們的貢獻說明與架構指南取得更多資源。

如果你沒有時間親自貢獻，但仍想支持專案的長期發展，部分維護者有 GitHub 贊助頁面：

授權條款

PyO3 採用 Apache-2.0 授權 或 MIT 授權，可自行選擇。

Python 採用 Python License 授權。

除非你明確另行說明，否則你提交且意圖納入 PyO3 的任何貢獻（依 Apache License 定義）將採用上述雙重授權，不附加任何額外條款或條件。

Deploys by Netlify

安裝

要開始使用 PyO3，你需要三樣東西：Rust 工具鏈、Python 環境，以及建置方式。下面將逐一說明。

[!TIP] 如果你想與 PyO3 維護者和其他 PyO3 使用者聊天，請考慮加入 PyO3 Discord 伺服器。我們很想聽聽你開始使用時的體驗，讓 PyO3 對每個人都更容易上手！

Rust

First, make sure you have Rust installed on your system. If you haven’t already done so, try following the instructions on the Rust website. PyO3 runs on both the stable and nightly versions so you can choose whichever one fits you best. The minimum required Rust version is 1.83.

如果你能執行 rustc --version 並且版本足夠新，就可以開始了！

Python

要使用 PyO3，至少需要 Python 3.7。雖然可直接使用系統預設的 Python 直譯器，但建議使用虛擬環境。

虛擬環境

While you can use any virtualenv manager you like, we recommend the use of pyenv in particular if you want to develop or test for multiple different Python versions, so that is what the examples in this book will use. The installation instructions for pyenv can be found in the pyenv GitHub repository. (Note: To get the pyenv activate and pyenv virtualenv commands, you will also need to install the pyenv-virtualenv plugin. The pyenv installer will install both together.)

使用 pyenv 安裝時保留來源檔可能對日後除錯有幫助，可在 pyenv install 命令中加入 --keep 旗標。

例如：

pyenv install 3.12 --keep

建置

There are a number of build and Python package management systems such as setuptools-rust or manually. We recommend the use of maturin, which you can install as per the maturin documentation. It is developed to work with PyO3 and provides the most “batteries included” experience, especially if you are aiming to publish to PyPI. maturin is just a Python package, so you can add it in the same way you already install Python packages.

系統 Python：

pip install maturin --user

pipx：

pipx install maturin

pyenv：

pyenv activate pyo3
pip install maturin

poetry：

poetry add -G dev maturin

安裝完成後，可執行 maturin --version 以確認是否安裝成功。

開始新專案

首先建立要放置新專案的資料夾與虛擬環境。此處將使用推薦的 pyenv：

mkdir pyo3-example
cd pyo3-example
pyenv virtualenv pyo3
pyenv local pyo3

接著安裝建置工具。這個例子使用 maturin。啟用 virtualenv 後，將 maturin 安裝進去：

pip install maturin

現在可以初始化新專案：

maturin init

如果已安裝 maturin，也可直接用它建立新專案：

maturin new -b pyo3 pyo3-example
cd pyo3-example
pyenv virtualenv pyo3
pyenv local pyo3

加入既有專案

可惜目前 maturin 無法在既有專案中執行，因此若要在既有專案中使用 Python，基本上有兩個選擇：

  1. 如上建立新專案並將既有程式碼移入
  2. 依需求手動編輯專案設定

若選擇第二種作法，請注意以下事項：

Cargo.toml

請確保你要讓 Python 存取的 Rust 軟體箱是以函式庫形式編譯。你也可以同時輸出二進位檔，但要讓 Python 存取的程式碼必須在函式庫部分。此外，請確保軟體箱型別為 cdylib，並依下列方式加入 PyO3 相依性：

# If you already have [package] information in `Cargo.toml`, you can ignore
# this section!
[package]
# `name` here is name of the package.
name = "pyo3_start"
# these are good defaults:
version = "0.1.0"
edition = "2021"

[lib]
# The name of the native library. This is the name which will be used in Python to import the
# library (i.e. `import string_sum`). If you change this, you must also change the name of the
# `#[pymodule]` in `src/lib.rs`.
name = "pyo3_example"

# "cdylib" is necessary to produce a shared library for Python to import from.
crate-type = ["cdylib"]

[dependencies]
pyo3 = git = "https://github.com/pyo3/pyo3"

pyproject.toml

你也應建立一個 pyproject.toml，內容如下：

[build-system]
requires = ["maturin>=1.9.4,<2"]
build-backend = "maturin"

[project]
name = "pyo3_example"
requires-python = ">=3.7"
classifiers = [
    "Programming Language :: Rust",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

執行程式碼

接著可依下列方式設定讓 Python 可使用 Rust 程式碼；例如將程式碼放在 src/lib.rs：

/// 以 Rust 實作的 Python 模組。此函式名稱必須
/// 與 `Cargo.toml` 中的 `lib.name` 設定一致，
/// 否則 Python 將無法匯入該模組。
#[pyo3::pymodule]
mod pyo3_example {
    use pyo3::prelude::*;

    /// 將兩個數字的總和格式化為字串。
    #[pyfunction]
    fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
        Ok((a + b).to_string())
    }
}

現在可以執行 maturin develop 準備 Python 軟體包，之後可如下使用：

$ maturin develop
# maturin 執行編譯時會輸出大量進度訊息...
$ python
>>> import pyo3_example
>>> pyo3_example.sum_as_string(5, 20)
'25'

更多關於如何從 Rust 使用 Python 程式碼的說明，請見 Python from Rust。

Maturin 匯入掛鉤

In development, any changes in the code would require running maturin develop before testing. To streamline the development process, you may want to install Maturin Import Hook which will run maturin develop automatically when the library with code changes is being imported.

從 Python 使用 Rust

本章專門說明如何將 Rust 程式碼封裝成 Python 物件。

PyO3 使用 Rust 的「程序巨集」提供強大而簡單的 API，用來標示哪些 Rust 程式碼要對應為 Python 物件。

PyO3 可建立三種 Python 物件：

  • Python 模組，透過 #[pymodule] 巨集
  • Python 函式，透過 #[pyfunction] 巨集
  • Python 類別，透過 #[pyclass] 巨集（另用 #[pymethods] 為這些類別定義方法）

The following subchapters go through each of these in turn.

Python 模組

您可以使用 #[pymodule] 建立模組：

mod declarative_module_basic_test {
use pyo3::prelude::*;

#[pyfunction]
fn double(x: usize) -> usize {
    x * 2
}

/// This module is implemented in Rust.
#[pymodule]
mod my_extension {
    use pyo3::prelude::*;

    #[pymodule_export]
    use super::double; // The double function is made available from Python, works also with classes

    #[pyfunction] // Inline definition of a pyfunction, also made available to Python
    fn triple(x: usize) -> usize {
        x * 3
    }
}
}

The #[pymodule] procedural macro takes care of creating the initialization function of your module and exposing it to Python.

The module’s name defaults to the name of the Rust module. You can override the module name by using #[pyo3(name = "custom_name")]:

mod declarative_module_custom_name_test {
use pyo3::prelude::*;

#[pyfunction]
fn double(x: usize) -> usize {
    x * 2
}

#[pymodule(name = "custom_name")]
mod my_extension {
    #[pymodule_export]
    use super::double;
}
}

The name of the module must match the name of the .so or .pyd file. Otherwise, you will get an import error in Python with the following message: ImportError: dynamic module does not define module export function (PyInit_name_of_your_module)

To import the module, either:

文件

The Rust doc comments of the Rust module will be applied automatically as the Python docstring of your module.

For example, building off of the above code, this will print This module is implemented in Rust.:

import my_extension

print(my_extension.__doc__)

Python 子模組

You can create a module hierarchy within a single extension module by just useing modules like functions or classes. For example, you could define the modules parent_module and parent_module.child_module:

use pyo3::prelude::*;

#[pymodule]
mod parent_module {
    #[pymodule_export]
    use super::child_module;
}

#[pymodule]
mod child_module {
    #[pymodule_export]
    use super::func;
}

#[pyfunction]
fn func() -> String {
    "func".to_string()
}

fn main() {
  Python::attach(|py| {
      use pyo3::wrap_pymodule;
      use pyo3::types::IntoPyDict;
      let parent_module = wrap_pymodule!(parent_module)(py);
      let ctx = [("parent_module", parent_module)].into_py_dict(py).unwrap();

     py.run(c"assert parent_module.child_module.func() == 'func'", None, Some(&ctx)).unwrap();
  })
}

Note that this does not define a package, so this won’t allow Python code to directly import submodules by using from parent_module import child_module. For more information, see #759 and #1517.

You can provide the submodule argument to #[pymodule()] for modules that are not top-level modules in order for them to properly generate the #[pyclass] module attribute automatically.

Inline declaration

It is possible to declare functions, classes, sub-modules and constants inline in a module:

例如：

mod declarative_module_test {
#[pyo3::pymodule]
mod my_extension {
    use pyo3::prelude::*;

    #[pymodule_export]
    const PI: f64 = std::f64::consts::PI; // Exports PI constant as part of the module

    #[pyfunction] // This will be part of the module
    fn double(x: usize) -> usize {
        x * 2
    }

    #[pyclass] // This will be part of the module
    struct Unit;

    #[pymodule]
    mod submodule {
        // This is a submodule
        use pyo3::prelude::*;

        #[pyclass]
        struct Nested;
    }
}
}

In this case, #[pymodule] macro automatically sets the module attribute of the #[pyclass] macros declared inside of it with its name. For nested modules, the name of the parent module is automatically added. In the previous example, the Nested class will have for module my_extension.submodule.

Procedural initialization

If the macros provided by PyO3 are not enough, it is possible to run code at the module initialization:

mod procedural_module_test {
#[pyo3::pymodule]
mod my_extension {
    use pyo3::prelude::*;

    #[pyfunction]
    fn double(x: usize) -> usize {
        x * 2
    }

    #[pymodule_init]
    fn init(m: &Bound<'_, PyModule>) -> PyResult<()> {
        // Arbitrary code to run at the module initialization
        m.add("double2", m.getattr("double")?)
    }
}
}

Python 函式

The #[pyfunction] attribute is used to define a Python function from a Rust function. Once defined, the function needs to be added to a module.

The following example defines a function called double in a Python module called my_extension:

#[pyo3::pymodule]
mod my_extension {
    use pyo3::prelude::*;

    #[pyfunction]
    fn double(x: usize) -> usize {
        x * 2
    }
}

This chapter of the guide explains full usage of the #[pyfunction] attribute. In this first section, the following topics are covered:

There are also additional sections on the following topics:

Function options

The #[pyo3] attribute can be used to modify properties of the generated Python function. It can take any combination of the following options:

  • #[pyo3(name = "...")]

    Overrides the name exposed to Python.

    In the following example, the Rust function no_args_py will be added to the Python module module_with_functions as the Python function no_args:

    use pyo3::prelude::*;
#[pyo3::pymodule]
mod module_with_functions {
    use pyo3::prelude::*;

    #[pyfunction]
    #[pyo3(name = "no_args")]
    fn no_args_py() -> usize {
        42
    }
}

Python::attach(|py| {
    let m = pyo3::wrap_pymodule!(module_with_functions)(py);
    assert!(m.getattr(py, "no_args").is_ok());
    assert!(m.getattr(py, "no_args_py").is_err());
});

  • #[pyo3(signature = (...))]

    Defines the function signature in Python. See Function Signatures.

  • #[pyo3(text_signature = "...")]

    Overrides the PyO3-generated function signature visible in Python tooling (such as via inspect.signature). See the corresponding topic in the Function Signatures subchapter.

  • #[pyo3(pass_module)]

    Set this option to make PyO3 pass the containing module as the first argument to the function. It is then possible to use the module in the function body. The first argument must be of type &Bound<'_, PyModule>, Bound<'_, PyModule>, or Py<PyModule>.

    The following example creates a function pyfunction_with_module which returns the containing module’s name (i.e. module_with_fn):

    #[pyo3::pymodule]
mod module_with_fn {
    use pyo3::prelude::*;
    use pyo3::types::PyString;

    #[pyfunction]
    #[pyo3(pass_module)]
    fn pyfunction_with_module<'py>(
        module: &Bound<'py, PyModule>,
    ) -> PyResult<Bound<'py, PyString>> {
        module.name()
    }
}

  • #[pyo3(warn(message = "...", category = ...))]

    This option is used to display a warning when the function is used in Python. It is equivalent to warnings.warn(message, category). The message parameter is a string that will be displayed when the function is called, and the category parameter is optional and has to be a subclass of Warning. When the category parameter is not provided, the warning will be defaulted to UserWarning.

    Note: when used with #[pymethods], this attribute does not work with #[classattr] nor __traverse__ magic method.

    The following are examples of using the #[pyo3(warn)] attribute:

    use pyo3::prelude::*;

#[pymodule]
mod raising_warning_fn {
    use pyo3::prelude::pyfunction;
    use pyo3::exceptions::PyFutureWarning;

    #[pyfunction]
    #[pyo3(warn(message = "This is a warning message"))]
    fn function_with_warning() -> usize {
        42
    }

    #[pyfunction]
    #[pyo3(warn(message = "This function is warning with FutureWarning", category = PyFutureWarning))]
    fn function_with_warning_and_custom_category() -> usize {
        42
    }
}

use pyo3::exceptions::{PyFutureWarning, PyUserWarning};
use pyo3::types::{IntoPyDict, PyList};
use pyo3::PyTypeInfo;

fn catch_warning(py: Python<'_>, f: impl FnOnce(&Bound<'_, PyList>) -> ()) -> PyResult<()> {
    let warnings = py.import("warnings")?;
    let kwargs = [("record", true)].into_py_dict(py)?;
    let catch_warnings = warnings
        .getattr("catch_warnings")?
        .call((), Some(&kwargs))?;
    let list = catch_warnings.call_method0("__enter__")?.cast_into()?;
    warnings.getattr("simplefilter")?.call1(("always",))?;  // show all warnings
    f(&list);
    catch_warnings
        .call_method1("__exit__", (py.None(), py.None(), py.None()))
        .unwrap();
    Ok(())
}

macro_rules! assert_warnings {
    ($py:expr, $body:expr, [$(($category:ty, $message:literal)),+] $(,)? ) => {
        catch_warning($py, |list| {
            $body;
            let expected_warnings = [$((<$category as PyTypeInfo>::type_object($py), $message)),+];
            assert_eq!(list.len(), expected_warnings.len());
            for (warning, (category, message)) in list.iter().zip(expected_warnings) {
                assert!(warning.getattr("category").unwrap().is(&category));
                assert_eq!(
                    warning.getattr("message").unwrap().str().unwrap().to_string_lossy(),
                    message
                );
            }
        }).unwrap();
    };
}

Python::attach(|py| {
    assert_warnings!(
        py,
        {
            let m = pyo3::wrap_pymodule!(raising_warning_fn)(py);
            let f1 = m.getattr(py, "function_with_warning").unwrap();
            let f2 = m.getattr(py, "function_with_warning_and_custom_category").unwrap();
            f1.call0(py).unwrap();
            f2.call0(py).unwrap();
        },
        [
            (PyUserWarning, "This is a warning message"),
            (
                PyFutureWarning,
                "This function is warning with FutureWarning"
            )
        ]
    );
});

    When the functions are called as the following, warnings will be displayed.

    import warnings
from raising_warning_fn import function_with_warning, function_with_warning_and_custom_category

function_with_warning()
function_with_warning_and_custom_category()

    The warning output will be:

    UserWarning: This is a warning message
FutureWarning: This function is warning with FutureWarning

Per-argument options

The #[pyo3] attribute can be used on individual arguments to modify properties of them in the generated function. It can take any combination of the following options:

  • #[pyo3(from_py_with = ...)]

    Set this on an option to specify a custom function to convert the function argument from Python to the desired Rust type, instead of using the default FromPyObject extraction. The function signature must be fn(&Bound<'_, PyAny>) -> PyResult<T> where T is the Rust type of the argument.

    The following example uses from_py_with to convert the input Python object to its length:

    use pyo3::prelude::*;

fn get_length(obj: &Bound<'_, PyAny>) -> PyResult<usize> {
    obj.len()
}

#[pyfunction]
fn object_length(#[pyo3(from_py_with = get_length)] argument: usize) -> usize {
    argument
}

Python::attach(|py| {
    let f = pyo3::wrap_pyfunction!(object_length)(py).unwrap();
    assert_eq!(f.call1((vec![1, 2, 3],)).unwrap().extract::<usize>().unwrap(), 3);
});

Advanced function patterns

Calling Python functions in Rust

You can pass Python def’d functions and built-in functions to Rust functions PyFunction corresponds to regular Python functions while PyCFunction describes built-ins such as repr().

You can also use Bound<'_, PyAny>::is_callable to check if you have a callable object. is_callable will return true for functions (including lambdas), methods and objects with a __call__ method. You can call the object with Bound<'_, PyAny>::call with the args as first parameter and the kwargs (or None) as second parameter. There are also Bound<'_, PyAny>::call0 with no args and Bound<'_, PyAny>::call1 with only positional args.

Calling Rust functions in Python

The ways to convert a Rust function into a Python object vary depending on the function:

  • Named functions, e.g. fn foo(): add #[pyfunction] and then use wrap_pyfunction! to get the corresponding PyCFunction.
  • Anonymous functions (or closures), e.g. foo: fn() either:
    • use a #[pyclass] struct which stores the function as a field and implement __call__ to call the stored function.
    • use PyCFunction::new_closure to create an object directly from the function.

Accessing the FFI functions

In order to make Rust functions callable from Python, PyO3 generates an extern "C" function whose exact signature depends on the Rust signature. (PyO3 chooses the optimal Python argument passing convention.) It then embeds the call to the Rust function inside this FFI-wrapper function. This wrapper handles extraction of the regular arguments and the keyword arguments from the input PyObjects.

The wrap_pyfunction macro can be used to directly get a Bound<PyCFunction> given a #[pyfunction] and a Bound<PyModule>: wrap_pyfunction!(rust_fun, module).

函式簽名

The #[pyfunction] attribute also accepts parameters to control how the generated Python function accepts arguments. Just like in Python, arguments can be positional-only, keyword-only, or accept either. *args lists and **kwargs dicts can also be accepted. These parameters also work for #[pymethods] which will be introduced in the Python Classes section of the guide.

Like Python, by default PyO3 accepts all arguments as either positional or keyword arguments. All arguments are required by default. This behaviour can be configured by the #[pyo3(signature = (...))] option which allows writing a signature in Python syntax.

This section of the guide goes into detail about use of the #[pyo3(signature = (...))] option and its related option #[pyo3(text_signature = "...")]

Using #[pyo3(signature = (...))]

For example, below is a function that accepts arbitrary keyword arguments (**kwargs in Python syntax) and returns the number that was passed:

#[pyo3::pymodule]
mod module_with_functions {
    use pyo3::prelude::*;
    use pyo3::types::PyDict;

    #[pyfunction]
    #[pyo3(signature = (**kwds))]
    fn num_kwds(kwds: Option<&Bound<'_, PyDict>>) -> usize {
        kwds.map_or(0, |dict| dict.len())
    }
}

Just like in Python, the following constructs can be part of the signature::

  • /: positional-only arguments separator, each parameter defined before / is a positional-only parameter.
  • *: var arguments separator, each parameter defined after * is a keyword-only parameter.
  • *args: “args” is var args. Type of the args parameter has to be &Bound<'_, PyTuple>.
  • **kwargs: “kwargs” receives keyword arguments. The type of the kwargs parameter has to be Option<&Bound<'_, PyDict>>.
  • arg=Value: arguments with default value. If the arg argument is defined after var arguments, it is treated as a keyword-only argument. Note that Value has to be valid rust code, PyO3 just inserts it into the generated code unmodified.

Example:

use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};

#[pyclass]
struct MyClass {
    num: i32,
}
#[pymethods]
impl MyClass {
    #[new]
    #[pyo3(signature = (num=-1))]
    fn new(num: i32) -> Self {
        MyClass { num }
    }

    #[pyo3(signature = (num=10, *py_args, name="Hello", **py_kwargs))]
    fn method(
        &mut self,
        num: i32,
        py_args: &Bound<'_, PyTuple>,
        name: &str,
        py_kwargs: Option<&Bound<'_, PyDict>>,
    ) -> String {
        let num_before = self.num;
        self.num = num;
        format!(
            "num={} (was previously={}), py_args={:?}, name={}, py_kwargs={:?} ",
            num, num_before, py_args, name, py_kwargs,
        )
    }

    fn make_change(&mut self, num: i32) -> PyResult<String> {
        self.num = num;
        Ok(format!("num={}", self.num))
    }
}

Arguments of type Python must not be part of the signature:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyfunction]
#[pyo3(signature = (lambda))]
pub fn simple_python_bound_function(py: Python<'_>, lambda: Py<PyAny>) -> PyResult<()> {
    Ok(())
}

N.B. the position of the / and * arguments (if included) control the system of handling positional and keyword arguments. In Python:

import mymodule

mc = mymodule.MyClass()
print(mc.method(44, False, "World", 666, x=44, y=55))
print(mc.method(num=-1, name="World"))
print(mc.make_change(44))

Produces output:

num=44 (was previously=-1), py_args=(False, 'World', 666), name=Hello, py_kwargs=Some({'x': 44, 'y': 55})
num=-1 (was previously=44), py_args=(), name=World, py_kwargs=None
num=44

[!NOTE] To use keywords like struct as a function argument, use “raw identifier” syntax r#struct in both the signature and the function definition:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyfunction(signature = (r#struct = "foo"))]
fn function_with_keyword(r#struct: &str) {
    let _ = r#struct;
    /* ... */
}

Making the function signature available to Python

The function signature is exposed to Python via the __text_signature__ attribute. PyO3 automatically generates this for every #[pyfunction] and all #[pymethods] directly from the Rust function, taking into account any override done with the #[pyo3(signature = (...))] option.

This automatic generation can only display the value of default arguments for strings, integers, boolean types, and None. Any other default arguments will be displayed as .... (.pyi type stub files commonly also use ... for default arguments in the same way.)

In cases where the automatically-generated signature needs adjusting, it can be overridden using the #[pyo3(text_signature)] option.)

The example below creates a function add which accepts two positional-only arguments a and b, where b has a default value of zero.

use pyo3::prelude::*;

/// This function adds two unsigned 64-bit integers.
#[pyfunction]
#[pyo3(signature = (a, b=0, /))]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(add, py)?;

        let doc: String = fun.getattr("__doc__")?.extract()?;
        assert_eq!(doc, "This function adds two unsigned 64-bit integers.");

        let inspect = PyModule::import(py, "inspect")?.getattr("signature")?;
        let sig: String = inspect
            .call1((fun,))?
            .call_method0("__str__")?
            .extract()?;

        #[cfg(Py_3_8)]  // on 3.7 the signature doesn't render b, upstream bug?
        assert_eq!(sig, "(a, b=0, /)");

        Ok(())
    })
}

The following IPython output demonstrates how this generated signature will be seen from Python tooling:

>>> pyo3_test.add.__text_signature__
'(a, b=..., /)'
>>> pyo3_test.add?
Signature: pyo3_test.add(a, b=0, /)
Docstring: This function adds two unsigned 64-bit integers.
Type:      builtin_function_or_method

Overriding the generated signature

The #[pyo3(text_signature = "(<some signature>)")] attribute can be used to override the default generated signature.

In the snippet below, the text signature attribute is used to include the default value of 0 for the argument b, instead of the automatically-generated default value of ...:

use pyo3::prelude::*;

/// This function adds two unsigned 64-bit integers.
#[pyfunction]
#[pyo3(signature = (a, b=0, /), text_signature = "(a, b=0, /)")]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(add, py)?;

        let doc: String = fun.getattr("__doc__")?.extract()?;
        assert_eq!(doc, "This function adds two unsigned 64-bit integers.");

        let inspect = PyModule::import(py, "inspect")?.getattr("signature")?;
        let sig: String = inspect
            .call1((fun,))?
            .call_method0("__str__")?
            .extract()?;
        assert_eq!(sig, "(a, b=0, /)");

        Ok(())
    })
}

PyO3 will include the contents of the annotation unmodified as the __text_signature__. Below shows how IPython will now present this (see the default value of 0 for b):

>>> pyo3_test.add.__text_signature__
'(a, b=0, /)'
>>> pyo3_test.add?
Signature: pyo3_test.add(a, b=0, /)
Docstring: This function adds two unsigned 64-bit integers.
Type:      builtin_function_or_method

If no signature is wanted at all, #[pyo3(text_signature = None)] will disable the built-in signature. The snippet below demonstrates use of this:

use pyo3::prelude::*;

/// This function adds two unsigned 64-bit integers.
#[pyfunction]
#[pyo3(signature = (a, b=0, /), text_signature = None)]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(add, py)?;

        let doc: String = fun.getattr("__doc__")?.extract()?;
        assert_eq!(doc, "This function adds two unsigned 64-bit integers.");
        assert!(fun.getattr("__text_signature__")?.is_none());

        Ok(())
    })
}

Now the function’s __text_signature__ will be set to None, and IPython will not display any signature in the help:

>>> pyo3_test.add.__text_signature__ == None
True
>>> pyo3_test.add?
Docstring: This function adds two unsigned 64-bit integers.
Type:      builtin_function_or_method

簽章中的型別註解

When the experimental-inspect Cargo feature is enabled, the signature attribute can also contain type hints:

#[cfg(feature = "experimental-inspect")] {
use pyo3::prelude::*;

#[pymodule]
pub mod example {
   use pyo3::prelude::*;

   #[pyfunction]
   #[pyo3(signature = (arg: "list[int]") -> "list[int]")]
   fn list_of_int_identity(arg: Bound<'_, PyAny>) -> Bound<'_, PyAny> {
      arg
   }
}
}

It enables the work-in-progress capacity of PyO3 to autogenerate type stubs to generate a file with the correct type hints:

def list_of_int_identity(arg: list[int]) -> list[int]: ...

instead of the generic:

import typing

def list_of_int_identity(arg: typing.Any) -> typing.Any: ...

Note that currently type annotations must be written as Rust strings.

錯誤處理

This chapter contains a little background of error handling in Rust and how PyO3 integrates this with Python exceptions.

This covers enough detail to create a #[pyfunction] which raises Python exceptions from errors originating in Rust.

There is a later section of the guide on Python exceptions which covers exception types in more detail.

Representing Python exceptions

Rust code uses the generic Result<T, E> enum to propagate errors. The error type E is chosen by the code author to describe the possible errors which can happen.

PyO3 has the PyErr type which represents a Python exception. If a PyO3 API could result in a Python exception being raised, the return type of that API will be PyResult<T>, which is an alias for the type Result<T, PyErr>.

In summary:

  • When Python exceptions are raised and caught by PyO3, the exception will be stored in the Err variant of the PyResult.
  • Passing Python exceptions through Rust code then uses all the “normal” techniques such as the ? operator, with PyErr as the error type.
  • Finally, when a PyResult crosses from Rust back to Python via PyO3, if the result is an Err variant the contained exception will be raised.

(There are many great tutorials on Rust error handling and the ? operator, so this guide will not go into detail on Rust-specific topics.)

Raising an exception from a function

As indicated in the previous section, when a PyResult containing an Err crosses from Rust to Python, PyO3 will raise the exception contained within.

Accordingly, to raise an exception from a #[pyfunction], change the return type T to PyResult<T>. When the function returns an Err it will raise a Python exception. (Other Result<T, E> types can be used as long as the error E has a From conversion for PyErr, see custom Rust error types below.)

This also works for functions in #[pymethods].

For example, the following check_positive function raises a ValueError when the input is negative:

use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;

#[pyfunction]
fn check_positive(x: i32) -> PyResult<()> {
    if x < 0 {
        Err(PyValueError::new_err("x is negative"))
    } else {
        Ok(())
    }
}

fn main(){
	Python::attach(|py|{
		let fun = pyo3::wrap_pyfunction!(check_positive, py).unwrap();
		fun.call1((-1,)).unwrap_err();
		fun.call1((1,)).unwrap();
	});
}

All built-in Python exception types are defined in the pyo3::exceptions module. They have a new_err constructor to directly build a PyErr, as seen in the example above.

Custom Rust error types

PyO3 will automatically convert a Result<T, E> returned by a #[pyfunction] into a PyResult<T> as long as there is an implementation of std::from::From<E> for PyErr. Many error types in the Rust standard library have a From conversion defined in this way.

If the type E you are handling is defined in a third-party crate, see the section on foreign rust error types below for ways to work with this error.

The following example makes use of the implementation of From<ParseIntError> for PyErr to raise exceptions encountered when parsing strings as integers:

use pyo3::prelude::*;
use std::num::ParseIntError;

#[pyfunction]
fn parse_int(x: &str) -> Result<usize, ParseIntError> {
    x.parse()
}

fn main() {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(parse_int, py).unwrap();
        let value: usize = fun.call1(("5",)).unwrap().extract().unwrap();
        assert_eq!(value, 5);
    });
}

When passed a string which doesn’t contain a floating-point number, the exception raised will look like the below:

>>> parse_int("bar")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid digit found in string

As a more complete example, the following snippet defines a Rust error named CustomIOError. It then defines a From<CustomIOError> for PyErr, which returns a PyErr representing Python’s OSError. Therefore, it can use this error in the result of a #[pyfunction] directly, relying on the conversion if it has to be propagated into a Python exception.

use pyo3::exceptions::PyOSError;
use pyo3::prelude::*;
use std::fmt;

#[derive(Debug)]
struct CustomIOError;

impl std::error::Error for CustomIOError {}

impl fmt::Display for CustomIOError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "Oh no!")
    }
}

impl std::convert::From<CustomIOError> for PyErr {
    fn from(err: CustomIOError) -> PyErr {
        PyOSError::new_err(err.to_string())
    }
}

pub struct Connection {/* ... */}

fn bind(addr: String) -> Result<Connection, CustomIOError> {
    if &addr == "0.0.0.0" {
        Err(CustomIOError)
    } else {
        Ok(Connection{ /* ... */})
    }
}

#[pyfunction]
fn connect(s: String) -> Result<(), CustomIOError> {
    bind(s)?;
    // etc.
    Ok(())
}

fn main() {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(connect, py).unwrap();
        let err = fun.call1(("0.0.0.0",)).unwrap_err();
        assert!(err.is_instance_of::<PyOSError>(py));
    });
}

If lazy construction of the Python exception instance is desired, the PyErrArguments trait can be implemented instead of From. In that case, actual exception argument creation is delayed until the PyErr is needed.

A final note is that any errors E which have a From conversion can be used with the ? (“try”) operator with them. An alternative implementation of the above parse_int which instead returns PyResult is below:

use pyo3::prelude::*;

fn parse_int(s: String) -> PyResult<usize> {
    let x = s.parse()?;
    Ok(x)
}

use pyo3::exceptions::PyValueError;

fn main() {
    Python::attach(|py| {
        assert_eq!(parse_int(String::from("1")).unwrap(), 1);
        assert_eq!(parse_int(String::from("1337")).unwrap(), 1337);

        assert!(parse_int(String::from("-1"))
            .unwrap_err()
            .is_instance_of::<PyValueError>(py));
        assert!(parse_int(String::from("foo"))
            .unwrap_err()
            .is_instance_of::<PyValueError>(py));
        assert!(parse_int(String::from("13.37"))
            .unwrap_err()
            .is_instance_of::<PyValueError>(py));
    })
}

Foreign Rust error types

The Rust compiler will not permit implementation of traits for types outside of the crate where the type is defined. (This is known as the “orphan rule”.)

Given a type OtherError which is defined in third-party code, there are two main strategies available to integrate it with PyO3:

  • Create a newtype wrapper, e.g. MyOtherError. Then implement From<MyOtherError> for PyErr (or PyErrArguments), as well as From<OtherError> for MyOtherError.
  • Use Rust’s Result combinators such as map_err to write code freely to convert OtherError into whatever is needed. This requires boilerplate at every usage however gives unlimited flexibility.

To detail the newtype strategy a little further, the key trick is to return Result<T, MyOtherError> from the #[pyfunction]. This means that PyO3 will make use of From<MyOtherError> for PyErr to create Python exceptions while the #[pyfunction] implementation can use ? to convert OtherError to MyOtherError automatically.

The following example demonstrates this for some imaginary third-party crate some_crate with a function get_x returning Result<i32, OtherError>:

mod some_crate {
  pub struct OtherError(());
  impl OtherError {
      pub fn message(&self) -> &'static str { "some error occurred" }
  }
  pub fn get_x() -> Result<i32, OtherError> { Ok(5) }
}

use pyo3::prelude::*;
use pyo3::exceptions::PyValueError;
use some_crate::{OtherError, get_x};

struct MyOtherError(OtherError);

impl From<MyOtherError> for PyErr {
    fn from(error: MyOtherError) -> Self {
        PyValueError::new_err(error.0.message())
    }
}

impl From<OtherError> for MyOtherError {
    fn from(other: OtherError) -> Self {
        Self(other)
    }
}

#[pyfunction]
fn wrapped_get_x() -> Result<i32, MyOtherError> {
    // get_x is a function returning Result<i32, OtherError>
    let x: i32 = get_x()?;
    Ok(x)
}

fn main() {
    Python::attach(|py| {
        let fun = pyo3::wrap_pyfunction!(wrapped_get_x, py).unwrap();
        let value: usize = fun.call0().unwrap().extract().unwrap();
        assert_eq!(value, 5);
    });
}

Notes

In Python 3.11 and up, notes can be added to Python exceptions to provide additional debugging information when printing the exception. In PyO3, you can use the add_note method on PyErr to accomplish this functionality.

Python 類別

PyO3 exposes a group of attributes powered by Rust’s proc macro system for defining Python classes as Rust structs.

The main attribute is #[pyclass], which is placed upon a Rust struct or enum to generate a Python type for it. They will usually also have one #[pymethods]-annotated impl block for the struct, which is used to define Python methods and constants for the generated Python type. (If the multiple-pymethods feature is enabled, each #[pyclass] is allowed to have multiple #[pymethods] blocks.) #[pymethods] may also have implementations for Python magic methods such as __str__.

This chapter will discuss the functionality and configuration these attributes offer. Below is a list of links to the relevant section of this chapter for each:

定義新類別

To define a custom Python class, add the #[pyclass] attribute to a Rust struct or enum.

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyclass]
struct MyClass {
    inner: i32,
}

// A "tuple" struct
#[pyclass]
struct Number(i32);

// PyO3 supports unit-only enums (which contain only unit variants)
// These simple enums behave similarly to Python's enumerations (enum.Enum)
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum MyEnum {
    Variant,
    OtherVariant = 30, // PyO3 supports custom discriminants.
}

// PyO3 supports custom discriminants in unit-only enums
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum HttpResponse {
    Ok = 200,
    NotFound = 404,
    Teapot = 418,
    // ...
}

// PyO3 also supports enums with Struct and Tuple variants
// These complex enums have slightly different behavior from the simple enums above
// They are meant to work with instance checks and match statement patterns
// The variants can be mixed and matched
// Struct variants have named fields while tuple enums generate generic names for fields in order _0, _1, _2, ...
// Apart from this both types are functionally identical
#[pyclass]
enum Shape {
    Circle { radius: f64 },
    Rectangle { width: f64, height: f64 },
    RegularPolygon(u32, f64),
    Nothing(),
}

The above example generates implementations for PyTypeInfo and PyClass for MyClass, Number, MyEnum, HttpResponse, and Shape. To see these generated implementations, refer to the implementation details at the end of this chapter.

Restrictions

To integrate Rust types with Python, PyO3 needs to place some restrictions on the types which can be annotated with #[pyclass]. In particular, they must have no lifetime parameters, no generic parameters, and must be thread-safe. The reason for each of these is explained below.

No lifetime parameters

Rust lifetimes are used by the Rust compiler to reason about a program’s memory safety. They are a compile-time only concept; there is no way to access Rust lifetimes at runtime from a dynamic language like Python.

As soon as Rust data is exposed to Python, there is no guarantee that the Rust compiler can make on how long the data will live. Python is a reference-counted language and those references can be held for an arbitrarily long time which is untraceable by the Rust compiler. The only possible way to express this correctly is to require that any #[pyclass] does not borrow data for any lifetime shorter than the 'static lifetime, i.e. the #[pyclass] cannot have any lifetime parameters.

When you need to share ownership of data between Python and Rust, instead of using borrowed references with lifetimes consider using reference-counted smart pointers such as Arc or Py.

No generic parameters

A Rust struct Foo<T> with a generic parameter T generates new compiled implementations each time it is used with a different concrete type for T. These new implementations are generated by the compiler at each usage site. This is incompatible with wrapping Foo in Python, where there needs to be a single compiled implementation of Foo which is integrated with the Python interpreter.

Currently, the best alternative is to write a macro which expands to a new #[pyclass] for each instantiation you want:

#![allow(dead_code)]
use pyo3::prelude::*;

struct GenericClass<T> {
    data: T,
}

macro_rules! create_interface {
    ($name: ident, $type: ident) => {
        #[pyclass]
        pub struct $name {
            inner: GenericClass<$type>,
        }
        #[pymethods]
        impl $name {
            #[new]
            pub fn new(data: $type) -> Self {
                Self {
                    inner: GenericClass { data: data },
                }
            }
        }
    };
}

create_interface!(IntClass, i64);
create_interface!(FloatClass, String);

Must be thread-safe

Python objects are freely shared between threads by the Python interpreter. This means that:

  • Python objects may be created and destroyed by different Python threads; therefore #[pyclass] objects must be Send.
  • Python objects may be accessed by multiple Python threads simultaneously; therefore #[pyclass] objects must be Sync.

For now, don’t worry about these requirements; simple classes will already be thread-safe. There is a detailed discussion on thread-safety later in the guide.

Constructor

By default, it is not possible to create an instance of a custom class from Python code. To declare a constructor, you need to define a method and annotate it with the #[new] attribute. A constructor is accessible as Python’s __new__ method.

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    #[new]
    fn new(value: i32) -> Self {
        Number(value)
    }
}

Alternatively, if your new method may fail you can return PyResult<Self>.

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::exceptions::PyValueError;
#[pyclass]
struct Nonzero(i32);

#[pymethods]
impl Nonzero {
    #[new]
    fn py_new(value: i32) -> PyResult<Self> {
        if value == 0 {
            Err(PyValueError::new_err("cannot be zero"))
        } else {
            Ok(Nonzero(value))
        }
    }
}

If you want to return an existing object (for example, because your new method caches the values it returns), new can return pyo3::Py<Self>.

As you can see, the Rust method name is not important here; this way you can still, use new() for a Rust-level constructor.

If no method marked with #[new] is declared, object instances can only be created from Rust, but not from Python.

For arguments, see the Method arguments section below.

Initializer

An initializer implements Python’s __init__ method.

It may be required when it’s needed to control an object initalization flow on the Rust code. If possible handling this in __new__ should be preferred, but in some cases, like subclassing native types, overwriting __init__ might be necessary. For example, you define a class that extends PyDict and don’t want that the original __init__ method of PyDict been called. In this case by defining an own __init__ method it’s possible to stop initialization flow.

If you declare an __init__ method you may need to call a super class’ __init__ method explicitly like in Python code.

To declare an initializer, you need to define the __init__ method. Like in Python __init__ must have the self receiver as the first argument, followed by the same arguments as the constructor. It can either return () or PyResult<()>.

#![allow(dead_code)]
use pyo3::prelude::*;
#[cfg(not(any(Py_LIMITED_API, GraalPy)))]
use pyo3::types::{PyDict, PyTuple, PySuper};
#[cfg(not(any(Py_LIMITED_API, GraalPy)))]
use crate::pyo3::PyTypeInfo;

#[cfg(not(any(Py_LIMITED_API, GraalPy)))]
#[pyclass(extends = PyDict)]
struct MyDict;

#[cfg(not(any(Py_LIMITED_API, GraalPy)))]
#[pymethods]
impl MyDict {
  #[allow(unused_variables)]
    #[new]
    #[pyo3(signature = (*args, **kwargs))]
    fn __new__(
        args: &Bound<'_, PyTuple>,
        kwargs: Option<&Bound<'_, PyDict>>,
    ) -> PyResult<Self> {
        Ok(Self)
    }

    #[pyo3(signature = (*args, **kwargs))]
    fn __init__(
        slf: &Bound<'_, Self>,
        args: &Bound<'_, PyTuple>,
        kwargs: Option<&Bound<'_, PyDict>>,
    ) -> PyResult<()> {
        // call the super types __init__
        PySuper::new(&PyDict::type_object(slf.py()), slf)?
            .call_method("__init__", args.to_owned(), kwargs)?;
        // Note: if `MyDict` allows further subclassing, and this is called from such a subclass,
        // then this will not that any overrides into account that such a subclass may have defined.
        // In such a case it may be preferred to just call `slf.set_item` and let Python figure it out.
        slf.as_super().set_item("my_key", "always insert this key")?;
        Ok(())
    }
}

#[cfg(not(any(Py_LIMITED_API, GraalPy)))]
fn main() {
    Python::attach(|py| {
        let typeobj = py.get_type::<MyDict>();
        let obj = typeobj.call((), None).unwrap().cast_into::<MyDict>().unwrap();
        // check __init__ was called
        assert_eq!(obj.get_item("my_key").unwrap().extract::<&str>().unwrap(), "always insert this key");
    });
}
#[cfg(any(Py_LIMITED_API, GraalPy))]
fn main() {}

Adding the class to a module

The next step is to create the Python module and add our class to it:

#![allow(dead_code)]
use pyo3::prelude::*;
fn main() {}
#[pyclass]
struct Number(i32);

#[pymodule]
mod my_module {
    #[pymodule_export]
    use super::Number;
}

Bound<T> and interior mutability

It is often useful to turn a #[pyclass] type T into a Python object and access it from Rust code. The Py<T> and Bound<'py, T> smart pointers are the ways to represent a Python object in PyO3’s API. More detail can be found about them in the Python objects section of the guide.

Most Python objects do not offer exclusive (&mut) access (see the section on Python’s memory model). However, Rust structs wrapped as Python objects (called pyclass types) often do need &mut access. However, the Rust borrow checker cannot reason about &mut references once an object’s ownership has been passed to the Python interpreter.

To solve this, PyO3 does borrow checking at runtime using a scheme very similar to std::cell::RefCell<T>. This is known as interior mutability.

Users who are familiar with RefCell<T> can use Py<T> and Bound<'py, T> just like RefCell<T>.

For users who are not very familiar with RefCell<T>, here is a reminder of Rust’s rules of borrowing:

  • At any given time, you can have either (but not both of) one mutable reference or any number of immutable references.
  • References can never outlast the data they refer to.

Py<T> and Bound<'py, T>, like RefCell<T>, ensure these borrowing rules by tracking references at runtime.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    #[pyo3(get)]
    num: i32,
}
Python::attach(|py| {
    let obj = Bound::new(py, MyClass { num: 3 }).unwrap();
    {
        let obj_ref = obj.borrow(); // Get PyRef
        assert_eq!(obj_ref.num, 3);
        // You cannot get PyRefMut unless all PyRefs are dropped
        assert!(obj.try_borrow_mut().is_err());
    }
    {
        let mut obj_mut = obj.borrow_mut(); // Get PyRefMut
        obj_mut.num = 5;
        // You cannot get any other refs until the PyRefMut is dropped
        assert!(obj.try_borrow().is_err());
        assert!(obj.try_borrow_mut().is_err());
    }

    // You can convert `Bound` to a Python object
    pyo3::py_run!(py, obj, "assert obj.num == 5");
});

A Bound<'py, T> is restricted to the Python lifetime 'py. To make the object longer lived (for example, to store it in a struct on the Rust side), use Py<T>. Py<T> needs a Python<'_> token to allow access:

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    num: i32,
}

fn return_myclass() -> Py<MyClass> {
    Python::attach(|py| Py::new(py, MyClass { num: 1 }).unwrap())
}

let obj = return_myclass();

Python::attach(move |py| {
    let bound = obj.bind(py); // Py<MyClass>::bind returns &Bound<'py, MyClass>
    let obj_ref = bound.borrow(); // Get PyRef<T>
    assert_eq!(obj_ref.num, 1);
});

frozen classes: Opting out of interior mutability

As detailed above, runtime borrow checking is currently enabled by default. But a class can opt of out it by declaring itself frozen. It can still use interior mutability via standard Rust types like RefCell or Mutex, but it is not bound to the implementation provided by PyO3 and can choose the most appropriate strategy on field-by-field basis.

Classes which are frozen and also Sync, e.g. they do use Mutex but not RefCell, can be accessed without needing a Python token via the Bound::get and Py::get methods:

use std::sync::atomic::{AtomicUsize, Ordering};
use pyo3::prelude::*;

#[pyclass(frozen)]
struct FrozenCounter {
    value: AtomicUsize,
}

let py_counter: Py<FrozenCounter> = Python::attach(|py| {
    let counter = FrozenCounter {
        value: AtomicUsize::new(0),
    };

    Py::new(py, counter).unwrap()
});

py_counter.get().value.fetch_add(1, Ordering::Relaxed);

Python::attach(move |_py| drop(py_counter));

Frozen classes are likely to become the default thereby guiding the PyO3 ecosystem towards a more deliberate application of interior mutability. Eventually, this should enable further optimizations of PyO3’s internals and avoid downstream code paying the cost of interior mutability when it is not actually required.

Customizing the class

#[pyclass] can be used with the following parameters:

參數說明
constructorThis is currently only allowed on variants of complex enums. It allows customization of the generated class constructor for each variant. It uses the same syntax and supports the same options as the signature attribute of functions and methods.
crate = "some::path"Path to import the pyo3 crate, if it’s not accessible at ::pyo3.
dictGives instances of this class an empty __dict__ to store custom attributes.
eqImplements __eq__ using the PartialEq implementation of the underlying Rust datatype.
eq_intImplements __eq__ using __int__ for simple enums.
extends = BaseTypeUse a custom baseclass. Defaults to PyAny
freelist = NImplements a free list of size N. This can improve performance for types that are often created and deleted in quick succession. Profile your code to see whether freelist is right for you.
from_py_objectImplement FromPyObject for this pyclass. Requires the pyclass to be Clone.
frozenDeclares that your pyclass is immutable. It removes the borrow checker overhead when retrieving a shared reference to the Rust struct, but disables the ability to get a mutable reference.
genericImplements runtime parametrization for the class following PEP 560.
get_allGenerates getters for all fields of the pyclass.
hashImplements __hash__ using the Hash implementation of the underlying Rust datatype. Requires eq and frozen
immutable_typeMakes the type object immutable. Supported on 3.14+ with the abi3 feature active, or 3.10+ otherwise.
mappingInform PyO3 that this class is a Mapping, and so leave its implementation of sequence C-API slots empty.
module = "module_name"Python code will see the class as being defined in this module. Defaults to builtins.
name = "python_name"Sets the name that Python sees this class as. Defaults to the name of the Rust struct.
ordImplements __lt__, __gt__, __le__, & __ge__ using the PartialOrd implementation of the underlying Rust datatype. Requires eq
rename_all = "renaming_rule"Applies renaming rules to every getters and setters of a struct, or every variants of an enum. Possible values are: “camelCase”, “kebab-case”, “lowercase”, “PascalCase”, “SCREAMING-KEBAB-CASE”, “SCREAMING_SNAKE_CASE”, “snake_case”, “UPPERCASE”.
sequenceInform PyO3 that this class is a Sequence, and so leave its C-API mapping length slot empty.
set_allGenerates setters for all fields of the pyclass.
new = "from_fields"Generates a default __new__ constructor with all fields as parameters in the new() method.
skip_from_py_objectPrevents this PyClass from participating in the FromPyObject: PyClass + Clone blanket implementation. This allows a custom FromPyObject impl, even if self is Clone.
strImplements __str__ using the Display implementation of the underlying Rust datatype or by passing an optional format string str="<format string>". Note: The optional format string is only allowed for structs. name and rename_all are incompatible with the optional format string. Additional details can be found in the discussion on this PR.
subclassAllows other Python classes and #[pyclass] to inherit from this class. Enums cannot be subclassed.
unsendableRequired if your struct is not Send. Rather than using unsendable, consider implementing your struct in a thread-safe way by e.g. substituting Rc with Arc. By using unsendable, your class will panic when accessed by another thread. Also note the Python’s GC is multi-threaded and while unsendable classes will not be traversed on foreign threads to avoid UB, this can lead to memory leaks.
weakrefAllows this class to be weakly referenceable.

All of these parameters can either be passed directly on the #[pyclass(...)] annotation, or as one or more accompanying #[pyo3(...)] annotations, e.g.:

// Argument supplied directly to the `#[pyclass]` annotation.
#[pyclass(name = "SomeName", subclass)]
struct MyClass {}

// Argument supplied as a separate annotation.
#[pyclass]
#[pyo3(name = "SomeName", subclass)]
struct MyClass {}

These parameters are covered in various sections of this guide.

Return type

Generally, #[new] methods have to return T: Into<PyClassInitializer<Self>> or PyResult<T> where T: Into<PyClassInitializer<Self>>.

For constructors that may fail, you should wrap the return type in a PyResult as well. Consult the table below to determine which type your constructor should return:

Cannot failMay fail
No inheritanceTPyResult<T>
Inheritance(T Inherits U)(T, U)PyResult<(T, U)>
Inheritance(General Case)PyClassInitializer<T>PyResult<PyClassInitializer<T>>

Inheritance

By default, object, i.e. PyAny is used as the base class. To override this default, use the extends parameter for pyclass with the full path to the base class. Currently, only classes defined in Rust and builtins provided by PyO3 can be inherited from; inheriting from other classes defined in Python is not yet supported (#991).

For convenience, (T, U) implements Into<PyClassInitializer<T>> where U is the base class of T. But for a more deeply nested inheritance, you have to return PyClassInitializer<T> explicitly.

To get a parent class from a child, use PyRef instead of &self for methods, or PyRefMut instead of &mut self. Then you can access a parent class by self_.as_super() as &PyRef<Self::BaseClass>, or by self_.into_super() as PyRef<Self::BaseClass> (and similar for the PyRefMut case). For convenience, self_.as_ref() can also be used to get &Self::BaseClass directly; however, this approach does not let you access base classes higher in the inheritance hierarchy, for which you would need to chain multiple as_super or into_super calls.

use pyo3::prelude::*;

#[pyclass(subclass)]
struct BaseClass {
    val1: usize,
}

#[pymethods]
impl BaseClass {
    #[new]
    fn new() -> Self {
        BaseClass { val1: 10 }
    }

    pub fn method1(&self) -> PyResult<usize> {
        Ok(self.val1)
    }
}

#[pyclass(extends=BaseClass, subclass)]
struct SubClass {
    val2: usize,
}

#[pymethods]
impl SubClass {
    #[new]
    fn new() -> (Self, BaseClass) {
        (SubClass { val2: 15 }, BaseClass::new())
    }

    fn method2(self_: PyRef<'_, Self>) -> PyResult<usize> {
        let super_ = self_.as_super(); // Get &PyRef<BaseClass>
        super_.method1().map(|x| x * self_.val2)
    }
}

#[pyclass(extends=SubClass)]
struct SubSubClass {
    val3: usize,
}

#[pymethods]
impl SubSubClass {
    #[new]
    fn new() -> PyClassInitializer<Self> {
        PyClassInitializer::from(SubClass::new()).add_subclass(SubSubClass { val3: 20 })
    }

    fn method3(self_: PyRef<'_, Self>) -> PyResult<usize> {
        let base = self_.as_super().as_super(); // Get &PyRef<'_, BaseClass>
        base.method1().map(|x| x * self_.val3)
    }

    fn method4(self_: PyRef<'_, Self>) -> PyResult<usize> {
        let v = self_.val3;
        let super_ = self_.into_super(); // Get PyRef<'_, SubClass>
        SubClass::method2(super_).map(|x| x * v)
    }

      fn get_values(self_: PyRef<'_, Self>) -> (usize, usize, usize) {
          let val1 = self_.as_super().as_super().val1;
          let val2 = self_.as_super().val2;
          (val1, val2, self_.val3)
      }

    fn double_values(mut self_: PyRefMut<'_, Self>) {
        self_.as_super().as_super().val1 *= 2;
        self_.as_super().val2 *= 2;
        self_.val3 *= 2;
    }

    #[staticmethod]
    fn factory_method(py: Python<'_>, val: usize) -> PyResult<Py<PyAny>> {
        let base = PyClassInitializer::from(BaseClass::new());
        let sub = base.add_subclass(SubClass { val2: val });
        if val % 2 == 0 {
            Ok(Py::new(py, sub)?.into_any())
        } else {
            let sub_sub = sub.add_subclass(SubSubClass { val3: val });
            Ok(Py::new(py, sub_sub)?.into_any())
        }
    }
}
Python::attach(|py| {
    let subsub = pyo3::Py::new(py, SubSubClass::new()).unwrap();
    pyo3::py_run!(py, subsub, "assert subsub.method1() == 10");
    pyo3::py_run!(py, subsub, "assert subsub.method2() == 150");
    pyo3::py_run!(py, subsub, "assert subsub.method3() == 200");
    pyo3::py_run!(py, subsub, "assert subsub.method4() == 3000");
    pyo3::py_run!(py, subsub, "assert subsub.get_values() == (10, 15, 20)");
    pyo3::py_run!(py, subsub, "assert subsub.double_values() == None");
    pyo3::py_run!(py, subsub, "assert subsub.get_values() == (20, 30, 40)");
    let subsub = SubSubClass::factory_method(py, 2).unwrap();
    let subsubsub = SubSubClass::factory_method(py, 3).unwrap();
    let cls = py.get_type::<SubSubClass>();
    pyo3::py_run!(py, subsub cls, "assert not isinstance(subsub, cls)");
    pyo3::py_run!(py, subsubsub cls, "assert isinstance(subsubsub, cls)");
});

You can inherit native types such as PyDict, if they implement PySizedLayout. This is not supported when building for the Python limited API (aka the abi3 feature of PyO3).

To convert between the Rust type and its native base class, you can take slf as a Python object. To access the Rust fields use slf.borrow() or slf.borrow_mut(), and to access the base class use slf.cast::<BaseClass>().

#[cfg(any(not(Py_LIMITED_API), Py_3_12))] {
use pyo3::prelude::*;
use pyo3::types::PyDict;
use std::collections::HashMap;

#[pyclass(extends=PyDict)]
#[derive(Default)]
struct DictWithCounter {
    counter: HashMap<String, usize>,
}

#[pymethods]
impl DictWithCounter {
    #[new]
    fn new() -> Self {
        Self::default()
    }

    fn set(slf: &Bound<'_, Self>, key: String, value: Bound<'_, PyAny>) -> PyResult<()> {
        slf.borrow_mut().counter.entry(key.clone()).or_insert(0);
        let dict = slf.cast::<PyDict>()?;
        dict.set_item(key, value)
    }
}
Python::attach(|py| {
    let cnt = pyo3::Py::new(py, DictWithCounter::new()).unwrap();
    pyo3::py_run!(py, cnt, "cnt.set('abc', 10); assert cnt['abc'] == 10")
});
}

If SubClass does not provide a base class initialization, the compilation fails.

use pyo3::prelude::*;

#[pyclass]
struct BaseClass {
    val1: usize,
}

#[pyclass(extends=BaseClass)]
struct SubClass {
    val2: usize,
}

#[pymethods]
impl SubClass {
    #[new]
    fn new() -> Self {
        SubClass { val2: 15 }
    }
}

The __new__ constructor of a native base class is called implicitly when creating a new instance from Python. Be sure to accept arguments in the #[new] method that you want the base class to get, even if they are not used in that fn:

#[allow(dead_code)]
#[cfg(any(not(Py_LIMITED_API), Py_3_12))] {
use pyo3::prelude::*;
use pyo3::types::PyDict;

#[pyclass(extends=PyDict)]
struct MyDict {
    private: i32,
}

#[pymethods]
impl MyDict {
    #[new]
    #[pyo3(signature = (*args, **kwargs))]
    fn new(args: &Bound<'_, PyAny>, kwargs: Option<&Bound<'_, PyAny>>) -> Self {
        Self { private: 0 }
    }

    // some custom methods that use `private` here...
}
Python::attach(|py| {
    let cls = py.get_type::<MyDict>();
    pyo3::py_run!(py, cls, "cls(a=1, b=2)")
});
}

Here, the args and kwargs allow creating instances of the subclass passing initial items, such as MyDict(item_sequence) or MyDict(a=1, b=2).

Object properties

PyO3 supports two ways to add properties to your #[pyclass]:

  • For simple struct fields with no side effects, a #[pyo3(get, set)] attribute can be added directly to the field definition in the #[pyclass].
  • For properties which require computation you can define #[getter], #[setter] and #[deleter] functions in the #[pymethods] block.

We’ll cover each of these in the following sections.

Object properties using #[pyo3(get, set)]

For simple cases where a member variable is just read and written with no side effects, you can declare getters and setters in your #[pyclass] field definition using the pyo3 attribute, like in the example below:

use pyo3::prelude::*;
#[allow(dead_code)]
#[pyclass]
struct MyClass {
    #[pyo3(get, set)]
    num: i32,
}

The above would make the num field available for reading and writing as a self.num Python property. To expose the property with a different name to the field, specify this alongside the rest of the options, e.g. #[pyo3(get, set, name = "custom_name")].

Properties can be readonly or writeonly by using just #[pyo3(get)] or #[pyo3(set)] respectively.

To use these annotations, your field type must implement some conversion traits:

  • For get the field type T must implement either &T: IntoPyObject or T: IntoPyObject + Clone.
  • For set the field type must implement FromPyObject.

For example, implementations of those traits are provided for the Cell type, if the inner type also implements the trait. This means you can use #[pyo3(get, set)] on fields wrapped in a Cell.

Object properties using #[getter] and #[setter]

For cases which don’t satisfy the #[pyo3(get, set)] trait requirements, or need side effects, descriptor methods can be defined in a #[pymethods] impl block.

This is done using the #[getter] and #[setter] attributes, like in the example below:

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    num: i32,
}

#[pymethods]
impl MyClass {
    #[getter]
    fn num(&self) -> PyResult<i32> {
        Ok(self.num)
    }

    #[setter]
    fn set_num(&mut self, num: i32) {
        self.num = num;
    }
}

The #[deleter] attribute is also available to delete the property. This corresponds to the del my_object.my_property python operation.

use pyo3::prelude::*;
use pyo3::exceptions::PyAttributeError;
#[pyclass]
struct MyClass {
    num: Option<i32>,
}

#[pymethods]
impl MyClass {
    #[getter]
    fn num(&self) -> PyResult<i32> {
        self.num.ok_or_else(|| PyAttributeError::new_err("num has been deleted"))
    }

    #[deleter]
    fn delete_num(&mut self) {
        self.num = None;
    }
}

A getter, setter or deleters’s function name is used as the property name by default. There are several ways how to override the name.

If a function name starts with get_, set_ or delete_ for getter, setter or deleter, respectively, the descriptor name becomes the function name with this prefix removed. This is also useful in case of Rust keywords like type (raw identifiers can be used since Rust 2018).

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    num: i32,
}
#[pymethods]
impl MyClass {
    #[getter]
    fn get_num(&self) -> PyResult<i32> {
        Ok(self.num)
    }

    #[setter]
    fn set_num(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
    }
}

In this case, a property num is defined and available from Python code as self.num.

The #[getter], #[setter] and #[deleter] attributes accept one parameter. If this parameter is specified, it is used as the property name, i.e.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
   num: i32,
}
#[pymethods]
impl MyClass {
    #[getter(number)]
    fn num(&self) -> PyResult<i32> {
        Ok(self.num)
    }

    #[setter(number)]
    fn set_num(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
    }
}

In this case, the property number is defined and available from Python code as self.number.

Instance methods

To define a Python compatible method, an impl block for your struct has to be annotated with the #[pymethods] attribute. PyO3 generates Python compatible wrappers for all functions in this block with some variations, like descriptors, class method static methods, etc.

Since Rust allows any number of impl blocks, you can easily split methods between those accessible to Python (and Rust) and those accessible only to Rust. However to have multiple #[pymethods]-annotated impl blocks for the same struct you must enable the multiple-pymethods feature of PyO3.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    num: i32,
}
#[pymethods]
impl MyClass {
    fn method1(&self) -> PyResult<i32> {
        Ok(10)
    }

    fn set_method(&mut self, value: i32) -> PyResult<()> {
        self.num = value;
        Ok(())
    }
}

Both &self and &mut self can be used, due to the use of runtime borrow checking.

The return type must be PyResult<T> or T for some T that implements IntoPyObject; the latter is allowed if the method cannot raise Python exceptions.

A Python parameter can be specified as part of method signature, in this case the py argument gets injected by the method wrapper, e.g.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
#[allow(dead_code)]
    num: i32,
}
#[pymethods]
impl MyClass {
    fn method2(&self, py: Python<'_>) -> PyResult<i32> {
        Ok(10)
    }
}

From the Python perspective, the method2 in this example does not accept any arguments.

Class methods

To create a class method for a custom class, the method needs to be annotated with the #[classmethod] attribute. This is the equivalent of the Python decorator @classmethod.

use pyo3::prelude::*;
use pyo3::types::PyType;
#[pyclass]
struct MyClass {
    #[allow(dead_code)]
    num: i32,
}
#[pymethods]
impl MyClass {
    #[classmethod]
    fn cls_method(cls: &Bound<'_, PyType>) -> PyResult<i32> {
        Ok(10)
    }
}

Declares a class method callable from Python.

  • The first parameter is the type object of the class on which the method is called. This may be the type object of a derived class.
  • The first parameter implicitly has type &Bound<'_, PyType>.
  • For details on parameter-list, see the documentation of Method arguments section.
  • The return type must be PyResult<T> or T for some T that implements IntoPyObject.

Constructors which accept a class argument

To create a constructor which takes a positional class argument, you can combine the #[classmethod] and #[new] modifiers:

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyType;
#[pyclass]
struct BaseClass(Py<PyAny>);

#[pymethods]
impl BaseClass {
    #[new]
    #[classmethod]
    fn py_new(cls: &Bound<'_, PyType>) -> PyResult<Self> {
        // Get an abstract attribute (presumably) declared on a subclass of this class.
        let subclass_attr: Bound<'_, PyAny> = cls.getattr("a_class_attr")?;
        Ok(Self(subclass_attr.unbind()))
    }
}

Static methods

To create a static method for a custom class, the method needs to be annotated with the #[staticmethod] attribute. The return type must be T or PyResult<T> for some T that implements IntoPyObject.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    #[allow(dead_code)]
    num: i32,
}
#[pymethods]
impl MyClass {
    #[staticmethod]
    fn static_method(param1: i32, param2: &str) -> PyResult<i32> {
        Ok(10)
    }
}

Class attributes

To create a class attribute (also called class variable), a method without any arguments can be annotated with the #[classattr] attribute.

use pyo3::prelude::*;
#[pyclass]
struct MyClass {}
#[pymethods]
impl MyClass {
    #[classattr]
    fn my_attribute() -> String {
        "hello".to_string()
    }
}

Python::attach(|py| {
    let my_class = py.get_type::<MyClass>();
    pyo3::py_run!(py, my_class, "assert my_class.my_attribute == 'hello'")
});

[!NOTE] If the method has a Result return type and returns an Err, PyO3 will panic during class creation.

[!NOTE] #[classattr] does not work with #[pyo3(warn(...))] attribute.

If the class attribute is defined with const code only, one can also annotate associated constants:

use pyo3::prelude::*;
#[pyclass]
struct MyClass {}
#[pymethods]
impl MyClass {
    #[classattr]
    const MY_CONST_ATTRIBUTE: &'static str = "foobar";
}

Classes as function arguments

Class objects can be used as arguments to #[pyfunction]s and #[pymethods] in the same way as the self parameters of instance methods, i.e. they can be passed as:

  • Py<T> or Bound<'py, T> smart pointers to the class Python object,
  • &T or &mut T references to the Rust data contained in the Python object, or
  • PyRef<T> and PyRefMut<T> reference wrappers.

Examples of each of these below:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyclass]
struct MyClass {
    my_field: i32,
}

// Take a reference to Rust data when the Python object is irrelevant.
#[pyfunction]
fn increment_field(my_class: &mut MyClass) {
    my_class.my_field += 1;
}

// Take a reference wrapper when borrowing should be automatic,
// but access to the Python object is still needed
#[pyfunction]
fn print_field_and_return_me(my_class: PyRef<'_, MyClass>) -> PyRef<'_, MyClass> {
    println!("{}", my_class.my_field);
    my_class
}

// Take (a reference to) a Python object smart pointer when borrowing needs to be managed manually.
#[pyfunction]
fn increment_then_print_field(my_class: &Bound<'_, MyClass>) {
    my_class.borrow_mut().my_field += 1;

    println!("{}", my_class.borrow().my_field);
}

// When the Python object smart pointer needs to be stored elsewhere prefer `Py<T>` over `Bound<'py, T>`
// to avoid the lifetime restrictions.
#[pyfunction]
fn print_is_none(my_class: Py<MyClass>, py: Python<'_>) {
    println!("{}", my_class.is_none(py));
}

Classes can also be passed by value if they can be cloned, i.e. they automatically implement FromPyObject if they implement Clone, e.g. via #[derive(Clone)]:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyclass(from_py_object)]
#[derive(Clone)]
struct MyClass {
    my_field: Box<i32>,
}

#[pyfunction]
fn disassemble_clone(my_class: MyClass) {
    let MyClass { mut my_field } = my_class;
    *my_field += 1;
}

Note that #[derive(FromPyObject)] on a class is usually not useful as it tries to construct a new Rust value by filling in the fields by looking up attributes of any given Python value.

Method arguments

Similar to #[pyfunction], the #[pyo3(signature = (...))] attribute can be used to specify the way that #[pymethods] accept arguments. Consult the documentation for function signatures to see the parameters this attribute accepts.

The following example defines a class MyClass with a method method. This method has a signature that sets default values for num and name, and indicates that py_args should collect all extra positional arguments and py_kwargs all extra keyword arguments:

use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};

#[pyclass]
struct MyClass {
    num: i32,
}
#[pymethods]
impl MyClass {
    #[new]
    #[pyo3(signature = (num=-1))]
    fn new(num: i32) -> Self {
        MyClass { num }
    }

    #[pyo3(signature = (num=10, *py_args, name="Hello", **py_kwargs))]
    fn method(
        &mut self,
        num: i32,
        py_args: &Bound<'_, PyTuple>,
        name: &str,
        py_kwargs: Option<&Bound<'_, PyDict>>,
    ) -> String {
        let num_before = self.num;
        self.num = num;
        format!(
            "num={} (was previously={}), py_args={:?}, name={}, py_kwargs={:?} ",
            num, num_before, py_args, name, py_kwargs,
        )
    }
}

In Python, this might be used like:

>>> import mymodule
>>> mc = mymodule.MyClass()
>>> print(mc.method(44, False, "World", 666, x=44, y=55))
py_args=('World', 666), py_kwargs=Some({'x': 44, 'y': 55}), name=Hello, num=44, num_before=-1
>>> print(mc.method(num=-1, name="World"))
py_args=(), py_kwargs=None, name=World, num=-1, num_before=44

The #[pyo3(text_signature = "...") option for #[pyfunction] also works for #[pymethods].

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyType;

#[pyclass]
struct MyClass {}

#[pymethods]
impl MyClass {
    #[new]
    #[pyo3(text_signature = "(c, d)")]
    fn new(c: i32, d: &str) -> Self {
        Self {}
    }
    // the self argument should be written $self
    #[pyo3(text_signature = "($self, e, f)")]
    fn my_method(&self, e: i32, f: i32) -> i32 {
        e + f
    }
    // similarly for classmethod arguments, use $cls
    #[classmethod]
    #[pyo3(text_signature = "($cls, e, f)")]
    fn my_class_method(cls: &Bound<'_, PyType>, e: i32, f: i32) -> i32 {
        e + f
    }
    #[staticmethod]
    #[pyo3(text_signature = "(e, f)")]
    fn my_static_method(e: i32, f: i32) -> i32 {
        e + f
    }
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let inspect = PyModule::import(py, "inspect")?.getattr("signature")?;
        let module = PyModule::new(py, "my_module")?;
        module.add_class::<MyClass>()?;
        let class = module.getattr("MyClass")?;

        if cfg!(not(Py_LIMITED_API)) || py.version_info() >= (3, 10)  {
            let doc: String = class.getattr("__doc__")?.extract()?;
            assert_eq!(doc, "");

            let sig: String = inspect
                .call1((&class,))?
                .call_method0("__str__")?
                .extract()?;
            assert_eq!(sig, "(c, d)");
        } else {
            let doc: String = class.getattr("__doc__")?.extract()?;
            assert_eq!(doc, "");

            inspect.call1((&class,)).expect_err("`text_signature` on classes is not compatible with compilation in `abi3` mode until Python 3.10 or greater");
         }

        {
            let method = class.getattr("my_method")?;

            assert!(method.getattr("__doc__")?.is_none());

            let sig: String = inspect
                .call1((method,))?
                .call_method0("__str__")?
                .extract()?;
            assert_eq!(sig, "(self, /, e, f)");
        }

        {
            let method = class.getattr("my_class_method")?;

            assert!(method.getattr("__doc__")?.is_none());

            let sig: String = inspect
                .call1((method,))?
                .call_method0("__str__")?
                .extract()?;
            assert_eq!(sig, "(e, f)");  // inspect.signature skips the $cls arg
        }

        {
            let method = class.getattr("my_static_method")?;

            assert!(method.getattr("__doc__")?.is_none());

            let sig: String = inspect
                .call1((method,))?
                .call_method0("__str__")?
                .extract()?;
            assert_eq!(sig, "(e, f)");
        }

        Ok(())
    })
}

Note that text_signature on #[new] is not compatible with compilation in abi3 mode until Python 3.10 or greater.

Method receivers and lifetime elision

PyO3 supports writing instance methods using the normal method receivers for shared &self and unique &mut self references. This interacts with lifetime elision insofar as the lifetime of a such a receiver is assigned to all elided output lifetime parameters.

This is a good default for general Rust code where return values are more likely to borrow from the receiver than from the other arguments, if they contain any lifetimes at all. However, when returning bound references Bound<'py, T> in PyO3-based code, the Python lifetime 'py should usually be derived from a py: Python<'py> token passed as an argument instead of the receiver.

Specifically, signatures like

fn frobnicate(&self, py: Python) -> Bound<Foo>;

will not work as they are inferred as

fn frobnicate<'a, 'py>(&'a self, py: Python<'py>) -> Bound<'a, Foo>;

instead of the intended

fn frobnicate<'a, 'py>(&'a self, py: Python<'py>) -> Bound<'py, Foo>;

and should usually be written as

fn frobnicate<'py>(&self, py: Python<'py>) -> Bound<'py, Foo>;

The same problem does not exist for #[pyfunction]s as the special case for receiver lifetimes does not apply and indeed a signature like

fn frobnicate(bar: &Bar, py: Python) -> Bound<Foo>;

will yield compiler error E0106 “missing lifetime specifier”.

#[pyclass] enums

Enum support in PyO3 comes in two flavors, depending on what kind of variants the enum has: simple and complex.

Simple enums

A simple enum (a.k.a. C-like enum) has only unit variants.

PyO3 adds a class attribute for each variant, so you can access them in Python without defining #[new]. PyO3 also provides default implementations of __richcmp__ and __int__, so they can be compared using ==:

use pyo3::prelude::*;
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum MyEnum {
    Variant,
    OtherVariant,
}

Python::attach(|py| {
    let x = Py::new(py, MyEnum::Variant).unwrap();
    let y = Py::new(py, MyEnum::OtherVariant).unwrap();
    let cls = py.get_type::<MyEnum>();
    pyo3::py_run!(py, x y cls, r#"
        assert x == cls.Variant
        assert y == cls.OtherVariant
        assert x != y
    "#)
})

You can also convert your simple enums into int:

use pyo3::prelude::*;
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum MyEnum {
    Variant,
    OtherVariant = 10,
}

Python::attach(|py| {
    let cls = py.get_type::<MyEnum>();
    let x = MyEnum::Variant as i32; // The exact value is assigned by the compiler.
    pyo3::py_run!(py, cls x, r#"
        assert int(cls.Variant) == x
        assert int(cls.OtherVariant) == 10
    "#)
})

PyO3 also provides __repr__ for enums:

use pyo3::prelude::*;
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum MyEnum{
    Variant,
    OtherVariant,
}

Python::attach(|py| {
    let cls = py.get_type::<MyEnum>();
    let x = Py::new(py, MyEnum::Variant).unwrap();
    pyo3::py_run!(py, cls x, r#"
        assert repr(x) == 'MyEnum.Variant'
        assert repr(cls.OtherVariant) == 'MyEnum.OtherVariant'
    "#)
})

All methods defined by PyO3 can be overridden. For example here’s how you override __repr__:

use pyo3::prelude::*;
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum MyEnum {
    Answer = 42,
}

#[pymethods]
impl MyEnum {
    fn __repr__(&self) -> &'static str {
        "42"
    }
}

Python::attach(|py| {
    let cls = py.get_type::<MyEnum>();
    pyo3::py_run!(py, cls, "assert repr(cls.Answer) == '42'")
})

Enums and their variants can also be renamed using #[pyo3(name)].

use pyo3::prelude::*;
#[pyclass(eq, eq_int, name = "RenamedEnum")]
#[derive(PartialEq)]
enum MyEnum {
    #[pyo3(name = "UPPERCASE")]
    Variant,
}

Python::attach(|py| {
    let x = Py::new(py, MyEnum::Variant).unwrap();
    let cls = py.get_type::<MyEnum>();
    pyo3::py_run!(py, x cls, r#"
        assert repr(x) == 'RenamedEnum.UPPERCASE'
        assert x == cls.UPPERCASE
    "#)
})

Ordering of enum variants is optionally added using #[pyo3(ord)]. Note: Implementation of the PartialOrd trait is required when passing the ord argument. If not implemented, a compile time error is raised.

use pyo3::prelude::*;
#[pyclass(eq, ord)]
#[derive(PartialEq, PartialOrd)]
enum MyEnum{
    A,
    B,
    C,
}

Python::attach(|py| {
    let cls = py.get_type::<MyEnum>();
    let a = Py::new(py, MyEnum::A).unwrap();
    let b = Py::new(py, MyEnum::B).unwrap();
    let c = Py::new(py, MyEnum::C).unwrap();
    pyo3::py_run!(py, cls a b c, r#"
        assert (a < b) == True
        assert (c <= b) == False
        assert (c > a) == True
    "#)
})

You may not use enums as a base class or let enums inherit from other classes.

use pyo3::prelude::*;
#[pyclass(subclass)]
enum BadBase {
    Var1,
}
use pyo3::prelude::*;

#[pyclass(subclass)]
struct Base;

#[pyclass(extends=Base)]
enum BadSubclass {
    Var1,
}

#[pyclass] enums are currently not interoperable with IntEnum in Python.

Complex enums

An enum is complex if it has any non-unit (struct or tuple) variants.

PyO3 supports only struct and tuple variants in a complex enum. Unit variants aren’t supported at present (the recommendation is to use an empty tuple enum instead).

PyO3 adds a class attribute for each variant, which may be used to construct values and in match patterns. PyO3 also provides getter methods for all fields of each variant.

use pyo3::prelude::*;
#[pyclass]
enum Shape {
    Circle { radius: f64 },
    Rectangle { width: f64, height: f64 },
    RegularPolygon(u32, f64),
    Nothing { },
}

#[cfg(Py_3_10)]
Python::attach(|py| {
    let circle = Shape::Circle { radius: 10.0 }.into_pyobject(py)?;
    let square = Shape::RegularPolygon(4, 10.0).into_pyobject(py)?;
    let cls = py.get_type::<Shape>();
    pyo3::py_run!(py, circle square cls, r#"
        assert isinstance(circle, cls)
        assert isinstance(circle, cls.Circle)
        assert circle.radius == 10.0

        assert isinstance(square, cls)
        assert isinstance(square, cls.RegularPolygon)
        assert square[0] == 4 # Gets _0 field
        assert square[1] == 10.0 # Gets _1 field

        def count_vertices(cls, shape):
            match shape:
                case cls.Circle():
                    return 0
                case cls.Rectangle():
                    return 4
                case cls.RegularPolygon(n):
                    return n
                case cls.Nothing():
                    return 0

        assert count_vertices(cls, circle) == 0
        assert count_vertices(cls, square) == 4
    "#);
  Ok::<_, PyErr>(())
})
.unwrap();

WARNING: Py::new and .into_pyobject are currently inconsistent. Note how the constructed value is not an instance of the specific variant. For this reason, constructing values is only recommended using .into_pyobject.

use pyo3::prelude::*;
#[pyclass]
enum MyEnum {
    Variant { i: i32 },
}

Python::attach(|py| {
    let x = Py::new(py, MyEnum::Variant { i: 42 }).unwrap();
    let cls = py.get_type::<MyEnum>();
    pyo3::py_run!(py, x cls, r#"
        assert isinstance(x, cls)
        assert not isinstance(x, cls.Variant)
    "#)
})

The constructor of each generated class can be customized using the #[pyo3(constructor = (...))] attribute. This uses the same syntax as the #[pyo3(signature = (...))] attribute on function and methods and supports the same options. To apply this attribute simply place it on top of a variant in a #[pyclass] complex enum as shown below:

use pyo3::prelude::*;
#[pyclass]
enum Shape {
    #[pyo3(constructor = (radius=1.0))]
    Circle { radius: f64 },
    #[pyo3(constructor = (*, width, height))]
    Rectangle { width: f64, height: f64 },
    #[pyo3(constructor = (side_count, radius=1.0))]
    RegularPolygon { side_count: u32, radius: f64 },
    Nothing { },
}

#[cfg(Py_3_10)]
Python::attach(|py| {
    let cls = py.get_type::<Shape>();
    pyo3::py_run!(py, cls, r#"
        circle = cls.Circle()
        assert isinstance(circle, cls)
        assert isinstance(circle, cls.Circle)
        assert circle.radius == 1.0

        square = cls.Rectangle(width = 1, height = 1)
        assert isinstance(square, cls)
        assert isinstance(square, cls.Rectangle)
        assert square.width == 1
        assert square.height == 1

        hexagon = cls.RegularPolygon(6)
        assert isinstance(hexagon, cls)
        assert isinstance(hexagon, cls.RegularPolygon)
        assert hexagon.side_count == 6
        assert hexagon.radius == 1
    "#)
})

Implementation details

The #[pyclass] macros rely on a lot of conditional code generation: each #[pyclass] can optionally have a #[pymethods] block.

To support this flexibility the #[pyclass] macro expands to a blob of boilerplate code which sets up the structure for “dtolnay specialization”. This implementation pattern enables the Rust compiler to use #[pymethods] implementations when they are present, and fall back to default (empty) definitions when they are not.

This simple technique works for the case when there is zero or one implementations. To support multiple #[pymethods] for a #[pyclass] (in the multiple-pymethods feature), a registry mechanism provided by the inventory crate is used instead. This collects impls at library load time, but isn’t supported on all platforms. See inventory: how it works for more details.

The #[pyclass] macro expands to roughly the code seen below. The PyClassImplCollector is the type used internally by PyO3 for dtolnay specialization:

#[cfg(not(feature = "multiple-pymethods"))] {
use pyo3::prelude::*;
// Note: the implementation differs slightly with the `multiple-pymethods` feature enabled.
#[allow(dead_code)]
struct MyClass {
    #[allow(dead_code)]
    num: i32,
}

impl pyo3::types::DerefToPyAny for MyClass {}

unsafe impl pyo3::type_object::PyTypeInfo for MyClass {
    const NAME: &'static str = "MyClass";
    const MODULE: ::std::option::Option<&'static str> = ::std::option::Option::None;

    #[inline]
    fn type_object_raw(py: pyo3::Python<'_>) -> *mut pyo3::ffi::PyTypeObject {
        <Self as pyo3::impl_::pyclass::PyClassImpl>::lazy_type_object()
            .get_or_try_init(py)
            .unwrap_or_else(|e| pyo3::impl_::pyclass::type_object_init_failed(
                py,
                e,
                <Self as pyo3::PyClass>::NAME
            ))
            .as_type_ptr()
    }
}

impl pyo3::PyClass for MyClass {
    const NAME: &str = "MyClass";
    type Frozen = pyo3::pyclass::boolean_struct::False;
}

impl pyo3::impl_::pyclass::PyClassImpl for MyClass {
    const MODULE: Option<&str> = None;
    const IS_BASETYPE: bool = false;
    const IS_SUBCLASS: bool = false;
    const IS_MAPPING: bool = false;
    const IS_SEQUENCE: bool = false;
    type Layout = <Self::BaseNativeType as pyo3::impl_::pyclass::PyClassBaseType>::Layout<Self>;
    type BaseType = PyAny;
    type ThreadChecker = pyo3::impl_::pyclass::NoopThreadChecker;
    type PyClassMutability = <<pyo3::PyAny as pyo3::impl_::pyclass::PyClassBaseType>::PyClassMutability as pyo3::impl_::pycell::PyClassMutability>::MutableChild;
    type Dict = pyo3::impl_::pyclass::PyClassDummySlot;
    type WeakRef = pyo3::impl_::pyclass::PyClassDummySlot;
    type BaseNativeType = pyo3::PyAny;

    const RAW_DOC: &'static std::ffi::CStr = c"...";
    const DOC: &'static std::ffi::CStr = c"...";

    fn items_iter() -> pyo3::impl_::pyclass::PyClassItemsIter {
        use pyo3::impl_::pyclass::*;
        let collector = PyClassImplCollector::<MyClass>::new();
        static INTRINSIC_ITEMS: PyClassItems = PyClassItems { slots: &[], methods: &[] };
        PyClassItemsIter::new(&INTRINSIC_ITEMS, collector.py_methods())
    }

    fn lazy_type_object() -> &'static pyo3::impl_::pyclass::LazyTypeObject<MyClass> {
        use pyo3::impl_::pyclass::LazyTypeObject;
        static TYPE_OBJECT: LazyTypeObject<MyClass> = LazyTypeObject::new();
        &TYPE_OBJECT
    }
}

Python::attach(|py| {
    let cls = py.get_type::<MyClass>();
    pyo3::py_run!(py, cls, "assert cls.__name__ == 'MyClass'")
});
}

類別自訂

Python’s object model defines several protocols for different object behavior, such as the sequence, mapping, and number protocols. Python classes support these protocols by implementing “magic” methods, such as __str__ or __repr__. Because of the double-underscores surrounding their name, these are also known as “dunder” methods.

PyO3 makes it possible for every magic method to be implemented in #[pymethods] just as they would be done in a regular Python class, with a few notable differences:

  • __new__ is replaced by the #[new] attribute.
  • __del__ is not yet supported, but may be in the future.
  • __buffer__ and __release_buffer__ are currently not supported and instead PyO3 supports __getbuffer__ and __releasebuffer__ methods (these predate PEP 688), again this may change in the future.
  • PyO3 adds __traverse__ and __clear__ methods for controlling garbage collection.
  • The Python C-API which PyO3 is implemented upon requires many magic methods to have a specific function signature in C and be placed into special “slots” on the class type object. This limits the allowed argument and return types for these methods. They are listed in detail in the section below.

If a magic method is not on the list above (for example __init_subclass__), then it should just work in PyO3. If this is not the case, please file a bug report.

Magic Methods handled by PyO3

If a function name in #[pymethods] is a magic method which is known to need special handling, it will be automatically placed into the correct slot in the Python type object. The function name is taken from the usual rules for naming #[pymethods]: the #[pyo3(name = "...")] attribute is used if present, otherwise the Rust function name is used.

The magic methods handled by PyO3 are very similar to the standard Python ones on this page - in particular they are the subset which have slots as defined here.

When PyO3 handles a magic method, a couple of changes apply compared to other #[pymethods]:

  • The Rust function signature is restricted to match the magic method.
  • The #[pyo3(signature = (...)] and #[pyo3(text_signature = "...")] attributes are not allowed.

The following sections list all magic methods for which PyO3 implements the necessary special handling. The given signatures should be interpreted as follows:

  • All methods take a receiver as first argument, shown as <self>. It can be &self, &mut self or a Bound reference like self_: PyRef<'_, Self> and self_: PyRefMut<'_, Self>, as described in the parent section.
  • An optional Python<'py> argument is always allowed as the first argument.
  • Return values can be optionally wrapped in PyResult.
  • object means that any type is allowed that can be extracted from a Python object (if argument) or converted to a Python object (if return value).
  • Other types must match what’s given, e.g. pyo3::basic::CompareOp for __richcmp__’s second argument.
  • For the comparison and arithmetic methods, extraction errors are not propagated as exceptions, but lead to a return of NotImplemented.
  • For some magic methods, the return values are not restricted by PyO3, but checked by the Python interpreter. For example, __str__ needs to return a string object. This is indicated by object (Python type).

基本物件自訂

  • __str__(<self>) -> object (str)
  • __repr__(<self>) -> object (str)

  • __hash__(<self>) -> isize

    Objects that compare equal must have the same hash value. Any type up to 64 bits may be returned instead of isize, PyO3 will convert to an isize automatically (wrapping unsigned types like u64 and usize).

    Disabling Python's default hash

    By default, all #[pyclass] types have a default hash implementation from Python. Types which should not be hashable can override this by setting __hash__ to None. This is the same mechanism as for a pure-Python class. This is done like so:

    use pyo3::prelude::*;

#[pyclass]
struct NotHashable {}

#[pymethods]
impl NotHashable {
    #[classattr]
    const __hash__: Option<Py<PyAny>> = None;
}

  • __lt__(<self>, object) -> object

  • __le__(<self>, object) -> object

  • __eq__(<self>, object) -> object

  • __ne__(<self>, object) -> object

  • __gt__(<self>, object) -> object

  • __ge__(<self>, object) -> object

    The implementations of Python’s “rich comparison” operators <, <=, ==, !=, > and >= respectively.

    Note that implementing any of these methods will cause Python not to generate a default __hash__ implementation, so consider also implementing __hash__.

    Return type

    The return type will normally be bool or PyResult<bool>, however any Python object can be returned.

  • __richcmp__(<self>, object, pyo3::basic::CompareOp) -> object

    Implements Python comparison operations (==, !=, <, <=, >, and >=) in a single method. The CompareOp argument indicates the comparison operation being performed. You can use CompareOp::matches to adapt a Rust std::cmp::Ordering result to the requested comparison.

    This method cannot be implemented in combination with any of __lt__, __le__, __eq__, __ne__, __gt__, or __ge__.

    Note that implementing __richcmp__ will cause Python not to generate a default __hash__ implementation, so consider implementing __hash__ when implementing __richcmp__.

    Return type

    The return type will normally be PyResult<bool>, but any Python object can be returned.

    If you want to leave some operations unimplemented, you can return py.NotImplemented() for some of the operations:

    use pyo3::class::basic::CompareOp;
use pyo3::types::PyNotImplemented;

use pyo3::prelude::*;
use pyo3::BoundObject;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __richcmp__<'py>(&self, other: &Self, op: CompareOp, py: Python<'py>) -> PyResult<Borrowed<'py, 'py, PyAny>> {
        match op {
            CompareOp::Eq => Ok((self.0 == other.0).into_pyobject(py)?.into_any()),
            CompareOp::Ne => Ok((self.0 != other.0).into_pyobject(py)?.into_any()),
            _ => Ok(PyNotImplemented::get(py).into_any()),
        }
    }
}

    If the second argument object is not of the type specified in the signature, the generated code will automatically return NotImplemented.

  • __getattr__(<self>, object) -> object

  • __getattribute__(<self>, object) -> object

    Differences between __getattr__ and __getattribute__

    As in Python, __getattr__ is only called if the attribute is not found by normal attribute lookup. __getattribute__, on the other hand, is called for every attribute access. If it wants to access existing attributes on self, it needs to be very careful not to introduce infinite recursion, and use baseclass.__getattribute__().

  • __setattr__(<self>, value: object) -> ()

  • __delattr__(<self>, object) -> ()

    Overrides attribute access.

  • __bool__(<self>) -> bool

    Determines the “truthyness” of an object.

  • __call__(<self>, ...) -> object - here, any argument list can be defined as for normal pymethods

Iterable objects

Iterators can be defined using these methods:

  • __iter__(<self>) -> object
  • __next__(<self>) -> Option<object> or IterNextOutput (see details)

Returning None from __next__ indicates that that there are no further items.

Example:

use pyo3::prelude::*;

use std::sync::Mutex;

#[pyclass]
struct MyIterator {
    iter: Mutex<Box<dyn Iterator<Item = Py<PyAny>> + Send>>,
}

#[pymethods]
impl MyIterator {
    fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> {
        slf
    }
    fn __next__(slf: PyRefMut<'_, Self>) -> Option<Py<PyAny>> {
        slf.iter.lock().unwrap().next()
    }
}

In many cases you’ll have a distinction between the type being iterated over (i.e. the iterable) and the iterator it provides. In this case, the iterable only needs to implement __iter__() while the iterator must implement both __iter__() and __next__(). For example:

use pyo3::prelude::*;

#[pyclass]
struct Iter {
    inner: std::vec::IntoIter<usize>,
}

#[pymethods]
impl Iter {
    fn __iter__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> {
        slf
    }

    fn __next__(mut slf: PyRefMut<'_, Self>) -> Option<usize> {
        slf.inner.next()
    }
}

#[pyclass]
struct Container {
    iter: Vec<usize>,
}

#[pymethods]
impl Container {
    fn __iter__(slf: PyRef<'_, Self>) -> PyResult<Py<Iter>> {
        let iter = Iter {
            inner: slf.iter.clone().into_iter(),
        };
        Py::new(slf.py(), iter)
    }
}

Python::attach(|py| {
    let container = Container { iter: vec![1, 2, 3, 4] };
    let inst = pyo3::Py::new(py, container).unwrap();
    pyo3::py_run!(py, inst, "assert list(inst) == [1, 2, 3, 4]");
    pyo3::py_run!(py, inst, "assert list(iter(iter(inst))) == [1, 2, 3, 4]");
});

For more details on Python’s iteration protocols, check out the “Iterator Types” section of the library documentation.

Returning a value from iteration

This guide has so far shown how to use Option<T> to implement yielding values during iteration. In Python a generator can also return a value. This is done by raising a StopIteration exception. To express this in Rust, return PyResult::Err with a PyStopIteration as the error.

Awaitable objects

  • __await__(<self>) -> object
  • __aiter__(<self>) -> object
  • __anext__(<self>) -> Option<object>

Mapping & Sequence types

The magic methods in this section can be used to implement Python container types. They are two main categories of container in Python: “mappings” such as dict, with arbitrary keys, and “sequences” such as list and tuple, with integer keys.

The Python C-API which PyO3 is built upon has separate “slots” for sequences and mappings. When writing a class in pure Python, there is no such distinction in the implementation - a __getitem__ implementation will fill the slots for both the mapping and sequence forms, for example.

By default PyO3 reproduces the Python behaviour of filling both mapping and sequence slots. This makes sense for the “simple” case which matches Python, and also for sequences, where the mapping slot is used anyway to implement slice indexing.

Mapping types usually will not want the sequence slots filled. Having them filled will lead to outcomes which may be unwanted, such as:

  • The mapping type will successfully cast to PySequence. This may lead to consumers of the type handling it incorrectly.
  • Python provides a default implementation of __iter__ for sequences, which calls __getitem__ with consecutive positive integers starting from 0 until an IndexError is returned. Unless the mapping only contains consecutive positive integer keys, this __iter__ implementation will likely not be the intended behavior.

Use the #[pyclass(mapping)] annotation to instruct PyO3 to only fill the mapping slots, leaving the sequence ones empty. This will apply to __getitem__, __setitem__, and __delitem__.

Use the #[pyclass(sequence)] annotation to instruct PyO3 to fill the sq_length slot instead of the mp_length slot for __len__. This will help libraries such as numpy recognise the class as a sequence, however will also cause CPython to automatically add the sequence length to any negative indices before passing them to __getitem__. (__getitem__, __setitem__ and __delitem__ mapping slots are still used for sequences, for slice operations.)

  • __len__(<self>) -> usize

    Implements the built-in function len().

  • __contains__(<self>, object) -> bool

    Implements membership test operators. Should return true if item is in self, false otherwise. For objects that don’t define __contains__(), the membership test simply traverses the sequence until it finds a match.

    Disabling Python's default contains

    By default, all #[pyclass] types with an __iter__ method support a default implementation of the in operator. Types which do not want this can override this by setting __contains__ to None. This is the same mechanism as for a pure-Python class. This is done like so:

    use pyo3::prelude::*;

#[pyclass]
struct NoContains {}

#[pymethods]
impl NoContains {
    #[classattr]
    const __contains__: Option<Py<PyAny>> = None;
}

  • __getitem__(<self>, object) -> object

    Implements retrieval of the self[a] element.

    Note: Negative integer indexes are not handled specially by PyO3. However, for classes with #[pyclass(sequence)], when a negative index is accessed via PySequence::get_item, the underlying C API already adjusts the index to be positive.

  • __setitem__(<self>, object, object) -> ()

    Implements assignment to the self[a] element. Should only be implemented if elements can be replaced.

    Same behavior regarding negative indices as for __getitem__.

  • __delitem__(<self>, object) -> ()

    Implements deletion of the self[a] element. Should only be implemented if elements can be deleted.

    Same behavior regarding negative indices as for __getitem__.

  • fn __concat__(&self, other: impl FromPyObject) -> PyResult<impl ToPyObject>

    Concatenates two sequences. Used by the + operator, after trying the numeric addition via the __add__ and __radd__ methods.

  • fn __repeat__(&self, count: isize) -> PyResult<impl ToPyObject>

    Repeats the sequence count times. Used by the * operator, after trying the numeric multiplication via the __mul__ and __rmul__ methods.

  • fn __inplace_concat__(&self, other: impl FromPyObject) -> PyResult<impl ToPyObject>

    Concatenates two sequences. Used by the += operator, after trying the numeric addition via the __iadd__ method.

  • fn __inplace_repeat__(&self, count: isize) -> PyResult<impl ToPyObject>

    Concatenates two sequences. Used by the *= operator, after trying the numeric multiplication via the __imul__ method.

Descriptors

  • __get__(<self>, object, object) -> object
  • __set__(<self>, object, object) -> ()
  • __delete__(<self>, object) -> ()

Numeric types

Binary arithmetic operations (+, -, *, @, /, //, %, divmod(), pow() and **, <<, >>, &, ^, and |) and their reflected versions:

(If the object is not of the type specified in the signature, the generated code will automatically return NotImplemented.)

  • __add__(<self>, object) -> object
  • __radd__(<self>, object) -> object
  • __sub__(<self>, object) -> object
  • __rsub__(<self>, object) -> object
  • __mul__(<self>, object) -> object
  • __rmul__(<self>, object) -> object
  • __matmul__(<self>, object) -> object
  • __rmatmul__(<self>, object) -> object
  • __floordiv__(<self>, object) -> object
  • __rfloordiv__(<self>, object) -> object
  • __truediv__(<self>, object) -> object
  • __rtruediv__(<self>, object) -> object
  • __divmod__(<self>, object) -> object
  • __rdivmod__(<self>, object) -> object
  • __mod__(<self>, object) -> object
  • __rmod__(<self>, object) -> object
  • __lshift__(<self>, object) -> object
  • __rlshift__(<self>, object) -> object
  • __rshift__(<self>, object) -> object
  • __rrshift__(<self>, object) -> object
  • __and__(<self>, object) -> object
  • __rand__(<self>, object) -> object
  • __xor__(<self>, object) -> object
  • __rxor__(<self>, object) -> object
  • __or__(<self>, object) -> object
  • __ror__(<self>, object) -> object
  • __pow__(<self>, object, object) -> object
  • __rpow__(<self>, object, object) -> object

In-place assignment operations (+=, -=, *=, @=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=):

  • __iadd__(<self>, object) -> ()
  • __isub__(<self>, object) -> ()
  • __imul__(<self>, object) -> ()
  • __imatmul__(<self>, object) -> ()
  • __itruediv__(<self>, object) -> ()
  • __ifloordiv__(<self>, object) -> ()
  • __imod__(<self>, object) -> ()
  • __ipow__(<self>, object, object) -> ()
  • __ilshift__(<self>, object) -> ()
  • __irshift__(<self>, object) -> ()
  • __iand__(<self>, object) -> ()
  • __ixor__(<self>, object) -> ()
  • __ior__(<self>, object) -> ()

Unary operations (-, +, abs() and ~):

  • __pos__(<self>) -> object
  • __neg__(<self>) -> object
  • __abs__(<self>) -> object
  • __invert__(<self>) -> object

Coercions:

  • __index__(<self>) -> object (int)
  • __int__(<self>) -> object (int)
  • __float__(<self>) -> object (float)

Buffer objects

  • __getbuffer__(<self>, *mut ffi::Py_buffer, flags) -> ()
  • __releasebuffer__(<self>, *mut ffi::Py_buffer) -> () Errors returned from __releasebuffer__ will be sent to sys.unraiseablehook. It is strongly advised to never return an error from __releasebuffer__, and if it really is necessary, to make best effort to perform any required freeing operations before returning. __releasebuffer__ will not be called a second time; anything not freed will be leaked.

Garbage Collector Integration

If your type owns references to other Python objects, you will need to integrate with Python’s garbage collector so that the GC is aware of those references. To do this, implement the two methods __traverse__ and __clear__. These correspond to the slots tp_traverse and tp_clear in the Python C API. __traverse__ must call visit.call() for each reference to another Python object. __clear__ must clear out any mutable references to other Python objects (thus breaking reference cycles). Immutable references do not have to be cleared, as every cycle must contain at least one mutable reference.

  • __traverse__(<self>, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>
  • __clear__(<self>) -> ()

[!NOTE] __traverse__ does not work with #[pyo3(warn(...))].

Example:

use pyo3::prelude::*;
use pyo3::PyTraverseError;
use pyo3::gc::PyVisit;

#[pyclass]
struct ClassWithGCSupport {
    obj: Option<Py<PyAny>>,
}

#[pymethods]
impl ClassWithGCSupport {
    fn __traverse__(&self, visit: PyVisit<'_>) -> Result<(), PyTraverseError> {
        visit.call(&self.obj)?;
        Ok(())
    }

    fn __clear__(&mut self) {
        // Clear reference, this decrements ref counter.
        self.obj = None;
    }
}

[!NOTE] When a class inherits from either a Python builtins type or another type declared in Rust and implement either or both __traverse__ and __clear__, the parent class __traverse__ and __clear__ is called automatically. There is no need to explicitly call it from inside the class implementation.

Usually, an implementation of __traverse__ should do nothing but calls to visit.call. Most importantly, safe access to the interpreter is prohibited inside implementations of __traverse__, i.e. Python::attach will panic.

[!NOTE] These methods are part of the C API, PyPy does not necessarily honor them. If you are building for PyPy you should measure memory consumption to make sure you do not have runaway memory growth. See this issue on the PyPy bug tracker.

基本物件自訂

Recall the Number class from the previous chapter:

#![allow(dead_code)]
fn main() {}
use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    #[new]
    fn new(value: i32) -> Self {
        Self(value)
    }
}

#[pymodule]
mod my_module {
    #[pymodule_export]
    use super::Number;
}

At this point Python code can import the module, access the class and create class instances - but nothing else.

from my_module import Number

n = Number(5)
print(n)

<builtins.Number object at 0x000002B4D185D7D0>

字串表示法

It can’t even print an user-readable representation of itself! We can fix that by defining the __repr__ and __str__ methods inside a #[pymethods] block. We do this by accessing the value contained inside Number.

use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    // For `__repr__` we want to return a string that Python code could use to recreate
    // the `Number`, like `Number(5)` for example.
    fn __repr__(&self) -> String {
        // We use the `format!` macro to create a string. Its first argument is a
        // format string, followed by any number of parameters which replace the
        // `{}`'s in the format string.
        //
        //                       👇 Tuple field access in Rust uses a dot
        format!("Number({})", self.0)
    }
    // `__str__` is generally used to create an "informal" representation, so we
    // just forward to `i32`'s `ToString` trait implementation to print a bare number.
    fn __str__(&self) -> String {
        self.0.to_string()
    }
}

To automatically generate the __str__ implementation using a Display trait implementation, pass the str argument to pyclass.

use std::fmt::{Display, Formatter};
use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass(str)]
struct Coordinate {
    x: i32,
    y: i32,
    z: i32,
}

impl Display for Coordinate {
    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
        write!(f, "({}, {}, {})", self.x, self.y, self.z)
    }
}

For convenience, a shorthand format string can be passed to str as str="<format string>" for structs only. It expands and is passed into the format! macro in the following ways:

  • "{x}" -> "{}", self.x
  • "{0}" -> "{}", self.0
  • "{x:?}" -> "{:?}", self.x

Note: Depending upon the format string you use, this may require implementation of the Display or Debug traits for the given Rust types. Note: the pyclass args name and rename_all are incompatible with the shorthand format string and will raise a compile time error.

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass(str="({x}, {y}, {z})")]
struct Coordinate {
    x: i32,
    y: i32,
    z: i32,
}

Accessing the class name

In the __repr__, we used a hard-coded class name. This is sometimes not ideal, because if the class is subclassed in Python, we would like the repr to reflect the subclass name. This is typically done in Python code by accessing self.__class__.__name__. In order to be able to access the Python type information and the Rust struct, we need to use a Bound as the self argument.

use pyo3::prelude::*;
use pyo3::types::PyString;

#[allow(dead_code)]
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __repr__(slf: &Bound<'_, Self>) -> PyResult<String> {
        // This is the equivalent of `self.__class__.__name__` in Python.
        let class_name: Bound<'_, PyString> = slf.get_type().qualname()?;
        // To access fields of the Rust struct, we need to borrow from the Bound object.
        Ok(format!("{}({})", class_name, slf.borrow().0))
    }
}

Hashing

Let’s also implement hashing. We’ll just hash the i32. For that we need a Hasher. The one provided by std is DefaultHasher, which uses the SipHash algorithm.

use std::collections::hash_map::DefaultHasher;

// Required to call the `.hash` and `.finish` methods, which are defined on traits.
use std::hash::{Hash, Hasher};

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __hash__(&self) -> u64 {
        let mut hasher = DefaultHasher::new();
        self.0.hash(&mut hasher);
        hasher.finish()
    }
}

To implement __hash__ using the Rust Hash trait implementation, the hash option can be used. This option is only available for frozen classes to prevent accidental hash changes from mutating the object. If you need an __hash__ implementation for a mutable class, use the manual method from above. This option also requires eq: According to the Python docs “If a class does not define an __eq__() method it should not define a __hash__() operation either”

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass(frozen, eq, hash)]
#[derive(PartialEq, Hash)]
struct Number(i32);

[!NOTE] When implementing __hash__ and comparisons, it is important that the following property holds:

k1 == k2 -> hash(k1) == hash(k2)

In other words, if two keys are equal, their hashes must also be equal. In addition you must take care that your classes’ hash doesn’t change during its lifetime. In this tutorial we do that by not letting Python code change our Number class. In other words, it is immutable.

By default, all #[pyclass] types have a default hash implementation from Python. Types which should not be hashable can override this by setting __hash__ to None. This is the same mechanism as for a pure-Python class. This is done like so:

use pyo3::prelude::*;
#[pyclass]
struct NotHashable {}

#[pymethods]
impl NotHashable {
    #[classattr]
    const __hash__: Option<Py<PyAny>> = None;
}

Comparisons

PyO3 supports the usual magic comparison methods available in Python such as __eq__, __lt__ and so on. It is also possible to support all six operations at once with __richcmp__. This method will be called with a value of CompareOp depending on the operation.

use pyo3::class::basic::CompareOp;

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult<bool> {
        match op {
            CompareOp::Lt => Ok(self.0 < other.0),
            CompareOp::Le => Ok(self.0 <= other.0),
            CompareOp::Eq => Ok(self.0 == other.0),
            CompareOp::Ne => Ok(self.0 != other.0),
            CompareOp::Gt => Ok(self.0 > other.0),
            CompareOp::Ge => Ok(self.0 >= other.0),
        }
    }
}

If you obtain the result by comparing two Rust values, as in this example, you can take a shortcut using CompareOp::matches:

use pyo3::class::basic::CompareOp;

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __richcmp__(&self, other: &Self, op: CompareOp) -> bool {
        op.matches(self.0.cmp(&other.0))
    }
}

It checks that the std::cmp::Ordering obtained from Rust’s Ord matches the given CompareOp.

Alternatively, you can implement just equality using __eq__:

use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __eq__(&self, other: &Self) -> bool {
        self.0 == other.0
    }
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let x = &Bound::new(py, Number(4))?;
        let y = &Bound::new(py, Number(4))?;
        assert!(x.eq(y)?);
        assert!(!x.ne(y)?);
        Ok(())
    })
}

To implement __eq__ using the Rust PartialEq trait implementation, the eq option can be used.

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass(eq)]
#[derive(PartialEq)]
struct Number(i32);

To implement __lt__, __le__, __gt__, & __ge__ using the Rust PartialOrd trait implementation, the ord option can be used. Note: Requires eq.

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass(eq, ord)]
#[derive(PartialEq, PartialOrd)]
struct Number(i32);

Truthyness

We’ll consider Number to be True if it is nonzero:

use pyo3::prelude::*;

#[allow(dead_code)]
#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __bool__(&self) -> bool {
        self.0 != 0
    }
}

Final code

fn main() {}
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};

use pyo3::prelude::*;
use pyo3::class::basic::CompareOp;
use pyo3::types::PyString;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    #[new]
    fn new(value: i32) -> Self {
        Self(value)
    }

    fn __repr__(slf: &Bound<'_, Self>) -> PyResult<String> {
        let class_name: Bound<'_, PyString> = slf.get_type().qualname()?;
        Ok(format!("{}({})", class_name, slf.borrow().0))
    }

    fn __str__(&self) -> String {
        self.0.to_string()
    }

    fn __hash__(&self) -> u64 {
        let mut hasher = DefaultHasher::new();
        self.0.hash(&mut hasher);
        hasher.finish()
    }

    fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult<bool> {
        match op {
            CompareOp::Lt => Ok(self.0 < other.0),
            CompareOp::Le => Ok(self.0 <= other.0),
            CompareOp::Eq => Ok(self.0 == other.0),
            CompareOp::Ne => Ok(self.0 != other.0),
            CompareOp::Gt => Ok(self.0 > other.0),
            CompareOp::Ge => Ok(self.0 >= other.0),
        }
    }

    fn __bool__(&self) -> bool {
        self.0 != 0
    }
}

#[pymodule]
mod my_module {
    #[pymodule_export]
    use super::Number;
}

仿真數值型別

At this point we have a Number class that we can’t actually do any math on!

Before proceeding, we should think about how we want to handle overflows. There are three obvious solutions:

  • We can have infinite precision just like Python’s int. However that would be quite boring - we’d be reinventing the wheel.
  • We can raise exceptions whenever Number overflows, but that makes the API painful to use.
  • We can wrap around the boundary of i32. This is the approach we’ll take here. To do that we’ll just forward to i32’s wrapping_* methods.

Fixing our constructor

Let’s address the first overflow, in Number’s constructor:

from my_module import Number

n = Number(1 << 1337)

Traceback (most recent call last):
  File "example.py", line 3, in <module>
    n = Number(1 << 1337)
OverflowError: Python int too large to convert to C long

Instead of relying on the default FromPyObject extraction to parse arguments, we can specify our own extraction function, using the #[pyo3(from_py_with = ...)] attribute. Unfortunately PyO3 doesn’t provide a way to wrap Python integers out of the box, but we can do a Python call to mask it and cast it to an i32.

#![allow(dead_code)]
use pyo3::prelude::*;

fn wrap(obj: &Bound<'_, PyAny>) -> PyResult<i32> {
    let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?;
    let val: u32 = val.extract()?;
    //     👇 This intentionally overflows!
    Ok(val as i32)
}

We also add documentation, via /// comments, which are visible to Python users.

#![allow(dead_code)]
use pyo3::prelude::*;

fn wrap(obj: &Bound<'_, PyAny>) -> PyResult<i32> {
    let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?;
    let val: u32 = val.extract()?;
    Ok(val as i32)
}

/// Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not.
/// It's not a story C would tell you. It's a Rust legend.
#[pyclass(module = "my_module")]
struct Number(i32);

#[pymethods]
impl Number {
    #[new]
    fn new(#[pyo3(from_py_with = wrap)] value: i32) -> Self {
        Self(value)
    }
}

With that out of the way, let’s implement some operators:

use pyo3::exceptions::{PyZeroDivisionError, PyValueError};

use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __add__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_add(other.0))
    }

    fn __sub__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_sub(other.0))
    }

    fn __mul__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_mul(other.0))
    }

    fn __truediv__(&self, other: &Self) -> PyResult<Self> {
        match self.0.checked_div(other.0) {
            Some(i) => Ok(Self(i)),
            None => Err(PyZeroDivisionError::new_err("division by zero")),
        }
    }

    fn __floordiv__(&self, other: &Self) -> PyResult<Self> {
        match self.0.checked_div(other.0) {
            Some(i) => Ok(Self(i)),
            None => Err(PyZeroDivisionError::new_err("division by zero")),
        }
    }

    fn __rshift__(&self, other: &Self) -> PyResult<Self> {
        match other.0.try_into() {
            Ok(rhs) => Ok(Self(self.0.wrapping_shr(rhs))),
            Err(_) => Err(PyValueError::new_err("negative shift count")),
        }
    }

    fn __lshift__(&self, other: &Self) -> PyResult<Self> {
        match other.0.try_into() {
            Ok(rhs) => Ok(Self(self.0.wrapping_shl(rhs))),
            Err(_) => Err(PyValueError::new_err("negative shift count")),
        }
    }
}

一元算數運算

use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

#[pymethods]
impl Number {
    fn __pos__(slf: PyRef<'_, Self>) -> PyRef<'_, Self> {
        slf
    }

    fn __neg__(&self) -> Self {
        Self(-self.0)
    }

    fn __abs__(&self) -> Self {
        Self(self.0.abs())
    }

    fn __invert__(&self) -> Self {
        Self(!self.0)
    }
}

Support for the complex(), int() and float() built-in functions

use pyo3::prelude::*;

#[pyclass]
struct Number(i32);

use pyo3::types::PyComplex;

#[pymethods]
impl Number {
    fn __int__(&self) -> i32 {
        self.0
    }

    fn __float__(&self) -> f64 {
        self.0 as f64
    }

    fn __complex__<'py>(&self, py: Python<'py>) -> Bound<'py, PyComplex> {
        PyComplex::from_doubles(py, self.0 as f64, 0.0)
    }
}

We do not implement the in-place operations like __iadd__ because we do not wish to mutate Number. Similarly we’re not interested in supporting operations with different types, so we do not implement the reflected operations like __radd__ either.

Now Python can use our Number class:

from my_module import Number

def hash_djb2(s: str):
	'''
	A version of Daniel J. Bernstein's djb2 string hashing algorithm
	Like many hashing algorithms, it relies on integer wrapping.
	'''

	n = Number(0)
	five = Number(5)

	for x in s:
		n = Number(ord(x)) + ((n << five) - n)
	return n

assert hash_djb2('l50_50') == Number(-1152549421)

Final code

use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};

use pyo3::exceptions::{PyValueError, PyZeroDivisionError};
use pyo3::prelude::*;
use pyo3::class::basic::CompareOp;
use pyo3::types::{PyComplex, PyString};

fn wrap(obj: &Bound<'_, PyAny>) -> PyResult<i32> {
    let val = obj.call_method1("__and__", (0xFFFFFFFF_u32,))?;
    let val: u32 = val.extract()?;
    Ok(val as i32)
}
/// Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not.
/// It's not a story C would tell you. It's a Rust legend.
#[pyclass(module = "my_module")]
struct Number(i32);

#[pymethods]
impl Number {
    #[new]
    fn new(#[pyo3(from_py_with = wrap)] value: i32) -> Self {
        Self(value)
    }

    fn __repr__(slf: &Bound<'_, Self>) -> PyResult<String> {
       // Get the class name dynamically in case `Number` is subclassed
       let class_name: Bound<'_, PyString> = slf.get_type().qualname()?;
        Ok(format!("{}({})", class_name, slf.borrow().0))
    }

    fn __str__(&self) -> String {
        self.0.to_string()
    }

    fn __hash__(&self) -> u64 {
        let mut hasher = DefaultHasher::new();
        self.0.hash(&mut hasher);
        hasher.finish()
    }

    fn __richcmp__(&self, other: &Self, op: CompareOp) -> PyResult<bool> {
        match op {
            CompareOp::Lt => Ok(self.0 < other.0),
            CompareOp::Le => Ok(self.0 <= other.0),
            CompareOp::Eq => Ok(self.0 == other.0),
            CompareOp::Ne => Ok(self.0 != other.0),
            CompareOp::Gt => Ok(self.0 > other.0),
            CompareOp::Ge => Ok(self.0 >= other.0),
        }
    }

    fn __bool__(&self) -> bool {
        self.0 != 0
    }

    fn __add__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_add(other.0))
    }

    fn __sub__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_sub(other.0))
    }

    fn __mul__(&self, other: &Self) -> Self {
        Self(self.0.wrapping_mul(other.0))
    }

    fn __truediv__(&self, other: &Self) -> PyResult<Self> {
        match self.0.checked_div(other.0) {
            Some(i) => Ok(Self(i)),
            None => Err(PyZeroDivisionError::new_err("division by zero")),
        }
    }

    fn __floordiv__(&self, other: &Self) -> PyResult<Self> {
        match self.0.checked_div(other.0) {
            Some(i) => Ok(Self(i)),
            None => Err(PyZeroDivisionError::new_err("division by zero")),
        }
    }

    fn __rshift__(&self, other: &Self) -> PyResult<Self> {
        match other.0.try_into() {
            Ok(rhs) => Ok(Self(self.0.wrapping_shr(rhs))),
            Err(_) => Err(PyValueError::new_err("negative shift count")),
        }
    }

    fn __lshift__(&self, other: &Self) -> PyResult<Self> {
        match other.0.try_into() {
            Ok(rhs) => Ok(Self(self.0.wrapping_shl(rhs))),
            Err(_) => Err(PyValueError::new_err("negative shift count")),
        }
    }

    fn __xor__(&self, other: &Self) -> Self {
        Self(self.0 ^ other.0)
    }

    fn __or__(&self, other: &Self) -> Self {
        Self(self.0 | other.0)
    }

    fn __and__(&self, other: &Self) -> Self {
        Self(self.0 & other.0)
    }

    fn __int__(&self) -> i32 {
        self.0
    }

    fn __float__(&self) -> f64 {
        self.0 as f64
    }

    fn __complex__<'py>(&self, py: Python<'py>) -> Bound<'py, PyComplex> {
        PyComplex::from_doubles(py, self.0 as f64, 0.0)
    }
}

#[pymodule]
mod my_module {
    #[pymodule_export]
    use super::Number;
}
const SCRIPT: &'static std::ffi::CStr = cr#"
def hash_djb2(s: str):
    n = Number(0)
    five = Number(5)

    for x in s:
        n = Number(ord(x)) + ((n << five) - n)
    return n

assert hash_djb2('l50_50') == Number(-1152549421)
assert hash_djb2('logo') == Number(3327403)
assert hash_djb2('horizon') == Number(1097468315)


assert Number(2) + Number(2) == Number(4)
assert Number(2) + Number(2) != Number(5)

assert Number(13) - Number(7) == Number(6)
assert Number(13) - Number(-7) == Number(20)

assert Number(13) / Number(7) == Number(1)
assert Number(13) // Number(7) == Number(1)

assert Number(13) * Number(7) == Number(13*7)

assert Number(13) > Number(7)
assert Number(13) < Number(20)
assert Number(13) == Number(13)
assert Number(13) >= Number(7)
assert Number(13) <= Number(20)
assert Number(13) == Number(13)


assert (True if Number(1) else False)
assert (False if Number(0) else True)


assert int(Number(13)) == 13
assert float(Number(13)) == 13
assert Number.__doc__ == "Did you ever hear the tragedy of Darth Signed The Overfloweth? I thought not.\nIt's not a story C would tell you. It's a Rust legend."
assert Number(12345234523452) == Number(1498514748)
try:
    import inspect
    assert inspect.signature(Number).__str__() == '(value)'
except ValueError:
    # Not supported with `abi3` before Python 3.10
    pass
assert Number(1337).__str__() == '1337'
assert Number(1337).__repr__() == 'Number(1337)'
"#;


use pyo3::PyTypeInfo;

fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let globals = PyModule::import(py, "__main__")?.dict();
        globals.set_item("Number", Number::type_object(py))?;

        py.run(SCRIPT, Some(&globals), None)?;
        Ok(())
    })
}

附錄：編寫一些不安全的程式碼

At the beginning of this chapter we said that PyO3 doesn’t provide a way to wrap Python integers out of the box but that’s a half truth. There’s not a PyO3 API for it, but there’s a Python C API function that does:

unsigned long PyLong_AsUnsignedLongMask(PyObject *obj)

We can call this function from Rust by using pyo3::ffi::PyLong_AsUnsignedLongMask. This is an unsafe function, which means we have to use an unsafe block to call it and take responsibility for upholding the contracts of this function. Let’s review those contracts:

  • We must be attached to the interpreter. If we’re not, calling this function causes a data race.
  • The pointer must be valid, i.e. it must be properly aligned and point to a valid Python object.

Let’s create that helper function. The signature has to be fn(&Bound<'_, PyAny>) -> PyResult<T>.

  • &Bound<'_, PyAny> represents a checked bound reference, so the pointer derived from it is valid (and not null).
  • Whenever we have bound references to Python objects in scope, it is guaranteed that we’re attached to the interpreter. This reference is also where we can get a Python token to use in our call to PyErr::take.
#![allow(dead_code)]
use std::ffi::c_ulong;
use pyo3::prelude::*;
use pyo3::ffi;

fn wrap(obj: &Bound<'_, PyAny>) -> Result<i32, PyErr> {
    let py: Python<'_> = obj.py();

    unsafe {
        let ptr = obj.as_ptr();

        let ret: c_ulong = ffi::PyLong_AsUnsignedLongMask(ptr);
        if ret == c_ulong::MAX {
            if let Some(err) = PyErr::take(py) {
                return Err(err);
            }
        }

        Ok(ret as i32)
    }
}

仿真可呼叫物件

Classes can be callable if they have a #[pymethod] named __call__. This allows instances of a class to behave similar to functions.

This method’s signature must look like __call__(<self>, ...) -> object - here, any argument list can be defined as for normal pymethods

Example: Implementing a call counter

The following pyclass is a basic decorator - its constructor takes a Python object as argument and calls that object when called. An equivalent Python implementation is linked at the end.

An example crate containing this pyclass can be found in the PyO3 GitHub repository

use pyo3::prelude::*;
use pyo3::types::{PyDict, PyTuple};
use std::sync::atomic::{AtomicU64, Ordering};

/// A function decorator that keeps track how often it is called.
///
/// It otherwise doesn't do anything special.
#[pyclass(name = "Counter")]
pub struct PyCounter {
    // Keeps track of how many calls have gone through.
    //
    // See the discussion at the end for why `AtomicU64` is used.
    count: AtomicU64,

    // This is the actual function being wrapped.
    wraps: Py<PyAny>,
}

#[pymethods]
impl PyCounter {
    // Note that we don't validate whether `wraps` is actually callable.
    //
    // While we could use `PyAny::is_callable` for that, it has some flaws:
    //    1. It doesn't guarantee the object can actually be called successfully
    //    2. We still need to handle any exceptions that the function might raise
    #[new]
    fn __new__(wraps: Py<PyAny>) -> Self {
        PyCounter {
            count: AtomicU64::new(0),
            wraps,
        }
    }

    #[getter]
    fn count(&self) -> u64 {
        self.count.load(Ordering::Relaxed)
    }

    #[pyo3(signature = (*args, **kwargs))]
    fn __call__(
        &self,
        py: Python<'_>,
        args: &Bound<'_, PyTuple>,
        kwargs: Option<&Bound<'_, PyDict>>,
    ) -> PyResult<Py<PyAny>> {
        let new_count = self.count.fetch_add(1, Ordering::Relaxed);
        let name = self.wraps.getattr(py, "__name__")?;

        println!("{name} has been called {new_count} time(s).");

        // After doing something, we finally forward the call to the wrapped function
        let ret = self.wraps.call(py, args, kwargs)?;

        // We could do something with the return value of
        // the function before returning it
        Ok(ret)
    }
}

#[pymodule]
pub fn decorator(module: &Bound<'_, PyModule>) -> PyResult<()> {
    module.add_class::<PyCounter>()?;
    Ok(())
}

Python code:

from decorator import Counter


@Counter
def say_hello():
    print("hello")


say_hello()
say_hello()
say_hello()
say_hello()

assert say_hello.count == 4

輸出：

say_hello has been called 1 time(s).
hello
say_hello has been called 2 time(s).
hello
say_hello has been called 3 time(s).
hello
say_hello has been called 4 time(s).
hello

純 Python 實現

A Python implementation of this looks similar to the Rust version:

class Counter:
    def __init__(self, wraps):
        self.count = 0
        self.wraps = wraps

    def __call__(self, *args, **kwargs):
        self.count += 1
        print(f"{self.wraps.__name__} has been called {self.count} time(s)")
        self.wraps(*args, **kwargs)

Note that it can also be implemented as a higher order function:

def Counter(wraps):
    count = 0
    def call(*args, **kwargs):
        nonlocal count
        count += 1
        print(f"{wraps.__name__} has been called {count} time(s)")
        return wraps(*args, **kwargs)
    return call

What is the AtomicU64 for?

A previous implementation used a normal u64, which meant it required a &mut self receiver to update the count:

#[pyo3(signature = (*args, **kwargs))]
fn __call__(
    &mut self,
    py: Python<'_>,
    args: &Bound<'_, PyTuple>,
    kwargs: Option<&Bound<'_, PyDict>>,
) -> PyResult<Py<PyAny>> {
    self.count += 1;
    let name = self.wraps.getattr(py, "__name__")?;

    println!("{} has been called {} time(s).", name, self.count);

    // After doing something, we finally forward the call to the wrapped function
    let ret = self.wraps.call(py, args, kwargs)?;

    // We could do something with the return value of
    // the function before returning it
    Ok(ret)
}

The problem with this is that the &mut self receiver means PyO3 has to borrow it exclusively, and hold this borrow across the self.wraps.call(py, args, kwargs) call. This call returns control to the user’s Python code which is free to call arbitrary things, including the decorated function. If that happens PyO3 is unable to create a second unique borrow and will be forced to raise an exception.

As a result, something innocent like this will raise an exception:

@Counter
def say_hello():
    if say_hello.count < 2:
        print(f"hello from decorator")

say_hello()
# RuntimeError: Already borrowed

The implementation in this chapter fixes that by never borrowing exclusively; all the methods take &self as receivers, of which multiple may exist simultaneously. This requires a shared counter and the most straightforward way to implement thread-safe interior mutability (e.g. the type does not need to accept &mut self to modify the “interior” state) for a u64 is to use AtomicU64, so that’s what is used here.

This shows the dangers of running arbitrary Python code - note that “running arbitrary Python code” can be far more subtle than the example above:

  • Python’s asynchronous executor may park the current thread in the middle of Python code, even in Python code that you control, and let other Python code run.
  • Dropping arbitrary Python objects may invoke destructors defined in Python (__del__ methods).
  • Calling Python’s C-api (most PyO3 apis call C-api functions internally) may raise exceptions, which may allow Python code in signal handlers to run.
  • On the free-threaded build, users might use Python’s threading module to work with your types simultaneously from multiple OS threads.

This is especially important if you are writing unsafe code; Python code must never be able to cause undefined behavior. You must ensure that your Rust code is in a consistent state before doing any of the above things.

#[pyclass] 執行緒安全

Python objects are freely shared between threads by the Python interpreter. This means that:

  • there is no control which thread might eventually drop the #[pyclass] object, meaning Send is required.
  • multiple threads can potentially be reading the #[pyclass] data simultaneously, meaning Sync is required.

This section of the guide discusses various data structures which can be used to make types satisfy these requirements.

In special cases where it is known that your Python application is never going to use threads (this is rare!), these thread-safety requirements can be opted-out with #[pyclass(unsendable)], at the cost of making concurrent access to the Rust data be runtime errors. This is only for very specific use cases; it is almost always better to make proper thread-safe types.

Making #[pyclass] types thread-safe

The general challenge with thread-safety is to make sure that two threads cannot produce a data race, i.e. unsynchronized writes to the same data at the same time. A data race produces an unpredictable result and is forbidden by Rust.

By default, #[pyclass] employs an “interior mutability” pattern to allow for either multiple &T references or a single exclusive &mut T reference to access the data. This allows for simple #[pyclass] types to be thread-safe automatically, at the cost of runtime checking for concurrent access. Errors will be raised if the usage overlaps.

For example, the below simple class is thread-safe:

use pyo3::prelude::*;

#[pyclass]
struct MyClass {
    x: i32,
    y: i32,
}

#[pymethods]
impl MyClass {
    fn get_x(&self) -> i32 {
        self.x
    }

    fn set_y(&mut self, value: i32) {
        self.y = value;
    }
}

In the above example, if calls to get_x and set_y overlap (from two different threads) then at least one of those threads will experience a runtime error indicating that the data was “already borrowed”.

To avoid these errors, you can take control of the interior mutability yourself in one of the following ways.

使用原子資料結構

To remove the possibility of having overlapping &self and &mut self references produce runtime errors, consider using #[pyclass(frozen)] and use atomic data structures to control modifications directly.

For example, a thread-safe version of the above MyClass using atomic integers would be as follows:

use pyo3::prelude::*;
use std::sync::atomic::{AtomicI32, Ordering};

#[pyclass(frozen)]
struct MyClass {
    x: AtomicI32,
    y: AtomicI32,
}

#[pymethods]
impl MyClass {
    fn get_x(&self) -> i32 {
        self.x.load(Ordering::Relaxed)
    }

    fn set_y(&self, value: i32) {
        self.y.store(value, Ordering::Relaxed)
    }
}

Using locks

An alternative to atomic data structures is to use locks to make threads wait for access to shared data.

For example, a thread-safe version of the above MyClass using locks would be as follows:

use pyo3::prelude::*;
use std::sync::Mutex;

struct MyClassInner {
    x: i32,
    y: i32,
}

#[pyclass(frozen)]
struct MyClass {
    inner: Mutex<MyClassInner>
}

#[pymethods]
impl MyClass {
    fn get_x(&self) -> i32 {
        self.inner.lock().expect("lock not poisoned").x
    }

    fn set_y(&self, value: i32) {
        self.inner.lock().expect("lock not poisoned").y = value;
    }
}

If you need to lock around state stored in the Python interpreter or otherwise call into the Python C API while a lock is held, you might find the MutexExt trait useful. It provides a lock_py_attached method for std::sync::Mutex that avoids deadlocks with the GIL or other global synchronization events in the interpreter. Additionally, support for the parking_lot and lock_api synchronization libraries is gated behind the parking_lot and lock_api features. You can also enable the arc_lock feature if you need the arc_lock features of either library.

Wrapping unsynchronized data

In some cases, the data structures stored within a #[pyclass] may themselves not be thread-safe. Rust will therefore not implement Send and Sync on the #[pyclass] type.

To achieve thread-safety, a manual Send and Sync implementation is required which is unsafe and should only be done following careful review of the soundness of the implementation. Doing this for PyO3 types is no different than for any other Rust code, the Rustonomicon has a great discussion on this.

在 Rust 程式碼中呼叫 Python

本章說明從 Rust 與 Python 程式碼互動的幾種方式。

以下將介紹 'py 生命週期，以及 PyO3 API 如何思考 Python 程式碼的一些一般說明。

子章節也涵蓋以下主題：

  • PyO3 API 中可用的 Python 物件型別
  • 如何處理 Python 例外狀況
  • 如何呼叫 Python 函式
  • 如何執行既有的 Python 程式碼

'py 生命週期

要安全地與 Python 直譯器互動，Rust 執行緒必須附著於 Python 直譯器。PyO3 提供 Python<'py> token 用來證明已滿足這些條件，其生命週期 'py 是 PyO3 API 的核心部分。

Python<'py> token 有三個用途：

  • 它提供 Python 直譯器的全域 API，例如 py.eval() 與 py.import()。
  • 它可傳遞給需要附著證明的函式，例如 Py::clone_ref。
  • 其生命週期 'py 用來將多種 PyO3 型別綁定到 Python 直譯器，例如 Bound<'py, T>。

綁定 'py 生命週期的 PyO3 型別（例如 Bound<'py, T>）都包含 Python<'py> token。這表示它們能完整存取 Python 直譯器，並提供與 Python 物件互動的完整 API。

請參考 PyO3 API 文件 了解如何取得此 token。

全域直譯器鎖（GIL）

Prior to the introduction of free-threaded Python (first available in 3.13, fully supported in 3.14), the Python interpreter was made thread-safe by the global interpreter lock. This ensured that only one Python thread can use the Python interpreter and its API at the same time. Historically, Rust code was able to use the GIL as a synchronization guarantee, but the introduction of free-threaded Python removed this possibility.

pyo3::sync 模組提供同步工具，可抽象化兩種 Python 版本的差異。

為了在有 GIL 的版本中啟用並行、以及在自由執行緒版本中獲得最佳吞吐量，非 Python 的操作（系統呼叫與原生 Rust 程式碼）應考慮暫時與直譯器分離，以便其他工作繼續進行。如何使用 PyO3 API 進行分離，請見並行章節。

Python 的記憶體模型

Python’s memory model differs from Rust’s memory model in two key ways:

  • There is no concept of ownership; all Python objects are shared and usually implemented via reference counting
  • There is no concept of exclusive (&mut) references; any reference can mutate a Python object

PyO3’s API reflects this by providing smart pointer types, Py<T>, Bound<'py, T>, and (the very rarely used) Borrowed<'a, 'py, T>. These smart pointers all use Python reference counting. See the subchapter on types for more detail on these types.

Because of the lack of exclusive &mut references, PyO3’s APIs for Python objects, for example PyListMethods::append, use shared references. This is safe because Python objects have internal mechanisms to prevent data races (as of time of writing, the Python GIL).

Python 物件型別

PyO3 offers two main sets of types to interact with Python objects. This section of the guide expands into detail about these types and how to choose which to use.

The first set of types are the smart pointers which all Python objects are wrapped in. These are Py<T>, Bound<'py, T>, and Borrowed<'a, 'py, T>. The first section below expands on each of these in detail and why there are three of them.

The second set of types are types which fill in the generic parameter T of the smart pointers. The most common is PyAny, which represents any Python object (similar to Python’s typing.Any). There are also concrete types for many Python built-in types, such as PyList, PyDict, and PyTuple. User defined #[pyclass] types also fit this category. The second section below expands on how to use these types.

PyO3 的智慧指標

PyO3’s API offers three generic smart pointers: Py<T>, Bound<'py, T> and Borrowed<'a, 'py, T>. For each of these the type parameter T will be filled by a concrete Python type. For example, a Python list object can be represented by Py<PyList>, Bound<'py, PyList>, and Borrowed<'a, 'py, PyList>.

These smart pointers behave differently due to their lifetime parameters. Py<T> has no lifetime parameters, Bound<'py, T> has the 'py lifetime as a parameter, and Borrowed<'a, 'py, T> has the 'py lifetime plus an additional lifetime 'a to denote the lifetime it is borrowing data for. (You can read more about these lifetimes in the subsections below).

Python objects are reference counted, like std::sync::Arc. A major reason for these smart pointers is to bring Python’s reference counting to a Rust API.

The recommendation of when to use each of these smart pointers is as follows:

  • Use Bound<'py, T> for as much as possible, as it offers the most efficient and complete API.
  • Use Py<T> mostly just for storage inside Rust structs which do not want to or can’t add a lifetime parameter for Bound<'py, T>.
  • Borrowed<'a, 'py, T> is almost never used. It is occasionally present at the boundary between Rust and the Python interpreter, for example when borrowing data from Python tuples (which is safe because they are immutable).

The sections below also explain these smart pointers in a little more detail.

Py<T>

Py<T> is the foundational smart pointer in PyO3’s API. The type parameter T denotes the type of the Python object. Very frequently this is PyAny, meaning any Python object.

Because Py<T> is not bound to the 'py lifetime, it is the type to use when storing a Python object inside a Rust struct or enum which do not want to have a lifetime parameter. In particular, #[pyclass] types are not permitted to have a lifetime, so Py<T> is the correct type to store Python objects inside them.

The lack of binding to the 'py lifetime also carries drawbacks:

  • Almost all methods on Py<T> require a Python<'py> token as the first argument
  • Other functionality, such as Drop, needs to check at runtime for attachment to the Python interpreter, at a small performance cost

Because of the drawbacks Bound<'py, T> is preferred for many of PyO3’s APIs. In particular, Bound<'py, T> is better for function arguments.

To convert a Py<T> into a Bound<'py, T>, the Py::bind and Py::into_bound methods are available. Bound<'py, T> can be converted back into Py<T> using Bound::unbind.

Bound<'py, T>

Bound<'py, T> is the counterpart to Py<T> which is also bound to the 'py lifetime. It can be thought of as equivalent to the Rust tuple (Python<'py>, Py<T>).

By having the binding to the 'py lifetime, Bound<'py, T> can offer the complete PyO3 API at maximum efficiency. This means that Bound<'py, T> should usually be used whenever carrying this lifetime is acceptable, and Py<T> otherwise.

Bound<'py, T> engages in Python reference counting. This means that Bound<'py, T> owns a Python object. Rust code which just wants to borrow a Python object should use a shared reference &Bound<'py, T>. Just like std::sync::Arc, using .clone() and drop() will cheaply increment and decrement the reference count of the object (just in this case, the reference counting is implemented by the Python interpreter itself).

To give an example of how Bound<'py, T> is PyO3’s primary API type, consider the following Python code:

def example():
    x = list()   # create a Python list
    x.append(1)  # append the integer 1 to it
    y = x        # create a second reference to the list
    del x        # delete the original reference

Using PyO3’s API, and in particular Bound<'py, PyList>, this code translates into the following Rust code:

use pyo3::prelude::*;
use pyo3::types::PyList;

fn example<'py>(py: Python<'py>) -> PyResult<()> {
    let x: Bound<'py, PyList> = PyList::empty(py);
    x.append(1)?;
    let y: Bound<'py, PyList> = x.clone(); // y is a new reference to the same list
    drop(x); // release the original reference x
    Ok(())
}
Python::attach(example).unwrap();

Or, without the type annotations:

use pyo3::prelude::*;
use pyo3::types::PyList;

fn example(py: Python<'_>) -> PyResult<()> {
    let x = PyList::empty(py);
    x.append(1)?;
    let y = x.clone();
    drop(x);
    Ok(())
}
Python::attach(example).unwrap();

Function argument lifetimes

Because the 'py lifetime often appears in many function arguments as part of the Bound<'py, T> smart pointer, the Rust compiler will often require annotations of input and output lifetimes. This occurs when the function output has at least one lifetime, and there is more than one lifetime present on the inputs.

To demonstrate, consider this function which takes accepts Python objects and applies the Python + operation to them:

use pyo3::prelude::*;
fn add(left: &'_ Bound<'_, PyAny>, right: &'_ Bound<'_, PyAny>) -> PyResult<Bound<'_, PyAny>> {
    left.add(right)
}

Because the Python + operation might raise an exception, this function returns PyResult<Bound<'_, PyAny>>. It doesn’t need ownership of the inputs, so it takes &Bound<'_, PyAny> shared references. To demonstrate the point, all lifetimes have used the wildcard '_ to allow the Rust compiler to attempt to infer them. Because there are four input lifetimes (two lifetimes of the shared references, and two 'py lifetimes unnamed inside the Bound<'_, PyAny> pointers), the compiler cannot reason about which must be connected to the output.

The correct way to solve this is to add the 'py lifetime as a parameter for the function, and name all the 'py lifetimes inside the Bound<'py, PyAny> smart pointers. For the shared references, it’s also fine to reduce &'_ to just &. The working end result is below:

use pyo3::prelude::*;
fn add<'py>(
    left: &Bound<'py, PyAny>,
    right: &Bound<'py, PyAny>,
) -> PyResult<Bound<'py, PyAny>> {
    left.add(right)
}
Python::attach(|py| {
    let s = pyo3::types::PyString::new(py, "s");
    assert!(add(&s, &s).unwrap().eq("ss").unwrap());
})

If naming the 'py lifetime adds unwanted complexity to the function signature, it is also acceptable to return Py<PyAny>, which has no lifetime. The cost is instead paid by a slight increase in implementation complexity, as seen by the introduction of a call to Bound::unbind:

use pyo3::prelude::*;
fn add(left: &Bound<'_, PyAny>, right: &Bound<'_, PyAny>) -> PyResult<Py<PyAny>> {
    let output: Bound<'_, PyAny> = left.add(right)?;
    Ok(output.unbind())
}
Python::attach(|py| {
    let s = pyo3::types::PyString::new(py, "s");
    assert!(add(&s, &s).unwrap().bind(py).eq("ss").unwrap());
})

Borrowed<'a, 'py, T>

Borrowed<'a, 'py, T> is an advanced type used just occasionally at the edge of interaction with the Python interpreter. It can be thought of as analogous to the shared reference &'a Bound<'py, T>. The difference is that Borrowed<'a, 'py, T> is just a smart pointer rather than a reference-to-a-smart-pointer, which is a helpful reduction in indirection in specific interactions with the Python interpreter.

Borrowed<'a, 'py, T> dereferences to Bound<'py, T>, so all methods on Bound<'py, T> are available on Borrowed<'a, 'py, T>.

An example where Borrowed<'a, 'py, T> is used is in PyTupleMethods::get_borrowed_item:

use pyo3::prelude::*;
use pyo3::types::PyTuple;

fn example<'py>(py: Python<'py>) -> PyResult<()> {
// Create a new tuple with the elements (0, 1, 2)
let t = PyTuple::new(py, [0, 1, 2])?;
for i in 0..=2 {
    let entry: Borrowed<'_, 'py, PyAny> = t.get_borrowed_item(i)?;
    // `PyAnyMethods::extract` is available on `Borrowed`
    // via the dereference to `Bound`
    let value: usize = entry.extract()?;
    assert_eq!(i, value);
}
Ok(())
}
Python::attach(example).unwrap();

Casting between smart pointer types

To convert between Py<T> and Bound<'py, T> use the bind() / into_bound() methods. Use the as_unbound() / unbind() methods to go back from Bound<'py, T> to Py<T>.

let obj: Py<PyAny> = ...;
let bound: &Bound<'py, PyAny> = obj.bind(py);
let bound: Bound<'py, PyAny> = obj.into_bound(py);

let obj: &Py<PyAny> = bound.as_unbound();
let obj: Py<PyAny> = bound.unbind();

To convert between Bound<'py, T> and Borrowed<'a, 'py, T> use the as_borrowed() method. Borrowed<'a, 'py, T> has a deref coercion to Bound<'py, T>. Use the to_owned() method to increment the Python reference count and to create a new Bound<'py, T> from the Borrowed<'a, 'py, T>.

let bound: Bound<'py, PyAny> = ...;
let borrowed: Borrowed<'_, 'py, PyAny> = bound.as_borrowed();

// deref coercion
let bound: &Bound<'py, PyAny> = &borrowed;

// create a new Bound by increase the Python reference count
let bound: Bound<'py, PyAny> = borrowed.to_owned();

To convert between Py<T> and Borrowed<'a, 'py, T> use the bind_borrowed() method. Use either as_unbound() or .to_owned().unbind() to go back to Py<T> from Borrowed<'a, 'py, T>, via Bound<'py, T>.

let obj: Py<PyAny> = ...;
let borrowed: Borrowed<'_, 'py, PyAny> = bound.as_borrowed();

// via deref coercion to Bound and then using Bound::as_unbound
let obj: &Py<PyAny> = borrowed.as_unbound();

// via a new Bound by increasing the Python reference count, and unbind it
let obj: Py<PyAny> = borrowed.to_owned().unbind().

Concrete Python types

In all of Py<T>, Bound<'py, T>, and Borrowed<'a, 'py, T>, the type parameter T denotes the type of the Python object referred to by the smart pointer.

This parameter T can be filled by:

  • PyAny, which represents any Python object,
  • Native Python types such as PyList, PyTuple, and PyDict, and
  • #[pyclass] types defined from Rust

The following subsections covers some further detail about how to work with these types:

  • the APIs that are available for these concrete types,
  • how to cast Bound<'py, T> to a specific concrete type, and
  • how to get Rust data out of a Bound<'py, T>.

Using APIs for concrete Python types

Each concrete Python type such as PyAny, PyTuple and PyDict exposes its API on the corresponding bound smart pointer Bound<'py, PyAny>, Bound<'py, PyTuple> and Bound<'py, PyDict>.

Each type’s API is exposed as a trait: PyAnyMethods, PyTupleMethods, PyDictMethods, and so on for all concrete types. Using traits rather than associated methods on the Bound smart pointer is done for a couple of reasons:

  • Clarity of documentation: each trait gets its own documentation page in the PyO3 API docs. If all methods were on the Bound smart pointer directly, the vast majority of PyO3’s API would be on a single, extremely long, documentation page.
  • Consistency: downstream code implementing Rust APIs for existing Python types can also follow this pattern of using a trait. Downstream code would not be allowed to add new associated methods directly on the Bound type.
  • Future design: it is hoped that a future Rust with arbitrary self types will remove the need for these traits in favour of placing the methods directly on PyAny, PyTuple, PyDict, and so on.

These traits are all included in the pyo3::prelude module, so with the glob import use pyo3::prelude::* the full PyO3 API is made available to downstream code.

The following function accesses the first item in the input Python list, using the .get_item() method from the PyListMethods trait:

use pyo3::prelude::*;
use pyo3::types::PyList;

fn get_first_item<'py>(list: &Bound<'py, PyList>) -> PyResult<Bound<'py, PyAny>> {
    list.get_item(0)
}
Python::attach(|py| {
    let l = PyList::new(py, ["hello world"]).unwrap();
    assert!(get_first_item(&l).unwrap().eq("hello world").unwrap());
})

Casting between Python object types

To cast Bound<'py, T> smart pointers to some other type, use the .cast() family of functions. This converts &Bound<'py, T> to a different &Bound<'py, U>, without transferring ownership. There is also .cast_into() to convert Bound<'py, T> to Bound<'py, U> with transfer of ownership. These methods are available for all types T which implement the PyTypeCheck trait.

Casting to Bound<'py, PyAny> can be done with .as_any() or .into_any().

For example, the following snippet shows how to cast Bound<'py, PyAny> to Bound<'py, PyTuple>:

use pyo3::prelude::*;
use pyo3::types::PyTuple;
fn example<'py>(py: Python<'py>) -> PyResult<()> {
// create a new Python `tuple`, and use `.into_any()` to erase the type
let obj: Bound<'py, PyAny> = PyTuple::empty(py).into_any();

// use `.cast()` to cast to `PyTuple` without transferring ownership
let _: &Bound<'py, PyTuple> = obj.cast()?;

// use `.cast_into()` to cast to `PyTuple` with transfer of ownership
let _: Bound<'py, PyTuple> = obj.cast_into()?;
Ok(())
}
Python::attach(example).unwrap()

Custom #[pyclass] types implement PyTypeCheck, so .cast() also works for these types. The snippet below is the same as the snippet above casting instead to a custom type MyClass:

use pyo3::prelude::*;

#[pyclass]
struct MyClass {}

fn example<'py>(py: Python<'py>) -> PyResult<()> {
// create a new Python `tuple`, and use `.into_any()` to erase the type
let obj: Bound<'py, PyAny> = Bound::new(py, MyClass {})?.into_any();

// use `.cast()` to cast to `MyClass` without transferring ownership
let _: &Bound<'py, MyClass> = obj.cast()?;

// use `.cast_into()` to cast to `MyClass` with transfer of ownership
let _: Bound<'py, MyClass> = obj.cast_into()?;
Ok(())
}
Python::attach(example).unwrap()

Extracting Rust data from Python objects

To extract Rust data from Python objects, use .extract() instead of .cast(). This method is available for all types which implement the FromPyObject trait.

For example, the following snippet extracts a Rust tuple of integers from a Python tuple:

use pyo3::prelude::*;
use pyo3::types::PyTuple;
fn example<'py>(py: Python<'py>) -> PyResult<()> {
// create a new Python `tuple`, and use `.into_any()` to erase the type
let obj: Bound<'py, PyAny> = PyTuple::new(py, [1, 2, 3])?.into_any();

// extracting the Python `tuple` to a rust `(i32, i32, i32)` tuple
let (x, y, z) = obj.extract::<(i32, i32, i32)>()?;
assert_eq!((x, y, z), (1, 2, 3));
Ok(())
}
Python::attach(example).unwrap()

To avoid copying data, #[pyclass] types can directly reference Rust data stored within the Python objects without needing to .extract(). See the corresponding documentation in the class section of the guide for more detail.

Python 例外狀況

Defining a new exception

Use the create_exception! macro:

use pyo3::create_exception;

create_exception!(module, MyError, pyo3::exceptions::PyException);
  • module is the name of the containing module.
  • MyError is the name of the new exception type.

例如：

use pyo3::prelude::*;
use pyo3::create_exception;
use pyo3::types::IntoPyDict;
use pyo3::exceptions::PyException;

create_exception!(mymodule, CustomError, PyException);

fn main() -> PyResult<()> {
Python::attach(|py| {
    let ctx = [("CustomError", py.get_type::<CustomError>())].into_py_dict(py)?;
    pyo3::py_run!(
        py,
        *ctx,
        "assert str(CustomError) == \"<class 'mymodule.CustomError'>\""
    );
    pyo3::py_run!(py, *ctx, "assert CustomError('oops').args == ('oops',)");
  Ok(())
})
}

When using PyO3 to create an extension module, you can add the new exception to the module like this, so that it is importable from Python:

fn main() {}
use pyo3::prelude::*;
use pyo3::exceptions::PyException;

pyo3::create_exception!(mymodule, CustomError, PyException);

#[pymodule]
mod mymodule {
    #[pymodule_export]
    use super::CustomError;

    // ... other elements added to module ...
}

拋出例外狀況

As described in the function error handling chapter, to raise an exception from a #[pyfunction] or #[pymethods], return an Err(PyErr). PyO3 will automatically raise this exception for you when returning the result to Python.

You can also manually write and fetch errors in the Python interpreter’s global state:

use pyo3::{Python, PyErr};
use pyo3::exceptions::PyTypeError;

Python::attach(|py| {
    PyTypeError::new_err("Error").restore(py);
    assert!(PyErr::occurred(py));
    drop(PyErr::fetch(py));
});

Checking exception types

Python has an isinstance method to check an object’s type. In PyO3 every object has the PyAny::is_instance and PyAny::is_instance_of methods which do the same thing.

use pyo3::prelude::*;
use pyo3::types::{PyBool, PyList};

fn main() -> PyResult<()> {
Python::attach(|py| {
    assert!(PyBool::new(py, true).is_instance_of::<PyBool>());
    let list = PyList::new(py, &[1, 2, 3, 4])?;
    assert!(!list.is_instance_of::<PyBool>());
    assert!(list.is_instance_of::<PyList>());
Ok(())
})
}

To check the type of an exception, you can similarly do:

use pyo3::exceptions::PyTypeError;
use pyo3::prelude::*;
Python::attach(|py| {
let err = PyTypeError::new_err(());
err.is_instance_of::<PyTypeError>(py);
});

使用 Python 程式碼中定義的例外狀況

It is possible to use an exception defined in Python code as a native Rust type. The import_exception! macro allows importing a specific exception class and defines a Rust type for that exception.

#![allow(dead_code)]
use pyo3::prelude::*;

mod io {
    pyo3::import_exception!(io, UnsupportedOperation);
}

fn tell(file: &Bound<'_, PyAny>) -> PyResult<u64> {
    match file.call_method0("tell") {
        Err(_) => Err(io::UnsupportedOperation::new_err("not supported: tell")),
        Ok(x) => x.extract::<u64>(),
    }
}

pyo3::exceptions defines exceptions for several standard library modules.

建立更多複雜的例外狀況

If you need to create an exception with more complex behavior, you can also manually create a subclass of PyException:

#![allow(dead_code)]
#[cfg(any(not(Py_LIMITED_API), Py_3_12))] {
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
use pyo3::exceptions::PyException;

#[pyclass(extends=PyException)]
struct CustomError {
    #[pyo3(get)]
    url: String,

    #[pyo3(get)]
    message: String,
}

#[pymethods]
impl CustomError {
    #[new]
    fn new(url: String, message: String) -> Self {
        Self { url, message }
    }
}

fn main() -> PyResult<()> {
Python::attach(|py| {
    let ctx = [("CustomError", py.get_type::<CustomError>())].into_py_dict(py)?;
    pyo3::py_run!(
        py,
        *ctx,
        "assert str(CustomError) == \"<class 'builtins.CustomError'>\", repr(CustomError)"
    );
    pyo3::py_run!(py, *ctx, "assert CustomError('https://example.com', 'something went bad').args == ('https://example.com', 'something went bad')");
    pyo3::py_run!(py, *ctx, "assert CustomError('https://example.com', 'something went bad').url == 'https://example.com'");
  Ok(())
})
}
}

Note that when the abi3 feature is enabled, subclassing PyException is only possible on Python 3.12 or greater.

呼叫 Python 函式

The Bound<'py, T> smart pointer (such as Bound<'py, PyAny>, Bound<'py, PyList>, or Bound<'py, MyClass>) can be used to call Python functions.

PyO3 offers two APIs to make function calls:

  • call - call any callable Python object.
  • call_method - call a method on the Python object.

Both of these APIs take args and kwargs arguments (for positional and keyword arguments respectively). There are variants for less complex calls:

For convenience the Py<T> smart pointer also exposes these same six API methods, but needs a Python token as an additional first argument to prove the thread is attached to the Python interpreter.

The example below calls a Python function behind a Py<PyAny> reference:

use pyo3::prelude::*;
use pyo3::types::PyTuple;

fn main() -> PyResult<()> {
    let arg1 = "arg1";
    let arg2 = "arg2";
    let arg3 = "arg3";

    Python::attach(|py| {
        let fun: Py<PyAny> = PyModule::from_code(
            py,
            c"def example(*args, **kwargs):
                if args != ():
                    print('called with args', args)
                if kwargs != {}:
                    print('called with kwargs', kwargs)
                if args == () and kwargs == {}:
                    print('called with no arguments')",
            c"example.py",
            c"",
        )?
        .getattr("example")?
        .into();

        // call object without any arguments
        fun.call0(py)?;

        // pass object with Rust tuple of positional arguments
        let args = (arg1, arg2, arg3);
        fun.call1(py, args)?;

        // call object with Python tuple of positional arguments
        let args = PyTuple::new(py, &[arg1, arg2, arg3])?;
        fun.call1(py, args)?;
        Ok(())
    })
}

Creating keyword arguments

For the call and call_method APIs, kwargs are Option<&Bound<'py, PyDict>>, so can either be None or Some(&dict). You can use the IntoPyDict trait to convert other dict-like containers, e.g. HashMap or BTreeMap, as well as tuples with up to 10 elements and Vecs where each element is a two-element tuple. To pass keyword arguments of different types, construct a PyDict object.

use pyo3::prelude::*;
use pyo3::types::{PyDict, IntoPyDict};
use std::collections::HashMap;

fn main() -> PyResult<()> {
    let key1 = "key1";
    let val1 = 1;
    let key2 = "key2";
    let val2 = 2;

    Python::attach(|py| {
        let fun: Py<PyAny> = PyModule::from_code(
            py,
            c"def example(*args, **kwargs):
                if args != ():
                    print('called with args', args)
                if kwargs != {}:
                    print('called with kwargs', kwargs)
                if args == () and kwargs == {}:
                    print('called with no arguments')",
            c"example.py",
            c"",
        )?
        .getattr("example")?
        .into();

        // call object with PyDict
        let kwargs = [(key1, val1)].into_py_dict(py)?;
        fun.call(py, (), Some(&kwargs))?;

        // pass arguments as Vec
        let kwargs = vec![(key1, val1), (key2, val2)];
        fun.call(py, (), Some(&kwargs.into_py_dict(py)?))?;

        // pass arguments as HashMap
        let mut kwargs = HashMap::<&str, i32>::new();
        kwargs.insert(key1, 1);
        fun.call(py, (), Some(&kwargs.into_py_dict(py)?))?;

        // pass arguments of different types as PyDict
        let kwargs = PyDict::new(py);
        kwargs.set_item(key1, val1)?;
        kwargs.set_item(key2, "string")?;
        fun.call(py, (), Some(&kwargs))?;

        Ok(())
    })
}

執行現有的 Python 程式碼

If you already have some existing Python code that you need to execute from Rust, the following FAQs can help you select the right PyO3 functionality for your situation:

Want to access Python APIs? Then use PyModule::import

PyModule::import can be used to get handle to a Python module from Rust. You can use this to import and use any Python module available in your environment.

use pyo3::prelude::*;

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let builtins = PyModule::import(py, "builtins")?;
        let total: i32 = builtins
            .getattr("sum")?
            .call1((vec![1, 2, 3],))?
            .extract()?;
        assert_eq!(total, 6);
        Ok(())
    })
}

Want to run just an expression? Then use eval

Python::eval is a method to execute a Python expression and return the evaluated value as a Bound<'py, PyAny> object.

use pyo3::prelude::*;

fn main() -> Result<(), ()> {
Python::attach(|py| {
    let result = py
        .eval(c"[i * 10 for i in range(5)]", None, None)
        .map_err(|e| {
            e.print_and_set_sys_last_vars(py);
        })?;
    let res: Vec<i64> = result.extract().unwrap();
    assert_eq!(res, vec![0, 10, 20, 30, 40]);
    Ok(())
})
}

Want to run statements? Then use run

Python::run is a method to execute one or more Python statements. This method returns nothing (like any Python statement), but you can get access to manipulated objects via the locals dict.

You can also use the py_run! macro, which is a shorthand for Python::run. Since py_run! panics on exceptions, we recommend you use this macro only for quickly testing your Python extensions.

use pyo3::prelude::*;
use pyo3::py_run;

fn main() {
#[pyclass]
struct UserData {
    id: u32,
    name: String,
}

#[pymethods]
impl UserData {
    fn as_tuple(&self) -> (u32, String) {
        (self.id, self.name.clone())
    }

    fn __repr__(&self) -> PyResult<String> {
        Ok(format!("User {}(id: {})", self.name, self.id))
    }
}

Python::attach(|py| {
    let userdata = UserData {
        id: 34,
        name: "Yu".to_string(),
    };
    let userdata = Py::new(py, userdata).unwrap();
    let userdata_as_tuple = (34, "Yu");
    py_run!(py, userdata userdata_as_tuple, r#"
assert repr(userdata) == "User Yu(id: 34)"
assert userdata.as_tuple() == userdata_as_tuple
    "#);
})
}

You have a Python file or code snippet? Then use PyModule::from_code

PyModule::from_code can be used to generate a Python module which can then be used just as if it was imported with PyModule::import.

Warning: This will compile and execute code. Never pass untrusted code to this function!

use pyo3::{prelude::*, types::IntoPyDict};

fn main() -> PyResult<()> {
Python::attach(|py| {
    let activators = PyModule::from_code(
        py,
        cr#"
def relu(x):
    """see https://en.wikipedia.org/wiki/Rectifier_(neural_networks)"""
    return max(0.0, x)

def leaky_relu(x, slope=0.01):
    return x if x >= 0 else x * slope
    "#,
        c"activators.py",
        c"activators",
    )?;

    let relu_result: f64 = activators.getattr("relu")?.call1((-1.0,))?.extract()?;
    assert_eq!(relu_result, 0.0);

    let kwargs = [("slope", 0.2)].into_py_dict(py)?;
    let lrelu_result: f64 = activators
        .getattr("leaky_relu")?
        .call((-1.0,), Some(&kwargs))?
        .extract()?;
    assert_eq!(lrelu_result, -0.2);
   Ok(())
})
}

Want to embed Python in Rust with additional modules?

Python maintains the sys.modules dict as a cache of all imported modules. An import in Python will first attempt to lookup the module from this dict, and if not present will use various strategies to attempt to locate and load the module.

The append_to_inittab macro can be used to add additional #[pymodule] modules to an embedded Python interpreter. The macro must be invoked before initializing Python.

As an example, the below adds the module foo to the embedded interpreter:

use pyo3::prelude::*;

#[pymodule]
mod foo {
    use pyo3::prelude::*;

    #[pyfunction]
    fn add_one(x: i64) -> i64 {
        x + 1
    }
}

fn main() -> PyResult<()> {
    pyo3::append_to_inittab!(foo);
    Python::attach(|py| Python::run(py, c"import foo; foo.add_one(6)", None, None))
}

If append_to_inittab cannot be used due to constraints in the program, an alternative is to create a module using PyModule::new and insert it manually into sys.modules:

use pyo3::prelude::*;
use pyo3::types::PyDict;

#[pyfunction]
pub fn add_one(x: i64) -> i64 {
    x + 1
}

fn main() -> PyResult<()> {
    Python::attach(|py| {
        // 建立新模組
        let foo_module = PyModule::new(py, "foo")?;
        foo_module.add_function(wrap_pyfunction!(add_one, &foo_module)?)?;

        // 匯入並獲取 sys.modules
        let sys = PyModule::import(py, "sys")?;
        let py_modules: Bound<'_, PyDict> = sys.getattr("modules")?.cast_into()?;

        // Insert foo into sys.modules
        py_modules.set_item("foo", foo_module)?;

        // Now we can import + run our python code
        Python::run(py, c"import foo; foo.add_one(6)", None, None)
    })
}

Include multiple Python files

You can include a file at compile time by using std::include_str macro.

Or you can load a file at runtime by using std::fs::read_to_string function.

Many Python files can be included and loaded as modules. If one file depends on another you must preserve correct order while declaring PyModule.

範例目錄結構：

.
├── Cargo.lock
├── Cargo.toml
├── python_app
│   ├── app.py
│   └── utils
│       └── foo.py
└── src
    └── main.rs

python_app/app.py:

from utils.foo import bar


def run():
    return bar()

python_app/utils/foo.py:

def bar():
    return "baz"

The example below shows:

  • how to include content of app.py and utils/foo.py into your rust binary
  • how to call function run() (declared in app.py) that needs function imported from utils/foo.py

src/main.rs:

use pyo3::prelude::*;
use pyo3_ffi::c_str;

fn main() -> PyResult<()> {
    let py_foo = c_str!(include_str!(concat!(
        env!("CARGO_MANIFEST_DIR"),
        "/python_app/utils/foo.py"
    )));
    let py_app = c_str!(include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/python_app/app.py")));
    let from_python = Python::attach(|py| -> PyResult<Py<PyAny>> {
        PyModule::from_code(py, py_foo, c"foo.py", c"utils.foo")?;
        let app: Py<PyAny> = PyModule::from_code(py, py_app, c"app.py", c"")?
            .getattr("run")?
            .into();
        app.call0(py)
    });

    println!("py: {}", from_python?);
    Ok(())
}

The example below shows:

  • how to load content of app.py at runtime so that it sees its dependencies automatically
  • how to call function run() (declared in app.py) that needs function imported from utils/foo.py

It is recommended to use absolute paths because then your binary can be run from anywhere as long as your app.py is in the expected directory (in this example that directory is /usr/share/python_app).

src/main.rs:

use pyo3::prelude::*;
use pyo3::types::PyList;
use std::fs;
use std::path::Path;
use std::ffi::CString;

fn main() -> PyResult<()> {
    let path = Path::new("/usr/share/python_app");
    let py_app = CString::new(fs::read_to_string(path.join("app.py"))?)?;
    let from_python = Python::attach(|py| -> PyResult<Py<PyAny>> {
        let syspath = py
            .import("sys")?
            .getattr("path")?
            .cast_into::<PyList>()?;
        syspath.insert(0, path)?;
        let app: Py<PyAny> = PyModule::from_code(py, py_app.as_c_str(), c"app.py", c"")?
            .getattr("run")?
            .into();
        app.call0(py)
    });

    println!("py: {}", from_python?);
    Ok(())
}

Need to use a context manager from Rust?

Use context managers by directly invoking __enter__ and __exit__.

use pyo3::prelude::*;

fn main() {
    Python::attach(|py| {
        let custom_manager = PyModule::from_code(
            py,
            cr#"
class House(object):
    def __init__(self, address):
        self.address = address
    def __enter__(self):
        print(f"Welcome to {self.address}!")
    def __exit__(self, type, value, traceback):
        if type:
            print(f"Sorry you had {type} trouble at {self.address}")
        else:
            print(f"Thank you for visiting {self.address}, come again soon!")

        "#,
            c"house.py",
            c"house",
        )
        .unwrap();

        let house_class = custom_manager.getattr("House").unwrap();
        let house = house_class.call1(("123 Main Street",)).unwrap();

        house.call_method0("__enter__").unwrap();

        let result = py.eval(c"undefined_variable + 1", None, None);

        // If the eval threw an exception we'll pass it through to the context manager.
        // Otherwise, __exit__  is called with empty arguments (Python "None").
        match result {
            Ok(_) => {
                let none = py.None();
                house
                    .call_method1("__exit__", (&none, &none, &none))
                    .unwrap();
            }
            Err(e) => {
                house
                    .call_method1(
                        "__exit__",
                        (
                            e.get_type(py),
                            e.value(py),
                            e.traceback(py),
                        ),
                    )
                    .unwrap();
            }
        }
    })
}

Handling system signals/interrupts (Ctrl-C)

The best way to handle system signals when running Rust code is to periodically call Python::check_signals to handle any signals captured by Python’s signal handler. See also the FAQ entry.

Alternatively, set Python’s signal module to take the default action for a signal:

use pyo3::prelude::*;

fn main() -> PyResult<()> {
Python::attach(|py| -> PyResult<()> {
    let signal = py.import("signal")?;
    // Set SIGINT to have the default action
    signal
        .getattr("signal")?
        .call1((signal.getattr("SIGINT")?, signal.getattr("SIG_DFL")?))?;
    Ok(())
})
}

型別轉換

本節指南將介紹 PyO3 提供的 Python 型別到 Rust 型別的對應，以及用於在兩者之間進行轉換的特徵。

另請參考轉換表格與特徵。

Rust 型別對應到 Python 型別

撰寫可由 Python 呼叫的函式（例如 #[pyfunction] 或 #[pymethods] 區塊）時，函式引數需要 FromPyObject 特徵，回傳值需要 IntoPyObject 特徵。

請參考下一節的表格，找出 PyO3 提供且實作這些特徵的 Rust 型別。

引數型別

接受函式引數時，可以使用 Rust 程式庫型別或 PyO3 的 Python 原生型別。（何時使用哪種型別，請見下一節。）

下表列出 Python 型別及其對應可接受的函式引數型別：

PythonRustRust（Python 原生）
object-PyAny
strString, Cow<str>, &str, char, OsString, PathBuf, PathPyString
bytesVec<u8>, &[u8], Cow<[u8]>PyBytes
boolboolPyBool
inti8, u8, i16, u16, i32, u32, i64, u64, i128, u128, isize, usize, num_bigint::BigInt1, num_bigint::BigUint1PyInt
floatf32, f64, ordered_float::NotNan2, ordered_float::OrderedFloat2PyFloat
complexnum_complex::Complex3PyComplex
fractions.Fractionnum_rational::Ratio4-
list[T]Vec<T>PyList
dict[K, V]HashMap<K, V>, BTreeMap<K, V>, hashbrown::HashMap<K, V>5, indexmap::IndexMap<K, V>6PyDict
tuple[T, U](T, U), Vec<T>PyTuple
set[T]HashSet<T>, BTreeSet<T>, hashbrown::HashSet<T>5PySet
frozenset[T]HashSet<T>, BTreeSet<T>, hashbrown::HashSet<T>5PyFrozenSet
bytearrayVec<u8>, Cow<[u8]>PyByteArray
slice-PySlice
type-PyType
module-PyModule
collections.abc.Buffer-PyBuffer<T>
datetime.datetimeSystemTime, chrono::DateTime<Tz>7, chrono::NaiveDateTime7PyDateTime
datetime.datechrono::NaiveDate7PyDate
datetime.timechrono::NaiveTime7PyTime
datetime.tzinfochrono::FixedOffset7, chrono::Utc7, chrono_tz::TimeZone8PyTzInfo
datetime.timedeltaDuration, chrono::Duration7PyDelta
decimal.Decimalrust_decimal::Decimal9-
decimal.Decimalbigdecimal::BigDecimal10-
ipaddress.IPv4Addressstd::net::IpAddr, std::net::Ipv4Addr-
ipaddress.IPv6Addressstd::net::IpAddr, std::net::Ipv6Addr-
os.PathLikePathBuf, PathPyString
pathlib.PathPathBuf, PathPyString
typing.Optional[T]Option<T>-
typing.Sequence[T]Vec<T>PySequence
typing.Mapping[K, V]HashMap<K, V>, BTreeMap<K, V>, hashbrown::HashMap<K, V>5, indexmap::IndexMap<K, V>6&PyMapping
typing.Iterator[Any]-PyIterator
typing.Union[...]See #[derive(FromPyObject)]-

另外也值得記住以下特殊型別：

項目說明
Python<'py>用來證明已附加到 Python 直譯器的 token。
Bound<'py, T>具有生命週期的 Python 物件，將其綁定到 Python 直譯器的附加狀態。可存取大多數 PyO3 API。
Py<T>不連結任何直譯器附加生命週期的 Python 物件，可傳送至其他執行緒。
PyRef<T>不可變借用的 #[pyclass]。
PyRefMut<T>可變借用的 #[pyclass]。

更多關於將 #[pyclass] 作為函式引數的細節，請見本指南的Python 類別章節。

使用 Rust 程式庫型別 vs Python 原生型別

使用 Rust 函式庫型別作為函式引數，相較於 Python 原生型別會產生轉換成本。使用 Python 原生型別幾乎是零成本（只需進行類似 Python 內建函式 isinstance() 的型別檢查）。

不過，一旦付出轉換成本，Rust 標準函式庫型別會帶來多項好處：

  • 你可以用原生速度的 Rust 程式碼實作功能（不受 Python 執行期成本影響）。
  • 可與 Rust 生態系的其他元件有更好的互通性。
  • 你可使用 Python::detach 從直譯器分離，讓其他 Python 執行緒在 Rust 程式碼執行時繼續前進。
  • 你也會受益於更嚴格的型別檢查。例如指定 Vec<i32>，只會接受包含整數的 Python list。相對的 Python 原生等價型別 &PyList 會接受包含任何型別 Python 物件的 list。

對於多數 PyO3 使用情境，付出轉換成本以換取上述好處是值得的。一如往常，若不確定是否值得，請進行基準測試！

將 Rust 值回傳給 Python

當從可由 Python 呼叫的函式回傳值時，可零成本使用 PyO3 的智慧指標（Py<T>、Bound<'py, T> 與 Borrowed<'a, 'py, T>）。

由於 Bound<'py, T> 與 Borrowed<'a, 'py, T> 具有生命週期參數，Rust 編譯器可能會要求你在函式中補上生命週期標註。請參考指南中專門章節。

若函式可能失敗，應回傳 PyResult<T> 或 Result<T, E>，其中 E 需實作 From<E> for PyErr。若回傳 Err 變體，將拋出 Python 例外。

最後，以下 Rust 型別也可作為回傳值轉換為 Python：

Rust 型別對應的 Python 型別
Stringstr
&strstr
boolbool
Any integer type (i32, u32, usize, etc)int
f32, f64float
Option<T>Optional[T]
(T, U)Tuple[T, U]
Vec<T>List[T]
Cow<[u8]>bytes
HashMap<K, V>Dict[K, V]
BTreeMap<K, V>Dict[K, V]
HashSet<T>Set[T]
BTreeSet<T>Set[T]
Py<T>T
Bound<T>T
PyRef<T: PyClass>T
PyRefMut<T: PyClass>T

  1. 需要 num-bigint 選用功能。 ↩ ↩2

  2. 需要 ordered-float 選用功能。 ↩ ↩2

  3. 需要 num-complex 選用功能。 ↩

  4. 需要 num-rational 選用功能。 ↩

  5. 需要 hashbrown 選用功能。 ↩ ↩2 ↩3 ↩4

  6. 需要 indexmap 選用功能。 ↩ ↩2

  7. 需要 chrono（以及可能的 chrono-local）選用功能。 ↩ ↩2 ↩3 ↩4 ↩5 ↩6 ↩7

  8. 需要 chrono-tz 選用功能。 ↩

  9. 需要 rust_decimal 選用功能。 ↩

  10. 需要 bigdecimal 選用功能。 ↩

轉換特徵

PyO3 提供一些方便的特徵，用於在 Python 與 Rust 型別之間進行轉換。

.extract() 與 FromPyObject 特徵

將 Python 物件轉為 Rust 值最簡單的方法是使用 .extract()。若轉換失敗會回傳帶有型別錯誤的 PyResult，因此通常會像這樣使用：

use pyo3::prelude::*;
use pyo3::types::PyList;
fn main() -> PyResult<()> {
    Python::attach(|py| {
        let list = PyList::new(py, b"foo")?;
let v: Vec<i32> = list.extract()?;
        assert_eq!(&v, &[102, 111, 111]);
        Ok(())
    })
}

這個方法可用於多種 Python 物件型別，並能產生各式各樣的 Rust 型別；你可以在 FromPyObject 的實作清單中查看。

FromPyObject 也可用於你自己的 Rust 型別（包裝成 Python 物件時，見類別章節）。在那裡，為了既能操作可變參照，_又_滿足 Rust 不允許可變參照別名的規則，你必須提取 PyO3 的參照包裝器 PyRef 與 PyRefMut。它們的行為類似 std::cell::RefCell 的參照包裝器，並在執行期確保 Rust 借用是允許的。

衍生 FromPyObject

若成員型別本身實作 FromPyObject，則可為多種結構與列舉自動衍生 FromPyObject。這甚至包含泛型成員 T: FromPyObject。不支援對空列舉、列舉變體與結構進行衍生。

為結構衍生 FromPyObject

The derivation generates code that will attempt to access the attribute my_string on the Python object, i.e. obj.getattr("my_string"), and call extract() on the attribute.

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyStruct {
    my_string: String,
}

fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let module = PyModule::from_code(
            py,
            c"class Foo:
            def __init__(self):
                self.my_string = 'test'",
            c"<string>",
            c"",
        )?;

        let class = module.getattr("Foo")?;
        let instance = class.call0()?;
        let rustystruct: RustyStruct = instance.extract()?;
        assert_eq!(rustystruct.my_string, "test");
        Ok(())
    })
}

在欄位上設定 #[pyo3(item)] 屬性時，PyO3 會嘗試透過呼叫 Python 物件的 get_item 方法來提取值。

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyStruct {
    #[pyo3(item)]
    my_string: String,
}

use pyo3::types::PyDict;
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let dict = PyDict::new(py);
        dict.set_item("my_string", "test")?;

        let rustystruct: RustyStruct = dict.extract()?;
        assert_eq!(rustystruct.my_string, "test");
        Ok(())
    })
}

傳給 getattr 與 get_item 的引數也可進行設定：

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyStruct {
    #[pyo3(item("key"))]
    string_in_mapping: String,
    #[pyo3(attribute("name"))]
    string_attr: String,
}

fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let module = PyModule::from_code(
            py,
            c"class Foo(dict):
            def __init__(self):
                self.name = 'test'
                self['key'] = 'test2'",
            c"<string>",
            c"",
        )?;

        let class = module.getattr("Foo")?;
        let instance = class.call0()?;
        let rustystruct: RustyStruct = instance.extract()?;
		assert_eq!(rustystruct.string_attr, "test");
        assert_eq!(rustystruct.string_in_mapping, "test2");

        Ok(())
    })
}

這會嘗試從 name 屬性提取 string_attr，並從鍵為 "key" 的對映中提取 string_in_mapping。attribute 的引數限制為非空字串常值，而 item 可接受任何實作 ToBorrowedObject 的有效常值。

你可以在結構上使用 #[pyo3(from_item_all)]，以 get_item 方法提取每個欄位。此時不能使用 #[pyo3(attribute)]，且幾乎不能在任何欄位上使用 #[pyo3(item)]。不過，仍可使用 #[pyo3(item("key"))] 指定欄位的鍵。

use pyo3::prelude::*;

#[derive(FromPyObject)]
#[pyo3(from_item_all)]
struct RustyStruct {
    foo: String,
    bar: String,
    #[pyo3(item("foobar"))]
    baz: String,
}

fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let py_dict = py.eval(c"{'foo': 'foo', 'bar': 'bar', 'foobar': 'foobar'}", None, None)?;
        let rustystruct: RustyStruct = py_dict.extract()?;
		  assert_eq!(rustystruct.foo, "foo");
        assert_eq!(rustystruct.bar, "bar");
        assert_eq!(rustystruct.baz, "foobar");

        Ok(())
    })
}

為元組結構衍生 FromPyObject

元組結構也受支援，但不允許自訂提取方式。輸入一律視為與 Rust 型別長度相同的 Python tuple，第 n 個欄位會從 Python tuple 的第 n 個項目提取。

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyTuple(String, String);

use pyo3::types::PyTuple;
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let tuple = PyTuple::new(py, vec!["test", "test2"])?;

        let rustytuple: RustyTuple = tuple.extract()?;
        assert_eq!(rustytuple.0, "test");
        assert_eq!(rustytuple.1, "test2");

        Ok(())
    })
}

只有單一欄位的元組結構會被視為包裝型別，並在下一節說明。若要覆寫此行為並確保輸入確實為 tuple，請將結構指定為

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyTuple((String,));

use pyo3::types::PyTuple;
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let tuple = PyTuple::new(py, vec!["test"])?;

        let rustytuple: RustyTuple = tuple.extract()?;
        assert_eq!((rustytuple.0).0, "test");

        Ok(())
    })
}

為包裝型別衍生 FromPyObject

pyo3(transparent) 屬性可用於只有一個欄位的結構。這會直接從輸入物件提取（即 obj.extract()），而不是嘗試存取項目或屬性。此行為預設啟用於 newtype 結構與單一欄位的元組變體。

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyTransparentTupleStruct(String);

#[derive(FromPyObject)]
#[pyo3(transparent)]
struct RustyTransparentStruct {
    inner: String,
}

use pyo3::types::PyString;
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        let s = PyString::new(py, "test");

        let tup: RustyTransparentTupleStruct = s.extract()?;
        assert_eq!(tup.0, "test");

        let stru: RustyTransparentStruct = s.extract()?;
        assert_eq!(stru.inner, "test");

        Ok(())
    })
}

為列舉衍生 FromPyObject

針對列舉的 FromPyObject 衍生會產生程式碼，依欄位順序嘗試提取各變體。一旦某個變體成功提取，就會回傳該變體。這使得提取 str | int 等 Python union 型別成為可能。

結構衍生的相同自訂與限制也適用於列舉變體：元組變體假設輸入為 Python tuple，結構變體預設以屬性提取欄位，但可用相同方式設定。transparent 屬性可套用於單一欄位的變體。

use pyo3::prelude::*;

#[derive(FromPyObject)]
#[derive(Debug)]
enum RustyEnum<'py> {
    Int(usize),                    // input is a positive int
    String(String),                // input is a string
    IntTuple(usize, usize),        // input is a 2-tuple with positive ints
    StringIntTuple(String, usize), // input is a 2-tuple with String and int
    Coordinates3d {
        // needs to be in front of 2d
        x: usize,
        y: usize,
        z: usize,
    },
    Coordinates2d {
        // only gets checked if the input did not have `z`
        #[pyo3(attribute("x"))]
        a: usize,
        #[pyo3(attribute("y"))]
        b: usize,
    },
    #[pyo3(transparent)]
    CatchAll(Bound<'py, PyAny>), // This extraction never fails
}

use pyo3::types::{PyBytes, PyString};
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        {
            let thing = 42_u8.into_pyobject(py)?;
            let rust_thing: RustyEnum<'_> = thing.extract()?;

            assert_eq!(
                42,
                match rust_thing {
                    RustyEnum::Int(i) => i,
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }
        {
            let thing = PyString::new(py, "text");
            let rust_thing: RustyEnum<'_> = thing.extract()?;

            assert_eq!(
                "text",
                match rust_thing {
                    RustyEnum::String(i) => i,
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }
        {
            let thing = (32_u8, 73_u8).into_pyobject(py)?;
            let rust_thing: RustyEnum<'_> = thing.extract()?;

            assert_eq!(
                (32, 73),
                match rust_thing {
                    RustyEnum::IntTuple(i, j) => (i, j),
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }
        {
            let thing = ("foo", 73_u8).into_pyobject(py)?;
            let rust_thing: RustyEnum<'_> = thing.extract()?;

            assert_eq!(
                (String::from("foo"), 73),
                match rust_thing {
                    RustyEnum::StringIntTuple(i, j) => (i, j),
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }
        {
            let module = PyModule::from_code(
                py,
                c"class Foo(dict):
            def __init__(self):
                self.x = 0
                self.y = 1
                self.z = 2",
                c"<string>",
                c"",
            )?;

            let class = module.getattr("Foo")?;
            let instance = class.call0()?;
            let rust_thing: RustyEnum<'_> = instance.extract()?;

            assert_eq!(
                (0, 1, 2),
                match rust_thing {
                    RustyEnum::Coordinates3d { x, y, z } => (x, y, z),
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }

        {
            let module = PyModule::from_code(
                py,
                c"class Foo(dict):
            def __init__(self):
                self.x = 3
                self.y = 4",
                c"<string>",
                c"",
            )?;

            let class = module.getattr("Foo")?;
            let instance = class.call0()?;
            let rust_thing: RustyEnum<'_> = instance.extract()?;

            assert_eq!(
                (3, 4),
                match rust_thing {
                    RustyEnum::Coordinates2d { a, b } => (a, b),
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }

        {
            let thing = PyBytes::new(py, b"text");
            let rust_thing: RustyEnum<'_> = thing.extract()?;

            assert_eq!(
                b"text",
                match rust_thing {
                    RustyEnum::CatchAll(ref i) => i.cast::<PyBytes>()?.as_bytes(),
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }
        Ok(())
    })
}

If none of the enum variants match, a PyTypeError containing the names of the tested variants is returned. The names reported in the error message can be customized through the #[pyo3(annotation = "name")] attribute, e.g. to use conventional Python type names:

use pyo3::prelude::*;

#[derive(FromPyObject)]
#[derive(Debug)]
enum RustyEnum {
    #[pyo3(transparent, annotation = "str")]
    String(String),
    #[pyo3(transparent, annotation = "int")]
    Int(isize),
}

fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        {
            let thing = 42_u8.into_pyobject(py)?;
            let rust_thing: RustyEnum = thing.extract()?;

            assert_eq!(
                42,
                match rust_thing {
                    RustyEnum::Int(i) => i,
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }

        {
            let thing = "foo".into_pyobject(py)?;
            let rust_thing: RustyEnum = thing.extract()?;

            assert_eq!(
                "foo",
                match rust_thing {
                    RustyEnum::String(i) => i,
                    other => unreachable!("Error extracting: {:?}", other),
                }
            );
        }

        {
            let thing = b"foo".into_pyobject(py)?;
            let error = thing.extract::<RustyEnum>().unwrap_err();
            assert!(error.is_instance_of::<pyo3::exceptions::PyTypeError>(py));
        }

        Ok(())
    })
}

If the input is neither a string nor an integer, the error message will be: "'<INPUT_TYPE>' is not an instance of 'str | int'".

#[derive(FromPyObject)] Container Attributes

  • pyo3(transparent)
    • extract the field directly from the object as obj.extract() instead of get_item() or getattr()
    • Newtype structs and tuple-variants are treated as transparent per default.
    • only supported for single-field structs and enum variants
  • pyo3(annotation = "name")
    • changes the name of the failed variant in the generated error message in case of failure.
    • e.g. pyo3("int") reports the variant’s type as int.
    • only supported for enum variants
  • pyo3(rename_all = "...")
    • renames all attributes/item keys according to the specified renaming rule
    • Possible values are: “camelCase”, “kebab-case”, “lowercase”, “PascalCase”, “SCREAMING-KEBAB-CASE”, “SCREAMING_SNAKE_CASE”, “snake_case”, “UPPERCASE”.
    • fields with an explicit renaming via attribute(...)/item(...) are not affected

#[derive(FromPyObject)] Field Attributes

  • pyo3(attribute), pyo3(attribute("name"))
    • retrieve the field from an attribute, possibly with a custom name specified as an argument
    • argument must be a string-literal.
  • pyo3(item), pyo3(item("key"))
    • retrieve the field from a mapping, possibly with the custom key specified as an argument.
    • can be any literal that implements ToBorrowedObject
  • pyo3(from_py_with = ...)
    • apply a custom function to convert the field from Python the desired Rust type.
    • the argument must be the path to the function.
    • the function signature must be fn(&Bound<PyAny>) -> PyResult<T> where T is the Rust type of the argument.
  • pyo3(default), pyo3(default = ...)
    • if the argument is set, uses the given default value.
    • in this case, the argument must be a Rust expression returning a value of the desired Rust type.
    • if the argument is not set, Default::default is used.
    • note that the default value is only used if the field is not set. If the field is set and the conversion function from Python to Rust fails, an exception is raised and the default value is not used.
    • this attribute is only supported on named fields.

For example, the code below applies the given conversion function on the "value" dict item to compute its length or fall back to the type default value (0):

use pyo3::prelude::*;

#[derive(FromPyObject)]
struct RustyStruct {
    #[pyo3(item("value"), default, from_py_with = Bound::<'_, PyAny>::len)]
    len: usize,
    #[pyo3(item)]
    other: usize,
}

use pyo3::types::PyDict;
fn main() -> PyResult<()> {
    Python::attach(|py| -> PyResult<()> {
        // Filled case
        let dict = PyDict::new(py);
        dict.set_item("value", (1,)).unwrap();
        dict.set_item("other", 1).unwrap();
        let result = dict.extract::<RustyStruct>()?;
        assert_eq!(result.len, 1);
        assert_eq!(result.other, 1);

        // Empty case
        let dict = PyDict::new(py);
        dict.set_item("other", 1).unwrap();
        let result = dict.extract::<RustyStruct>()?;
        assert_eq!(result.len, 0);
        assert_eq!(result.other, 1);
        Ok(())
    })
}

⚠ Phase-Out of FromPyObject blanket implementation for cloneable PyClasses ⚠

Historically PyO3 has provided a blanket implementation for #[pyclass] types that also implement Clone, to allow extraction of such types by value. Over time this has turned out problematic for a few reasons, the major one being the prevention of custom conversions by downstream crates if their type is Clone. Over the next few releases the blanket implementation is gradually phased out, and eventually replaced by an opt-in option. As a first step of this migration a new skip_from_py_object option for #[pyclass] was introduced, to opt-out of the blanket implementation and allow downstream users to provide their own implementation:

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyclass(skip_from_py_object)] // opt-out of the PyO3 FromPyObject blanket
#[derive(Clone)]
struct Number(i32);

impl<'py> FromPyObject<'_, 'py> for Number {
    type Error = PyErr;

    fn extract(obj: pyo3::Borrowed<'_, 'py, pyo3::PyAny>) -> Result<Self, Self::Error> {
        if let Ok(obj) = obj.cast::<Self>() { // first try extraction via class object
            Ok(obj.borrow().clone())
        } else {
            obj.extract::<i32>().map(Self) // otherwise try integer directly
        }
    }
}

As a second step the from_py_object option was introduced. This option also opts-out of the blanket implementation and instead generates a custom FromPyObject implementation for the pyclass which is functionally equivalent to the blanket.

IntoPyObject

The IntoPyObject trait defines the to-python conversion for a Rust type. All types in PyO3 implement this trait, as does a #[pyclass] which doesn’t use extends.

This trait defines a single method, into_pyobject(), which returns a Result with Ok and Err types depending on the input value. For convenience, there is a companion IntoPyObjectExt trait which adds methods such as into_py_any() which converts the Ok and Err types to commonly used types (in the case of into_py_any(), Py<PyAny> and PyErr respectively).

Occasionally you may choose to implement this for custom types which are mapped to Python types without having a unique python type.

derive macro

IntoPyObject can be implemented using our derive macro. Both structs and enums are supported.

structs will turn into a PyDict using the field names as keys, tuple structs will turn convert into PyTuple with the fields in declaration order.

#![allow(dead_code)]
use pyo3::prelude::*;
use std::collections::HashMap;
use std::hash::Hash;

// structs convert into `PyDict` with field names as keys
#[derive(IntoPyObject)]
struct Struct {
    count: usize,
    obj: Py<PyAny>,
}

// tuple structs convert into `PyTuple`
// lifetimes and generics are supported, the impl will be bounded by
// `K: IntoPyObject, V: IntoPyObject`
#[derive(IntoPyObject)]
struct Tuple<'a, K: Hash + Eq, V>(&'a str, HashMap<K, V>);

For structs with a single field (newtype pattern) the #[pyo3(transparent)] option can be used to forward the implementation to the inner type.

#![allow(dead_code)]
use pyo3::prelude::*;

// newtype tuple structs are implicitly `transparent`
#[derive(IntoPyObject)]
struct TransparentTuple(Py<PyAny>);

#[derive(IntoPyObject)]
#[pyo3(transparent)]
struct TransparentStruct<'py> {
    inner: Bound<'py, PyAny>, // `'py` lifetime will be used as the Python lifetime
}

For enums each variant is converted according to the rules for structs above.

#![allow(dead_code)]
use pyo3::prelude::*;
use std::collections::HashMap;
use std::hash::Hash;

#[derive(IntoPyObject)]
enum Enum<'a, 'py, K: Hash + Eq, V> { // enums are supported and convert using the same
    TransparentTuple(Py<PyAny>),       // rules on the variants as the structs above
    #[pyo3(transparent)]
    TransparentStruct { inner: Bound<'py, PyAny> },
    Tuple(&'a str, HashMap<K, V>),
    Struct { count: usize, obj: Py<PyAny> }
}

Additionally IntoPyObject can be derived for a reference to a struct or enum using the IntoPyObjectRef derive macro. All the same rules from above apply as well.

#[derive(IntoPyObject)]/#[derive(IntoPyObjectRef)] Field Attributes

  • pyo3(into_py_with = ...)

    • apply a custom function to convert the field from Rust into Python.

    • the argument must be the function identifier

    • the function signature must be fn(Cow<'_, T>, Python<'py>) -> PyResult<Bound<'py, PyAny>> where T is the Rust type of the argument.

      • #[derive(IntoPyObject)] will invoke the function with Cow::Owned
      • #[derive(IntoPyObjectRef)] will invoke the function with Cow::Borrowed
      use pyo3::prelude::*;
use pyo3::IntoPyObjectExt;
use std::borrow::Cow;
#[derive(Clone)]
struct NotIntoPy(usize);

#[derive(IntoPyObject, IntoPyObjectRef)]
struct MyStruct {
    #[pyo3(into_py_with = convert)]
    not_into_py: NotIntoPy,
}

/// Convert `NotIntoPy` into Python
fn convert<'py>(not_into_py: Cow<'_, NotIntoPy>, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {
    not_into_py.0.into_bound_py_any(py)
}

manual implementation

If the derive macro is not suitable for your use case, IntoPyObject can be implemented manually as demonstrated below.

use pyo3::prelude::*;
#[allow(dead_code)]
struct MyPyObjectWrapper(Py<PyAny>);

impl<'py> IntoPyObject<'py> for MyPyObjectWrapper {
    type Target = PyAny; // the Python type
    type Output = Bound<'py, Self::Target>; // in most cases this will be `Bound`
    type Error = std::convert::Infallible; // the conversion error type, has to be convertible to `PyErr`

    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        Ok(self.0.into_bound(py))
    }
}

// equivalent to former `ToPyObject` implementations
impl<'a, 'py> IntoPyObject<'py> for &'a MyPyObjectWrapper {
    type Target = PyAny;
    type Output = Borrowed<'a, 'py, Self::Target>; // `Borrowed` can be used to optimized reference counting
    type Error = std::convert::Infallible;

    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        Ok(self.0.bind_borrowed(py))
    }
}

BoundObject for conversions that may be Bound or Borrowed

IntoPyObject::into_py_object returns either Bound or Borrowed depending on the implementation for a concrete type. For example, the IntoPyObject implementation for u32 produces a Bound<'py, PyInt> and the bool implementation produces a Borrowed<'py, 'py, PyBool>:

use pyo3::prelude::*;
use pyo3::IntoPyObject;
use pyo3::types::{PyBool, PyInt};

let ints: Vec<u32> = vec![1, 2, 3, 4];
let bools = vec![true, false, false, true];

Python::attach(|py| {
    let ints_as_pyint: Vec<Bound<'_, PyInt>> = ints
        .iter()
        .map(|x| Ok(x.into_pyobject(py)?))
        .collect::<PyResult<_>>()
        .unwrap();

    let bools_as_pybool: Vec<Borrowed<'_, '_, PyBool>> = bools
        .iter()
        .map(|x| Ok(x.into_pyobject(py)?))
        .collect::<PyResult<_>>()
        .unwrap();
});

In this example if we wanted to combine ints_as_pyints and bools_as_pybool into a single Vec<Py<PyAny>> to return from the Python::attach closure, we would have to manually convert the concrete types for the smart pointers and the python types.

Instead, we can write a function that generically converts vectors of either integers or bools into a vector of Py<PyAny> using the BoundObject trait:

use pyo3::prelude::*;
use pyo3::BoundObject;
use pyo3::IntoPyObject;

let bools = vec![true, false, false, true];
let ints = vec![1, 2, 3, 4];

fn convert_to_vec_of_pyobj<'py, T>(py: Python<'py>, the_vec: Vec<T>) -> PyResult<Vec<Py<PyAny>>>
where
   T: IntoPyObject<'py> + Copy
{
    the_vec.iter()
        .map(|x| {
            Ok(
                // Note: the below is equivalent to `x.into_py_any()`
                // from the `IntoPyObjectExt` trait
                x.into_pyobject(py)
                .map_err(Into::into)?
                .into_any()
                .unbind()
            )
        })
        .collect()
}

let vec_of_pyobjs: Vec<Py<PyAny>> = Python::attach(|py| {
    let mut bools_as_pyany = convert_to_vec_of_pyobj(py, bools).unwrap();
    let mut ints_as_pyany = convert_to_vec_of_pyobj(py, ints).unwrap();
    let mut result: Vec<Py<PyAny>> = vec![];
    result.append(&mut bools_as_pyany);
    result.append(&mut ints_as_pyany);
    result
});

In the example above we used BoundObject::into_any and BoundObject::unbind to manipulate the python types and smart pointers into the result type we wanted to produce from the function.

使用 async 和 await

This feature is still in active development. See the related issue.

#[pyfunction] 與 #[pymethods] 屬性也支援 async fn。

#![allow(dead_code)]
#[cfg(feature = "experimental-async")] {
use std::{thread, time::Duration};
use futures::channel::oneshot;
use pyo3::prelude::*;

#[pyfunction]
#[pyo3(signature=(seconds, result=None))]
async fn sleep(seconds: f64, result: Option<Py<PyAny>>) -> Option<Py<PyAny>> {
    let (tx, rx) = oneshot::channel();
    thread::spawn(move || {
        thread::sleep(Duration::from_secs_f64(seconds));
        tx.send(()).unwrap();
    });
    rx.await.unwrap();
    result
}
}

Python awaitables instantiated with this method can only be awaited in asyncio context. Other Python async runtime may be supported in the future.

Send + 'static 限制

由 #[pyfunction] 修飾的 async fn 所產生的 future 必須是 Send + 'static，才能嵌入為 Python 物件。

因此，async fn 的參數與回傳型別也必須是 Send + 'static，所以無法使用像 async fn does_not_compile<'py>(arg: Bound<'py, PyAny>) -> Bound<'py, PyAny> 的簽名。

不過，方法接收者有例外，因此 async 方法可以接受 &self/&mut self。請注意，這表示類別實例會在回傳的 future 完成前一直被借用，甚至跨越 yield 點與等待 I/O 操作完成的期間。因此，在 future 仍被輪詢時，其他方法無法取得獨占借用。這與 Rust 中 async 方法的一般行為相同，但由於普遍的共享可變性，對與 Python 互動的 Rust 程式碼而言問題更大。強烈建議優先使用共享借用 &self 而非獨占借用 &mut self，以避免執行期的競態借用檢查失敗。

隱式附加到直譯器

即使無法將 py: Python<'py> token 傳給 async fn，我們在 future 執行期間仍會附加到直譯器——就像一般不帶 Python<'py>/Bound<'py, PyAny> 參數的 fn 一樣。

仍可透過 Python::attach 取得 Python 標記；由於 attach 具備可重入性且已最佳化，其成本可忽略不計。

跨越 .await 時從直譯器分離

目前沒有簡單的方法在等待 future 時從直譯器分離，但解法正在開發中。

目前建議的替代作法如下：

use std::{
    future::Future,
    pin::{Pin, pin},
    task::{Context, Poll},
};
use pyo3::prelude::*;

struct AllowThreads<F>(F);

impl<F> Future for AllowThreads<F>
where
    F: Future + Unpin + Send,
    F::Output: Send,
{
    type Output = F::Output;

    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let waker = cx.waker();
        Python::attach(|py| {
            py.detach(|| pin!(&mut self.0).poll(&mut Context::from_waker(waker)))
        })
    }
}

取消

可透過 CancelHandle 型別捕捉 Python 端的取消事件，方法是在函式參數上加註 #[pyo3(cancel_handle)]。

#![allow(dead_code)]
#[cfg(feature = "experimental-async")] {
use futures::FutureExt;
use pyo3::prelude::*;
use pyo3::coroutine::CancelHandle;

#[pyfunction]
async fn cancellable(#[pyo3(cancel_handle)] mut cancel: CancelHandle) {
    futures::select! {
        /* _ = ... => println!("done"), */
        _ = cancel.cancelled().fuse() => println!("cancelled"),
    }
}
}

Coroutine 型別

為了讓 Rust future 能在 Python 中被 await，PyO3 定義了 Coroutine 型別，並實作了 Python 的協程協定。

每次 coroutine.send 呼叫都會被轉換為 Future::poll 呼叫。若宣告了 CancelHandle 參數，傳給 coroutine.throw 的例外會儲存在其中，並可透過 CancelHandle::cancelled 取得；否則會取消 Rust future，並重新拋出該例外；

The type does not yet have a public constructor until the design is finalized.

平行化

歷史上，CPython 受全域直譯器鎖（GIL）限制，一次只允許單一執行緒驅動 Python 直譯器。這使得 Python 的執行緒對CPU-bound 任務不太合適，並常迫使開發者接受多行程的額外負擔。

Rust 很適合多執行緒程式碼，而像 rayon 這樣的函式庫能讓你以最小成本運用安全的平行化。Python::detach 方法可讓 Python 直譯器在 Rust 工作進行時處理其他工作。

若要在應用程式中啟用完整平行化，也可考慮使用自 Python 3.14 起支援的自由執行緒 Python。

Python GIL 下的平行化

我們來看看 word-count 範例，其中的 search 函式使用 rayon 軟體箱以平行方式計算字數。

#![allow(dead_code)]
use pyo3::prelude::*;

// 這些特徵讓我們可以使用 `par_lines` 和 `map`。
use rayon::str::ParallelString;
use rayon::iter::ParallelIterator;

/// 計算某行中 needle 出現次數，不區分大小寫
fn count_line(line: &str, needle: &str) -> usize {
    let mut total = 0;
    for word in line.split(' ') {
        if word == needle {
            total += 1;
        }
    }
    total
}

#[pyfunction]
fn search(contents: &str, needle: &str) -> usize {
    contents
        .par_lines()
        .map(|line| count_line(line, needle))
        .sum()
}

假設你有一個耗時的 Rust 函式，想要平行執行多次。以下以字數統計的序列版作為範例：

#![allow(dead_code)]
fn count_line(line: &str, needle: &str) -> usize {
    let mut total = 0;
    for word in line.split(' ') {
        if word == needle {
            total += 1;
        }
    }
    total
}

fn search_sequential(contents: &str, needle: &str) -> usize {
    contents.lines().map(|line| count_line(line, needle)).sum()
}

要讓此函式平行執行，可以使用 Python::detach 暫時釋放 GIL，讓其他 Python 執行緒得以運作。接著我們提供一個暴露給 Python 執行環境的函式，將 search_sequential 放在傳給 Python::detach 的閉包中，以實現真正的平行化：

#![allow(dead_code)]
use pyo3::prelude::*;

fn count_line(line: &str, needle: &str) -> usize {
    let mut total = 0;
    for word in line.split(' ') {
        if word == needle {
            total += 1;
        }
    }
    total
}

fn search_sequential(contents: &str, needle: &str) -> usize {
   contents.lines().map(|line| count_line(line, needle)).sum()
}
#[pyfunction]
fn search_sequential_detached(py: Python<'_>, contents: &str, needle: &str) -> usize {
    py.detach(|| search_sequential(contents, needle))
}

現在 Python 執行緒可使用多個 CPU 核心，解決了 Python 多執行緒通常只適合 I/O-bound 任務的限制：

from concurrent.futures import ThreadPoolExecutor
from word_count import search_sequential_detached

executor = ThreadPoolExecutor(max_workers=2)

future_1 = executor.submit(
    word_count.search_sequential_detached, contents, needle
)
future_2 = executor.submit(
    word_count.search_sequential_detached, contents, needle
)
result_1 = future_1.result()
result_2 = future_2.result()

基準測試

讓我們對 word-count 範例做基準測試，確認 PyO3 確實解鎖了平行化。

我們使用 pytest-benchmark 來測試四個字數統計函式：

  1. 純 Python 版本
  2. Rust 平行版本
  3. Rust 序列版本
  4. Rust 序列版本在兩個 Python 執行緒中執行兩次

The benchmark script can be found in the PyO3 GitHub repository, and we can run nox in the word-count folder to benchmark these functions.

雖然基準測試結果會因機器而異，但相對結果應與下圖（2020 年中）相近：

-------------------------------------------------------------------------------------------------- benchmark: 4 tests -------------------------------------------------------------------------------------------------
Name (time in ms)                                          Min                Max               Mean            StdDev             Median               IQR            Outliers       OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_word_count_rust_parallel                           1.7315 (1.0)       4.6495 (1.0)       1.9972 (1.0)      0.4299 (1.0)       1.8142 (1.0)      0.2049 (1.0)         40;46  500.6943 (1.0)         375           1
test_word_count_rust_sequential                         7.3348 (4.24)     10.3556 (2.23)      8.0035 (4.01)     0.7785 (1.81)      7.5597 (4.17)     0.8641 (4.22)         26;5  124.9457 (0.25)        121           1
test_word_count_rust_sequential_twice_with_threads      7.9839 (4.61)     10.3065 (2.22)      8.4511 (4.23)     0.4709 (1.10)      8.2457 (4.55)     0.3927 (1.92)        17;17  118.3274 (0.24)        114           1
test_word_count_python_sequential                      27.3985 (15.82)    45.4527 (9.78)     28.9604 (14.50)    4.1449 (9.64)     27.5781 (15.20)    0.4638 (2.26)          3;5   34.5299 (0.07)         35           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

可以看到 Python 執行緒版本沒有比 Rust 序列版本慢太多，這代表相較於單一 CPU 核心的執行，速度翻倍。

在 Rust 執行緒間共享 Python 物件

在上述範例中，我們為底層 Rust 函式建立了 Python 介面，並利用 Python 的 threading 模組平行執行該函式。也可以在 Rust 中啟動執行緒，取得 GIL 並操作 Python 物件。不過在這些情況下必須小心，避免寫出會與 GIL 死結的程式碼。

  • 注意：此範例用來示範如何釋放並重新取得 GIL 以避免死結。除非啟動的執行緒之後會釋放 GIL，或你使用的是 CPython 的自由執行緒建置，否則用 rayon 平行化會在整個執行期間取得並持有 GIL 的程式碼，並不會帶來多執行緒的速度提升。

在下方範例中，我們共享一個使用 pyclass 巨集定義的使用者 ID 物件 Vec，並啟動執行緒，透過 rayon 的平行迭代器以判斷條件將資料集合處理成 Vec 的布林值：

use pyo3::prelude::*;

// 這些特徵讓我們可以使用 int_par_iter 和 map
use rayon::iter::{IntoParallelRefIterator, ParallelIterator};

#[pyclass]
struct UserID {
    id: i64,
}

let allowed_ids: Vec<bool> = Python::attach(|outer_py| {
    let instances: Vec<Py<UserID>> = (0..10).map(|x| Py::new(outer_py, UserID { id: x }).unwrap()).collect();
    outer_py.detach(|| {
        instances.par_iter().map(|instance| {
            Python::attach(|inner_py| {
                instance.borrow(inner_py).id > 5
            })
        }).collect()
    })
});
assert!(allowed_ids.into_iter().filter(|b| *b).count() == 4);

請注意這裡同時有 outer_py 與 inner_py 兩個 Python token。在執行緒之間共享 Python token 是不允許的，執行緒必須各自附加到直譯器，才能存取 Python 物件包裹的資料。

此外，此範例使用 Python::detach 包裹透過 rayon 啟動 OS 執行緒的程式碼。若未使用 detach，rayon 工作執行緒會在取得 GIL 時阻塞，而擁有 GIL 的執行緒會無限等待 rayon 執行緒結果。呼叫 detach 可讓收集工作執行緒結果的執行緒釋放 GIL。只要會啟動工作執行緒，就應呼叫 detach，尤其在工作執行緒需要取得 GIL 的情況下，以避免死結。

支援自由執行緒的 CPython

CPython 3.14 宣告支援不依賴全域直譯器鎖（通常稱為 GIL）來提供執行緒安全的「自由執行緒」建置。自 0.23 版起，PyO3 支援為自由執行緒的 Python 建置 Rust 擴充，並可從 Rust 呼叫自由執行緒的 Python。

若想了解自由執行緒 Python 的更多背景，可參考 3.13 版更新說明中What’s New 的條目（首次以實驗模式加入「自由執行緒」建置）、CPython 文件中的自由執行緒 HOWTO 指南、社群維護的自由執行緒指南中的擴充移植指南，以及提供 CPython 自由執行緒實作技術背景的 PEP 703。

在啟用 GIL 的建置中（在「自由執行緒」建置出現之前的唯一選擇），全域直譯器鎖會序列化對 Python 執行環境的存取。因此，依據阿姆達爾定律，GIL 是多執行緒 Python 工作流程平行擴展的根本限制，因為任何只在單一執行內容中進行的平行處理，都不可能藉由平行化而加速。

自由執行緒建置移除了多執行緒 Python 擴展上的限制，意味著使用 Python 的 threading 模組更容易達成平行化。若你曾為了某些 Python 程式碼的平行加速而使用 multiprocessing，自由執行緒很可能讓相同工作流程改以 Python 執行緒達成。

PyO3 對自由執行緒 Python 的支援，將能撰寫天生就具備執行緒安全的原生 Python 擴充，並提供比 C 擴充更強的安全保證。我們的目標是以 Rust 的 Send 與 Sync 特徵為基礎，在原生 Python 執行環境中實現無畏並行。

本文提供將使用 PyO3 的 Rust 程式碼移植至自由執行緒 Python 的建議。

使用 PyO3 支援自由執行緒 Python

自 PyO3 0.28 起，PyO3 預設假設以它建立的 Python 模組是執行緒安全的。除非某些 Rust 程式碼使用 unsafe 錯誤地假設執行緒安全，否則應該成立。典型例子是曾基於歷史假設（Python 因 GIL 而是單執行緒）所撰寫的 unsafe 程式碼，並假定 PyO3 使用的 Python<'py> token 可保證執行緒安全。模組可在完成 unsafe 程式碼正確性稽核前，透過 #[pymodule(gil_used = true)] 宣告選擇不支援自由執行緒 Python（見下文）。

較複雜的 #[pyclass] 型別可能需要直接處理執行緒安全；指南中有專門章節討論此議題。

在較低層級，為模組加註會將擴充所定義模組的 Py_MOD_GIL slot 設為 Py_MOD_GIL_NOT_USED，讓直譯器在執行期得知擴充作者認為該擴充具備執行緒安全性。

若選擇不支援自由執行緒 Python，Python 直譯器會在匯入你的模組時於執行期重新啟用 GIL，並輸出含有觸發此行為之模組名稱的 RuntimeWarning。你可以設定環境變數 PYTHON_GIL=0 或在啟動 Python 時傳入 -Xgil=0 強制 GIL 維持關閉（0 代表關閉 GIL）。

如果你確定 PyModule 中暴露的所有資料結構都是執行緒安全的，請在宣告模組的 pymodule 程序巨集中傳入 gil_used = false，或在 PyModule 實例上呼叫 PyModule::gil_used。例如：

選擇支援範例

（注意：在 PyO3 0.23 至 0.27 版中，預設為 gil_used = true，因此必須反向設定；模組需要以 gil_used = false 選擇支援自由執行緒 Python。）

/// 此模組依賴 GIL 以確保執行緒安全
#[pyo3::pymodule(gil_used = true)]
mod my_extension {
    use pyo3::prelude::*;

    // 此型別不是執行緒安全
    #[pyclass]
    struct MyNotThreadSafeType {
        // 插入非執行緒安全的程式碼
    }
}

或是針對未使用 pymodule 巨集設定的模組：

use pyo3::prelude::*;

#[allow(dead_code)]
fn register_child_module(parent_module: &Bound<'_, PyModule>) -> PyResult<()> {
    let child_module = PyModule::new(parent_module.py(), "child_module")?;
    child_module.gil_used(true)?;
    parent_module.add_submodule(&child_module)
}

關於如何使用原始 FFI 呼叫為採用單階段初始化的模組宣告自由執行緒支援，請參考 string-sum 範例；採用多階段初始化的模組則可參考 sequential 範例。

若你想透過條件編譯在自由執行緒建置下觸發不同的程式路徑，可在設定軟體箱以產生必要的建置設定資料後，使用 Py_GIL_DISABLED 屬性。關於支援多個 Python 版本（含自由執行緒建置）的詳細內容，請參考指南章節。

自由執行緒建置的特別注意事項

自由執行緒直譯器沒有 GIL。許多既有擴充提供可變資料結構時仰賴 GIL 來鎖住 Python 物件，使內部可變性具備執行緒安全。

只有在作業系統執行緒明確附加到直譯器執行環境時，呼叫 CPython C API 才是合法的。在啟用 GIL 的建置中，這會在取得 GIL 時發生。自由執行緒建置中沒有 GIL，但在啟用 GIL 的建置裡負責釋放或取得 GIL 的同一組 C 巨集，會改為要求直譯器將該執行緒附加到 Python 執行環境，且可能同時附加多個執行緒。更多關於執行緒如何附加/分離直譯器執行環境（類似於在啟用 GIL 建置中釋放或取得 GIL）的背景，請參考 PEP 703。

在啟用 GIL 的建置中，PyO3 使用 Python<'py> 型別與 'py 生命週期來表示已持有全域直譯器鎖。在自由執行緒建置中，持有 'py 生命週期僅代表該執行緒目前已附加到 Python 直譯器——其他執行緒仍可同時與直譯器互動。

附加到執行環境

你仍需要取得 'py 生命週期才能與 Python 物件互動或呼叫 CPython C API。若尚未附加到 Python 執行環境，可使用 Python::attach 註冊執行緒。由 Python 的 threading 模組建立的執行緒不需要這麼做，當 CPython 呼叫你的擴充時，pyo3 會負責設定 Python<'py> token。

分離以避免卡住與死結

自由執行緒建置在以下情況會觸發全域同步事件：

  • 垃圾回收時，為取得參考計數與物件間參考的一致全域視圖
  • 在 Python 3.13 中，啟動第一個背景執行緒時用以將部分物件標記為不朽
  • 呼叫 sys.settrace 或 sys.setprofile 以對執行中的程式碼物件與執行緒進行追蹤時
  • 呼叫 os.fork() 時，為確保整個行程的一致狀態

以上並非完整清單，未來的 Python 版本可能還會有其他觸發全域同步事件的情況。

這表示你應在與啟用 GIL 建置相同的情境下，使用 Python::detach 從直譯器執行環境分離：當進行不需要 CPython 執行環境的長時間工作，或進行任何需要重新附加執行環境的工作時（請見指南章節）。在前者情況，等待長時間工作完成的執行緒會卡住；在後者情況，當執行環境觸發全域同步事件後，執行緒嘗試附加時會發生死結，因為啟動該執行緒的執行緒阻止同步事件完成。

多執行緒存取可變 pyclass 實例時的例外與恐慌

附加在 pyclass 實例上的資料會透過類似 RefCell 的執行期借用檢查來防止並發存取。如同 RefCell，PyO3 會拋出例外（或在某些情況下 panic）以強制對可變借用的獨占存取。過去在 PyO3 中就可能產生這類 panic，例如在使用 Python::detach 釋放 GIL 的程式碼裡，或從 &mut self 呼叫接收 &self 的 Python 方法（見內部可變性文件），但在自由執行緒 Python 中，因為沒有 GIL 來鎖住來自 Python 的可變借用資料的並發存取，觸發這些 panic 的機會更多。

最直接觸發此問題的方式，是使用 Python 的 threading 模組，在多個執行緒中同時呼叫會可變借用 pyclass 的 Rust 函式。例如，考慮以下實作：

use pyo3::prelude::*;
#[pyclass]
#[derive(Default)]
struct ThreadIter {
    count: usize,
}

#[pymethods]
impl ThreadIter {
    #[new]
    pub fn new() -> Self {
        Default::default()
    }

    fn __next__(&mut self, py: Python<'_>) -> usize {
        self.count += 1;
        self.count
    }
}

接著如果在 Python 中這樣做：

import concurrent.futures
from my_module import ThreadIter

i = ThreadIter()

def increment():
    next(i)

with concurrent.futures.ThreadPoolExecutor(max_workers=16) as tpe:
    futures = [tpe.submit(increment) for _ in range(100)]
    [f.result() for f in futures]

會看到以下例外：

Traceback (most recent call last)
  File "example.py", line 5, in <module>
    next(i)
RuntimeError: Already borrowed

未來版本的 PyO3 可能會允許為可變 pyclass 定義提供可由使用者選擇的語意，必要時可選擇性啟用鎖定以模擬 GIL。目前你應明確加入鎖定，可能透過條件編譯或臨界區 API，以避免與 GIL 造成死結。

無法使用 limited API 建置擴充

自由執行緒建置使用全新的 ABI，目前尚無與自由執行緒 ABI 對應的 limited API。這表示若你的軟體箱依賴 PyO3 並使用 abi3 功能或 abi3-pyxx 功能，PyO3 在使用自由執行緒直譯器建置擴充時會輸出警告並忽略該設定。

這表示若你的軟體包利用 limited API 提供的 ABI 向前相容性，只在每次發行時上傳一個 wheel，你就需要更新發佈流程，另外上傳針對自由執行緒版本的特定 wheel。

更多關於支援多個 Python 版本（含自由執行緒建置）的細節，請參考指南章節。

執行緒安全的單次初始化

若要只初始化一次資料，請使用 PyOnceLock 型別，它與 std::sync::OnceLock 類似，且在執行緒等待其他執行緒完成初始化時會從 Python 直譯器分離，以避免死結。若已使用 OnceLock，且不便改用 PyOnceLock，可使用 OnceLockExt 擴充特徵，其新增 OnceLockExt::get_or_init_py_attached 方法，在阻塞時以與 PyOnceLock 相同方式從直譯器分離。以下為使用 PyOnceLock 單次初始化持有 Py<PyDict> 之執行期快取的範例：

use pyo3::prelude::*;
use pyo3::sync::PyOnceLock;
use pyo3::types::PyDict;

let cache: PyOnceLock<Py<PyDict>> = PyOnceLock::new();

Python::attach(|py| {
    // 保證只會被呼叫一次
    cache.get_or_init(py, || PyDict::new(py).unbind())
});

若某函式必須恰好執行一次，可將 OnceExt 特徵引入範圍。OnceExt 特徵為 std::sync::Once 的 API 新增 OnceExt::call_once_py_attached 與 OnceExt::call_once_force_py_attached 函式，使 Once 能在執行緒已附加到 Python 直譯器的情境中使用。這些函式類似於 Once::call_once 與 Once::call_once_force，差別在於它們除了 FnOnce 外還接受 Python<'py> token。這些函式會在阻塞前先從直譯器分離，並在執行函式前重新附加，以避免在未使用 PyO3 擴充特徵時可能發生的死結。以下是使用 Once 而非 PyOnceLock 的相同範例：

use pyo3::prelude::*;
use std::sync::Once;
use pyo3::sync::OnceExt;
use pyo3::types::PyDict;

struct RuntimeCache {
    once: Once,
    cache: Option<Py<PyDict>>
}

let mut cache = RuntimeCache {
    once: Once::new(),
    cache: None
};

Python::attach(|py| {
    // 保證只會被呼叫一次
    cache.once.call_once_py_attached(py, || {
        cache.cache = Some(PyDict::new(py).unbind());
    });
});

GILProtected 已被移除

GILProtected 是 PyO3 的一種型別，透過 GIL 鎖住其他執行緒的並發存取，允許對靜態資料進行可變存取。在自由執行緒 Python 中沒有 GIL，因此此型別必須改以其他鎖定方式取代。多數情況下，使用 std::sync::atomic 的型別或 std::sync::Mutex 就足夠了。

之前：

fn main() {
#[cfg(not(Py_GIL_DISABLED))] {
use pyo3::prelude::*;
use pyo3::sync::GILProtected;
use pyo3::types::{PyDict, PyNone};
use std::cell::RefCell;

static OBJECTS: GILProtected<RefCell<Vec<Py<PyDict>>>> =
    GILProtected::new(RefCell::new(Vec::new()));

Python::attach(|py| {
    // 代表執行任意 Python 程式碼的範例
    let d = PyDict::new(py);
    d.set_item(PyNone::get(py), PyNone::get(py)).unwrap();
    OBJECTS.get(py).borrow_mut().push(d.unbind());
});
}}

之後（使用 Mutex）：

use pyo3::prelude::*;
fn main() {
use pyo3::types::{PyDict, PyNone};
use std::sync::Mutex;

static OBJECTS: Mutex<Vec<Py<PyDict>>> = Mutex::new(Vec::new());

Python::attach(|py| {
    // 代表執行任意 Python 程式碼的範例
    let d = PyDict::new(py);
    d.set_item(PyNone::get(py), PyNone::get(py)).unwrap();
    // 與任何 `Mutex` 用法相同，鎖住 mutex 的時間應盡可能短
    // 在此例中，只在把元素推入 `Vec` 時上鎖
    OBJECTS.lock().unwrap().push(d.unbind());
});
}

若在持有鎖的情況下執行任意 Python 程式碼，應引入 MutexExt 特徵並使用 lock_py_attached 方法取代 lock。這能確保由 Python 執行環境啟動的全域同步事件可以繼續，避免與直譯器產生死結。

除錯

巨集

PyO3’s attributes (#[pyclass], #[pymodule], etc.) are procedural macros, which means that they rewrite the source of the annotated item. You can view the generated source with the following command, which also expands a few other things:

cargo rustc --profile=check -- -Z unstable-options --pretty=expanded > expanded.rs; rustfmt expanded.rs

(You might need to install rustfmt if you don’t already have it.)

You can also debug classic !-macros by adding -Z trace-macros:

cargo rustc --profile=check -- -Z unstable-options --pretty=expanded -Z trace-macros > expanded.rs; rustfmt expanded.rs

Note that those commands require using the nightly build of rust and may occasionally have bugs. See cargo expand for a more elaborate and stable version of those commands.

Running with Valgrind

Valgrind is a tool to detect memory management bugs such as memory leaks.

You first need to install a debug build of Python, otherwise Valgrind won’t produce usable results. In Ubuntu there’s e.g. a python3-dbg package.

Activate an environment with the debug interpreter and recompile. If you’re on Linux, use ldd with the name of your binary and check that you’re linking e.g. libpython3.7d.so.1.0 instead of libpython3.7.so.1.0.

Download the suppressions file for CPython.

Run Valgrind with valgrind --suppressions=valgrind-python.supp ./my-command --with-options

Getting a stacktrace

The best start to investigate a crash such as an segmentation fault is a backtrace. You can set RUST_BACKTRACE=1 as an environment variable to get the stack trace on a panic!. Alternatively you can use a debugger such as gdb to explore the issue. Rust provides a wrapper, rust-gdb, which has pretty-printers for inspecting Rust variables. Since PyO3 uses cdylib for Python shared objects, it does not receive the pretty-print debug hooks in rust-gdb (rust-lang/rust#96365). The mentioned issue contains a workaround for enabling pretty-printers in this case.

  • Link against a debug build of python as described in the previous chapter
  • Run rust-gdb <my-binary>
  • Set a breakpoint (b) on rust_panic if you are investigating a panic!
  • Enter r to run
  • After the crash occurred, enter bt or bt full to print the stacktrace

Often it is helpful to run a small piece of Python code to exercise a section of Rust.

rust-gdb --args python -c "import my_package; my_package.sum_to_string(1, 2)"

Setting breakpoints in your Rust code

One of the preferred ways by developers to debug their code is by setting breakpoints. This can be achieved in PyO3 by using a debugger like rust-gdb or rust-lldb with your Python interpreter.

For more information about how to use both lldb and gdb you can read the gdb to lldb command map from the lldb documentation.

Common setup

  1. Compile your extension with debug symbols:

    # Debug is the default for maturin, but you can explicitly ensure debug symbols with:
RUSTFLAGS="-g" maturin develop

# For setuptools-rust users:
pip install -e .

    Note: When using debuggers, make sure that python resolves to an actual Python binary or symlink and not a shim script. Some tools like pyenv use shim scripts which can interfere with debugging.

Debugger specific setup

Depending on your OS and your preferences you can use two different debuggers, rust-gdb or rust-lldb.

  1. Launch rust-gdb with the Python interpreter:

    rust-gdb --args python

  2. Once in gdb, set a breakpoint in your Rust code:

    (gdb) break your_module.rs:42

  3. Run your Python script that imports and uses your Rust extension:

    # Option 1: Run an inline Python command
(gdb) run -c "import your_module; your_module.your_function()"

# 選項 2：執行 Python 腳本
(gdb) run your_script.py

# 選項 3：執行 pytest 測試
(gdb) run -m pytest tests/test_something.py::TestName

使用 VS Code

VS Code with the Rust and Python extensions provides an integrated debugging experience:

  1. First, install the necessary VS Code extensions:

  2. Create a .vscode/launch.json file with a configuration that uses the LLDB Debug Launcher:

    {
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug PyO3",
            "type": "lldb",
            "request": "attach",
            "program": "${workspaceFolder}/.venv/bin/python",
            "pid": "${command:pickProcess}",
            "sourceLanguages": [
                "rust"
            ]
        },
        {
            "name": "Launch Python with PyO3",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/.venv/bin/python",
            "args": ["${file}"],
            "cwd": "${workspaceFolder}",
            "sourceLanguages": ["rust"]
        },
        {
            "name": "Debug PyO3 with Args",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/.venv/bin/python",
            "args": ["path/to/your/script.py", "arg1", "arg2"],
            "cwd": "${workspaceFolder}",
            "sourceLanguages": ["rust"]
        },
        {
            "name": "Debug PyO3 Tests",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/.venv/bin/python",
            "args": ["-m", "pytest", "tests/your_test.py::test_function", "-v"],
            "cwd": "${workspaceFolder}",
            "sourceLanguages": ["rust"]
        }
     ]
}

    This configuration supports multiple debugging scenarios:

    • Attaching to a running Python process
    • Launching the currently open Python file
    • Running a specific script with command-line arguments
    • 執行 pytest 測試

  3. Set breakpoints in your Rust code by clicking in the gutter next to line numbers.

  4. 開始除錯：

    • For attaching to a running Python process: First start the process, then select the “Debug PyO3” configuration and click Start Debugging (F5). You’ll be prompted to select the Python process to attach to.
    • For launching a Python script: Open your Python script, select the “Launch Python with PyO3” configuration and click Start Debugging (F5).
    • For running with arguments: Select “Debug PyO3 with Args” (remember to edit the configuration with your actual script path and arguments).
    • For running tests: Select “Debug PyO3 Tests” (edit the test path as needed).

  5. When debugging PyO3 code:

    • You can inspect Rust variables and data structures
    • Use the debug console to evaluate expressions
    • Step through Rust code line by line using the step controls
    • Set conditional breakpoints for more complex debugging scenarios

進階除錯組態

For advanced debugging scenarios, you might want to add environment variables or enable specific Rust debug flags:

{
    "name": "Debug PyO3 with Environment",
    "type": "lldb",
    "request": "launch",
    "program": "${workspaceFolder}/.venv/bin/python",
    "args": ["${file}"],
    "env": {
        "RUST_BACKTRACE": "1",
        "PYTHONPATH": "${workspaceFolder}"
    },
    "sourceLanguages": ["rust"]
}

Debugging from Jupyter Notebooks

For Jupyter Notebooks run from VS Code, you can use the following helper functions to automate the launch configuration:

from pathlib import Path
import os
import json
import sys


def update_launch_json(vscode_config_file_path=None):
    """Update VSCode launch.json with the correct Jupyter kernel PID.

    Args:
        vscode_config_file_path (str, optional): Path to the .vscode/launch.json file.
            If not provided, will use the current working directory.
    """
    pid = get_jupyter_kernel_pid()
    if not pid:
        print("Could not determine Jupyter kernel PID.")
        return

    # Determine launch.json path
    if vscode_config_file_path:
        launch_json_path = vscode_config_file_path
    else:
        launch_json_path = os.path.join(Path(os.getcwd()), ".vscode", "launch.json")

    # 獲取 Python 直譯器路徑
    python_path = sys.executable

    # 預設除錯器組態
    debug_config = {
        "version": "0.2.0",
        "configurations": [
            {
                "name": "Debug PyO3 (Jupyter)",
                "type": "lldb",
                "request": "attach",
                "program": python_path,
                "pid": pid,
                "sourceLanguages": ["rust"],
            },
            {
                "name": "Launch Python with PyO3",
                "type": "lldb",
                "request": "launch",
                "program": python_path,
                "args": ["${file}"],
                "cwd": "${workspaceFolder}",
                "sourceLanguages": ["rust"]
            }
        ],
    }

    # Create .vscode directory if it doesn't exist
    try:
        os.makedirs(os.path.dirname(launch_json_path), exist_ok=True)

        # If launch.json already exists, try to update it instead of overwriting
        if os.path.exists(launch_json_path):
            try:
                with open(launch_json_path, "r") as f:
                    existing_config = json.load(f)

                # Check if our configuration already exists
                config_exists = False
                for config in existing_config.get("configurations", []):
                    if config.get("name") == "Debug PyO3 (Jupyter)":
                        config["pid"] = pid
                        config["program"] = python_path
                        config_exists = True

                if not config_exists:
                    existing_config.setdefault("configurations", []).append(debug_config["configurations"][0])

                debug_config = existing_config
            except Exception:
                # If reading fails, we'll just overwrite with our new configuration
                pass

        with open(launch_json_path, "w") as f:
            json.dump(debug_config, f, indent=4)
        print(f"Updated launch.json with PID: {pid} at {launch_json_path}")
    except Exception as e:
        print(f"Error updating launch.json: {e}")


def get_jupyter_kernel_pid():
    """Find the process ID (PID) of the running Jupyter kernel.

    Returns:
        int: The process ID of the Jupyter kernel, or None if not found.
    """
    # Check if we're running in a Jupyter environment
    if 'ipykernel' in sys.modules:
        pid = os.getpid()
        print(f"Jupyter kernel PID: {pid}")
        return pid
    else:
        print("Not running in a Jupyter environment.")
        return None

To use these functions:

  1. Run the cell containing these functions in your Jupyter notebook
  2. Run update_launch_json() in a cell
  3. In VS Code, select the “Debug PyO3 (Jupyter)” configuration and start debugging

執行緒安全與編譯器淨化器

PyO3 attempts to match the Rust language-level guarantees for thread safety, but that does not preclude other code outside of the control of PyO3 or buggy code managed by a PyO3 extension from creating a thread safety issue. Analyzing whether or not a piece of Rust code that uses the CPython C API is thread safe can be quite complicated, since many Python operations can lead to arbitrary Python code execution. Automated ways to discover thread safety issues can often be more fruitful than code analysis.

ThreadSanitizer is a thread safety checking runtime that can be used to detect data races triggered by thread safety bugs or incorrect use of thread-unsafe data structures. While it can only detect data races triggered by code at runtime, if it does detect something the reports often point to exactly where the problem is happening.

To use ThreadSanitizer with a library that depends on PyO3, you will need to install a nightly Rust toolchain, along with the rust-src component, since you will need to compile the Rust standard library:

rustup install nightly
rustup override set nightly
rustup component add rust-src

You will also need a version of CPython compiled using LLVM/Clang with the same major version of LLVM as is currently used to compile nightly Rust. As of March 2025, Rust nightly uses LLVM 20.

The cpython_sanity docker images contain a development environment with a pre-compiled version of CPython 3.13 or 3.14 as well as optionally NumPy and SciPy, all compiled using LLVM 20 and ThreadSanitizer.

After activating a nightly Rust toolchain, you can build your project using ThreadSanitizer with the following command:

RUSTFLAGS="-Zsanitizer=thread" maturin develop -Zbuild-std --target x86_64-unknown-linux-gnu

If you are not running on an x86_64 Linux machine, you should replace x86_64-unknown-linux-gnu with the target triple that is appropriate for your system. You can also replace maturin develop with cargo test to run cargo tests. Note that cargo runs tests in a thread pool, so cargo tests can be a good way to find thread safety issues.

You can also replace -Zsanitizer=thread with -Zsanitizer=address or any of the other sanitizers that are supported by Rust. Note that you’ll need to build CPython from source with the appropriate configure script flags to use the same sanitizer environment as you want to use for your Rust code.

功能參考

PyO3 provides a number of Cargo features to customize functionality. This chapter of the guide provides detail on each of them.

By default, only the macros feature is enabled.

Features for extension module authors

extension-module

Deprecated, users should remove this feature and upgrade to maturin >= 1.9.4 or setuptools-rust >= 1.12.

See the building and distribution section for further detail.

abi3

This feature is used when building Python extension modules to create wheels which are compatible with multiple Python versions.

It restricts PyO3’s API to a subset of the full Python API which is guaranteed by PEP 384 to be forwards-compatible with future Python versions.

See the building and distribution section for further detail.

The abi3-pyXY features

(abi3-py37, abi3-py38, abi3-py39, abi3-py310, abi3-py311, abi3-py312, abi3-py313 and abi3-py314)

These features are extensions of the abi3 feature to specify the exact minimum Python version which the multiple-version-wheel will support.

See the building and distribution section for further detail.

generate-import-lib

This feature is deprecated and has no effect. PyO3 now uses Rust’s raw-dylib linking feature to link against the Python DLL on Windows, eliminating the need for import library (.lib) files entirely. Cross-compiling for Windows targets works without any additional setup.

Features for embedding Python in Rust

auto-initialize

This feature changes Python::attach to automatically initialize a Python interpreter (by calling Python::initialize) if needed.

If you do not enable this feature, you should call Python::initialize() before attempting to call any other Python APIs.

Advanced Features

experimental-async

This feature adds support for async fn in #[pyfunction] and #[pymethods].

The feature has some unfinished refinements and performance improvements. To help finish this off, see issue #1632 and its associated draft PRs.

experimental-inspect

This feature adds to the built binaries introspection data that can be then retrieved using the pyo3-introspection crate to generate type stubs.

Also, this feature adds the pyo3::inspect module, as well as IntoPy::type_output and FromPyObject::type_input APIs to produce Python type “annotations” for Rust types.

This is a first step towards adding first-class support for generating type annotations automatically in PyO3, however work is needed to finish this off. All feedback and offers of help welcome on issue #2454.

py-clone

This feature was introduced to ease migration. It was found that delayed reference counting (which PyO3 used historically) could not be made sound and hence Clone-ing an instance of Py<T> is impossible when not attached to Python interpreter (it will panic). To avoid migrations introducing new panics without warning, the Clone implementation itself is now gated behind this feature.

pyo3_disable_reference_pool

This is a performance-oriented conditional compilation flag, e.g. set via $RUSTFLAGS, which disabled the global reference pool and the associated overhead for the crossing the Python-Rust boundary. However, if enabled, Dropping an instance of Py<T> when not attached to the Python interpreter will abort the process.

macros

This feature enables a dependency on the pyo3-macros crate, which provides the procedural macros portion of PyO3’s API:

  • #[pymodule]
  • #[pyfunction]
  • #[pyclass]
  • #[pymethods]
  • #[derive(FromPyObject)]

It also provides the py_run! macro.

These macros require a number of dependencies which may not be needed by users who just need PyO3 for Python FFI. Disabling this feature enables faster builds for those users, as these dependencies will not be built if this feature is disabled.

[!NOTE] This feature is enabled by default. To disable it, set default-features = false for the pyo3 entry in your Cargo.toml.

multiple-pymethods

This feature enables each #[pyclass] to have more than one #[pymethods] block.

Most users should only need a single #[pymethods] per #[pyclass]. In addition, not all platforms (e.g. Wasm) are supported by inventory, which is used in the implementation of the feature. For this reason this feature is not enabled by default, meaning fewer dependencies and faster compilation for the majority of users.

See the #[pyclass] implementation details for more information.

nightly

The nightly feature needs the nightly Rust compiler. This allows PyO3 to use the auto_traits and negative_impls features to fix the Python::detach function.

resolve-config

The resolve-config feature of the pyo3-build-config crate controls whether that crate’s build script automatically resolves a Python interpreter / build configuration. This feature is primarily useful when building PyO3 itself. By default this feature is not enabled, meaning you can freely use pyo3-build-config as a standalone library to read or write PyO3 build configuration files or resolve metadata about a Python interpreter.

Optional Dependencies

These features enable conversions between Python types and types from other Rust crates, enabling easy access to the rest of the Rust ecosystem.

anyhow

Adds a dependency on anyhow. Enables a conversion from anyhow’s Error type to PyErr, for easy error handling.

arc_lock

Enables Pyo3’s MutexExt trait for all Mutexes that extend on lock_api::Mutex or parking_lot::ReentrantMutex and are wrapped in an Arc type. Like Arc<parking_lot::Mutex>

bigdecimal

Adds a dependency on bigdecimal and enables conversions into its BigDecimal type.

bytes

Adds a dependency on bytes and enables conversions into its Bytes type.

chrono

Adds a dependency on chrono. Enables a conversion from chrono’s types to python:

chrono-local

Enables conversion from and to Local timezones. The current system timezone as determined by iana_time_zone::get_timezone() will be used for conversions.

chrono::DateTime<Local> will convert from either of:

  • datetime objects with tzinfo equivalent to the current system timezone.
  • “naive” datetime objects (those without a tzinfo), as it is a convention that naive datetime objects should be treated as using the system timezone.

When converting to Python, Local tzinfo is converted to a zoneinfo.ZoneInfo matching the current system timezone.

chrono-tz

Adds a dependency on chrono-tz. Enables conversion from and to Tz.

either

Adds a dependency on either. Enables a conversions into either’s Either type.

eyre

Adds a dependency on eyre. Enables a conversion from eyre’s Report type to PyErr, for easy error handling.

hashbrown

Adds a dependency on hashbrown and enables conversions into its HashMap and HashSet types.

indexmap

Adds a dependency on indexmap and enables conversions into its IndexMap type.

jiff-02

Adds a dependency on jiff@0.2. Enables a conversion from jiff’s types to python:

lock_api

Adds a dependency on lock_api and enables Pyo3’s MutexExt trait for all mutexes that extend on lock_api::Mutex and parking_lot::ReentrantMutex (like parking_lot or spin).

num-bigint

Adds a dependency on num-bigint and enables conversions into its BigInt and BigUint types.

num-complex

Adds a dependency on num-complex and enables conversions into its Complex type.

num-rational

Adds a dependency on num-rational and enables conversions into its Ratio type.

ordered-float

Adds a dependency on ordered-float and enables conversions between ordered-float’s types and Python:

parking-lot

Adds a dependency on parking_lot and enables Pyo3’s OnceExt & MutexExt traits for parking_lot::Once parking_lot::Mutex and parking_lot::ReentrantMutex types.

rust_decimal

Adds a dependency on rust_decimal and enables conversions into its Decimal type.

time

Adds a dependency on time. Enables conversions between time’s types and Python:

serde

Enables (de)serialization of Py<T> objects via serde. This allows to use #[derive(Serialize, Deserialize) on structs that hold references to #[pyclass] instances

#[cfg(feature = "serde")]
#[allow(dead_code)]
mod serde_only {
use pyo3::prelude::*;
use serde::{Deserialize, Serialize};

#[pyclass]
#[derive(Serialize, Deserialize)]
struct Permission {
    name: String,
}

#[pyclass]
#[derive(Serialize, Deserialize)]
struct User {
    username: String,
    permissions: Vec<Py<Permission>>,
}
}

smallvec

Adds a dependency on smallvec and enables conversions into its SmallVec type.

uuid

Adds a dependency on uuid and enables conversions into its Uuid type.

效能

為了達到最佳效能，了解 PyO3 API 的一些技巧與陷阱很有幫助。

extract 與 cast

使用 PyO3 實作的 Python 風格 API 通常是多型的，也就是接受 &Bound<'_, PyAny> 並嘗試轉換為多個更具體的型別以套用指定操作。這經常導致連串的 extract 呼叫，例如：

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::{exceptions::PyTypeError, types::PyList};

fn frobnicate_list<'py>(list: &Bound<'_, PyList>) -> PyResult<Bound<'py, PyAny>> {
    todo!()
}

fn frobnicate_vec<'py>(vec: Vec<Bound<'py, PyAny>>) -> PyResult<Bound<'py, PyAny>> {
    todo!()
}

#[pyfunction]
fn frobnicate<'py>(value: &Bound<'py, PyAny>) -> PyResult<Bound<'py, PyAny>> {
    if let Ok(list) = value.extract::<Bound<'_, PyList>>() {
        frobnicate_list(&list)
    } else if let Ok(vec) = value.extract::<Vec<Bound<'_, PyAny>>>() {
        frobnicate_vec(vec)
    } else {
        Err(PyTypeError::new_err("Cannot frobnicate that type."))
    }
}

這並不理想，因為 FromPyObject<T> 特徵要求 extract 回傳 Result<T, PyErr>。對於 PyList 這類原生型別，在忽略錯誤值時使用 cast（extract 內部會呼叫它）會更快。這可避免將 PyDowncastError 轉為 PyErr 的昂貴成本，以滿足 FromPyObject 合約，例如：

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::{exceptions::PyTypeError, types::PyList};
fn frobnicate_list<'py>(list: &Bound<'_, PyList>) -> PyResult<Bound<'py, PyAny>> { todo!() }
fn frobnicate_vec<'py>(vec: Vec<Bound<'py, PyAny>>) -> PyResult<Bound<'py, PyAny>> { todo!() }

#[pyfunction]
fn frobnicate<'py>(value: &Bound<'py, PyAny>) -> PyResult<Bound<'py, PyAny>> {
    // 使用 `cast` 取代 `extract`，因為將 `PyDowncastError` 轉為 `PyErr` 成本很高。
    if let Ok(list) = value.cast::<PyList>() {
        frobnicate_list(list)
    } else if let Ok(vec) = value.extract::<Vec<Bound<'_, PyAny>>>() {
        frobnicate_vec(vec)
    } else {
        Err(PyTypeError::new_err("Cannot frobnicate that type."))
    }
}

存取 Bound 代表可存取 Python token

當我們已附加到直譯器時，呼叫 Python::attach 幾乎是 no-op，但檢查這件事仍有成本。若無法存取既有的 Python token（例如在實作既有特徵時），但已持有 Python 綁定的參照，就可利用 Bound::py 以零成本取得 Python token，避免該成本。

例如，與其寫成

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;

struct Foo(Py<PyList>);

struct FooBound<'py>(Bound<'py, PyList>);

impl PartialEq<Foo> for FooBound<'_> {
    fn eq(&self, other: &Foo) -> bool {
        Python::attach(|py| {
            let len = other.0.bind(py).len();
            self.0.len() == len
        })
    }
}

請改用較有效率的

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;
struct Foo(Py<PyList>);
struct FooBound<'py>(Bound<'py, PyList>);

impl PartialEq<Foo> for FooBound<'_> {
    fn eq(&self, other: &Foo) -> bool {
        // 存取 `&Bound<'py, PyAny>` 代表可存取 `Python<'py>`。
        let py = self.0.py();
        let len = other.0.bind(py).len();
        self.0.len() == len
    }
}

呼叫 Python 可呼叫物件（__call__）

CPython 支援多種呼叫協定：tp_call 與 vectorcall。vectorcall 是更高效的協定，可加速呼叫。PyO3 會在可能時嘗試使用 vectorcall 呼叫慣例來達到最高效能，否則退回 tp_call。這是透過（內部的）PyCallArgs 特徵實作的，它定義 Rust 型別如何作為 Python call 參數使用。此特徵目前為以下項目實作：

  • Rust tuple，其中每個成員都實作 IntoPyObject，
  • Bound<'_, PyTuple>
  • Py<PyTuple>

Rust tuple 可使用 vectorcall，而 Bound<'_, PyTuple> 與 Py<PyTuple> 只能使用 tp_call。為求最佳效能，請優先使用 Rust tuple 作為參數。

長時間的純 Rust 工作請從直譯器分離

當執行不需要與 Python 直譯器互動的 Rust 程式碼時，請使用 Python::detach，讓 Python 直譯器在不等待目前執行緒的情況下繼續執行。

在啟用 GIL 的建置中，這對最佳效能至關重要，因為同一時間只能有單一執行緒附加。

在自由執行緒建置中，這仍是最佳實務，因為存在多個「stop the world」事件（如垃圾回收），會迫使所有附加到 Python 直譯器的執行緒等待。

一般而言，附加與分離 Python 直譯器所需時間不到 1 毫秒，因此預期耗時數毫秒以上的工作，多半可受益於先從直譯器分離。

停用全域參考池

PyO3 使用全域可變狀態來追蹤在未附加到直譯器時呼叫 impl<T> Drop for Py<T> 所隱含的延後參考計數更新。當 PyO3 程式碼下一次附加到直譯器時，需要進行同步以取得並套用這些參考計數更新，成本不低，且可能成為跨越 Python-Rust 邊界的重要開銷。

可透過設定 pyo3_disable_reference_pool 條件編譯旗標來避免此功能。這會完全移除全域參考池與相關成本。然而，它_不會_移除 Py<T> 的 Drop 實作，該實作對於與不以 PyO3 為前提撰寫的既有 Rust 程式碼互通是必要的。為了與更廣泛的 Rust 生態系相容，我們保留該實作，但在未附加到直譯器時呼叫 Drop 會中止。若另外啟用 pyo3_leak_on_drop_without_reference_pool，則在未附加到 Python 時被釋放的物件會改為洩漏，這永遠是安全的，但長期可能造成資源耗盡等不良影響。

使用此設定時務必留意這項限制，特別是在把 Python 程式碼嵌入 Rust 應用程式時，因為很容易在未先重新附加的情況下，意外釋放由 Python::attach 回傳的 Py<T>（或包含它的型別，如 PyErr、PyBackedStr 或 PyBackedBytes）。例如，下列程式碼

use pyo3::prelude::*;
use pyo3::types::PyList;
let numbers: Py<PyList> = Python::attach(|py| PyList::empty(py).unbind());

Python::attach(|py| {
    numbers.bind(py).append(23).unwrap();
});

Python::attach(|py| {
    numbers.bind(py).append(42).unwrap();
});

若清單未明確透過下列方式處理，將會中止

use pyo3::prelude::*;
use pyo3::types::PyList;
let numbers: Py<PyList> = Python::attach(|py| PyList::empty(py).unbind());

Python::attach(|py| {
    numbers.bind(py).append(23).unwrap();
});

Python::attach(|py| {
    numbers.bind(py).append(42).unwrap();
});

Python::attach(move |py| {
    drop(numbers);
});

型別 stub 產生（*.pyi 檔案）與內省

This feature is still in active development. See the related issue.

如需型別 stub 文件與在穩定版 PyO3 中的使用方式，請參考此頁

PyO3 正在開發產生型別 stub 檔案的支援。

其運作方式如下：

  1. PyO3 巨集（#[pyclass]）會產生固定的 JSON 字串，並在啟用 experimental-inspect 功能時由 rustc 內嵌到編譯出的二進位檔。
  2. pyo3-introspection 軟體箱可解析產生的二進位檔，抽取 JSON 字串並產生 stub 檔案。
  3. [尚未完成] 例如 maturin 等建置工具在其 CLI API 中暴露 pyo3-introspection 功能。

例如，以下 Rust 程式碼

#[pymodule]
pub mod example {
    use pyo3::prelude::*;

    #[pymodule_export]
    pub const CONSTANT: &str = "FOO";

    #[pyclass(eq)]
    #[derive(Eq)]
    struct Class {
        value: usize
    }

    #[pymethods]
    impl Class {
        #[new]
        fn new(value: usize) -> Self {
            Self { value }
        }

        #[getter]
        fn value(&self) -> usize {
            self.value
        }
    }

    #[pyfunction]
    #[pyo3(signature = (arg: "list[int]") -> "list[int]")]
    fn list_of_int_identity(arg: Bound<'_, PyAny>) -> Bound<'_, PyAny> {
        arg
    }
}

會產生以下 stub 檔案：

import typing

CONSTANT: typing.Final = "FOO"

class Class:
    def __init__(self, value: int) -> None: ...

    @property
    def value(self) -> int: ...

    def __eq__(self, other: Class) -> bool: ...
    def __ne__(self, other: Class) -> bool: ...

def list_of_int_identity(arg: list[int]) -> list[int]: ...

新增的語法只有一項：#[pyo3(signature = ...)] 屬性現在可以包含型別標註，例如 #[pyo3(signature = (arg: "list[int]") -> "list[int]")]（注意型別標註外層的 ""）。當 PyO3 無法自行推導適當的型別標註時，這會很有用。

約束與限制

  • 需要啟用 experimental-inspect 功能才能產生內省片段。
  • 許多功能尚未實作。清單請參考相關議題。
  • 內省僅適用於以內嵌 Rust 模組宣告的 Python 模組，使用函式宣告的模組不受支援。
  • 必須實作 FromPyObject::INPUT_TYPE 與 IntoPyObject::OUTPUT_TYPE，PyO3 才能取得正確的輸入／輸出型別標註。
  • PyO3 無法內省 #[pymodule] 與 #[pymodule_init] 函式的內容。若存在這些函式，模組會用一個假的 def __getattr__(name: str) -> Incomplete: ... 函式標記為 incomplete（遵循最佳實務）。

進階話題

FFI

PyO3 透過 ffi 模組公開了 Python C API 的大部分內容。

C API 天生不安全，必須由你自行管理引用計數、錯誤與特定不變式。在使用該 API 中的任何函式之前，請參考 C API 參考手冊 以及 The Rustonomicon。

建置與散布

本章將詳細說明如何使用 PyO3 建置與散布專案。具體做法會依專案是以 Rust 實作的 Python 模組，或是嵌入 Python 的 Rust 可執行檔而有很大差異。兩者也有共同問題，例如要建置的 Python 版本，以及要使用的連結器參數。

本章內容面向已閱讀 PyO3 README 的使用者。它依序說明 Python 模組與 Rust 可執行檔的選擇，最後還包含使用 PyO3 進行跨平台編譯的章節。

另有一個子章節專門說明支援多個 Python 版本。

設定 Python 版本

PyO3 使用建置腳本（由 pyo3-build-config crate 支援）來判斷 Python 版本並設定正確的連結器參數。預設會依序嘗試使用：

  • 任何已啟用的 Python 虛擬環境。
  • python 執行檔（若它是 Python 3 直譯器）。
  • python3 執行檔。

你可以設定 PYO3_PYTHON 環境變數來覆寫 Python 直譯器，例如 PYO3_PYTHON=python3.7、PYO3_PYTHON=/usr/bin/python3.9，甚至是 PyPy 直譯器 PYO3_PYTHON=pypy3。

找到 Python 直譯器後，pyo3-build-config 會執行它，以查詢 sysconfig 模組中的資訊，供後續編譯設定使用。

要驗證 PyO3 將採用的設定，你可以在編譯時設定 PYO3_PRINT_CONFIG=1 環境變數。其輸出範例如下：

$ PYO3_PRINT_CONFIG=1 cargo build
   Compiling pyo3 v0.14.1 (/home/david/dev/pyo3)
error: failed to run custom build command for `pyo3 v0.14.1 (/home/david/dev/pyo3)`

Caused by:
  process didn't exit successfully: `/home/david/dev/pyo3/target/debug/build/pyo3-7a8cf4fe22e959b7/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=PYO3_CROSS
  cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
  cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG

  -- PYO3_PRINT_CONFIG=1 is set, printing configuration and halting compile --
  implementation=CPython
  version=3.8
  shared=true
  abi3=false
  lib_name=python3.8
  lib_dir=/usr/lib
  executable=/usr/bin/python
  pointer_width=64
  build_flags=
  suppress_build_script_link_lines=false

PYO3_ENVIRONMENT_SIGNATURE 環境變數可在其值變更時觸發重新建置，除此之外沒有其他作用。

進階：設定檔

若將 PYO3_PRINT_CONFIG 的輸出設定儲存成檔案，你就能手動覆寫內容，並透過 PYO3_CONFIG_FILE 環境變數回饋給 PyO3。

若你的建置環境較為特殊，導致 PyO3 的一般設定偵測無法運作，使用這種設定檔可提供彈性以讓 PyO3 正常運作。要查看完整可用選項，請參考 InterpreterConfig struct 文件。

建置 Python 擴充模組

Python 擴充模組的編譯方式會依目標作業系統（與架構）而不同。除了多種作業系統（與架構），也有許多仍在支援的 Python 版本。上傳到 PyPI 的套件通常會提供涵蓋多種 OS/架構/版本組合的預建「wheels」，讓不同平台的使用者不必自行編譯。套件提供者也可選擇使用受限的 Python API「abi3」，讓單一 wheel 支援多個 Python 版本，以減少需要編譯的 wheel 數量，但功能也會受限。

實作方式有很多種：可以用 cargo 來建置擴充模組（並搭配一些手動步驟，會因 OS 而異）。PyO3 生態系提供兩個打包工具 maturin 和 setuptools-rust，可抽象化 OS 差異並支援建置可上傳 PyPI 的 wheels。

PyO3 提供一些功能用於建置 Python 擴充模組時的專案設定：

  • 建立 Python 擴充模組時必須設定 PYO3_BUILD_EXTENSION_MODULE 環境變數；maturin 與 setuptools-rust 會自動設定它。
  • abi3 Cargo feature 及其版本特定的 abi3-pyXY 伴隨 feature，用於選擇受限 Python API，以在單一 wheel 中支援多個 Python 版本。

本節先介紹打包工具，再說明如何不使用它們手動建置。接著說明 PYO3_BUILD_EXTENSION_MODULE 環境變數，最後介紹 PyO3 的 abi3 功能。

打包工具

PyO3 生態系提供兩個主要選項來抽象化 Python 擴充模組的開發流程：

  • maturin is a command-line tool to build, package and upload Python modules. It makes opinionated choices about project layout meaning it needs very little configuration. This makes it a great choice for users who are building a Python extension from scratch and don’t need flexibility.
  • setuptools-rust is an add-on for setuptools which adds extra keyword arguments to the setup.py configuration file. It requires more configuration than maturin, however this gives additional flexibility for users adding Rust to an existing Python package that can’t satisfy maturin’s constraints.

Consult each project’s documentation for full details on how to get started using them and how to upload wheels to PyPI. It should be noted that while maturin is able to build manylinux-compliant wheels out-of-the-box, setuptools-rust requires a bit more effort, relying on Docker for this purpose.

There are also maturin-starter and setuptools-rust-starter examples in the PyO3 repository.

手動建置

To build a PyO3-based Python extension manually, start by running cargo build as normal in a library project with the cdylib crate type while the PYO3_BUILD_EXTENSION_MODULE environment variable is set.

Once built, symlink (or copy) and rename the shared library from Cargo’s target/ directory to your desired output directory:

  • on macOS, rename libyour_module.dylib to your_module.so.
  • on Windows, rename libyour_module.dll to your_module.pyd.
  • on Linux, rename libyour_module.so to your_module.so.

You can then open a Python shell in the output directory and you’ll be able to run import your_module.

If you’re packaging your library for redistribution, you should indicate the Python interpreter your library is compiled for by including the platform tag in its name. This prevents incompatible interpreters from trying to import your library. If you’re compiling for PyPy you must include the platform tag, or PyPy will ignore the module.

Bazel builds

To use PyO3 with bazel one needs to manually configure PyO3, PyO3-ffi and PyO3-macros. In particular, one needs to make sure that it is compiled with the right python flags for the version you intend to use. For example see:

  1. github.com/abrisco/rules_pyo3 – General rules for building extension modules.
  2. github.com/OliverFM/pytorch_with_gazelle – for a minimal example of a repo that can use PyO3, PyTorch and Gazelle to generate python Build files.
  3. github.com/TheButlah/rules_pyo3 – is somewhat dated.

平臺標記

Rather than using just the .so or .pyd extension suggested above (depending on OS), you can prefix the shared library extension with a platform tag to indicate the interpreter it is compatible with. You can query your interpreter’s platform tag from the sysconfig module. Some example outputs of this are seen below:

# CPython 3.10 on macOS
.cpython-310-darwin.so

# PyPy 7.3 (Python 3.9) on Linux
$ python -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))'
.pypy39-pp73-x86_64-linux-gnu.so

So, for example, a valid module library name on CPython 3.10 for macOS is your_module.cpython-310-darwin.so, and its equivalent when compiled for PyPy 7.3 on Linux would be your_module.pypy38-pp73-x86_64-linux-gnu.so.

See PEP 3149 for more background on platform tags.

macOS

On macOS, because the PYO3_BUILD_EXTENSION_MODULE environment variable disables linking to libpython (see the next section), some additional linker arguments need to be set. maturin and setuptools-rust both pass these arguments for PyO3 automatically, but projects using manual builds will need to set these directly in order to support macOS.

The easiest way to set the correct linker arguments is to add a build.rs with the following content:

fn main() {
    pyo3_build_config::add_extension_module_link_args();
}

Remember to also add pyo3-build-config to the build-dependencies section in Cargo.toml.

An alternative to using pyo3-build-config is add the following to a cargo configuration file (e.g. .cargo/config.toml):

[target.x86_64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

[target.aarch64-apple-darwin]
rustflags = [
  "-C", "link-arg=-undefined",
  "-C", "link-arg=dynamic_lookup",
]

Using the MacOS system python3 (/usr/bin/python3, as opposed to python installed via homebrew, pyenv, nix, etc.) may result in runtime errors such as Library not loaded: @rpath/Python3.framework/Versions/3.8/Python3.

The easiest way to set the correct linker arguments is to add a build.rs with the following content:

fn main() {
    pyo3_build_config::add_python_framework_link_args();
}

Alternatively it can be resolved with another addition to .cargo/config.toml:

[build]
rustflags = [
  "-C", "link-args=-Wl,-rpath,/Library/Developer/CommandLineTools/Library/Frameworks",
]

For more discussion on and workarounds for MacOS linking problems see this issue.

Finally, don’t forget that on MacOS the extension-module feature will cause cargo test to fail without the --no-default-features flag (see the FAQ).

The PYO3_BUILD_EXTENSION_MODULE environment variable

By default PyO3 links to libpython. This makes binaries, tests, and examples “just work”. However, Python extensions on Unix must not link to libpython for manylinux compliance.

The downside of not linking to libpython is that binaries, tests, and examples (which usually embed Python) will fail to build. As a result, PyO3 uses an envionment variable PYO3_BUILD_EXTENSION_MODULE to disable linking to libpython. This should only be set when building a library for distribution. maturin >= 1.9.4 and setuptools-rust >= 1.12 will set this for you automatically.

[!NOTE] Historically PyO3 used an extension-module feature to perform the same function now done by the PYO3_BUILD_EXTENSION_MODULE env var. This feature caused linking to be disabled for all compile targets, including Rust tests and benchmarks.

Projects are encouraged to migrate off the feature, as it caused major development pain due to the lack of linking.

Py_LIMITED_API/abi3

By default, Python extension modules can only be used with the same Python version they were compiled against. For example, an extension module built for Python 3.5 can’t be imported in Python 3.8. PEP 384 introduced the idea of the limited Python API, which would have a stable ABI enabling extension modules built with it to be used against multiple Python versions. This is also known as abi3.

The advantage of building extension modules using the limited Python API is that package vendors only need to build and distribute a single copy (for each OS / architecture), and users can install it on all Python versions from the minimum version and up. The downside of this is that PyO3 can’t use optimizations which rely on being compiled against a known exact Python version. It’s up to you to decide whether this matters for your extension module. It’s also possible to design your extension module such that you can distribute abi3 wheels but allow users compiling from source to benefit from additional optimizations - see the support for multiple python versions section of this guide, in particular the #[cfg(Py_LIMITED_API)] flag.

There are three steps involved in making use of abi3 when building Python packages as wheels:

  1. Enable the abi3 feature in pyo3. This ensures pyo3 only calls Python C-API functions which are part of the stable API, and on Windows also ensures that the project links against the correct shared object (no special behavior is required on other platforms):

    [dependencies]
pyo3 = { git = "https://github.com/pyo3/pyo3", features = ["abi3"] }

  2. Ensure that the built shared objects are correctly marked as abi3. This is accomplished by telling your build system that you’re using the limited API. maturin >= 0.9.0 and setuptools-rust >= 0.11.4 support abi3 wheels.

    See the corresponding PRs for more.

  3. Ensure that the .whl is correctly marked as abi3. For projects using setuptools, this is accomplished by passing --py-limited-api=cp3x (where x is the minimum Python version supported by the wheel, e.g. --py-limited-api=cp35 for Python 3.5) to setup.py bdist_wheel.

Minimum Python version for abi3

Because a single abi3 wheel can be used with many different Python versions, PyO3 has feature flags abi3-py37, abi3-py38, abi3-py39 etc. to set the minimum required Python version for your abi3 wheel. For example, if you set the abi3-py37 feature, your extension wheel can be used on all Python 3 versions from Python 3.7 and up. maturin and setuptools-rust will give the wheel a name like my-extension-1.0-cp37-abi3-manylinux2020_x86_64.whl.

As your extension module may be run with multiple different Python versions you may occasionally find you need to check the Python version at runtime to customize behavior. See the relevant section of this guide on supporting multiple Python versions at runtime.

PyO3 is only able to link your extension module to abi3 version up to and including your host Python version. E.g., if you set abi3-py38 and try to compile the crate with a host of Python 3.7, the build will fail.

[!NOTE] If you set more that one of these abi3 version feature flags the lowest version always wins. For example, with both abi3-py37 and abi3-py38 set, PyO3 would build a wheel which supports Python 3.7 and up.

Building abi3 extensions without a Python interpreter

As an advanced feature, you can build PyO3 wheel without calling Python interpreter with the environment variable PYO3_NO_PYTHON set. Also, if the build host Python interpreter is not found or is too old or otherwise unusable, PyO3 will still attempt to compile abi3 extension modules after displaying a warning message.

Missing features

Due to limitations in the Python API, there are a few pyo3 features that do not work when compiling for abi3. These are:

  • #[pyo3(text_signature = "...")] does not work on classes until Python 3.10 or greater.
  • The dict and weakref options on classes are not supported until Python 3.9 or greater.
  • The buffer API is not supported until Python 3.11 or greater.
  • Subclassing native types (e.g. PyException) is not supported until Python 3.12 or greater.
  • Optimizations which rely on knowledge of the exact Python version compiled against.

Embedding Python in Rust

If you want to embed the Python interpreter inside a Rust program, there are two modes in which this can be done: dynamically and statically. We’ll cover each of these modes in the following sections. Each of them affect how you must distribute your program. Instead of learning how to do this yourself, you might want to consider using a project like PyOxidizer to ship your application and all of its dependencies in a single file.

PyO3 automatically switches between the two linking modes depending on whether the Python distribution you have configured PyO3 to use (see above) contains a shared library or a static library. The static library is most often seen in Python distributions compiled from source without the --enable-shared configuration option.

Dynamically embedding the Python interpreter

Embedding the Python interpreter dynamically is much easier than doing so statically. This is done by linking your program against a Python shared library (such as libpython.3.9.so on UNIX, or python39.dll on Windows). The implementation of the Python interpreter resides inside the shared library. This means that when the OS runs your Rust program it also needs to be able to find the Python shared library.

This mode of embedding works well for Rust tests which need access to the Python interpreter. It is also great for Rust software which is installed inside a Python virtualenv, because the virtualenv sets up appropriate environment variables to locate the correct Python shared library.

For distributing your program to non-technical users, you will have to consider including the Python shared library in your distribution as well as setting up wrapper scripts to set the right environment variables (such as LD_LIBRARY_PATH on UNIX, or PATH on Windows).

Note that PyPy cannot be embedded in Rust (or any other software). Support for this is tracked on the PyPy issue tracker.

Statically embedding the Python interpreter

Embedding the Python interpreter statically means including the contents of a Python static library directly inside your Rust binary. This means that to distribute your program you only need to ship your binary file: it contains the Python interpreter inside the binary!

On Windows static linking is almost never done, so Python distributions don’t usually include a static library. The information below applies only to UNIX.

The Python static library is usually called libpython.a.

Static linking has a lot of complications, listed below. For these reasons PyO3 does not yet have first-class support for this embedding mode. See issue 416 on PyO3’s GitHub for more information and to discuss any issues you encounter.

The auto-initialize feature is deliberately disabled when embedding the interpreter statically because this is often unintentionally done by new users to PyO3 running test programs. Trying out PyO3 is much easier using dynamic embedding.

The known complications are:

  • To import compiled extension modules (such as other Rust extension modules, or those written in C), your binary must have the correct linker flags set during compilation to export the original contents of libpython.a so that extensions can use them (e.g. -Wl,--export-dynamic).

  • The C compiler and flags which were used to create libpython.a must be compatible with your Rust compiler and flags, else you will experience compilation failures.

    Significantly different compiler versions may see errors like this:

    lto1: fatal error: bytecode stream in file 'rust-numpy/target/release/deps/libpyo3-6a7fb2ed970dbf26.rlib' generated with LTO version 6.0 instead of the expected 6.2

    Mismatching flags may lead to errors like this:

    /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpython3.9.a(zlibmodule.o): relocation R_X86_64_32 against `.data' can not be used when making a PIE object; recompile with -fPIE

If you encounter these or other complications when linking the interpreter statically, discuss them on issue 416 on PyO3’s GitHub. It is hoped that eventually that discussion will contain enough information and solutions that PyO3 can offer first-class support for static embedding.

Import your module when embedding the Python interpreter

When you run your Rust binary with an embedded interpreter, any #[pymodule] created modules won’t be accessible to import unless added to a table called PyImport_Inittab before the embedded interpreter is initialized. This will cause Python statements in your embedded interpreter such as import your_new_module to fail. You can call the macro append_to_inittab with your module before initializing the Python interpreter to add the module function into that table. (The Python interpreter will be initialized by calling Python::initialize, with_embedded_python_interpreter, or Python::attach with the auto-initialize feature enabled.)

Cross Compiling

Thanks to Rust’s great cross-compilation support, cross-compiling using PyO3 is relatively straightforward. To get started, you’ll need a few pieces of software:

  • A toolchain for your target.
  • The appropriate options in your Cargo .config for the platform you’re targeting and the toolchain you are using.
  • A Python interpreter that’s already been compiled for your target (optional when building “abi3” extension modules).
  • A Python interpreter that is built for your host and available through the PATH or setting the PYO3_PYTHON variable (optional when building “abi3” extension modules).

After you’ve obtained the above, you can build a cross-compiled PyO3 module by using Cargo’s --target flag. PyO3’s build script will detect that you are attempting a cross-compile based on your host machine and the desired target.

When cross-compiling, PyO3’s build script cannot execute the target Python interpreter to query the configuration, so there are a few additional environment variables you may need to set:

  • PYO3_CROSS: If present this variable forces PyO3 to configure as a cross-compilation.
  • PYO3_CROSS_LIB_DIR: This variable can be set to the directory containing the target’s libpython DSO and the associated _sysconfigdata*.py file for Unix-like targets. This variable is only needed when the output binary must link to libpython explicitly (e.g. when targeting Android or embedding a Python interpreter), or when it is absolutely required to get the interpreter configuration from _sysconfigdata*.py. On Windows, this variable is not needed because PyO3 uses raw-dylib linking.
  • PYO3_CROSS_PYTHON_VERSION: Major and minor version (e.g. 3.9) of the target Python installation. This variable is only needed if PyO3 cannot determine the version to target from abi3-py3* features, or if PYO3_CROSS_LIB_DIR is not set, or if there are multiple versions of Python present in PYO3_CROSS_LIB_DIR.
  • PYO3_CROSS_PYTHON_IMPLEMENTATION: Python implementation name (“CPython” or “PyPy”) of the target Python installation. CPython is assumed by default when this variable is not set, unless PYO3_CROSS_LIB_DIR is set for a Unix-like target and PyO3 can get the interpreter configuration from _sysconfigdata*.py.

An example might look like the following (assuming your target’s sysroot is at /home/pyo3/cross/sysroot and that your target is armv7):

export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib"

cargo build --target armv7-unknown-linux-gnueabihf

If there are multiple python versions at the cross lib directory and you cannot set a more precise location to include both the libpython DSO and _sysconfigdata*.py files, you can set the required version:

export PYO3_CROSS_PYTHON_VERSION=3.8
export PYO3_CROSS_LIB_DIR="/home/pyo3/cross/sysroot/usr/lib"

cargo build --target armv7-unknown-linux-gnueabihf

Or another example building for Windows (no PYO3_CROSS_LIB_DIR needed thanks to raw-dylib):

export PYO3_CROSS_PYTHON_VERSION=3.9

cargo build --target x86_64-pc-windows-gnu

Any of the abi3-py3* features can be enabled instead of setting PYO3_CROSS_PYTHON_VERSION in the above examples.

PYO3_CROSS_LIB_DIR can often be omitted when cross compiling extension modules for Unix, macOS, and Windows targets.

The following resources may also be useful for cross-compiling:

支援多種 Python 版本

PyO3 supports all actively-supported Python 3 and PyPy versions. As much as possible, this is done internally to PyO3 so that your crate’s code does not need to adapt to the differences between each version. However, as Python features grow and change between versions, PyO3 cannot offer a completely identical API for every Python version. This may require you to add conditional compilation to your crate or runtime checks for the Python version.

This section of the guide first introduces the pyo3-build-config crate, which you can use as a build-dependency to add additional #[cfg] flags which allow you to support multiple Python versions at compile-time.

Second, we’ll show how to check the Python version at runtime. This can be useful when building for multiple versions with the abi3 feature, where the Python API compiled against is not always the same as the one in use.

Conditional compilation for different Python versions

The pyo3-build-config exposes multiple #[cfg] flags which can be used to conditionally compile code for a given Python version. PyO3 itself depends on this crate, so by using it you can be sure that you are configured correctly for the Python version PyO3 is building against.

This allows us to write code like the following

#[cfg(Py_3_7)]
fn function_only_supported_on_python_3_7_and_up() {}

#[cfg(not(Py_3_8))]
fn function_only_supported_before_python_3_8() {}

#[cfg(not(Py_LIMITED_API))]
fn function_incompatible_with_abi3_feature() {}

The following sections first show how to add these #[cfg] flags to your build process, and then cover some common patterns flags in a little more detail.

To see a full reference of all the #[cfg] flags provided, see the pyo3-build-cfg docs.

使用 pyo3-build-config

You can use the #[cfg] flags in just two steps:

  1. Add pyo3-build-config with the resolve-config feature enabled to your crate’s build dependencies in Cargo.toml:

    [build-dependencies]
pyo3-build-config = { git = "https://github.com/pyo3/pyo3", features = ["resolve-config"] }

  2. Add a build.rs file to your crate with the following contents:

    fn main() {
    // If you have an existing build.rs file, just add this line to it.
    pyo3_build_config::use_pyo3_cfgs();
}

After these steps you are ready to annotate your code!

pyo3-build-cfg 旗標的常見用法

The #[cfg] flags added by pyo3-build-cfg can be combined with all of Rust’s logic in the #[cfg] attribute to create very precise conditional code generation. The following are some common patterns implemented using these flags:

#[cfg(Py_3_7)]

This #[cfg] marks code that will only be present on Python 3.7 and upwards. There are similar options Py_3_8, Py_3_9, Py_3_10 and so on for each minor version.

#[cfg(not(Py_3_7))]

This #[cfg] marks code that will only be present on Python versions before (but not including) Python 3.7.

#[cfg(not(Py_LIMITED_API))]

This #[cfg] marks code that is only available when building for the unlimited Python API (i.e. PyO3’s abi3 feature is not enabled). This might be useful if you want to ship your extension module as an abi3 wheel and also allow users to compile it from source to make use of optimizations only possible with the unlimited API.

#[cfg(any(Py_3_9, not(Py_LIMITED_API)))]

This #[cfg] marks code which is available when running Python 3.9 or newer, or when using the unlimited API with an older Python version. Patterns like this are commonly seen on Python APIs which were added to the limited Python API in a specific minor version.

#[cfg(PyPy)]

This #[cfg] marks code which is running on PyPy.

Checking the Python version at runtime

When building with PyO3’s abi3 feature, your extension module will be compiled against a specific minimum version of Python, but may be running on newer Python versions.

For example with PyO3’s abi3-py38 feature, your extension will be compiled as if it were for Python 3.8. If you were using pyo3-build-config, #[cfg(Py_3_8)] would be present. Your user could freely install and run your abi3 extension on Python 3.9.

There’s no way to detect your user doing that at compile time, so instead you need to fall back to runtime checks.

PyO3 provides the APIs Python::version() and Python::version_info() to query the running Python version. This allows you to do the following, for example:

use pyo3::Python;

Python::attach(|py| {
    // PyO3 supports Python 3.7 and up.
    assert!(py.version_info() >= (3, 7));
    assert!(py.version_info() >= (3, 7, 0));
});

PyO3 生態系

本指南此部分介紹主 PyO3 專案之外的軟體箱，它們提供你可能覺得有用的額外功能。

Because these projects evolve independently of the PyO3 repository the content of these articles may fall out of date over time; please file issues on the PyO3 GitHub to alert maintainers when this is the case.

日誌

理想情況是應用程式中的 Python 與 Rust 部分都使用相同設定，並將日誌輸出到同一位置。

本節簡要說明如何連結兩種語言的日誌生態系。對於 Python 擴充模組，建議的方式是使用 pyo3-log 軟體箱設定 Rust 的 logger，將日誌訊息送到 Python。若你想反向將 Python 日誌送到 Rust，請見本指南最後的說明。

使用 pyo3-log 將 Rust 日誌訊息送到 Python

pyo3-log 軟體箱允許將 Rust 端訊息傳送到 Python 的 logging 系統。這主要適用於為 Python 程式撰寫原生擴充。

使用 pyo3_log::init 以預設設定安裝 logger。也可以調整其設定（多半用於效能調校）。

#[pyo3::pymodule]
mod my_module {
    use log::info;
    use pyo3::prelude::*;

    #[pyfunction]
    fn log_something() {
        // 這會使用安裝在 `my_module` 的 logger，將 `info`
        // 訊息送到 Python 的 logging 系統。
        info!("Something!");
    }

    #[pymodule_init]
    fn init(m: &Bound<'_, PyModule>) -> PyResult<()> {
        // 安裝 Rust -> Python logger 的好位置。
        pyo3_log::init();
    }
}

接著由 Python 端負責將訊息實際輸出到某處。

import logging
import my_module

FORMAT = '%(levelname)s %(name)s %(asctime)-15s %(filename)s:%(lineno)d %(message)s'
logging.basicConfig(format=FORMAT)
logging.getLogger().setLevel(logging.INFO)
my_module.log_something()

在呼叫任何可能產生日誌的 Rust 函式之前，務必先初始化 Python 的 logger。若無法滿足此限制，仍可用替代方式處理，請閱讀快取相關文件。

Python 到 Rust 的方向

若要讓 Rust 處理 Python 日誌，只需註冊一個 Rust 函式來處理由 Python 核心 logging 模組送出的日誌。

這已由 pyo3-pylogger 軟體箱實作。

use log::{info, warn};
use pyo3::prelude::*;

fn main() -> PyResult<()> {
    // 在 python logger 中註冊主機端處理器，並提供 logger target
    // 這裡的名稱請設定為適合你的應用程式
    pyo3_pylogger::register("example_application_py_logger");

    // 初始化 logger
    env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("trace")).init();

    // 從 Rust 紀錄一些訊息。
    info!("Just some normal information!");
    warn!("Something spooky happened!");

    // 從 Python 紀錄一些訊息
    Python::attach(|py| {
        py.run(
            "
import logging
logging.error('Something bad happened')
",
            None,
            None,
        )
    })
}

追蹤

為了效能而撰寫擴充模組的 Python 專案，可能會想使用 Rust 的 tracing 生態系 來了解擴充模組的效能表現。

本節介紹幾個提供此能力的軟體箱。它們建立在 tracing_subscriber 之上，並需要同時修改 Python 與 Rust 程式碼來整合。請注意每個擴充模組都必須設定自己的 tracing 整合；一個擴充模組無法看到另一個模組的 tracing 資料。

pyo3-tracing-subscriber（文件）

pyo3-tracing-subscriber 提供 Python 專案設定 tracing_subscriber 的方式，並暴露幾個 tracing_subscriber layer：

  • tracing_subscriber::fmt：輸出易讀格式到檔案或 stdout
  • opentelemetry-stdout：輸出 OTLP 到檔案或 stdout
  • opentelemetry-otlp：輸出 OTLP 到 OTLP 端點

擴充模組必須呼叫 pyo3_tracing_subscriber::add_submodule 來導出設定與初始化 tracing 所需的 Python 類別。

在 Python 端，使用 Tracing 內容管理器來初始化 tracing，並在其區塊內執行 Rust 程式碼。Tracing 需要一個描述使用哪些 layer 的 GlobalTracingConfig 實例。

範例程式碼請見 crates.io 的 README。

pyo3-python-tracing-subscriber（文件）

名稱相近的 pyo3-python-tracing-subscriber 在 Rust 中實作一個 shim，將 tracing 資料轉送到由 Python 定義並傳入的 Layer 實作。

擴充模組可用多種方式整合 pyo3-python-tracing-subscriber，其中一個簡單做法可能如下：

#[tracing::instrument]
#[pyfunction]
fn fibonacci(index: usize, use_memoized: bool) -> PyResult<usize> {
    // ...
}

#[pyfunction]
pub fn initialize_tracing(py_impl: Bound<'_, PyAny>) {
    tracing_subscriber::registry()
        .with(pyo3_python_tracing_subscriber::PythonCallbackLayerBridge::new(py_impl))
        .init();
}

擴充模組必須提供某種方式，讓 Python 傳入一個或多個實作了 Layer 介面 的 Python 物件。接著應使用這些物件建立 pyo3_python_tracing_subscriber::PythonCallbackLayerBridge 實例，並如上所示初始化 tracing_subscriber。

Python 物件實作的是改良版的 Layer 介面：

  • on_new_span() 可回傳狀態，該狀態會儲存在 Rust span 中
  • 其他回呼會以額外的位置參數取得該狀態

一個假的 Layer 實作可能如下：

import rust_extension

class MyPythonLayer:
    def __init__(self):
        pass

    # `on_new_span` 可以回傳某些狀態
    def on_new_span(self, span_attrs: str, span_id: str) -> int:
        print(f"[on_new_span]: {span_attrs} | {span_id}")
        return random.randint(1, 1000)

    # 來自 `on_new_span` 的狀態會傳回其他特徵方法
    def on_event(self, event: str, state: int):
        print(f"[on_event]: {event} | {state}")

    def on_close(self, span_id: str, state: int):
        print(f"[on_close]: {span_id} | {state}")

    def on_record(self, span_id: str, values: str, state: int):
        print(f"[on_record]: {span_id} | {values} | {state}")

def main():
    rust_extension.initialize_tracing(MyPythonLayer())

    print("10th fibonacci number: ", rust_extension.fibonacci(10, True))

pyo3-python-tracing-subscriber 提供可運作的範例，展示 Rust 與 Python 端的整合方式。

使用 async 和 await

async/await support is currently being integrated in PyO3. See the dedicated documentation

若你使用的 Python 函式庫包含 async 函式，或希望為 async Rust 函式庫提供 Python 綁定，pyo3-async-runtimes 很可能就是你需要的工具。它提供 Python 與 Rust 之間 async 函式的轉換，並以對 tokio 與 async-std 等常見 Rust runtime 的一級支援為設計目標。此外，所有 async Python 程式碼都在預設的 asyncio 事件迴圈上執行，因此 pyo3-async-runtimes 應可與既有 Python 函式庫順利運作。

補充資訊

  • 在 pyo3-async-runtimes 中管理事件迴圈參照可能較棘手。請參考 API 文件中的 Event Loop References，以更了解此函式庫如何管理事件迴圈參照。
  • 測試 pyo3-async-runtimes 函式庫與應用程式需要自訂測試框架，因為 Python 需要掌控主執行緒。測試指南請見 testing 模組的 API 文件。

常見問題與疑難排解

很抱歉你在使用 PyO3 時遇到問題。如果在下列清單中找不到答案，也可以到 GitHub Discussions 或 Discord 尋求協助。

使用 PyO3 搭配 std::sync::OnceLock、std::sync::LazyLock、lazy_static 與 once_cell 時遇到死鎖

OnceLock、LazyLock 以及其第三方前身會透過阻塞來確保只有一個執行緒進行初始化。由於 Python 直譯器可能引入額外鎖（Python 的 GIL 與 GC 都可能要求其他執行緒暫停），因此可能以以下方式造成死鎖：

  1. 已附著到 Python 直譯器的執行緒（執行緒 A）開始初始化 OnceLock 值。
  2. 初始化程式碼呼叫了會暫時與直譯器分離的 Python API，例如 Python::import。
  3. 另一個執行緒（執行緒 B）附著到 Python 直譯器並嘗試存取相同的 OnceLock 值。
  4. 執行緒 B 被阻塞，因為它在等待 OnceLock 的初始化鎖釋放。
  5. 在非自由執行緒的 Python 中，執行緒 A 現在也會被阻塞，因為它等待重新附著到直譯器（必須取得執行緒 B 仍持有的 GIL）。
  6. 死鎖。

PyO3 提供 PyOnceLock 結構，基於這些型別實作單次初始化 API，可避免死鎖。你也可以使用 OnceExt 與 OnceLockExt 擴充 trait，透過為標準函式庫型別提供新方法來避免與 Python 直譯器死鎖的風險。當你遇到上述死鎖時，可用它們取代其他選擇。更多細節與使用範例請參考 PyOnceLock 與 OnceExt 文件。

無法執行 cargo test；或在 Cargo workspace 中無法建置：遇到連結器錯誤如「Symbol not found」或「Undefined reference to _PyExc_SystemError」

已棄用的 extension-module feature 會停用與 libpython 的連結，導致需要載入 libpython 符號才能執行的可執行檔與測試失敗。

移除 extension-module feature，並升級到 maturin >= 1.9.4 或 setuptools-rust 1.12。

若手動建置，請見 PYO3_BUILD_EXTENSION_MODULE 環境變數。

無法執行 cargo test：tests/ 目錄中的測試找不到我的 crate

Rust 書籍建議將整合測試放在 tests/ 目錄中。

對於在 Cargo.toml 中將 crate-type 設為 "cdylib" 的 PyO3 專案，編譯器無法找到你的 crate，並會顯示 E0432 或 E0463 等錯誤：

error[E0432]: unresolved import `my_crate`
 --> tests/test_my_crate.rs:1:5
  |
1 | use my_crate;
  |     ^^^^^^^^^^^^ no external crate `my_crate`

最佳解法是讓 crate 型別同時包含 rlib 與 cdylib：

# Cargo.toml
[lib]
crate-type = ["cdylib", "rlib"]

在 Rust 程式碼執行期間按 Ctrl-C 沒有效果

這是因為 Ctrl-C 會送出 SIGINT 訊號，呼叫端 Python 程序只會設定一個旗標以便稍後處理。當 Python 呼叫的 Rust 程式碼執行中時，不會檢查此旗標，直到控制權回到 Python 直譯器才會處理。

你可以呼叫 Python::check_signals 讓 Python 直譯器有機會正確處理訊號。若你的 Rust 函式會長時間執行，建議定期呼叫此函式，讓使用者能取消。

#[pyo3(get)] 會複製我的欄位

你可能有如下的巢狀結構：

use pyo3::prelude::*;
#[pyclass(from_py_object)]
#[derive(Clone)]
struct Inner {/* fields omitted */}

#[pyclass]
struct Outer {
    #[pyo3(get)]
    inner: Inner,
}

#[pymethods]
impl Outer {
    #[new]
    fn __new__() -> Self {
        Self { inner: Inner {} }
    }
}

當 Python 程式碼存取 Outer 的欄位時，PyO3 會在每次存取時回傳新的物件（注意位址不同）：

outer = Outer()

a = outer.inner
b = outer.inner

assert a is b, f"a: {a}\nb: {b}"

AssertionError: a: <builtins.Inner object at 0x00000238FFB9C7B0>
b: <builtins.Inner object at 0x00000238FFB9C830>

若欄位是可變的，這會特別令人困惑，因為取得欄位後再修改並不會持久化——下次存取只會拿到原始物件的新複本。不幸的是，Python 與 Rust 對所有權的理解不同——如果 PyO3 將（可能是暫時的）Rust 物件參照交給 Python，Python 可能會無限期保留該參照。因此回傳 Rust 物件需要進行複製。

If you don’t want that cloning to happen, a workaround is to allocate the field on the Python heap and store a reference to that, by using Py<...>:

use pyo3::prelude::*;
#[pyclass]
struct Inner {/* fields omitted */}

#[pyclass]
struct Outer {
    inner: Py<Inner>,
}

#[pymethods]
impl Outer {
    #[new]
    fn __new__(py: Python<'_>) -> PyResult<Self> {
        Ok(Self {
            inner: Py::new(py, Inner {})?,
        })
    }

    #[getter]
    fn inner(&self, py: Python<'_>) -> Py<Inner> {
        self.inner.clone_ref(py)
    }
}

This time a and b are the same object:

outer = Outer()

a = outer.inner
b = outer.inner

assert a is b, f"a: {a}\nb: {b}"
print(f"a: {a}\nb: {b}")

a: <builtins.Inner object at 0x0000020044FCC670>
b: <builtins.Inner object at 0x0000020044FCC670>

The downside to this approach is that any Rust code working on the Outer struct potentially has to attach to the Python interpreter to do anything with the inner field. (If Inner is #[pyclass(frozen)] and implements Sync, then Py::get may be used to access the Inner contents from Py<Inner> without needing to attach to the interpreter.)

I want to use the pyo3 crate re-exported from dependency but the proc-macros fail

All PyO3 proc-macros (#[pyclass], #[pyfunction], #[derive(FromPyObject)] and so on) expect the pyo3 crate to be available under that name in your crate root, which is the normal situation when pyo3 is a direct dependency of your crate.

However, when the dependency is renamed, or your crate only indirectly depends on pyo3, you need to let the macro code know where to find the crate. This is done with the crate attribute:

use pyo3::prelude::*;
pub extern crate pyo3;
mod reexported { pub use ::pyo3; }
#[allow(dead_code)]
#[pyclass]
#[pyo3(crate = "reexported::pyo3")]
struct MyClass;

I’m trying to call Python from Rust but I get STATUS_DLL_NOT_FOUND or STATUS_ENTRYPOINT_NOT_FOUND

This happens on Windows when linking to the python DLL fails or the wrong one is linked. The Python DLL on Windows will usually be called something like:

  • python3X.dll for Python 3.X, e.g. python310.dll for Python 3.10
  • python3.dll when using PyO3’s abi3 feature

The DLL needs to be locatable using the Windows DLL search order. Some ways to achieve this are:

  • Put the Python DLL in the same folder as your build artifacts
  • Add the directory containing the Python DLL to your PATH environment variable, for example C:\Users\<You>\AppData\Local\Programs\Python\Python310
  • If this happens when you are distributing your program, consider using PyOxidizer to package it with your binary.

If the wrong DLL is linked it is possible that this happened because another program added itself and its own Python DLLs to PATH. Rearrange your PATH variables to give the correct DLL priority.

[!NOTE] Changes to PATH (or any other environment variable) are not visible to existing shells. Restart it for changes to take effect.

For advanced troubleshooting, Dependency Walker can be used to diagnose linking errors.

從舊版 PyO3 遷移

本指南可協助你跨越 PyO3 版本間的破壞性變更來升級程式碼。完整變更清單請見 CHANGELOG。

從 0.27.* 到 0.28

預設支援自由執行緒的 Python

Click to expand

當 PyO3 0.23 加入對自由執行緒 Python 的支援時，模組必須透過標註 #[pymodule(gil_used = false)] 來選擇加入這項功能。

隨著支援逐漸成熟，且 PyO3 的 API 也演進到不再依賴 GIL，現在正是切換預設行為的好時機。模組現在會自動允許在自由執行緒的 Python 上使用，除非以 #[pymodule(gil_used = true)] 明確表示需要 GIL。

對實作 Clone 的 #[pyclass] 型別，自動 FromPyObject 已棄用

Click to expand

#[pyclass] types which implement Clone used to also implement FromPyObject automatically. This behavior is being phased out and replaced by an explicit opt-in, which will allow better error messages and more user control. Affected types will by marked by a deprecation message.

To migrate use either

  • #[pyclass(from_py_object)] to keep the automatic derive, or
  • #[pyclass(skip_from_py_object)] to accept the new behavior.

之前：

#![allow(deprecated)]
use pyo3::prelude::*;
#[pyclass]
#[derive(Clone)]
struct PyClass {}

After:

use pyo3::prelude::*;
// If the automatic implementation of `FromPyObject` is desired, opt in:
#[pyclass(from_py_object)]
#[derive(Clone)]
struct PyClass {}

// or if the `FromPyObject` implementation is not needed:
#[pyclass(skip_from_py_object)]
#[derive(Clone)]
struct PyClassWithoutFromPyObject {}

The #[pyclass(skip_from_py_object)] option will eventually be deprecated and removed as it becomes the default behavior.

從原始指標建立 Py<T> 的建構子已棄用

Click to expand

建構子 Py::from_owned_ptr、Py::from_owned_ptr_or_opt、Py::from_owned_ptr_or_err（以及類似的「borrowed」變體）會對目標型別 T 執行未檢查的轉型。對於主要關注如何從原始指標正確建構 PyO3 安全智慧指標型別的 API 而言，這種未檢查轉型是容易出錯的陷阱。

Bound 上的等價建構子一律產生 Bound<PyAny>，並鼓勵之後的轉型明確選擇要做檢查或不檢查。應改用這些建構子。

之前：

#![allow(deprecated)]
use pyo3::prelude::*;
use pyo3::types::PyNone;
Python::attach(|py| {
let raw_ptr = py.None().into_ptr();

let _: Py<PyNone> = unsafe { Py::from_borrowed_ptr(py, raw_ptr) };
let _: Py<PyNone> = unsafe { Py::from_owned_ptr(py, raw_ptr) };
})

After:

use pyo3::prelude::*;
use pyo3::types::PyNone;
Python::attach(|py| {
let raw_ptr = py.None().into_ptr();

// Bound API 需要選擇做未檢查或檢查轉型。必要時可以使用 `.unbind()`
// 產生 `Py<T>` 值。
let _: Bound<'_, PyNone> = unsafe { Bound::from_borrowed_ptr(py, raw_ptr).cast_into_unchecked() };
let _: Bound<'_, PyNone> = unsafe { Bound::from_owned_ptr(py, raw_ptr).cast_into_unchecked() };
})

移除 From<Bound<'_, T> 與 From<Py<T>> for PyClassInitializer<T>

Click to expand

作為初始化程式碼重構的一部分，這些實作被移除，其功能改移到 #[new] 產生的程式碼中。作為小小的副作用，下列模式將不再被接受：

use pyo3::prelude::*;
Python::attach(|py| {
let existing_py: Py<PyAny> = py.None();
let obj_1 = Py::new(py, existing_py);

let existing_bound: Bound<'_, PyAny> = py.None().into_bound(py);
let obj_2 = Bound::new(py, existing_bound);
})

遷移時改用 clone 或 clone_ref：

use pyo3::prelude::*;
Python::attach(|py| {
let existing_py: Py<PyAny> = py.None();
let obj_1 = existing_py.clone_ref(py);

let existing_bound: Bound<'_, PyAny> = py.None().into_bound(py);
let obj_2 = existing_bound.clone();
})

Untyped buffer API moved to PyUntypedBuffer

Click to expand

PyBuffer<T> now is a typed wrapper around PyUntypedBuffer. Many methods such as PyBuffer::format have been moved to PyUntypedBuffer::format. PyBuffer<T> dereferences to PyUntypedBuffer, so method call syntax will continue to work as-is. Users may need to update references to the moved functions.

內部改為使用多階段初始化

Click to expand

PEP 489 為擴充模組引入「多階段初始化」，提供配置與清理每個模組狀態的方法。這是支援 Python「子直譯器」的重要一步，子直譯器會在自己的狀態副本上執行。

自 PyO3 0.28 起，#[pymodule] 巨集機制已重構為使用多階段初始化。建立與使用每模組狀態（以及支援子直譯器）的可能性留待未來版本。此變更不需要遷移，也預期不會造成破壞性影響。

不過它會影響初始化的順序，因此在本指南中特別註記。

從 0.26.* 到 0.27

FromPyObject 重構以提升彈性與效率

Click to expand

隨著 PyO3 0.23 移除 gil-ref API，現在可以完全區分 Python 的生命週期 'py 與輸入生命週期 'a。這讓你能借用輸入資料，而不必延長附著於直譯器的生命週期。

FromPyObject 現在多了一個描述輸入生命週期的 'a。extract 方法的參數型別從 &Bound<'py, PyAny> 改為 Borrowed<'a, 'py, PyAny>。這是因為 &'a Bound<'py, PyAny> 由於參照型別會隱含 'py: 'a 的限制。

這個新形式在 0.22 已以內部 FromPyObjectBound trait 部分實作，現在擴展到所有型別。

大多數實作只要補上省略的生命週期即可完成遷移。

此外，FromPyObject 新增了關聯型別 Error，用於轉換錯誤時的錯誤型別。遷移期間使用 PyErr 是很好的預設值，之後可改用自訂錯誤型別，以避免不必要的 Python 例外物件建立並提升型別安全。

之前：

impl<'py> FromPyObject<'py> for IpAddr {
    fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
        ...
    }
}

After

impl<'py> FromPyObject<'_, 'py> for IpAddr {
    type Error = PyErr;

    fn extract(obj: Borrowed<'_, 'py, PyAny>) -> Result<Self, Self::Error> {
        ...
        // since `Borrowed` derefs to `&Bound`, the body often
        // needs no changes, or adding an occasional `&`
    }
}

Occasionally, more steps are necessary. For generic types, the bounds need to be adjusted. The correct bound depends on how the type is used.

For simple wrapper types usually it’s possible to just forward the bound.

之前：

struct MyWrapper<T>(T);

impl<'py, T> FromPyObject<'py> for MyWrapper<T>
where
    T: FromPyObject<'py>
{
    fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
        ob.extract().map(MyWrapper)
    }
}

After:

use pyo3::prelude::*;
#[allow(dead_code)]
pub struct MyWrapper<T>(T);
impl<'a, 'py, T> FromPyObject<'a, 'py> for MyWrapper<T>
where
    T: FromPyObject<'a, 'py>
{
    type Error = T::Error;

    fn extract(obj: Borrowed<'a, 'py, PyAny>) -> Result<Self, Self::Error> {
        obj.extract().map(MyWrapper)
    }
}

Container types that need to create temporary Python references during extraction, for example extracting from a PyList, requires a stronger bound. For these the FromPyObjectOwned trait was introduced. It is automatically implemented for any type that implements FromPyObject and does not borrow from the input. It is intended to be used as a trait bound in these situations.

之前：

struct MyVec<T>(Vec<T>);
impl<'py, T> FromPyObject<'py> for Vec<T>
where
    T: FromPyObject<'py>,
{
    fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
        let mut v = MyVec(Vec::new());
        for item in obj.try_iter()? {
            v.0.push(item?.extract::<T>()?);
        }
        Ok(v)
    }
}

After:

use pyo3::prelude::*;
#[allow(dead_code)]
pub struct MyVec<T>(Vec<T>);
impl<'py, T> FromPyObject<'_, 'py> for MyVec<T>
where
    T: FromPyObjectOwned<'py> // 👈 can only extract owned values, because each `item` below
                              //    is a temporary short lived owned reference
{
    type Error = PyErr;

    fn extract(obj: Borrowed<'_, 'py, PyAny>) -> Result<Self, Self::Error> {
        let mut v = MyVec(Vec::new());
        for item in obj.try_iter()? {
            v.0.push(item?.extract::<T>().map_err(Into::into)?); // `map_err` is needed because `?` uses `From`, not `Into` 🙁
        }
        Ok(v)
    }
}

This is very similar to serdes Deserialize and DeserializeOwned traits, see the serde docs.

.downcast() and DowncastError replaced with .cast() and CastError

Click to expand

The .downcast() family of functions were only available on Bound<PyAny>. In corner cases (particularly related to .downcast_into()) this would require use of .as_any().downcast() or .into_any().downcast_into() chains. Additionally, DowncastError produced Python exception messages which are not very Pythonic due to use of Rust type names in the error messages.

The .cast() family of functions are available on all Bound and Borrowed smart pointers, whatever the type, and have error messages derived from the actual type at runtime. This produces a nicer experience for both PyO3 module authors and consumers.

To migrate, replace .downcast() with .cast() and DowncastError with CastError (and similar with .downcast_into() / DowncastIntoError etc).

CastError requires a Python type object (or other “classinfo” object compatible with isinstance()) as the second object, so in the rare case where DowncastError was manually constructed, small adjustments to code may apply.

PyTypeCheck is now an unsafe trait

Click to expand

Because PyTypeCheck is the trait used to guard the .cast() functions to treat Python objects as specific concrete types, the trait is unsafe to implement.

This should always have been the case, it was an unfortunate omission from its original implementation which is being corrected in this release.

from 0.25.* to 0.26

Rename of Python::with_gil, Python::allow_threads, and pyo3::prepare_freethreaded_python

Click to expand

The names for these APIs were created when the global interpreter lock (GIL) was mandatory. With the introduction of free-threading in Python 3.13 this is no longer the case, and the naming has no universal meaning anymore. For this reason, we chose to rename these to more modern terminology introduced in free-threading:

  • Python::with_gil is now called Python::attach, it attaches a Python thread-state to the current thread. In GIL enabled builds there can only be 1 thread attached to the interpreter, in free-threading there can be more.
  • Python::allow_threads is now called Python::detach, it detaches a previously attached thread-state.
  • pyo3::prepare_freethreaded_python is now called Python::initialize.

Deprecation of PyObject type alias

Click to expand

The type alias PyObject (aka Py<PyAny>) is often confused with the identically named FFI definition pyo3::ffi::PyObject. For this reason we are deprecating its usage. To migrate simply replace its usage by the target type Py<PyAny>.

Replacement of GILOnceCell with PyOnceLock

Click to expand

Similar to the above renaming of Python::with_gil and related APIs, the GILOnceCell type was designed for a Python interpreter which was limited by the GIL. Aside from its name, it allowed for the “once” initialization to race because the racing was mediated by the GIL and was extremely unlikely to manifest in practice.

With the introduction of free-threaded Python the racy initialization behavior is more likely to be problematic and so a new type PyOnceLock has been introduced which performs true single-initialization correctly while attached to the Python interpreter. It exposes the same API as GILOnceCell, so should be a drop-in replacement with the notable exception that if the racy initialization of GILOnceCell was inadvertently relied on (e.g. due to circular references) then the stronger once-ever guarantee of PyOnceLock may lead to deadlocking which requires refactoring.

之前：

use pyo3::prelude::*;
use pyo3::sync::GILOnceCell;
use pyo3::types::PyType;
fn main() -> PyResult<()> {
Python::attach(|py| {
static DECIMAL_TYPE: GILOnceCell<Py<PyType>> = GILOnceCell::new();
DECIMAL_TYPE.import(py, "decimal", "Decimal")?;
Ok(())
})
}

After:

use pyo3::prelude::*;
use pyo3::sync::PyOnceLock;
use pyo3::types::PyType;
fn main() -> PyResult<()> {
Python::attach(|py| {
static DECIMAL_TYPE: PyOnceLock<Py<PyType>> = PyOnceLock::new();
DECIMAL_TYPE.import(py, "decimal", "Decimal")?;
Ok(())
})
}

Deprecation of GILProtected

Click to expand

As another cleanup related to concurrency primitives designed for a Python constrained by the GIL, the GILProtected type is now deprecated. Prefer to use concurrency primitives which are compatible with free-threaded Python, such as std::sync::Mutex (in combination with PyO3’s MutexExt trait).

之前：

use pyo3::prelude::*;
fn main() {
#[cfg(not(Py_GIL_DISABLED))] {
use pyo3::sync::GILProtected;
use std::cell::RefCell;
Python::attach(|py| {
static NUMBERS: GILProtected<RefCell<Vec<i32>>> = GILProtected::new(RefCell::new(Vec::new()));
Python::attach(|py| {
    NUMBERS.get(py).borrow_mut().push(42);
});
})
}
}

After:

use pyo3::prelude::*;
use pyo3::sync::MutexExt;
use std::sync::Mutex;
fn main() {
Python::attach(|py| {
static NUMBERS: Mutex<Vec<i32>> = Mutex::new(Vec::new());
Python::attach(|py| {
    NUMBERS.lock_py_attached(py).expect("no poisoning").push(42);
});
})
}

PyMemoryError now maps to io::ErrorKind::OutOfMemory when converted to io::Error

Click to expand

Previously, converting a PyMemoryError into a Rust io::Error would result in an error with kind Other. Now, it produces an error with kind OutOfMemory. Similarly, converting an io::Error with kind OutOfMemory back into a Python error would previously yield a generic PyOSError. Now, it yields a PyMemoryError.

This change makes error conversions more precise and matches the semantics of out-of-memory errors between Python and Rust.

from 0.24.* to 0.25

AsPyPointer removal

Click to expand

The AsPyPointer trait is mostly a leftover from the now removed gil-refs API. The last remaining uses were the GC API, namely PyVisit::call, and identity comparison (PyAnyMethods::is and Py::is).

PyVisit::call has been updated to take T: Into<Option<&Py<T>>>, which allows for arguments of type &Py<T>, &Option<Py<T>> and Option<&Py<T>>. It is unlikely any changes are needed here to migrate.

PyAnyMethods::is/ Py::is has been updated to take T: AsRef<Py<PyAny>>>. Additionally AsRef<Py<PyAny>>> implementations were added for Py, Bound and Borrowed. Because of the existing AsRef<Bound<PyAny>> for Bound<T> implementation this may cause inference issues in non-generic code. This can be easily migrated by switching to as_any instead of as_ref for these calls.

from 0.23.* to 0.24

Click to expand There were no significant changes from 0.23 to 0.24 which required documenting in this guide.

from 0.22.* to 0.23

Click to expand

PyO3 0.23 is a significant rework of PyO3’s internals for two major improvements:

  • Support of Python 3.13’s new freethreaded build (aka “3.13t”)
  • Rework of to-Python conversions with a new IntoPyObject trait.

These changes are both substantial and reasonable efforts have been made to allow as much code as possible to continue to work as-is despite the changes. The impacts are likely to be seen in three places when upgrading:

The sections below discuss the rationale and details of each change in more depth.

Free-threaded Python Support

Click to expand

PyO3 0.23 introduces initial support for the new free-threaded build of CPython 3.13, aka “3.13t”.

Because this build allows multiple Python threads to operate simultaneously on underlying Rust data, the #[pyclass] macro now requires that types it operates on implement Sync.

Aside from the change to #[pyclass], most features of PyO3 work unchanged, as the changes have been to the internal data structures to make them thread-safe. An example of this is the GILOnceCell type, which used the GIL to synchronize single-initialization. It now uses internal locks to guarantee that only one write ever succeeds, however it allows for multiple racing runs of the initialization closure. It may be preferable to instead use std::sync::OnceLock in combination with the pyo3::sync::OnceLockExt trait which adds OnceLock::get_or_init_py_attached for single-initialization where the initialization closure is guaranteed only ever to run once and without deadlocking with the GIL.

Future PyO3 versions will likely add more traits and data structures to make working with free-threaded Python easier.

Some features are inaccessible on the free-threaded build:

  • The GILProtected type, which relied on the GIL to expose synchronized access to inner contents
  • PyList::get_item_unchecked, which cannot soundly be used due to races between time-of-check and time-of-use

If you make use of these features then you will need to account for the unavailability of the API in the free-threaded build. One way to handle it is via conditional compilation – extensions can use pyo3-build-config to get access to a #[cfg(Py_GIL_DISABLED)] guard.

See the guide section on free-threaded Python for more details about supporting free-threaded Python in your PyO3 extensions.

New IntoPyObject trait unifies to-Python conversions

Click to expand

PyO3 0.23 introduces a new IntoPyObject trait to convert Rust types into Python objects which replaces both IntoPy and ToPyObject. Notable features of this new trait include:

  • conversions can now return an error
  • it is designed to work efficiently for both T owned types and &T references
  • compared to IntoPy<T> the generic T moved into an associated type, so
    • there is now only one way to convert a given type
    • the output type is stronger typed and may return any Python type instead of just PyAny
  • byte collections are specialized to convert into PyBytes now, see below
  • () (unit) is now only specialized in return position of #[pyfunction] and #[pymethods] to return None, in normal usage it converts into an empty PyTuple

All PyO3 provided types as well as #[pyclass]es already implement IntoPyObject. Other types will need to adapt an implementation of IntoPyObject to stay compatible with the Python APIs. In many cases the new #[derive(IntoPyObject)] macro can be used instead of manual implementations.

Since IntoPyObject::into_pyobject may return either a Bound or Borrowed, you may find the BoundObject trait to be useful to write code that generically handles either type of smart pointer.

Together with the introduction of IntoPyObject the old conversion traits ToPyObject and IntoPy are deprecated and will be removed in a future PyO3 version.

IntoPyObject and IntoPyObjectRef derive macros

To implement the new trait you may use the new IntoPyObject and IntoPyObjectRef derive macros as below.

use pyo3::prelude::*;
#[derive(IntoPyObject, IntoPyObjectRef)]
struct Struct {
    count: usize,
    obj: Py<PyAny>,
}

The IntoPyObjectRef derive macro derives implementations for references (e.g. for &Struct in the example above), which is a replacement for the ToPyObject trait.

IntoPyObject manual implementation

之前：

use pyo3::prelude::*;
#[allow(dead_code)]
struct MyPyObjectWrapper(PyObject);

impl IntoPy<PyObject> for MyPyObjectWrapper {
    fn into_py(self, py: Python<'_>) -> PyObject {
        self.0
    }
}

impl ToPyObject for MyPyObjectWrapper {
    fn to_object(&self, py: Python<'_>) -> PyObject {
        self.0.clone_ref(py)
    }
}

After:

use pyo3::prelude::*;
#[allow(dead_code)]
struct MyPyObjectWrapper(PyObject);

impl<'py> IntoPyObject<'py> for MyPyObjectWrapper {
    type Target = PyAny; // the Python type
    type Output = Bound<'py, Self::Target>; // in most cases this will be `Bound`
    type Error = std::convert::Infallible;

    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        Ok(self.0.into_bound(py))
    }
}

// `ToPyObject` implementations should be converted to implementations on reference types
impl<'a, 'py> IntoPyObject<'py> for &'a MyPyObjectWrapper {
    type Target = PyAny;
    type Output = Borrowed<'a, 'py, Self::Target>; // `Borrowed` can be used to optimized reference counting
    type Error = std::convert::Infallible;

    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        Ok(self.0.bind_borrowed(py))
    }
}

To-Python conversions changed for byte collections (Vec<u8>, [u8; N] and SmallVec<[u8; N]>)

Click to expand

With the introduction of the IntoPyObject trait, PyO3’s macros now prefer IntoPyObject implementations over IntoPy<PyObject> when producing Python values. This applies to #[pyfunction] and #[pymethods] return values and also fields accessed via #[pyo3(get)].

This change has an effect on functions and methods returning byte collections like

  • Vec<u8>
  • [u8; N]
  • SmallVec<[u8; N]>

In their new IntoPyObject implementation these will now turn into PyBytes rather than a PyList. All other Ts are unaffected and still convert into a PyList.

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyfunction]
fn foo() -> Vec<u8> { // would previously turn into a `PyList`, now `PyBytes`
    vec![0, 1, 2, 3]
}

#[pyfunction]
fn bar() -> Vec<u16> { // unaffected, returns `PyList`
    vec![0, 1, 2, 3]
}

If this conversion is not desired, consider building a list manually using PyList::new.

The following types were previously only implemented for u8 and now allow other Ts turn into PyList:

  • &[T]
  • Cow<[T]>

This is purely additional and should just extend the possible return types.

gil-refs feature removed

Click to expand

PyO3 0.23 completes the removal of the “GIL Refs” API in favour of the new “Bound” API introduced in PyO3 0.21.

With the removal of the old API, many “Bound” API functions which had been introduced with _bound suffixes no longer need the suffixes as these names have been freed up. For example, PyTuple::new_bound is now just PyTuple::new (the existing name remains but is deprecated).

之前：

#![allow(deprecated)]
use pyo3::prelude::*;
use pyo3::types::PyTuple;
fn main() {
Python::attach(|py| {
// For example, for PyTuple. Many such APIs have been changed.
let tup = PyTuple::new_bound(py, [1, 2, 3]);
})
}

After:

use pyo3::prelude::*;
use pyo3::types::PyTuple;
fn main() {
Python::attach(|py| {
// For example, for PyTuple. Many such APIs have been changed.
let tup = PyTuple::new(py, [1, 2, 3]);
})
}

IntoPyDict trait adjusted for removal of gil-refs

As part of this API simplification, the IntoPyDict trait has had a small breaking change: IntoPyDict::into_py_dict_bound method has been renamed to IntoPyDict::into_py_dict. It is also now fallible as part of the IntoPyObject trait addition.

If you implemented IntoPyDict for your type, you should implement into_py_dict instead of into_py_dict_bound. The old name is still available for calling but deprecated.

之前：

use pyo3::prelude::*;
use pyo3::types::{PyDict, IntoPyDict};
use std::collections::HashMap;

struct MyMap<K, V>(HashMap<K, V>);

impl<K, V> IntoPyDict for MyMap<K, V>
where
    K: ToPyObject,
    V: ToPyObject,
{
    fn into_py_dict_bound(self, py: Python<'_>) -> Bound<'_, PyDict> {
        let dict = PyDict::new_bound(py);
        for (key, value) in self.0 {
            dict.set_item(key, value)
                .expect("Failed to set_item on dict");
        }
        dict
    }
}

After:

use pyo3::prelude::*;
use pyo3::types::{PyDict, IntoPyDict};
use std::collections::HashMap;

#[allow(dead_code)]
struct MyMap<K, V>(HashMap<K, V>);

impl<'py, K, V> IntoPyDict<'py> for MyMap<K, V>
where
    K: IntoPyObject<'py>,
    V: IntoPyObject<'py>,
{
    fn into_py_dict(self, py: Python<'py>) -> PyResult<Bound<'py, PyDict>> {
        let dict = PyDict::new(py);
        for (key, value) in self.0 {
            dict.set_item(key, value)?;
        }
        Ok(dict)
    }
}

from 0.21.* to 0.22

Deprecation of gil-refs feature continues

Click to expand

Following the introduction of the “Bound” API in PyO3 0.21 and the planned removal of the “GIL Refs” API, all functionality related to GIL Refs is now gated behind the gil-refs feature and emits a deprecation warning on use.

See the 0.21 migration entry for help upgrading.

Deprecation of implicit default for trailing optional arguments

Click to expand

With pyo3 0.22 the implicit None default for trailing Option<T> type argument is deprecated. To migrate, place a #[pyo3(signature = (...))] attribute on affected functions or methods and specify the desired behavior. The migration warning specifies the corresponding signature to keep the current behavior. With 0.23 the signature will be required for any function containing Option<T> type parameters to prevent accidental and unnoticed changes in behavior. With 0.24 this restriction will be lifted again and Option<T> type arguments will be treated as any other argument without special handling.

之前：

#![allow(deprecated, dead_code)]
use pyo3::prelude::*;
#[pyfunction]
fn increment(x: u64, amount: Option<u64>) -> u64 {
    x + amount.unwrap_or(1)
}

After:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyfunction]
#[pyo3(signature = (x, amount=None))]
fn increment(x: u64, amount: Option<u64>) -> u64 {
    x + amount.unwrap_or(1)
}

Py::clone is now gated behind the py-clone feature

Click to expand

If you rely on impl<T> Clone for Py<T> to fulfil trait requirements imposed by existing Rust code written without PyO3-based code in mind, the newly introduced feature py-clone must be enabled.

However, take care to note that the behaviour is different from previous versions. If Clone was called without the GIL being held, we tried to delay the application of these reference count increments until PyO3-based code would re-acquire it. This turned out to be impossible to implement in a sound manner and hence was removed. Now, if Clone is called without the GIL being held, we panic instead for which calling code might not be prepared.

It is advised to migrate off the py-clone feature. The simplest way to remove dependency on impl<T> Clone for Py<T> is to wrap Py<T> as Arc<Py<T>> and use cloning of the arc.

Related to this, we also added a pyo3_disable_reference_pool conditional compilation flag which removes the infrastructure necessary to apply delayed reference count decrements implied by impl<T> Drop for Py<T>. They do not appear to be a soundness hazard as they should lead to memory leaks in the worst case. However, the global synchronization adds significant overhead to cross the Python-Rust boundary. Enabling this feature will remove these costs and make the Drop implementation abort the process if called without the GIL being held instead.

Require explicit opt-in for comparison for simple enums

Click to expand

With pyo3 0.22 the new #[pyo3(eq)] options allows automatic implementation of Python equality using Rust’s PartialEq. Previously simple enums automatically implemented equality in terms of their discriminants. To make PyO3 more consistent, this automatic equality implementation is deprecated in favour of having opt-ins for all #[pyclass] types. Similarly, simple enums supported comparison with integers, which is not covered by Rust’s PartialEq derive, so has been split out into the #[pyo3(eq_int)] attribute.

To migrate, place a #[pyo3(eq, eq_int)] attribute on simple enum classes.

之前：

#![allow(deprecated, dead_code)]
use pyo3::prelude::*;
#[pyclass]
enum SimpleEnum {
    VariantA,
    VariantB = 42,
}

After:

#![allow(dead_code)]
use pyo3::prelude::*;
#[pyclass(eq, eq_int)]
#[derive(PartialEq)]
enum SimpleEnum {
    VariantA,
    VariantB = 42,
}

PyType::name reworked to better match Python __name__

Click to expand

This function previously would try to read directly from Python type objects’ C API field (tp_name), in which case it would return a Cow::Borrowed. However the contents of tp_name don’t have well-defined semantics.

Instead PyType::name() now returns the equivalent of Python __name__ and returns PyResult<Bound<'py, PyString>>.

The closest equivalent to PyO3 0.21’s version of PyType::name() has been introduced as a new function PyType::fully_qualified_name(), which is equivalent to __module__ and __qualname__ joined as module.qualname.

之前：

#![allow(deprecated, dead_code)]
use pyo3::prelude::*;
use pyo3::types::{PyBool};
fn main() -> PyResult<()> {
Python::with_gil(|py| {
    let bool_type = py.get_type_bound::<PyBool>();
    let name = bool_type.name()?.into_owned();
    println!("Hello, {}", name);

    let mut name_upper = bool_type.name()?;
    name_upper.to_mut().make_ascii_uppercase();
    println!("Hello, {}", name_upper);

    Ok(())
})
}

After:

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::{PyBool};
fn main() -> PyResult<()> {
Python::with_gil(|py| {
    let bool_type = py.get_type_bound::<PyBool>();
    let name = bool_type.name()?;
    println!("Hello, {}", name);

    // (if the full dotted path was desired, switch from `name()` to `fully_qualified_name()`)
    let mut name_upper = bool_type.fully_qualified_name()?.to_string();
    name_upper.make_ascii_uppercase();
    println!("Hello, {}", name_upper);

    Ok(())
})
}

from 0.20.* to 0.21

Click to expand

PyO3 0.21 introduces a new Bound<'py, T> smart pointer which replaces the existing “GIL Refs” API to interact with Python objects. For example, in PyO3 0.20 the reference &'py PyAny would be used to interact with Python objects. In PyO3 0.21 the updated type is Bound<'py, PyAny>. Making this change moves Rust ownership semantics out of PyO3’s internals and into user code. This change fixes a known soundness edge case of interaction with gevent as well as improves CPU and memory performance. For a full history of discussion see . For a full history of discussion see https://github.com/PyO3/pyo3/issues/3382.

The “GIL Ref” &'py PyAny and similar types such as &'py PyDict continue to be available as a deprecated API. Due to the advantages of the new API it is advised that all users make the effort to upgrade as soon as possible.

In addition to the major API type overhaul, PyO3 has needed to make a few small breaking adjustments to other APIs to close correctness and soundness gaps.

The recommended steps to update to PyO3 0.21 is as follows:

  1. Enable the gil-refs feature to silence deprecations related to the API change
  2. Fix all other PyO3 0.21 migration steps
  3. Disable the gil-refs feature and migrate off the deprecated APIs

The following sections are laid out in this order.

Enable the gil-refs feature

Click to expand

To make the transition for the PyO3 ecosystem away from the GIL Refs API as smooth as possible, in PyO3 0.21 no APIs consuming or producing GIL Refs have been altered. Instead, variants using Bound<T> smart pointers have been introduced, for example PyTuple::new_bound which returns Bound<PyTuple> is the replacement form of PyTuple::new. The GIL Ref APIs have been deprecated, but to make migration easier it is possible to disable these deprecation warnings by enabling the gil-refs feature.

[!TIP] The one single exception where an existing API was changed in-place is the pyo3::intern! macro. Almost all uses of this macro did not need to update code to account it changing to return &Bound<PyString> immediately, and adding an intern_bound! replacement was perceived as adding more work for users.

It is recommended that users do this as a first step of updating to PyO3 0.21 so that the deprecation warnings do not get in the way of resolving the rest of the migration steps.

之前：

# Cargo.toml
[dependencies]
pyo3 = "0.20"

After:

# Cargo.toml
[dependencies]
pyo3 = { version = "0.21", features = ["gil-refs"] }

PyTypeInfo and PyTryFrom have been adjusted

Click to expand

The PyTryFrom trait has aged poorly, its try_from method now conflicts with TryFrom::try_from in the 2021 edition prelude. A lot of its functionality was also duplicated with PyTypeInfo.

To tighten up the PyO3 traits as part of the deprecation of the GIL Refs API the PyTypeInfo trait has had a simpler companion PyTypeCheck. The methods PyAny::downcast and PyAny::downcast_exact no longer use PyTryFrom as a bound, instead using PyTypeCheck and PyTypeInfo respectively.

To migrate, switch all type casts to use obj.downcast() instead of try_from(obj) (and similar for downcast_exact).

之前：

#![allow(deprecated)]
use pyo3::prelude::*;
use pyo3::types::{PyInt, PyList};
fn main() -> PyResult<()> {
Python::with_gil(|py| {
    let list = PyList::new(py, 0..5);
    let b = <PyInt as PyTryFrom>::try_from(list.get_item(0).unwrap())?;
    Ok(())
})
}

After:

use pyo3::prelude::*;
use pyo3::types::{PyInt, PyList};
fn main() -> PyResult<()> {
Python::with_gil(|py| {
    // Note that PyList::new is deprecated for PyList::new_bound as part of the GIL Refs API removal,
    // see the section below on migration to Bound<T>.
    #[allow(deprecated)]
    let list = PyList::new(py, 0..5);
    let b = list.get_item(0).unwrap().downcast::<PyInt>()?;
    Ok(())
})
}

Iter(A)NextOutput are deprecated

Click to expand

The __next__ and __anext__ magic methods can now return any type convertible into Python objects directly just like all other #[pymethods]. The IterNextOutput used by __next__ and IterANextOutput used by __anext__ are subsequently deprecated. Most importantly, this change allows returning an awaitable from __anext__ without non-sensically wrapping it into Yield or Some. Only the return types Option<T> and Result<Option<T>, E> are still handled in a special manner where Some(val) yields val and None stops iteration.

Starting with an implementation of a Python iterator using IterNextOutput, e.g.

use pyo3::prelude::*;
use pyo3::iter::IterNextOutput;

#[pyclass]
struct PyClassIter {
    count: usize,
}

#[pymethods]
impl PyClassIter {
    fn __next__(&mut self) -> IterNextOutput<usize, &'static str> {
        if self.count < 5 {
            self.count += 1;
            IterNextOutput::Yield(self.count)
        } else {
            IterNextOutput::Return("done")
        }
    }
}

If returning "done" via StopIteration is not really required, this should be written as

use pyo3::prelude::*;

#[pyclass]
struct PyClassIter {
    count: usize,
}

#[pymethods]
impl PyClassIter {
    fn __next__(&mut self) -> Option<usize> {
        if self.count < 5 {
            self.count += 1;
            Some(self.count)
        } else {
            None
        }
    }
}

This form also has additional benefits: It has already worked in previous PyO3 versions, it matches the signature of Rust’s Iterator trait and it allows using a fast path in CPython which completely avoids the cost of raising a StopIteration exception. Note that using Option::transpose and the Result<Option<T>, E> variant, this form can also be used to wrap fallible iterators.

Alternatively, the implementation can also be done as it would in Python itself, i.e. by “raising” a StopIteration exception

use pyo3::prelude::*;
use pyo3::exceptions::PyStopIteration;

#[pyclass]
struct PyClassIter {
    count: usize,
}

#[pymethods]
impl PyClassIter {
    fn __next__(&mut self) -> PyResult<usize> {
        if self.count < 5 {
            self.count += 1;
            Ok(self.count)
        } else {
            Err(PyStopIteration::new_err("done"))
        }
    }
}

Finally, an asynchronous iterator can directly return an awaitable without confusing wrapping

use pyo3::prelude::*;

#[pyclass]
struct PyClassAwaitable {
    number: usize,
}

#[pymethods]
impl PyClassAwaitable {
    fn __next__(&self) -> usize {
        self.number
    }

    fn __await__(slf: Py<Self>) -> Py<Self> {
        slf
    }
}

#[pyclass]
struct PyClassAsyncIter {
    number: usize,
}

#[pymethods]
impl PyClassAsyncIter {
    fn __anext__(&mut self) -> PyClassAwaitable {
        self.number += 1;
        PyClassAwaitable {
            number: self.number,
        }
    }

    fn __aiter__(slf: Py<Self>) -> Py<Self> {
        slf
    }
}

PyType::name has been renamed to PyType::qualname

Click to expand

PyType::name has been renamed to PyType::qualname to indicate that it does indeed return the qualified name, matching the __qualname__ attribute. The newly added PyType::name yields the full name including the module name now which corresponds to __module__.__name__ on the level of attributes.

PyCell has been deprecated

Click to expand

Interactions with Python objects implemented in Rust no longer need to go though PyCell<T>. Instead interactions with Python object now consistently go through Bound<T> or Py<T> independently of whether T is native Python object or a #[pyclass] implemented in Rust. Use Bound::new or Py::new respectively to create and Bound::borrow(_mut) / Py::borrow(_mut) to borrow the Rust object.

Migrating from the GIL Refs API to Bound<T>

Click to expand

To minimise breakage of code using the GIL Refs API, the Bound<T> smart pointer has been introduced by adding complements to all functions which accept or return GIL Refs. This allows code to migrate by replacing the deprecated APIs with the new ones.

To identify what to migrate, temporarily switch off the gil-refs feature to see deprecation warnings on almost all uses of APIs accepting and producing GIL Refs . Over one or more PRs it should be possible to follow the deprecation hints to update code. Depending on your development environment, switching off the gil-refs feature may introduce some very targeted breakages, so you may need to fixup those first.

For example, the following APIs have gained updated variants:

  • PyList::new, PyTuple::new and similar constructors have replacements PyList::new_bound, PyTuple::new_bound etc.
  • FromPyObject::extract has a new FromPyObject::extract_bound (see the section below)
  • The PyTypeInfo trait has had new _bound methods added to accept / return Bound<T>.

Because the new Bound<T> API brings ownership out of the PyO3 framework and into user code, there are a few places where user code is expected to need to adjust while switching to the new API:

  • Code will need to add the occasional & to borrow the new smart pointer as &Bound<T> to pass these types around (or use .clone() at the very small cost of increasing the Python reference count)
  • Bound<PyList> and Bound<PyTuple> cannot support indexing with list[0], you should use list.get_item(0) instead.
  • Bound<PyTuple>::iter_borrowed is slightly more efficient than Bound<PyTuple>::iter. The default iteration of Bound<PyTuple> cannot return borrowed references because Rust does not (yet) have “lending iterators”. Similarly Bound<PyTuple>::get_borrowed_item is more efficient than Bound<PyTuple>::get_item for the same reason.
  • &Bound<T> does not implement FromPyObject (although it might be possible to do this in the future once the GIL Refs API is completely removed). Use bound_any.downcast::<T>() instead of bound_any.extract::<&Bound<T>>().
  • Bound<PyString>::to_str now borrows from the Bound<PyString> rather than from the 'py lifetime, so code will need to store the smart pointer as a value in some cases where previously &PyString was just used as a temporary. (There are some more details relating to this in the section below.)
  • .extract::<&str>() now borrows from the source Python object. The simplest way to update is to change to .extract::<PyBackedStr>(), which retains ownership of the Python reference. See more information in the section on deactivating the gil-refs feature.

To convert between &PyAny and &Bound<PyAny> use the as_borrowed() method:

let gil_ref: &PyAny = ...;
let bound: &Bound<PyAny> = &gil_ref.as_borrowed();

To convert between Py<T> and Bound<T> use the bind() / into_bound() methods, and as_unbound() / unbind() to go back from Bound<T> to Py<T>.

let obj: Py<PyList> = ...;
let bound: &Bound<'py, PyList> = obj.bind(py);
let bound: Bound<'py, PyList> = obj.into_bound(py);

let obj: &Py<PyList> = bound.as_unbound();
let obj: Py<PyList> = bound.unbind();

[!WARNING] Dangling pointer trap 💣

Because of the ownership changes, code which uses .as_ptr() to convert &PyAny and other GIL Refs to a *mut pyo3_ffi::PyObject should take care to avoid creating dangling pointers now that Bound<PyAny> carries ownership.

For example, the following pattern with Option<&PyAny> can easily create a dangling pointer when migrating to the Bound<PyAny> smart pointer:

let opt: Option<&PyAny> = ...;
let p: *mut ffi::PyObject = opt.map_or(std::ptr::null_mut(), |any| any.as_ptr());

The correct way to migrate this code is to use .as_ref() to avoid dropping the Bound<PyAny> in the map_or closure:

let opt: Option<Bound<PyAny>> = ...;
let p: *mut ffi::PyObject = opt.as_ref().map_or(std::ptr::null_mut(), Bound::as_ptr);

Migrating FromPyObject implementations

FromPyObject has had a new method extract_bound which takes &Bound<'py, PyAny> as an argument instead of &PyAny. Both extract and extract_bound have been given default implementations in terms of the other, to avoid breaking code immediately on update to 0.21.

All implementations of FromPyObject should be switched from extract to extract_bound.

之前：

impl<'py> FromPyObject<'py> for MyType {
    fn extract(obj: &'py PyAny) -> PyResult<Self> {
        /* ... */
    }
}

After:

impl<'py> FromPyObject<'py> for MyType {
    fn extract_bound(obj: &Bound<'py, PyAny>) -> PyResult<Self> {
        /* ... */
    }
}

The expectation is that in 0.22 extract_bound will have the default implementation removed and in 0.23 extract will be removed.

Cases where PyO3 cannot emit GIL Ref deprecation warnings

Despite a large amount of deprecations warnings produced by PyO3 to aid with the transition from GIL Refs to the Bound API, there are a few cases where PyO3 cannot automatically warn on uses of GIL Refs. It is worth checking for these cases manually after the deprecation warnings have all been addressed:

  • Individual implementations of the FromPyObject trait cannot be deprecated, so PyO3 cannot warn about uses of code patterns like .extract<&PyAny>() which produce a GIL Ref.
  • GIL Refs in #[pyfunction] arguments emit a warning, but if the GIL Ref is wrapped inside another container such as Vec<&PyAny> then PyO3 cannot warn against this.
  • The wrap_pyfunction!(function)(py) deferred argument form of the wrap_pyfunction macro taking py: Python<'py> produces a GIL Ref, and due to limitations in type inference PyO3 cannot warn against this specific case.

Deactivating the gil-refs feature

Click to expand

As a final step of migration, deactivating the gil-refs feature will set up code for best performance and is intended to set up a forward-compatible API for PyO3 0.22.

At this point code that needed to manage GIL Ref memory can safely remove uses of GILPool (which are constructed by calls to Python::new_pool and Python::with_pool). Deprecation warnings will highlight these cases.

There is just one case of code that changes upon disabling these features: FromPyObject trait implementations for types that borrow directly from the input data cannot be implemented by PyO3 without GIL Refs (while the GIL Refs API is in the process of being removed). The main types affected are &str, Cow<'_, str>, &[u8], Cow<'_, u8>.

To make PyO3’s core functionality continue to work while the GIL Refs API is in the process of being removed, disabling the gil-refs feature moves the implementations of FromPyObject for &str, Cow<'_, str>, &[u8], Cow<'_, u8> to a new temporary trait FromPyObjectBound. This trait is the expected future form of FromPyObject and has an additional lifetime 'a to enable these types to borrow data from Python objects.

PyO3 0.21 has introduced the PyBackedStr and PyBackedBytes types to help with this case. The easiest way to avoid lifetime challenges from extracting &str is to use these. For more complex types like Vec<&str>, is now impossible to extract directly from a Python object and Vec<PyBackedStr> is the recommended upgrade path.

A key thing to note here is because extracting to these types now ties them to the input lifetime, some extremely common patterns may need to be split into multiple Rust lines. For example, the following snippet of calling .extract::<&str>() directly on the result of .getattr() needs to be adjusted when deactivating the gil-refs feature.

之前：

#[cfg(feature = "gil-refs")] {
use pyo3::prelude::*;
use pyo3::types::{PyList, PyType};
fn example<'py>(py: Python<'py>) -> PyResult<()> {
#[allow(deprecated)] // GIL Ref API
let obj: &'py PyType = py.get_type::<PyList>();
let name: &'py str = obj.getattr("__name__")?.extract()?;
assert_eq!(name, "list");
Ok(())
}
Python::with_gil(example).unwrap();
}

After:

#[cfg(any(not(Py_LIMITED_API), Py_3_10))] {
use pyo3::prelude::*;
use pyo3::types::{PyList, PyType};
fn example<'py>(py: Python<'py>) -> PyResult<()> {
let obj: Bound<'py, PyType> = py.get_type_bound::<PyList>();
let name_obj: Bound<'py, PyAny> = obj.getattr("__name__")?;
// the lifetime of the data is no longer `'py` but the much shorter
// lifetime of the `name_obj` smart pointer above
let name: &'_ str = name_obj.extract()?;
assert_eq!(name, "list");
Ok(())
}
Python::with_gil(example).unwrap();
}

To avoid needing to worry about lifetimes at all, it is also possible to use the new PyBackedStr type, which stores a reference to the Python str without a lifetime attachment. In particular, PyBackedStr helps for abi3 builds for Python older than 3.10. Due to limitations in the abi3 CPython API for those older versions, PyO3 cannot offer a FromPyObjectBound implementation for &str on those versions. The easiest way to migrate for older abi3 builds is to replace any cases of .extract::<&str>() with .extract::<PyBackedStr>(). Alternatively, use .extract::<Cow<str>>(), .extract::<String>() to copy the data into Rust.

The following example uses the same snippet as those just above, but this time the final extracted type is PyBackedStr:

use pyo3::prelude::*;
use pyo3::types::{PyList, PyType};
fn example<'py>(py: Python<'py>) -> PyResult<()> {
use pyo3::pybacked::PyBackedStr;
let obj: Bound<'py, PyType> = py.get_type_bound::<PyList>();
let name: PyBackedStr = obj.getattr("__name__")?.extract()?;
assert_eq!(&*name, "list");
Ok(())
}
Python::with_gil(example).unwrap();

from 0.19.* to 0.20

Drop support for older technologies

Click to expand

PyO3 0.20 has increased minimum Rust version to 1.56. This enables use of newer language features and simplifies maintenance of the project.

PyDict::get_item now returns a Result

Click to expand

PyDict::get_item in PyO3 0.19 and older was implemented using a Python API which would suppress all exceptions and return None in those cases. This included errors in __hash__ and __eq__ implementations of the key being looked up.

Newer recommendations by the Python core developers advise against using these APIs which suppress exceptions, instead allowing exceptions to bubble upwards. PyDict::get_item_with_error already implemented this recommended behavior, so that API has been renamed to PyDict::get_item.

之前：

use pyo3::prelude::*;
use pyo3::exceptions::PyTypeError;
use pyo3::types::{PyDict, IntoPyDict};

fn main() {
let _ =
Python::with_gil(|py| {
    let dict: &PyDict = [("a", 1)].into_py_dict(py);
    // `a` is in the dictionary, with value 1
    assert!(dict.get_item("a").map_or(Ok(false), |x| x.eq(1))?);
    // `b` is not in the dictionary
    assert!(dict.get_item("b").is_none());
    // `dict` is not hashable, so this fails with a `TypeError`
    assert!(dict
        .get_item_with_error(dict)
        .unwrap_err()
        .is_instance_of::<PyTypeError>(py));
});
}

After:

use pyo3::prelude::*;
use pyo3::exceptions::PyTypeError;
use pyo3::types::{PyDict, IntoPyDict};

fn main() {
let _ =
Python::with_gil(|py| -> PyResult<()> {
    let dict: &PyDict = [("a", 1)].into_py_dict(py);
    // `a` is in the dictionary, with value 1
    assert!(dict.get_item("a")?.map_or(Ok(false), |x| x.eq(1))?);
    // `b` is not in the dictionary
    assert!(dict.get_item("b")?.is_none());
    // `dict` is not hashable, so this fails with a `TypeError`
    assert!(dict
        .get_item(dict)
        .unwrap_err()
        .is_instance_of::<PyTypeError>(py));

    Ok(())
});
}

Required arguments are no longer accepted after optional arguments

Click to expand

Trailing Option<T> arguments have an automatic default of None. To avoid unwanted changes when modifying function signatures, in PyO3 0.18 it was deprecated to have a required argument after an Option<T> argument without using #[pyo3(signature = (...))] to specify the intended defaults. In PyO3 0.20, this becomes a hard error.

之前：

#[pyfunction]
fn x_or_y(x: Option<u64>, y: u64) -> u64 {
    x.unwrap_or(y)
}

After:

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyfunction]
#[pyo3(signature = (x, y))] // both x and y have no defaults and are required
fn x_or_y(x: Option<u64>, y: u64) -> u64 {
    x.unwrap_or(y)
}

Remove deprecated function forms

Click to expand

In PyO3 0.18 the #[args] attribute for #[pymethods], and directly specifying the function signature in #[pyfunction], was deprecated. This functionality has been removed in PyO3 0.20.

之前：

#[pyfunction]
#[pyo3(a, b = "0", "/")]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

After:

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyfunction]
#[pyo3(signature = (a, b=0, /))]
fn add(a: u64, b: u64) -> u64 {
    a + b
}

IntoPyPointer trait removed

Click to expand

The trait IntoPyPointer, which provided the into_ptr method on many types, has been removed. into_ptr is now available as an inherent method on all types that previously implemented this trait.

AsPyPointer now unsafe trait

Click to expand

The trait AsPyPointer is now unsafe trait, meaning any external implementation of it must be marked as unsafe impl, and ensure that they uphold the invariant of returning valid pointers.

from 0.18.* to 0.19

Access to Python inside __traverse__ implementations are now forbidden

Click to expand

During __traverse__ implementations for Python’s Garbage Collection it is forbidden to do anything other than visit the members of the #[pyclass] being traversed. This means making Python function calls or other API calls are forbidden.

Previous versions of PyO3 would allow access to Python (e.g. via Python::with_gil), which could cause the Python interpreter to crash or otherwise confuse the garbage collection algorithm.

Attempts to acquire the GIL will now panic. See #3165 for more detail.

use pyo3::prelude::*;

#[pyclass]
struct SomeClass {}

impl SomeClass {
    fn __traverse__(&self, pyo3::class::gc::PyVisit<'_>) -> Result<(), pyo3::class::gc::PyTraverseError>` {
        Python::with_gil(|| { /*...*/ })  // ERROR: this will panic
    }
}

Smarter anyhow::Error / eyre::Report conversion when inner error is “simple” PyErr

Click to expand

When converting from anyhow::Error or eyre::Report to PyErr, if the inner error is a “simple” PyErr (with no source error), then the inner error will be used directly as the PyErr instead of wrapping it in a new PyRuntimeError with the original information converted into a string.

#[cfg(feature = "anyhow")]
#[allow(dead_code)]
mod anyhow_only {
use pyo3::prelude::*;
use pyo3::exceptions::PyValueError;
#[pyfunction]
fn raise_err() -> anyhow::Result<()> {
    Err(PyValueError::new_err("original error message").into())
}

fn main() {
    Python::with_gil(|py| {
        let rs_func = wrap_pyfunction!(raise_err, py).unwrap();
        pyo3::py_run!(
            py,
            rs_func,
            r"
        try:
            rs_func()
        except Exception as e:
            print(repr(e))
        "
        );
    })
}
}

Before, the above code would have printed RuntimeError('ValueError: original error message'), which might be confusing.

After, the same code will print ValueError: original error message, which is more straightforward.

However, if the anyhow::Error or eyre::Report has a source, then the original exception will still be wrapped in a PyRuntimeError.

The deprecated Python::acquire_gil was removed and Python::with_gil must be used instead

Click to expand

While the API provided by Python::acquire_gil seems convenient, it is somewhat brittle as the design of the Python token relies on proper nesting and panics if not used correctly, e.g.

#![allow(dead_code, deprecated)]
use pyo3::prelude::*;

#[pyclass]
struct SomeClass {}

struct ObjectAndGuard {
    object: Py<SomeClass>,
    guard: GILGuard,
}

impl ObjectAndGuard {
    fn new() -> Self {
        let guard = Python::acquire_gil();
        let object = Py::new(guard.python(), SomeClass {}).unwrap();

        Self { object, guard }
    }
}

let first = ObjectAndGuard::new();
let second = ObjectAndGuard::new();
// Panics because the guard within `second` is still alive.
drop(first);
drop(second);

The replacement is Python::with_gil which is more cumbersome but enforces the proper nesting by design, e.g.

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyclass]
struct SomeClass {}

struct Object {
    object: Py<SomeClass>,
}

impl Object {
    fn new(py: Python<'_>) -> Self {
        let object = Py::new(py, SomeClass {}).unwrap();

        Self { object }
    }
}

// It either forces us to release the GIL before acquiring it again.
let first = Python::with_gil(|py| Object::new(py));
let second = Python::with_gil(|py| Object::new(py));
drop(first);
drop(second);

// Or it ensures releasing the inner lock before the outer one.
Python::with_gil(|py| {
    let first = Object::new(py);
    let second = Python::with_gil(|py| Object::new(py));
    drop(first);
    drop(second);
});

Furthermore, Python::acquire_gil provides ownership of a GILGuard which can be freely stored and passed around. This is usually not helpful as it may keep the lock held for a long time thereby blocking progress in other parts of the program. Due to the generative lifetime attached to the Python token supplied by Python::with_gil, the problem is avoided as the Python token can only be passed down the call chain. Often, this issue can also be avoided entirely as any GIL-bound reference &'py PyAny implies access to a Python token Python<'py> via the PyAny::py method.

from 0.17.* to 0.18

Required arguments after Option<_> arguments will no longer be automatically inferred

Click to expand

In #[pyfunction] and #[pymethods], if a “required” function input such as i32 came after an Option<_> input, then the Option<_> would be implicitly treated as required. (All trailing Option<_> arguments were treated as optional with a default value of None).

Starting with PyO3 0.18, this is deprecated and a future PyO3 version will require a #[pyo3(signature = (...))] option to explicitly declare the programmer’s intention.

Before, x in the below example would be required to be passed from Python code:

#![allow(dead_code)]
use pyo3::prelude::*;

#[pyfunction]
fn required_argument_after_option(x: Option<i32>, y: i32) {}

After, specify the intended Python signature explicitly:

#![allow(dead_code)]
use pyo3::prelude::*;

// If x really was intended to be required
#[pyfunction(signature = (x, y))]
fn required_argument_after_option_a(x: Option<i32>, y: i32) {}

// If x was intended to be optional, y needs a default too
#[pyfunction(signature = (x=None, y=0))]
fn required_argument_after_option_b(x: Option<i32>, y: i32) {}

__text_signature__ is now automatically generated for #[pyfunction] and #[pymethods]

Click to expand

The #[pyo3(text_signature = "...")] option was previously the only supported way to set the __text_signature__ attribute on generated Python functions.

PyO3 is now able to automatically populate __text_signature__ for all functions automatically based on their Rust signature (or the new #[pyo3(signature = (...))] option). These automatically-generated __text_signature__ values will currently only render ... for all default values. Many #[pyo3(text_signature = "...")] options can be removed from functions when updating to PyO3 0.18, however in cases with default values a manual implementation may still be preferred for now.

As examples:

use pyo3::prelude::*;

// The `text_signature` option here is no longer necessary, as PyO3 will automatically
// generate exactly the same value.
#[pyfunction(text_signature = "(a, b, c)")]
fn simple_function(a: i32, b: i32, c: i32) {}

// The `text_signature` still provides value here as of PyO3 0.18, because the automatically
// generated signature would be "(a, b=..., c=...)".
#[pyfunction(signature = (a, b = 1, c = 2), text_signature = "(a, b=1, c=2)")]
fn function_with_defaults(a: i32, b: i32, c: i32) {}

fn main() {
    Python::attach(|py| {
        let simple = wrap_pyfunction!(simple_function, py).unwrap();
        assert_eq!(simple.getattr("__text_signature__").unwrap().to_string(), "(a, b, c)");
        let defaulted = wrap_pyfunction!(function_with_defaults, py).unwrap();
        assert_eq!(defaulted.getattr("__text_signature__").unwrap().to_string(), "(a, b=1, c=2)");
    })
}

from 0.16.* to 0.17

Type checks have been changed for PyMapping and PySequence types

Click to expand

Previously the type checks for PyMapping and PySequence (implemented in PyTryFrom) used the Python C-API functions PyMapping_Check and PySequence_Check. Unfortunately these functions are not sufficient for distinguishing such types, leading to inconsistent behavior (see pyo3/pyo3#2072).

PyO3 0.17 changes these downcast checks to explicitly test if the type is a subclass of the corresponding abstract base class collections.abc.Mapping or collections.abc.Sequence. Note this requires calling into Python, which may incur a performance penalty over the previous method. If this performance penalty is a problem, you may be able to perform your own checks and use try_from_unchecked (unsafe).

Another side-effect is that a pyclass defined in Rust with PyO3 will need to be registered with the corresponding Python abstract base class for downcasting to succeed. PySequence::register and PyMapping:register have been added to make it easy to do this from Rust code. These are equivalent to calling collections.abc.Mapping.register(MappingPyClass) or collections.abc.Sequence.register(SequencePyClass) from Python.

For example, for a mapping class defined in Rust:

use pyo3::prelude::*;
use std::collections::HashMap;

#[pyclass(mapping)]
struct Mapping {
    index: HashMap<String, usize>,
}

#[pymethods]
impl Mapping {
    #[new]
    fn new(elements: Option<&PyList>) -> PyResult<Self> {
    // ...
    // truncated implementation of this mapping pyclass - basically a wrapper around a HashMap
}

You must register the class with collections.abc.Mapping before the downcast will work:

let m = Py::new(py, Mapping { index }).unwrap();
assert!(m.as_ref(py).downcast::<PyMapping>().is_err());
PyMapping::register::<Mapping>(py).unwrap();
assert!(m.as_ref(py).downcast::<PyMapping>().is_ok());

Note that this requirement may go away in the future when a pyclass is able to inherit from the abstract base class directly (see pyo3/pyo3#991).

The multiple-pymethods feature now requires Rust 1.62

Click to expand

Due to limitations in the inventory crate which the multiple-pymethods feature depends on, this feature now requires Rust 1.62. For more information see dtolnay/inventory#32.

Added impl IntoPy<Py<PyString>> for &str

Click to expand

This may cause inference errors.

之前：

use pyo3::prelude::*;

fn main() {
Python::with_gil(|py| {
    // Cannot infer either `Py<PyAny>` or `Py<PyString>`
    let _test = "test".into_py(py);
});
}

After, some type annotations may be necessary:

#![allow(deprecated)]
use pyo3::prelude::*;

fn main() {
Python::with_gil(|py| {
    let _test: Py<PyAny> = "test".into_py(py);
});
}

The pyproto feature is now disabled by default

Click to expand

In preparation for removing the deprecated #[pyproto] attribute macro in a future PyO3 version, it is now gated behind an opt-in feature flag. This also gives a slight saving to compile times for code which does not use the deprecated macro.

PyTypeObject trait has been deprecated

Click to expand

The PyTypeObject trait already was near-useless; almost all functionality was already on the PyTypeInfo trait, which PyTypeObject had a blanket implementation based upon. In PyO3 0.17 the final method, PyTypeObject::type_object was moved to PyTypeInfo::type_object.

To migrate, update trait bounds and imports from PyTypeObject to PyTypeInfo.

之前：

use pyo3::Python;
use pyo3::type_object::PyTypeObject;
use pyo3::types::PyType;

fn get_type_object<T: PyTypeObject>(py: Python<'_>) -> &PyType {
    T::type_object(py)
}

After

use pyo3::{Python, PyTypeInfo};
use pyo3::types::PyType;

fn get_type_object<T: PyTypeInfo>(py: Python<'_>) -> &PyType {
    T::type_object(py)
}

Python::with_gil(|py| { get_type_object::<pyo3::types::PyList>(py); });

impl<T, const N: usize> IntoPy<PyObject> for [T; N] now requires T: IntoPy rather than T: ToPyObject

Click to expand

If this leads to errors, simply implement IntoPy. Because pyclasses already implement IntoPy, you probably don’t need to worry about this.

Each #[pymodule] can now only be initialized once per process

Click to expand

To make PyO3 modules sound in the presence of Python sub-interpreters, for now it has been necessary to explicitly disable the ability to initialize a #[pymodule] more than once in the same process. Attempting to do this will now raise an ImportError.

from 0.15.* to 0.16

Drop support for older technologies

Click to expand

PyO3 0.16 has increased minimum Rust version to 1.48 and minimum Python version to 3.7. This enables use of newer language features (enabling some of the other additions in 0.16) and simplifies maintenance of the project.

#[pyproto] has been deprecated

Click to expand

In PyO3 0.15, the #[pymethods] attribute macro gained support for implementing “magic methods” such as __str__ (aka “dunder” methods). This implementation was not quite finalized at the time, with a few edge cases to be decided upon. The existing #[pyproto] attribute macro was left untouched, because it covered these edge cases.

In PyO3 0.16, the #[pymethods] implementation has been completed and is now the preferred way to implement magic methods. To allow the PyO3 project to move forward, #[pyproto] has been deprecated (with expected removal in PyO3 0.18).

Migration from #[pyproto] to #[pymethods] is straightforward; copying the existing methods directly from the #[pyproto] trait implementation is all that is needed in most cases.

之前：

use pyo3::prelude::*;
use pyo3::class::{PyObjectProtocol, PyIterProtocol};
use pyo3::types::PyString;

#[pyclass]
struct MyClass {}

#[pyproto]
impl PyObjectProtocol for MyClass {
    fn __str__(&self) -> &'static [u8] {
        b"hello, world"
    }
}

#[pyproto]
impl PyIterProtocol for MyClass {
    fn __iter__(slf: PyRef<self>) -> PyResult<&PyAny> {
        PyString::new(slf.py(), "hello, world").iter()
    }
}

After

use pyo3::prelude::*;
use pyo3::types::PyString;

#[pyclass]
struct MyClass {}

#[pymethods]
impl MyClass {
    fn __str__(&self) -> &'static [u8] {
        b"hello, world"
    }

    fn __iter__(slf: PyRef<self>) -> PyResult<&PyAny> {
        PyString::new(slf.py(), "hello, world").iter()
    }
}

Removed PartialEq for object wrappers

Click to expand

The Python object wrappers Py and PyAny had implementations of PartialEq so that object_a == object_b would compare the Python objects for pointer equality, which corresponds to the is operator, not the == operator in Python. This has been removed in favor of a new method: use object_a.is(object_b). This also has the advantage of not requiring the same wrapper type for object_a and object_b; you can now directly compare a Py<T> with a &PyAny without having to convert.

To check for Python object equality (the Python == operator), use the new method eq().

Container magic methods now match Python behavior

Click to expand

In PyO3 0.15, __getitem__, __setitem__ and __delitem__ in #[pymethods] would generate only the mapping implementation for a #[pyclass]. To match the Python behavior, these methods now generate both the mapping and sequence implementations.

This means that classes implementing these #[pymethods] will now also be treated as sequences, same as a Python class would be. Small differences in behavior may result:

  • PyO3 will allow instances of these classes to be cast to PySequence as well as PyMapping.
  • Python will provide a default implementation of __iter__ (if the class did not have one) which repeatedly calls __getitem__ with integers (starting at 0) until an IndexError is raised.

To explain this in detail, consider the following Python class:

class ExampleContainer:

    def __len__(self):
        return 5

    def __getitem__(self, idx: int) -> int:
        if idx < 0 or idx > 5:
            raise IndexError()
        return idx

This class implements a Python sequence.

The __len__ and __getitem__ methods are also used to implement a Python mapping. In the Python C-API, these methods are not shared: the sequence __len__ and __getitem__ are defined by the sq_length and sq_item slots, and the mapping equivalents are mp_length and mp_subscript. There are similar distinctions for __setitem__ and __delitem__.

Because there is no such distinction from Python, implementing these methods will fill the mapping and sequence slots simultaneously. A Python class with __len__ implemented, for example, will have both the sq_length and mp_length slots filled.

The PyO3 behavior in 0.16 has been changed to be closer to this Python behavior by default.

wrap_pymodule! and wrap_pyfunction! now respect privacy correctly

Click to expand

Prior to PyO3 0.16 the wrap_pymodule! and wrap_pyfunction! macros could use modules and functions whose defining fn was not reachable according Rust privacy rules.

For example, the following code was legal before 0.16, but in 0.16 is rejected because the wrap_pymodule! macro cannot access the private_submodule function:

mod foo {
    use pyo3::prelude::*;

    #[pymodule]
    fn private_submodule(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
        Ok(())
    }
}

use pyo3::prelude::*;
use foo::*;

#[pymodule]
fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pymodule!(private_submodule))?;
    Ok(())
}

To fix it, make the private submodule visible, e.g. with pub or pub(crate).

mod foo {
    use pyo3::prelude::*;

    #[pymodule]
    pub(crate) fn private_submodule(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
        Ok(())
    }
}

use pyo3::prelude::*;
use pyo3::wrap_pymodule;
use foo::*;

#[pymodule]
fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pymodule!(private_submodule))?;
    Ok(())
}

from 0.14.* to 0.15

Changes in sequence indexing

Click to expand

For all types that take sequence indices (PyList, PyTuple and PySequence), the API has been made consistent to only take usize indices, for consistency with Rust’s indexing conventions. Negative indices, which were only sporadically supported even in APIs that took isize, now aren’t supported anywhere.

Further, the get_item methods now always return a PyResult instead of panicking on invalid indices. The Index trait has been implemented instead, and provides the same panic behavior as on Rust vectors.

Note that slice indices (accepted by PySequence::get_slice and other) still inherit the Python behavior of clamping the indices to the actual length, and not panicking/returning an error on out of range indices.

An additional advantage of using Rust’s indexing conventions for these types is that these types can now also support Rust’s indexing operators as part of a consistent API:

#![allow(deprecated)]
use pyo3::{Python, types::PyList};

Python::with_gil(|py| {
    let list = PyList::new(py, &[1, 2, 3]);
    assert_eq!(list[0..2].to_string(), "[1, 2]");
});

from 0.13.* to 0.14

auto-initialize feature is now opt-in

Click to expand

For projects embedding Python in Rust, PyO3 no longer automatically initializes a Python interpreter on the first call to Python::with_gil (or Python::acquire_gil) unless the auto-initialize feature is enabled.

New multiple-pymethods feature

Click to expand

#[pymethods] have been reworked with a simpler default implementation which removes the dependency on the inventory crate. This reduces dependencies and compile times for the majority of users.

The limitation of the new default implementation is that it cannot support multiple #[pymethods] blocks for the same #[pyclass]. If you need this functionality, you must enable the multiple-pymethods feature which will switch #[pymethods] to the inventory-based implementation.

Deprecated #[pyproto] methods

Click to expand

Some protocol (aka __dunder__) methods such as __bytes__ and __format__ have been possible to implement two ways in PyO3 for some time: via a #[pyproto] (e.g. PyObjectProtocol for the methods listed here), or by writing them directly in #[pymethods]. This is only true for a handful of the #[pyproto] methods (for technical reasons to do with the way PyO3 currently interacts with the Python C-API).

In the interest of having only one way to do things, the #[pyproto] forms of these methods have been deprecated.

To migrate just move the affected methods from a #[pyproto] to a #[pymethods] block.

之前：

use pyo3::prelude::*;
use pyo3::class::basic::PyObjectProtocol;

#[pyclass]
struct MyClass {}

#[pyproto]
impl PyObjectProtocol for MyClass {
    fn __bytes__(&self) -> &'static [u8] {
        b"hello, world"
    }
}

After:

use pyo3::prelude::*;

#[pyclass]
struct MyClass {}

#[pymethods]
impl MyClass {
    fn __bytes__(&self) -> &'static [u8] {
        b"hello, world"
    }
}

from 0.12.* to 0.13

Minimum Rust version increased to Rust 1.45

Click to expand

PyO3 0.13 makes use of new Rust language features stabilized between Rust 1.40 and Rust 1.45. If you are using a Rust compiler older than Rust 1.45, you will need to update your toolchain to be able to continue using PyO3.

Runtime changes to support the CPython limited API

Click to expand

In PyO3 0.13 support was added for compiling against the CPython limited API. This had a number of implications for all PyO3 users, described here.

The largest of these is that all types created from PyO3 are what CPython calls “heap” types. The specific implications of this are:

  • If you wish to subclass one of these types from Rust you must mark it #[pyclass(subclass)], as you would if you wished to allow subclassing it from Python code.
  • Type objects are now mutable - Python code can set attributes on them.
  • __module__ on types without #[pyclass(module="mymodule")] no longer returns builtins, it now raises AttributeError.

from 0.11.* to 0.12

PyErr has been reworked

Click to expand

In PyO3 0.12 the PyErr type has been re-implemented to be significantly more compatible with the standard Rust error handling ecosystem. Specifically PyErr now implements Error + Send + Sync, which are the standard traits used for error types.

While this has necessitated the removal of a number of APIs, the resulting PyErr type should now be much more easier to work with. The following sections list the changes in detail and how to migrate to the new APIs.

PyErr::new and PyErr::from_type now require Send + Sync for their argument

Click to expand

For most uses no change will be needed. If you are trying to construct PyErr from a value that is not Send + Sync, you will need to first create the Python object and then use PyErr::from_instance.

Similarly, any types which implemented PyErrArguments will now need to be Send + Sync.

PyErr’s contents are now private

Click to expand

It is no longer possible to access the fields .ptype, .pvalue and .ptraceback of a PyErr. You should instead now use the new methods PyErr::ptype, PyErr::pvalue and PyErr::ptraceback.

PyErrValue and PyErr::from_value have been removed

Click to expand

As these were part the internals of PyErr which have been reworked, these APIs no longer exist.

If you used this API, it is recommended to use PyException::new_err (see the section on Exception types).

Into<PyResult<T>> for PyErr has been removed

Click to expand

This implementation was redundant. Just construct the Result::Err variant directly.

之前：

let result: PyResult<()> = PyErr::new::<TypeError, _>("error message").into();

After (also using the new reworked exception types; see the following section):

use pyo3::{PyResult, exceptions::PyTypeError};
let result: PyResult<()> = Err(PyTypeError::new_err("error message"));

Exception types have been reworked

Click to expand

Previously exception types were zero-sized marker types purely used to construct PyErr. In PyO3 0.12, these types have been replaced with full definitions and are usable in the same way as PyAny, PyDict etc. This makes it possible to interact with Python exception objects.

The new types also have names starting with the “Py” prefix. For example, before:

let err: PyErr = TypeError::py_err("error message");

After:

use pyo3::{PyErr, PyResult, Python, type_object::PyTypeObject};
use pyo3::exceptions::{PyBaseException, PyTypeError};
Python::with_gil(|py| -> PyResult<()> {
let err: PyErr = PyTypeError::new_err("error message");

// Uses Display for PyErr, new for PyO3 0.12
assert_eq!(err.to_string(), "TypeError: error message");

// Now possible to interact with exception instances, new for PyO3 0.12
let instance: &PyBaseException = err.instance(py);
assert_eq!(
    instance.getattr("__class__")?,
    PyTypeError::type_object(py).as_ref()
);
Ok(())
}).unwrap();

FromPy has been removed

Click to expand

To simplify the PyO3 conversion traits, the FromPy trait has been removed. Previously there were two ways to define the to-Python conversion for a type: FromPy<T> for PyObject and IntoPy<PyObject> for T.

Now there is only one way to define the conversion, IntoPy, so downstream crates may need to adjust accordingly.

之前：

use pyo3::prelude::*;
struct MyPyObjectWrapper(PyObject);

impl FromPy<MyPyObjectWrapper> for PyObject {
    fn from_py(other: MyPyObjectWrapper, _py: Python<'_>) -> Self {
        other.0
    }
}

After

use pyo3::prelude::*;
#[allow(dead_code)]
struct MyPyObjectWrapper(PyObject);

#[allow(deprecated)]
impl IntoPy<PyObject> for MyPyObjectWrapper {
    fn into_py(self, _py: Python<'_>) -> PyObject {
        self.0
    }
}

Similarly, code which was using the FromPy trait can be trivially rewritten to use IntoPy.

之前：

use pyo3::prelude::*;
Python::with_gil(|py| {
let obj = PyObject::from_py(1.234, py);
})

After:

#![allow(deprecated)]
use pyo3::prelude::*;
Python::with_gil(|py| {
let obj: PyObject = 1.234.into_py(py);
})

PyObject is now a type alias of Py<PyAny>

Click to expand

This should change very little from a usage perspective. If you implemented traits for both PyObject and Py<T>, you may find you can just remove the PyObject implementation.

AsPyRef has been removed

Click to expand

As PyObject has been changed to be just a type alias, the only remaining implementor of AsPyRef was Py<T>. This removed the need for a trait, so the AsPyRef::as_ref method has been moved to Py::as_ref.

This should require no code changes except removing use pyo3::AsPyRef for code which did not use pyo3::prelude::*.

之前：

use pyo3::{AsPyRef, Py, types::PyList};
pyo3::Python::with_gil(|py| {
let list_py: Py<PyList> = PyList::empty(py).into();
let list_ref: &PyList = list_py.as_ref(py);
})

After:

use pyo3::{Py, types::PyList};
pyo3::Python::with_gil(|py| {
let list_py: Py<PyList> = PyList::empty(py).into();
let list_ref: &PyList = list_py.as_ref(py);
})

from 0.10.* to 0.11

Stable Rust

Click to expand

PyO3 now supports the stable Rust toolchain. The minimum required version is 1.39.0.

#[pyclass] structs must now be Send or unsendable

Click to expand

Because #[pyclass] structs can be sent between threads by the Python interpreter, they must implement Send or declared as unsendable (by #[pyclass(unsendable)]). Note that unsendable is added in PyO3 0.11.1 and Send is always required in PyO3 0.11.0.

This may “break” some code which previously was accepted, even though it could be unsound. There can be two fixes:

  1. If you think that your #[pyclass] actually must be Sendable, then let’s implement Send. A common, safer way is using thread-safe types. E.g., Arc instead of Rc, Mutex instead of RefCell, and Box<dyn Send + T> instead of Box<dyn T>.

    之前：

    use pyo3::prelude::*;
use std::rc::Rc;
use std::cell::RefCell;

#[pyclass]
struct NotThreadSafe {
    shared_bools: Rc<RefCell<Vec<bool>>>,
    closure: Box<dyn Fn()>,
}

    After:

    #![allow(dead_code)]
use pyo3::prelude::*;
use std::sync::{Arc, Mutex};

#[pyclass]
struct ThreadSafe {
    shared_bools: Arc<Mutex<Vec<bool>>>,
    closure: Box<dyn Fn() + Send>,
}

    In situations where you cannot change your #[pyclass] to automatically implement Send (e.g., when it contains a raw pointer), you can use unsafe impl Send. In such cases, care should be taken to ensure the struct is actually thread safe. See the Rustonomicon for more.

  2. If you think that your #[pyclass] should not be accessed by another thread, you can use unsendable flag. A class marked with unsendable panics when accessed by another thread, making it thread-safe to expose an unsendable object to the Python interpreter.

    之前：

    use pyo3::prelude::*;

#[pyclass]
struct Unsendable {
    pointers: Vec<*mut std::ffi::c_char>,
}

    After:

    #![allow(dead_code)]
use pyo3::prelude::*;

#[pyclass(unsendable)]
struct Unsendable {
    pointers: Vec<*mut std::ffi::c_char>,
}

All PyObject and Py<T> methods now take Python as an argument

Click to expand

Previously, a few methods such as Object::get_refcnt did not take Python as an argument (to ensure that the Python GIL was held by the current thread). Technically, this was not sound. To migrate, just pass a py argument to any calls to these methods.

之前：

pyo3::Python::attach(|py| {
py.None().get_refcnt();
})

After:

pyo3::Python::attach(|py| {
py.None().get_refcnt(py);
})

from 0.9.* to 0.10

ObjectProtocol is removed

Click to expand

All methods are moved to PyAny. And since now all native types (e.g., PyList) implements Deref<Target=PyAny>, all you need to do is remove ObjectProtocol from your code. Or if you use ObjectProtocol by use pyo3::prelude::*, you have to do nothing.

之前：

use pyo3::ObjectProtocol;

pyo3::Python::with_gil(|py| {
let obj = py.eval("lambda: 'Hi :)'", None, None).unwrap();
let hi: &pyo3::types::PyString = obj.call0().unwrap().downcast().unwrap();
assert_eq!(hi.len().unwrap(), 5);
})

After:

pyo3::Python::with_gil(|py| {
let obj = py.eval("lambda: 'Hi :)'", None, None).unwrap();
let hi: &pyo3::types::PyString = obj.call0().unwrap().downcast().unwrap();
assert_eq!(hi.len().unwrap(), 5);
})

No #![feature(specialization)] in user code

Click to expand

While PyO3 itself still requires specialization and nightly Rust, now you don’t have to use #![feature(specialization)] in your crate.

from 0.8.* to 0.9

#[new] interface

Click to expand

PyRawObject is now removed and our syntax for constructors has changed.

之前：

#[pyclass]
struct MyClass {}

#[pymethods]
impl MyClass {
    #[new]
    fn new(obj: &PyRawObject) {
        obj.init(MyClass {})
    }
}

After:

use pyo3::prelude::*;
#[pyclass]
struct MyClass {}

#[pymethods]
impl MyClass {
    #[new]
    fn new() -> Self {
        MyClass {}
    }
}

Basically you can return Self or Result<Self> directly. For more, see the constructor section of this guide.

PyCell

Click to expand

PyO3 0.9 introduces PyCell, which is a RefCell-like object wrapper for ensuring Rust’s rules regarding aliasing of references are upheld. For more detail, see the Rust Book’s section on Rust’s rules of references

For #[pymethods] or #[pyfunction]s, your existing code should continue to work without any change. Python exceptions will automatically be raised when your functions are used in a way which breaks Rust’s rules of references.

Here is an example.

use pyo3::prelude::*;

#[pyclass]
struct Names {
    names: Vec<String>,
}

#[pymethods]
impl Names {
    #[new]
    fn new() -> Self {
        Names { names: vec![] }
    }
    fn merge(&mut self, other: &mut Names) {
        self.names.append(&mut other.names)
    }
}
Python::attach(|py| {
    let names = Py::new(py, Names::new()).unwrap();
    pyo3::py_run!(py, names, r"
    try:
       names.merge(names)
       assert False, 'Unreachable'
    except RuntimeError as e:
       assert str(e) == 'Already borrowed'
    ");
})

Names has a merge method, which takes &mut self and another argument of type &mut Self. Given this #[pyclass], calling names.merge(names) in Python raises a PyBorrowMutError exception, since it requires two mutable borrows of names.

However, for #[pyproto] and some functions, you need to manually fix the code.

Object creation

In 0.8 object creation was done with PyRef::new and PyRefMut::new. In 0.9 these have both been removed. To upgrade code, please use PyCell::new instead. If you need PyRef or PyRefMut, just call .borrow() or .borrow_mut() on the newly-created PyCell.

之前：

use pyo3::prelude::*;
#[pyclass]
struct MyClass {}
Python::with_gil(|py| {
let obj_ref = PyRef::new(py, MyClass {}).unwrap();
})

After:

use pyo3::prelude::*;
#[pyclass]
struct MyClass {}
Python::with_gil(|py| {
let obj = PyCell::new(py, MyClass {}).unwrap();
let obj_ref = obj.borrow();
})

Object extraction

For PyClass types T, &T and &mut T no longer have FromPyObject implementations. Instead you should extract PyRef<T> or PyRefMut<T>, respectively. If T implements Clone, you can extract T itself. In addition, you can also extract &PyCell<T>, though you rarely need it.

之前：

let obj: &PyAny = create_obj();
let obj_ref: &MyClass = obj.extract().unwrap();
let obj_ref_mut: &mut MyClass = obj.extract().unwrap();

After:

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
#[pyclass] #[derive(Clone)] struct MyClass {}
#[pymethods] impl MyClass { #[new]fn new() -> Self { MyClass {} }}
Python::with_gil(|py| {
let typeobj = py.get_type::<MyClass>();
let d = [("c", typeobj)].into_py_dict(py);
let create_obj = || py.eval("c()", None, Some(d)).unwrap();
let obj: &PyAny = create_obj();
let obj_cell: &PyCell<MyClass> = obj.extract().unwrap();
let obj_cloned: MyClass = obj.extract().unwrap(); // extracted by cloning the object
{
    let obj_ref: PyRef<'_, MyClass> = obj.extract().unwrap();
    // we need to drop obj_ref before we can extract a PyRefMut due to Rust's rules of references
}
let obj_ref_mut: PyRefMut<'_, MyClass> = obj.extract().unwrap();
})

#[pyproto]

Most of the arguments to methods in #[pyproto] impls require a FromPyObject implementation. So if your protocol methods take &T or &mut T (where T: PyClass), please use PyRef or PyRefMut instead.

之前：

use pyo3::prelude::*;
use pyo3::class::PySequenceProtocol;
#[pyclass]
struct ByteSequence {
    elements: Vec<u8>,
}
#[pyproto]
impl PySequenceProtocol for ByteSequence {
    fn __concat__(&self, other: &Self) -> PyResult<Self> {
        let mut elements = self.elements.clone();
        elements.extend_from_slice(&other.elements);
        Ok(Self { elements })
    }
}

After:

use pyo3::prelude::*;
use pyo3::class::PySequenceProtocol;
#[pyclass]
struct ByteSequence {
    elements: Vec<u8>,
}
#[pyproto]
impl PySequenceProtocol for ByteSequence {
    fn __concat__(&self, other: PyRef<'p, Self>) -> PyResult<Self> {
        let mut elements = self.elements.clone();
        elements.extend_from_slice(&other.elements);
        Ok(Self { elements })
    }
}

在 Python 中使用帶有 trait bounds 的 Rust 函式

PyO3 讓某些函式與類別可輕鬆在 Rust 與 Python 間轉換（見轉換表）。不過，當 Rust 程式碼需要以特定 trait 實作作為參數時，轉換就未必那麼直接。

本教學說明如何將以 trait 作為參數的 Rust 函式轉換成可在 Python 中使用，並讓具備相同方法的 Python 類別能夠配合使用。

為什麼這很有用？

優點

  • 讓你的 Rust 程式碼可供 Python 使用者使用
  • 在 Rust 中編寫複雜演算法並受惠於借用檢查器

缺點

  • 不如原生 Rust 快（需要進行型別轉換，且部分程式碼在 Python 端執行）
  • 你需要調整程式碼以便對外暴露

範例

讓我們以一個基本範例開始：一個在給定模型上運作的最佳化求解器實作。

假設我們有一個 solve 函式，會在模型上運作並改變其狀態。該函式的參數可以是任何實作 Model trait 的模型：

#![allow(dead_code)]
pub trait Model {
    fn set_variables(&mut self, inputs: &Vec<f64>);
    fn compute(&mut self);
    fn get_results(&self) -> Vec<f64>;
}

pub fn solve<T: Model>(model: &mut T) {
    println!("Magic solver that mutates the model into a resolved state");
}

假設我們有以下限制：

  • 我們不能修改該程式碼，因為它已用在許多 Rust 模型上。
  • 我們也有許多 Python 模型無法被求解，因為該求解器在那個語言中不可用。

由於一切都已在 Rust 中可用，把它改寫成 Python 既繁瑣又容易出錯。

那我們該如何透過 PyO3 將此求解器提供給 Python 使用？

為 Python 類別實作 trait bounds

如果某個 Python 類別實作了與 Model trait 相同的三個方法，直覺上它應該能配合此求解器。但實際上無法傳入 Py<PyAny>，因為它沒有實作 Rust 的 trait（即便 Python 模型具備所需方法也一樣）。

為了實作該 trait，我們必須在 Rust 端撰寫一個包裝器來呼叫 Python 模型。方法簽章必須與 trait 相同，因為 Rust trait 不能為了讓程式碼可在 Python 使用而被修改。

以下是我們想要對外暴露的 Python 模型，已包含所有必要方法：

class Model:
    def set_variables(self, inputs):
        self.inputs = inputs
    def compute(self):
        self.results = [elt**2 - 3 for elt in self.inputs]
    def get_results(self):
        return self.results

以下包裝器會從 Rust 呼叫 Python 模型，使用結構來持有該模型的 PyAny 物件：

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;

pub trait Model {
  fn set_variables(&mut self, inputs: &Vec<f64>);
  fn compute(&mut self);
  fn get_results(&self) -> Vec<f64>;
}

struct UserModel {
    model: Py<PyAny>,
}

impl Model for UserModel {
    fn set_variables(&mut self, var: &Vec<f64>) {
        println!("Rust calling Python to set the variables");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("set_variables", (PyList::new(py, var).unwrap(),), None)
                .unwrap();
        })
    }

    fn get_results(&self) -> Vec<f64> {
        println!("Rust calling Python to get the results");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("get_results", (), None)
                .unwrap()
                .extract()
                .unwrap()
        })
    }

    fn compute(&mut self) {
        println!("Rust calling Python to perform the computation");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("compute", (), None)
                .unwrap();
        })
    }
}

上述部分完成後，讓我們將模型包裝器對外暴露給 Python。加上 PyO3 標註並新增建構子：

#![allow(dead_code)]
fn main() {}
pub trait Model {
  fn set_variables(&mut self, inputs: &Vec<f64>);
  fn compute(&mut self);
  fn get_results(&self) -> Vec<f64>;
}
use pyo3::prelude::*;

#[pyclass]
struct UserModel {
    model: Py<PyAny>,
}

#[pymethods]
impl UserModel {
    #[new]
    pub fn new(model: Py<PyAny>) -> Self {
        UserModel { model }
    }
}

#[pymodule]
mod trait_exposure {
    #[pymodule_export]
    use super::UserModel;
}

接著在 trait 實作上加入 PyO3 標註：

#[pymethods]
impl Model for UserModel {
    // 前述的 trait 實作
}

然而，前述程式碼無法編譯。錯誤訊息如下：error: #[pymethods] cannot be used on trait impl blocks

這很可惜！不過，我們可以再寫一個包裝器來直接呼叫這些函式。該包裝器也會負責在 Python 與 Rust 之間進行型別轉換。

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;

pub trait Model {
  fn set_variables(&mut self, inputs: &Vec<f64>);
  fn compute(&mut self);
  fn get_results(&self) -> Vec<f64>;
}

#[pyclass]
struct UserModel {
    model: Py<PyAny>,
}

impl Model for UserModel {
 fn set_variables(&mut self, var: &Vec<f64>) {
     println!("Rust calling Python to set the variables");
     Python::attach(|py| {
         self.model.bind(py)
             .call_method("set_variables", (PyList::new(py, var).unwrap(),), None)
             .unwrap();
     })
 }

 fn get_results(&self) -> Vec<f64> {
     println!("Rust calling Python to get the results");
     Python::attach(|py| {
         self.model
             .bind(py)
             .call_method("get_results", (), None)
             .unwrap()
             .extract()
             .unwrap()
     })
 }

 fn compute(&mut self) {
     println!("Rust calling Python to perform the computation");
     Python::attach(|py| {
         self.model
             .bind(py)
             .call_method("compute", (), None)
             .unwrap();
     })

 }
}

#[pymethods]
impl UserModel {
    pub fn set_variables(&mut self, var: Vec<f64>) {
        println!("從 Python 呼叫 Rust 來設定變數");
        Model::set_variables(self, &var)
    }

    pub fn get_results(&mut self) -> Vec<f64> {
        println!("從 Python 呼叫 Rust 取得結果");
        Model::get_results(self)
    }

    pub fn compute(&mut self) {
        println!("從 Python 呼叫 Rust 執行計算");
        Model::compute(self)
    }
}

這個包裝器會處理 PyO3 要求與 trait 之間的型別轉換。為了符合 PyO3 的要求，此包裝器必須：

  • 回傳 PyResult 型別的物件
  • 在方法簽章中僅使用值，而非參照

讓我們執行這個 Python 檔案：

class Model:
    def set_variables(self, inputs):
        self.inputs = inputs
    def compute(self):
        self.results = [elt**2 - 3 for elt in self.inputs]
    def get_results(self):
        return self.results

if __name__=="__main__":
  import trait_exposure

  myModel = Model()
  my_rust_model = trait_exposure.UserModel(myModel)
  my_rust_model.set_variables([2.0])
  print("Print value from Python: ", myModel.inputs)
  my_rust_model.compute()
  print("Print value from Python through Rust: ", my_rust_model.get_results())
  print("Print value directly from Python: ", myModel.get_results())

This outputs:

Set variables from Python calling Rust
Set variables from Rust calling Python
Print value from Python:  [2.0]
Compute from Python calling Rust
Compute from Rust calling Python
Get results from Python calling Rust
Get results from Rust calling Python
Print value from Python through Rust:  [1.0]
Print value directly from Python:  [1.0]

We have now successfully exposed a Rust model that implements the Model trait to Python!

We will now expose the solve function, but before, let’s talk about types errors.

Type errors in Python

What happens if you have type errors when using Python and how can you improve the error messages?

Wrong types in Python function arguments

Let’s assume in the first case that you will use in your Python file my_rust_model.set_variables(2.0) instead of my_rust_model.set_variables([2.0]).

The Rust signature expects a vector, which corresponds to a list in Python. What happens if instead of a vector, we pass a single value ?

At the execution of Python, we get :

File "main.py", line 15, in <module>
   my_rust_model.set_variables(2)
TypeError

It is a type error and Python points to it, so it’s easy to identify and solve.

Wrong types in Python method signatures

Let’s assume now that the return type of one of the methods of our Model class is wrong, for example the get_results method that is expected to return a Vec<f64> in Rust, a list in Python.

class Model:
    def set_variables(self, inputs):
        self.inputs = inputs
    def compute(self):
        self.results = [elt**2 -3 for elt in self.inputs]
    def get_results(self):
        return self.results[0]
        #return self.results <-- this is the expected output

This call results in the following panic:

pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: PyErr { type: Py(0x10dcf79f0, PhantomData) }

This error code is not helpful for a Python user that does not know anything about Rust, or someone that does not know PyO3 was used to interface the Rust code.

However, as we are responsible for making the Rust code available to Python, we can do something about it.

The issue is that we called unwrap anywhere we could, and therefore any panic from PyO3 will be directly forwarded to the end user.

Let’s modify the code performing the type conversion to give a helpful error message to the Python user:

We used in our get_results method the following call that performs the type conversion:

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;

pub trait Model {
  fn set_variables(&mut self, inputs: &Vec<f64>);
  fn compute(&mut self);
  fn get_results(&self) -> Vec<f64>;
}

#[pyclass]
struct UserModel {
    model: Py<PyAny>,
}

impl Model for UserModel {
    fn get_results(&self) -> Vec<f64> {
        println!("Rust calling Python to get the results");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("get_results", (), None)
                .unwrap()
                .extract()
                .unwrap()
        })
    }
    fn set_variables(&mut self, var: &Vec<f64>) {
        println!("Rust calling Python to set the variables");
        Python::attach(|py| {
            self.model.bind(py)
                .call_method("set_variables", (PyList::new(py, var).unwrap(),), None)
                .unwrap();
        })
    }

    fn compute(&mut self) {
        println!("Rust calling Python to perform the computation");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("compute", (), None)
                .unwrap();
        })
    }
}

Let’s break it down in order to perform better error handling:

#![allow(dead_code)]
use pyo3::prelude::*;
use pyo3::types::PyList;

pub trait Model {
  fn set_variables(&mut self, inputs: &Vec<f64>);
  fn compute(&mut self);
  fn get_results(&self) -> Vec<f64>;
}

#[pyclass]
struct UserModel {
    model: Py<PyAny>,
}

impl Model for UserModel {
    fn get_results(&self) -> Vec<f64> {
        println!("Get results from Rust calling Python");
        Python::attach(|py| {
            let py_result: Bound<'_, PyAny> = self
                .model
                .bind(py)
                .call_method("get_results", (), None)
                .unwrap();

            if py_result.get_type().name().unwrap() != "list" {
                panic!(
                    "Expected a list for the get_results() method signature, got {}",
                    py_result.get_type().name().unwrap()
                );
            }
            py_result.extract()
        })
        .unwrap()
    }
    fn set_variables(&mut self, var: &Vec<f64>) {
        println!("Rust calling Python to set the variables");
        Python::attach(|py| {
            let py_model = self.model.bind(py)
                .call_method("set_variables", (PyList::new(py, var).unwrap(),), None)
                .unwrap();
        })
    }

    fn compute(&mut self) {
        println!("Rust calling Python to perform the computation");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("compute", (), None)
                .unwrap();
        })
    }
}

By doing so, you catch the result of the Python computation and check its type in order to be able to deliver a better error message before performing the unwrapping.

Of course, it does not cover all the possible wrong outputs: the user could return a list of strings instead of a list of floats. In this case, a runtime panic would still occur due to PyO3, but with an error message much more difficult to decipher for non-rust user.

It is up to the developer exposing the rust code to decide how much effort to invest into Python type error handling and improved error messages.

The final code

Now let’s expose the solve() function to make it available from Python.

It is not possible to directly expose the solve function to Python, as the type conversion cannot be performed. It requires an object implementing the Model trait as input.

However, the UserModel already implements this trait. Because of this, we can write a function wrapper that takes the UserModel--which has already been exposed to Python–as an argument in order to call the core function solve.

It is also required to make the struct public.

#![allow(dead_code)]
fn main() {}
use pyo3::prelude::*;
use pyo3::types::PyList;

pub trait Model {
    fn set_variables(&mut self, var: &Vec<f64>);
    fn get_results(&self) -> Vec<f64>;
    fn compute(&mut self);
}

pub fn solve<T: Model>(model: &mut T) {
    println!("Magic solver that mutates the model into a resolved state");
}

#[pyfunction]
#[pyo3(name = "solve")]
pub fn solve_wrapper(model: &mut UserModel) {
    solve(model);
}

#[pyclass]
pub struct UserModel {
    model: Py<PyAny>,
}

#[pymethods]
impl UserModel {
    #[new]
    pub fn new(model: Py<PyAny>) -> Self {
        UserModel { model }
    }

    pub fn set_variables(&mut self, var: Vec<f64>) {
        println!("從 Python 呼叫 Rust 來設定變數");
        Model::set_variables(self, &var)
    }

    pub fn get_results(&mut self) -> Vec<f64> {
        println!("從 Python 呼叫 Rust 取得結果");
        Model::get_results(self)
    }

    pub fn compute(&mut self) {
        Model::compute(self)
    }
}

#[pymodule]
mod trait_exposure {
    #[pymodule_export]
    use super::{UserModel, solve_wrapper};
}

impl Model for UserModel {
    fn set_variables(&mut self, var: &Vec<f64>) {
        println!("Rust calling Python to set the variables");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("set_variables", (PyList::new(py, var).unwrap(),), None)
                .unwrap();
        })
    }

    fn get_results(&self) -> Vec<f64> {
        println!("Get results from Rust calling Python");
        Python::attach(|py| {
            let py_result: Bound<'_, PyAny> = self
                .model
                .bind(py)
                .call_method("get_results", (), None)
                .unwrap();

            if py_result.get_type().name().unwrap() != "list" {
                panic!(
                    "Expected a list for the get_results() method signature, got {}",
                    py_result.get_type().name().unwrap()
                );
            }
            py_result.extract()
        })
        .unwrap()
    }

    fn compute(&mut self) {
        println!("Rust calling Python to perform the computation");
        Python::attach(|py| {
            self.model
                .bind(py)
                .call_method("compute", (), None)
                .unwrap();
        })
    }
}

為你的 Python 套件提供型別與 IDE 提示

PyO3 提供易用介面，可用 Rust 撰寫原生 Python 函式庫。搭配的 Maturin 讓你能將其建置並發布為套件。然而為了更好的使用體驗，Python 函式庫應為所有公開項目提供型別提示與文件，讓 IDE 能在開發時顯示，並讓 mypy 等型別分析工具正確驗證程式碼。

目前最佳的解法是手動維護 *.pyi 檔案，並與套件一同發布。

PyO3 正在進行自動產生的工作。請見型別 stub 產生文件以了解目前的自動產生狀態。

pyi 檔案介紹

pyi 檔案（Python Interface 的縮寫）在相關文件中多被稱為「stub 檔案」。對其定義可參考舊版 MyPy 文件：

Stub 檔案只包含模組公開介面的描述，不包含任何實作。

也可參考官方 Python typing 文件中關於 type stubs 的完整說明。

多數 Python 開發者可能在對內建型別使用 IDE 的「前往定義」功能時就已遇過它們。例如，幾個標準例外的定義如下：

class BaseException(object):
    args: Tuple[Any, ...]
    __cause__: BaseException | None
    __context__: BaseException | None
    __suppress_context__: bool
    __traceback__: TracebackType | None
    def __init__(self, *args: object) -> None: ...
    def __str__(self) -> str: ...
    def __repr__(self) -> str: ...
    def with_traceback(self: _TBE, tb: TracebackType | None) -> _TBE: ...

class SystemExit(BaseException):
    code: int

class Exception(BaseException): ...

class StopIteration(Exception):
    value: Any

如你所見，這些並非包含實作的完整定義，而只是介面描述。這通常已足以滿足函式庫使用者的需求。

PEP 有什麼說法？

在撰寫本文件時，pyi 檔案在四份 PEP 中被引用。

PEP8 - Python 程式碼風格指南 - #Function Annotations（最後一點）建議所有第三方函式庫作者提供 stub 檔案，作為型別檢查工具對套件的知識來源。

（……）預期第三方函式庫的使用者可能會對這些套件執行型別檢查。為此，PEP 484 建議使用 stub 檔案：型別檢查器會優先讀取 .pyi 檔案而不是對應的 .py 檔案。（……）

PEP484 - Type Hints - #Stub Files 對 stub 檔案的定義如下。

Stub 檔案是只提供給型別檢查器使用、而非執行階段使用的型別提示檔案。

其中包含對 stub 檔案的規範（強烈建議閱讀，因為至少有一項是一般 Python 程式碼不會用到的）以及關於存放位置的一般資訊。

PEP561 - Distributing and Packaging Type Information 詳細描述如何建置能啟用型別檢查的套件，特別是如何散布 stub 檔案以供型別檢查器使用。

PEP560 - Core support for typing module and generic types 描述了 Python 型別系統如何在內部支援泛型，包括執行階段行為與靜態型別檢查器的整合。

該怎麼做？

PEP561 提出三種散布型別資訊的方式：

  • inline - 型別直接放在原始碼（py）檔案中；
  • separate package with stub files - 型別放在 pyi 檔案中，並以獨立套件發布；
  • in-package stub files - 型別放在 pyi 檔案中，並與原始碼同套件發布。

第一種方式在 PyO3 中較困難，因為我們沒有 py 檔案。待完成研究與必要改動後，本文件將更新。

第二種方式很容易完成，而且可與主程式庫程式碼完全分離。帶有 stub 檔案的軟體包範例可在 PEP561 參考資料中找到：Stub 軟體包儲存庫

第三種方式如下所述。

在 PyO3/Maturin 建置套件中包含 pyi 檔案

當原始碼與 stub 檔案位於同一套件時，應該把它們放在彼此相鄰的位置。我們需要能用 Maturin 達成這點的方法。此外，為了標示套件具有型別資訊，我們需要在套件中新增一個名為 py.typed 的空檔案。

若你沒有其他 Python 檔案

若除了 pyi 之外不需要加入其他 Python 檔案，Maturin 提供了方法替你完成大部分工作。如同 Maturin 指南 所述，你只需在專案根目錄建立名為 <module_name>.pyi 的 stub 檔案，其餘由 Maturin 處理。

my-rust-project/
├── Cargo.toml
├── my_project.pyi  # <<< add type stubs for Rust functions in the my_project module here
├── pyproject.toml
└── src
    └── lib.rs

如需 pyi 範例檔案，請見 my_project.pyi 內容 一節。

If you need other Python files

If you need to add other Python files apart from pyi to the package, you can do it also, but that requires some more work. Maturin provides an easy way to add files to a package (documentation). You just need to create a folder with the name of your module next to the Cargo.toml file (for customization see documentation linked above).

The folder structure would be:

my-project
├── Cargo.toml
├── my_project
│   ├── __init__.py
│   ├── my_project.pyi
│   ├── other_python_file.py
│   └── py.typed
├── pyproject.toml
├── Readme.md
└── src
    └── lib.rs

Let’s go a little bit more into detail regarding the files inside the package folder.

__init__.py content

As we now specify our own package content, we have to provide the __init__.py file, so the folder is treated as a package and we can import things from it. We can always use the same content that Maturin creates for us if we do not specify a Python source folder. For PyO3 bindings it would be:

from .my_project import *

That way everything that is exposed by our native module can be imported directly from the package.

py.typed requirement

As stated in PEP561:

Package maintainers who wish to support type checking of their code MUST add a marker file named py.typed to their package supporting typing. This marker applies recursively: if a top-level package includes it, all its sub-packages MUST support type checking as well.

If we do not include that file, some IDEs might still use our pyi files to show hints, but the type checkers might not. MyPy will raise an error in this situation:

error: Skipping analyzing "my_project": found module but no type hints or library stubs

The file is just a marker file, so it should be empty.

my_project.pyi content

Our module stub file. This document does not aim at describing how to write them, since you can find a lot of documentation on it, starting from the already quoted PEP484.

The example can look like this:

class Car:
    """
    A class representing a car.

    :param body_type: the name of body type, e.g. hatchback, sedan
    :param horsepower: power of the engine in horsepower
    """
    def __init__(self, body_type: str, horsepower: int) -> None: ...

    @classmethod
    def from_unique_name(cls, name: str) -> 'Car':
        """
        Creates a Car based on unique name

        :param name: model name of a car to be created
        :return: a Car instance with default data
        """

    def best_color(self) -> str:
        """
        Gets the best color for the car.

        :return: the name of the color our great algorithm thinks is the best for this car
        """

Supporting Generics

Type annotations can also be made generic in Python. They are useful for working with different types while maintaining type safety. Usually, generic classes inherit from the typing.Generic metaclass.

Take for example the following .pyi file that specifies a Car that can accept multiple types of wheels:

from typing import Generic, TypeVar

W = TypeVar('W')

class Car(Generic[W]):
    def __init__(self, wheels: list[W]) -> None: ...

    def get_wheels(self) -> list[W]: ...

    def change_wheel(self, wheel_number: int, wheel: W) -> None: ...

This way, the end-user can specify the type with variables such as truck: Car[SteelWheel] = ... and f1_car: Car[AlloyWheel] = ....

There is also a special syntax for specifying generic types in Python 3.12+:

class Car[W]:
    def __init__(self, wheels: list[W]) -> None: ...

    def get_wheels(self) -> list[W]: ...

Runtime Behaviour

Stub files (pyi) are only useful for static type checkers and ignored at runtime. Therefore, PyO3 classes do not inherit from typing.Generic even if specified in the stub files.

This can cause some runtime issues, as annotating a variable like f1_car: Car[AlloyWheel] = ... can make Python call magic methods that are not defined.

To overcome this limitation, implementers can pass the generic parameter to pyclass in Rust:

#[pyclass(generic)]

Advanced Users

#[pyclass(generic)] implements a very simple runtime behavior that accepts any generic argument. Advanced users can opt to manually implement __class_getitem__ for the generic class to have more control.

impl MyClass {
    #[classmethod]
    #[pyo3(signature = (key, /))]
    pub fn __class_getitem__(
        cls: &Bound<'_, PyType>,
        key: &Bound<'_, PyAny>,
    ) -> PyResult<Py<PyAny>> {
        /* implementation details */
    }
}

Note that pyo3::types::PyGenericAlias can be helpful when implementing __class_getitem__ as it can create types.GenericAlias objects from Rust.

變更日誌

All notable changes to this project will be documented in this file. For help with updating to new PyO3 versions, please see the migration guide.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

To see unreleased changes, please see the CHANGELOG on the main branch guide.

0.28.2 - 2026-02-18

Fixed

  • Fix complex enum __qualname__ not using python name #5815
  • Fix FFI definition PyType_GetTypeDataSize (was incorrectly named PyObject_GetTypeDataSize). #5819
  • Fix memory corruption when subclassing native types with abi3 feature on Python 3.12+ (newly enabled in PyO3 0.28.0). #5823

0.28.1 - 2026-02-14

Fixed

  • Fix *args / **kwargs support in experimental-async feature (regressed in 0.28.0). #5771
  • Fix clippy::declare_interior_mutable_const warning inside #[pyclass] generated code on enums. #5772
  • Fix ambiguous_associated_items compilation error when deriving FromPyObject or using #[pyclass(from_py_object)] macro on enums with Error variant. #5784
  • Fix __qualname__ for complex #[pyclass] enum variants to include the enum name. #5796
  • Fix missing std::sync::atomic::Ordering import for targets without atomic64. #5808

0.28.0 - 2026-02-01

Packaging

  • Bump MSRV to Rust 1.83. #5531
  • Bump minimum supported quote version to 1.0.37. #5531
  • Bump supported GraalPy version to 25.0. #5542
  • Drop memoffset dependency. #5545
  • Support for free-threaded Python is now opt-out rather than opt-in. #5564
  • Bump target-lexicon dependency to 0.13.3. #5571
  • Drop indoc and unindent dependencies. #5608

Added

  • Add __init__ support in #[pymethods]. #4951
  • Expose PySuper on PyPy, GraalPy and ABI3 #4951
  • Add PyString::from_fmt and py_format! macro. #5199
  • Add #[pyclass(new = "from_fields")] option. #5421
  • Add pyo3::buffer::PyUntypedBuffer, a type-erased form of PyBuffer<T>. #5458
  • Add PyBytes::new_with_writer #5517
  • Add PyClass::NAME. #5579
  • Add pyo3_build_config::add_libpython_rpath_link_args. #5624
  • Add PyBackedStr::clone_ref and PyBackedBytes::clone_ref methods. #5654
  • Add PyCapsule::new_with_pointer and PyCapsule::new_with_pointer_and_destructor for creating capsules with raw pointers. #5689
  • Add #[deleter] attribute to implement property deleters in #[methods]. #5699
  • Add IntoPyObject and FromPyObject implementations for uuid::NonNilUuid. #5707
  • Add PyBackedStr::as_str and PyBackedStr::as_py_str methods. #5723
  • Add support for subclassing native types (PyDict, exceptions, …) when building for abi3 on Python 3.12+. #5733
  • Add support for subclassing PyList when building for Python 3.12+. #5734
  • FFI definitions:
    • Add FFI definitions PyEval_GetFrameBuiltins, PyEval_GetFrameGlobals and PyEval_GetFrameLocals on Python 3.13 and up. #5590
    • Add FFI definitions PyObject_New, PyObject_NewVar, PyObject_GC_Resize, PyObject_GC_New, and PyObject_GC_NewVar. #5591
    • Added FFI definitions and an unsafe Rust API wrapping Py_BEGIN_CRITICAL_SECTION_MUTEX and Py_BEGIN_CRITICAL_SECTION_MUTEX2. #5642
    • Add FFI definition PyDict_GetItemStringRef on Python 3.13 and up. #5659
    • Add FFI definition PyIter_NextItem on Python 3.14 and up, and compat::PyIter_NextItem for older versions. #5661
    • Add FFI definitions PyThreadState_GetInterpreter and PyThreadState_GetID on Python 3.9+, PyThreadState_EnterTracing and PyThreadState_LeaveTracing on Python 3.11+, PyThreadState_GetUnchecked on Python 3.13+, and compat::PyThreadState_GetUnchecked. #5711
    • Add FFI definitions PyImport_ImportModuleAttr and PyImport_ImportModuleAttrString on Python 3.14+. #5737
    • Add FFI definitions for the PyABIInfo and PyModExport APIs available in Python 3.15. #5746
  • experimental-inspect:
    • Emit base classes. #5331
    • Emit @typing.final on final classes. #5552
    • Generate nested classes for complex enums. #5708
    • Emit async keyword for async functions. #5731

Changed

  • Call sys.unraisablehook instead of PyErr_Print if panicking on null FFI pointer in Bound, Borrowed and Py constructors. #5496
  • Use PEP-489 multi-phase initialization for #[pymodule]. #5525
  • Deprecate implicit by-value implementation of FromPyObject for #[pyclass]. #5550
  • Deprecate PyTypeInfo::NAME and PyTypeInfo::MODULE. #5579
  • Deprecate Py<T>::from_{owned,borrowed}[or_{err,opt}] constructors from raw pointer. #5585
  • Deprecate FFI definitions PyEval_AcquireLock and PyEval_ReleaseLock. #5590
  • Relax 'py: 'a bound in Py::extract. #5594
  • Add a T: PyTypeCheck bound to the IntoPyObject implementations on Bound<T>, Borrowed<T> and Py<T>. #5640
  • The with_critical_section and with_critical_section2 functions are moved to pyo3::sync::critical_section. #5642
  • Use PyIter_NextItem in PyIterator::next implementation. #5661
  • IntoPyObject for simple enums now uses a singleton value, allowing identity (python is) comparisons. #5665
  • Allow any Sequence[int] in FromPyObject on Cow<[u8]> and change the error type to PyErr. #5667
  • async pymethods now borrow self only for the duration of awaiting the future, not the entire method call. #5684
  • Change CastError formatted message to directly describe the “is not an instance of” failure condition. #5693
  • Add #[inline] hints to many methods on PyBackedStr. #5723
  • Remove redundant internal counters from BoundSetIterator and BoundFrozenSetIterator. #5725
  • Implement PyIterator::size_hint on abi3 builds (previously was only on unlimited API builds). #5727
  • Deprecate FFI definition PyImport_ImportModuleNoBlock (deprecated in Python 3.13). #5737
  • #[new] can now return arbitrary Python objects. #5739
  • experimental-inspect:
    • Introduce TypeHint and make use of it to encode type hint annotations. #5438
    • Rename PyType{Info,Check}::TYPE_INFO into PyType{Info,Check}::TYPE_HINT. #5438 #5619 #5641
    • Fill annotations on function arguments and return values for all types supported natively by PyO3. #5634 #5637 #5639
    • Use _typeshed.Incomplete instead of typing.Any as default type hint, to make it easier to spot incomplete trait implementations. #5744
    • Use general Python expression syntax for type hints. #5671

Removed

  • Remove all functionality deprecated in PyO3 0.25 and 0.26. #5740
  • FFI definitions:
    • Remove FFI definition PyEval_GetCallStats (removed from CPython in Python 3.7). #5590
    • Remove FFI definitions PyEval_AcquireLock and PyEval_ReleaseLock on Python 3.13 and up. #5590
    • Remove private FFI definitions _PyObject_New, _PyObject_NewVar, _PyObject_GC_Resize, _PyObject_GC_New, and _PyObject_GC_NewVar. #5591
    • Remove private FFI definitions _PyDict_SetItem_KnownHash, _PyDict_Next, _PyDict_NewPresized, _PyDict_Contains_KnownHash, and _PyDict_Contains. #5659
    • Remove private FFI definitions _PyFrameEvalFunction, _PyInterpreterState_GetEvalFrameFunc and _PyInterpreterState_SetEvalFrameFunc. #5711
    • Remove private FFI definitions _PyImport_IsInitialized, _PyImport_SetModule, _PyImport_SetModuleString, _PyImport_AcquireLock, _PyImport_ReleaseLock, _PyImport_FindBuiltin, _PyImport_FindExtensionObject, _PyImport_FixupBuiltin, and _PyImport_FixupExtensionObject. #5737

Fixed

  • Fix PyModuleMethods::add_submodule() to use the last segment of the submodule name as the attribute name on the parent module instead of using the full name. #5375
  • Link with libpython for Cygwin extension modules. #5571
  • Link against the limited API DLL for Cygwin when abi3 is used. #5574
  • Handle errors in PyIterator when calling size_hint #5604
  • Link with libpython for iOS extension modules. #5605
  • Correct IntoPyObject output type of PyBackedStr to be PyString, not PyAny. #5655
  • Fix async functions to return None rather than empty tuple (). #5685
  • Fix compile error when using references to #[pyclass] types (e.g. &MyClass) as arguments to async #[pyfunction]s. #5725
  • FFI definitions:
    • Fix FFI definition PyMemberDescrObject.d_member to use PyMemberDef for Python 3.11+ (was incorrectly PyGetSetDef). #5647
    • Mark FFI definition PyThreadState_GetFrame available with abi3 in 3.10+. #5711
    • Fix FFI definition PyImport_GetModule on PyPy. #5737
  • experimental-inspect:
    • fix __new__ return type to be the built object type and not None. #5555
    • fix imports of decorators. #5618
    • fix the return type annotation of PyResult<()> (must be None and not tuple) #5674

0.27.2 - 2025-11-30

Changed

  • Disable subclassing PyDict on GraalPy (unsupported for now, may crash at runtime). #5653

Fixed

  • Fix crash when compiling on Rust 1.92+ with both debug assertions and optimizations enabled. #5638
  • Fix FFI definition of PyDictObject on PyPy. #5653

0.27.1 - 2025-10-21

Fixed

  • Fix clippy:declare_interior_mutable_const warning from #[pyfunction]. #5538
  • Expose pyo3::types::PySendResult in public API. #5539

0.27.0 - 2025-10-19

Packaging

  • Extend range of supported versions of hashbrown optional dependency to include version 0.16. #5428
  • Bump optional num-bigint dependency minimum version to 0.4.4. #5471
  • Test against Python 3.14 final release. #5499
  • Drop support for PyPy 3.9 and 3.10. #5516
  • Provide a better error message when building an outdated PyO3 for a too-new Python version. #5519

Added

  • Add FromPyObjectOwned as convenient trait bound for FromPyObject when the data is not borrowed from Python. #4390
  • Add Borrowed::extract, same as PyAnyMethods::extract, but does not restrict the lifetime by deref. #4390
  • experimental-inspect: basic support for #[derive(IntoPyObject)] (no struct fields support yet). #5365
  • experimental-inspect: support #[pyo3(get, set)] and #[pyclass(get_all, set_all)]. #5370
  • Add PyTypeCheck::classinfo_object that returns an object that can be used as parameter in isinstance or issubclass. #5387
  • Implement PyTypeInfo on datetime.* types even when the limited API is enabled. #5388
  • Implement PyTypeInfo on PyIterator, PyMapping and PySequence. #5402
  • Implement PyTypeInfo on PyCode when using the stable ABI. #5403
  • Implement PyTypeInfo on PyWeakrefReference when using the stable ABI. #5404
  • Add pyo3::sync::RwLockExt trait, analogous to pyo3::sync::MutexExt for readwrite locks. #5435
  • Add PyString::from_bytes. #5437
  • Implement AsRef<[u8]> for PyBytes. #5445
  • Add CastError and CastIntoError. #5468
  • Add PyCapsuleMethods::pointer_checked and PyCapsuleMethods::is_valid_checked. #5474
  • Add Borrowed::cast, Borrowed::cast_exact and Borrowed::cast_unchecked. #5475
  • Add conversions for jiff::civil::ISOWeekDate. #5478
  • Add conversions for &Cstr, Cstring and Cow<Cstr>. #5482
  • add #[pyclass(skip_from_py_object)] option, to opt-out of the FromPyObject: PyClass + Clone blanket impl. #5488
  • Add PyErr::add_note. #5489
  • Add FromPyObject impl for Cow<Path> & Cow<OsStr>. #5497
  • Add #[pyclass(from_py_object)] pyclass option, to opt-in to the extraction of pyclasses by value (requires Clone). #5506

Changed

  • Rework FromPyObject trait for flexibility and performance: #4390
    • Add a second lifetime to FromPyObject, to allow borrowing data from Python objects (e.g. &str from Python str).
    • Replace extract_bound with extract, which takes Borrowed<'a, 'py, PyAny>.
  • Optimize FromPyObject implementations for Vec<u8> and [u8; N] from bytes and bytearray. #5244
  • Deprecate #[pyfn] attribute. #5384
  • Fetch type name dynamically on cast errors instead of using PyTypeCheck::NAME. #5387
  • Deprecate PyTypeCheck::NAME in favour of PyTypeCheck::classinfo_object which provides the type information at runtime. #5387
  • PyClassGuard(Mut) and PyRef(Mut) extraction now returns an opaque Rust error #5413
  • Fetch type name dynamically when exporting types implementing PyTypeInfo with #[pymodule_use]. #5414
  • Improve Debug representation of PyBuffer<T>. #5442
  • experimental-inspect: change the way introspection data is emitted in the binaries to avoid a pointer indirection and simplify parsing. #5450
  • Optimize Py<T>::drop for the case when attached to the Python interpreter. #5454
  • Replace DowncastError and DowncastIntoError with CastError and CastIntoError. #5468
  • Enable fast-path for 128-bit integer conversions on GraalPy. #5471
  • Deprecate PyAnyMethods::downcast functions in favour of Bound::cast functions. #5472
  • Make PyTypeCheck an unsafe trait. #5473
  • Deprecate unchecked PyCapsuleMethods: pointer(), reference(), and is_valid(). #5474
  • Reduce lifetime of return value in PyCapsuleMethods::reference. #5474
  • PyCapsuleMethods::name now returns CapsuleName wrapper instead of &CStr. #5474
  • Deprecate import_exception_bound in favour of import_exception. #5480
  • PyList::get_item_unchecked, PyTuple::get_item_unchecked, and PyTuple::get_borrowed_item_unchecked no longer check for null values at the provided index. #5494
  • Allow converting naive datetime into chrono DateTime<Local>. #5507

Removed

  • Removed FromPyObjectBound trait. #4390

Fixed

  • Fix compilation failure on wasm32-wasip2. #5368
  • Fix OsStr conversion for non-utf8 strings on Windows. #5444
  • Fix issue with cargo vendor caused by gitignored build artifact emscripten/pybuilddir.txt. #5456
  • Stop leaking PyMethodDef instances inside #[pyfunction] macro generated code. #5459
  • Don’t export definition of FFI struct PyObjectObFlagsAndRefcnt on 32-bit Python 3.14 (doesn’t exist). #5499
  • Fix failure to build for abi3 interpreters on Windows using maturin’s built-in sysconfig in combination with the generate-import-lib feature. #5503
  • Fix FFI definitions PyModule_ExecDef and PyModule_FromDefAndSpec2 on PyPy. #5529

0.26.0 - 2025-08-29

Packaging

  • Bump hashbrown dependency to 0.15. #5152
  • Update MSRV to 1.74. #5171
  • Set the same maximum supported version for alternative interpreters as for CPython. #5192
  • Add optional bytes dependency to add conversions for bytes::Bytes. #5252
  • Publish new crate pyo3-introspection to pair with the experimental-inspect feature. #5300
  • The PYO3_BUILD_EXTENSION_MODULE now causes the same effect as the extension-module feature. Eventually we expect maturin and setuptools-rust to set this environment variable automatically. Users with their own build systems will need to do the same. #5343

Added

  • Add #[pyo3(warn(message = "...", category = ...))] attribute for automatic warnings generation for #[pyfunction] and #[pymethods]. #4364
  • Add PyMutex, available on Python 3.13 and newer. #4523
  • Add FFI definition PyMutex_IsLocked, available on Python 3.14 and newer. #4523
  • Add PyString::from_encoded_object. #5017
  • experimental-inspect: add basic input type annotations. #5089
  • Add FFI function definitions for PyFrameObject from CPython 3.13. #5154
  • experimental-inspect: tag modules created using #[pymodule] or #[pymodule_init] functions as incomplete. #5207
  • experimental-inspect: add basic return type support. #5208
  • Add PyCode::compile and PyCodeMethods::run to create and execute code objects. #5217
  • Add PyOnceLock type for thread-safe single-initialization. #5223
  • Add PyClassGuard(Mut) pyclass holders. In the future they will replace PyRef(Mut). #5233
  • experimental-inspect: allow annotations in #[pyo3(signature)] signature attribute. #5241
  • Implement MutexExt for parking_lot’s/lock_api ReentrantMutex. #5258
  • experimental-inspect: support class associated constants. #5272
  • Add Bound::cast family of functions superseding the PyAnyMethods::downcast family. #5289
  • Add FFI definitions Py_Version and Py_IsFinalizing. #5317
  • experimental-inspect: add output type annotation for #[pyclass]. #5320
  • experimental-inspect: support #[pyclass(eq, eq_int, ord, hash, str)]. #5338
  • experimental-inspect: add basic support for #[derive(FromPyObject)] (no struct fields support yet). #5339
  • Add Python::try_attach. #5342

Changed

  • Use Py_TPFLAGS_DISALLOW_INSTANTIATION instead of a __new__ which always fails for a #[pyclass] without a #[new] on Python 3.10 and up. #4568
  • PyModule::from_code now defaults file_name to <string> if empty. #4777
  • Deprecate PyString::from_object in favour of PyString::from_encoded_object. #5017
  • When building with abi3 for a Python version newer than pyo3 supports, automatically fall back to an abi3 build for the latest supported version. #5144
  • Change is_instance_of trait bound from PyTypeInfo to PyTypeCheck. #5146
  • Many PyO3 proc macros now report multiple errors instead of only the first one. #5159
  • Change MutexExt return type to be an associated type. #5201
  • Use PyCallArgs for Py::call and friends so they’re equivalent to their Bound counterpart. #5206
  • Rename Python::with_gil to Python::attach. #5209
  • Rename Python::allow_threads to Python::detach #5221
  • Deprecate GILOnceCell type in favour of PyOnceLock. #5223
  • Rename pyo3::prepare_freethreaded_python to Python::initialize. #5247
  • Convert PyMemoryError into/from io::ErrorKind::OutOfMemory. #5256
  • Deprecate GILProtected. #5285
  • Move #[pyclass] docstring formatting from import time to compile time. #5286
  • Python::attach will now panic if the Python interpreter is in the process of shutting down. #5317
  • Add fast-path to PyTypeInfo::type_object for #[pyclass] types. #5324
  • Deprecate PyObject type alias for Py<PyAny>. #5325
  • Rename Python::with_gil_unchecked to Python::attach_unchecked. #5340
  • Rename Python::assume_gil_acquired to Python::assume_attached. #5354

Removed

  • Remove FFI definition of internals of PyFrameObject. #5154
  • Remove Eq and PartialEq implementations on PyGetSetDef FFI definition. #5196
  • Remove private FFI definitions _Py_IsCoreInitialized and _Py_InitializeMain. #5317

Fixed

  • Use critical section in PyByteArray::to_vec on freethreaded build to replicate GIL-enabled “soundness”. #4742
  • Fix precision loss when converting bigdecimal into Python. #5198
  • Don’t treat win7 target as a cross-compilation. #5210
  • WASM targets no longer require exception handling support for Python < 3.14. #5239
  • Fix segfault when dropping PyBuffer<T> after the Python interpreter has been finalized. #5242
  • experimental-inspect: better automated imports generation. #5251
  • experimental-inspect: fix introspection of __richcmp__, __concat__, __repeat__, __inplace_concat__ and __inplace_repeat__. #5273
  • fixed a leaked borrow, when converting a mutable sub class into a frozen base class using PyRef::into_super #5281
  • Fix FFI definition Py_Exit (never returns, was () return value, now !). #5317
  • experimental-inspect: fix handling of module members gated behind #[cfg(...)] attributes. #5318

0.25.1 - 2025-06-12

Packaging

  • Add support for Windows on ARM64. #5145
  • Add chrono-local feature for optional conversions for chrono’s Local timezone & DateTime<Local> instances. #5174

Added

  • Add FFI definition PyBytes_AS_STRING. #5121
  • Add support for module associated consts introspection. #5150

Changed

  • Enable “vectorcall” FFI definitions on GraalPy. #5121
  • Use Py_Is function on GraalPy #5121

Fixed

  • Report a better compile error for async declarations when not using experimental-async feature. #5156
  • Fix implementation of FromPyObject for uuid::Uuid on big-endian architectures. #5161
  • Fix segmentation faults on 32-bit x86 with Python 3.14. #5180

0.25.0 - 2025-05-14

Packaging

  • Support Python 3.14.0b1. #4811
  • Bump supported GraalPy version to 24.2. #5116
  • Add optional bigdecimal dependency to add conversions for bigdecimal::BigDecimal. #5011
  • Add optional time dependency to add conversions for time types. #5057
  • Remove cfg-if dependency. #5110
  • Add optional ordered_float dependency to add conversions for ordered_float::NotNan and ordered_float::OrderedFloat. #5114

Added

  • Add initial type stub generation to the experimental-inspect feature. #3977
  • Add #[pyclass(generic)] option to support runtime generic typing. #4926
  • Implement OnceExt & MutexExt for parking_lot & lock_api. Use the new extension traits by enabling the arc_lock, lock_api, or parking_lot cargo features. #5044
  • Implement From/Into for Borrowed<T> -> Py<T>. #5054
  • Add PyTzInfo constructors. #5055
  • Add FFI definition PY_INVALID_STACK_EFFECT. #5064
  • Implement AsRef<Py<PyAny>> for Py<T>, Bound<T> and Borrowed<T>. #5071
  • Add FFI definition PyModule_Add and compat::PyModule_Add. #5085
  • Add FFI definitions Py_HashBuffer, Py_HashPointer, and PyObject_GenericHash. #5086
  • Support #[pymodule_export] on const items in declarative modules. #5096
  • Add #[pyclass(immutable_type)] option (on Python 3.14+ with abi3, or 3.10+ otherwise) for immutable type objects. #5101
  • Support #[pyo3(rename_all)] support on #[derive(IntoPyObject)]. #5112
  • Add PyRange wrapper. #5117

Changed

  • Enable use of datetime types with abi3 feature enabled. #4970
  • Deprecate timezone_utc in favor of PyTzInfo::utc. #5055
  • Reduce visibility of some CPython implementation details: #5064
    • The FFI definition PyCodeObject is now an opaque struct on all Python versions.
    • The FFI definition PyFutureFeatures is now only defined up until Python 3.10 (it was present in CPython headers but unused in 3.11 and 3.12).
  • Change PyAnyMethods::is to take other: &Bound<T>. #5071
  • Change Py::is to take other: &Py<T>. #5071
  • Change PyVisit::call to take T: Into<Option<&Py<T>>>. #5071
  • Expose PyDateTime_DATE_GET_TZINFO and PyDateTime_TIME_GET_TZINFO on PyPy 3.10 and later. #5079
  • Add #[track_caller] to with_gil and with_gil_unchecked. #5109
  • Use std::thread::park() instead of libc::pause() or sleep(9999999). #5115

Removed

  • Remove all functionality deprecated in PyO3 0.23. #4982
  • Remove deprecated IntoPy and ToPyObject traits. #5010
  • Remove private types from pyo3-ffi (i.e. starting with _Py) which are not referenced by public APIs: _PyLocalMonitors, _Py_GlobalMonitors, _PyCoCached, _PyCoLineInstrumentationData, _PyCoMonitoringData, _PyCompilerSrcLocation, _PyErr_StackItem. #5064
  • Remove FFI definition PyCode_GetNumFree (PyO3 cannot support it due to knowledge of the code object). #5064
  • Remove AsPyPointer trait. #5071
  • Remove support for the deprecated string form of from_py_with. #5097
  • Remove FFI definitions of private static variables: _PyMethodWrapper_Type, _PyCoroWrapper_Type, _PyImport_FrozenBootstrap, _PyImport_FrozenStdlib, _PyImport_FrozenTest, _PyManagedBuffer_Type, _PySet_Dummy, _PyWeakref_ProxyType, and _PyWeakref_CallableProxyType. #5105
  • Remove FFI definitions PyASCIIObjectState, PyUnicode_IS_ASCII, PyUnicode_IS_COMPACT, and PyUnicode_IS_COMPACT_ASCII on Python 3.14 and newer. #5133

Fixed

  • Correctly pick up the shared state for conda-based Python installation when reading information from sysconfigdata. #5037
  • Fix compile failure with #[derive(IntoPyObject, FromPyObject)] when using #[pyo3()] options recognised by only one of the two derives. #5070
  • Fix various compile errors from missing FFI definitions using certain feature combinations on PyPy and GraalPy. #5091
  • Fallback on backports.zoneinfo for python <3.9 when converting timezones into python. #5120

0.24.2 - 2025-04-21

Fixed

  • Fix unused_imports lint of #[pyfunction] and #[pymethods] expanded in macro_rules context. #5030
  • Fix size of PyCodeObject::_co_instrumentation_version ffi struct member on Python 3.13 for systems where uintptr_t is not 64 bits. #5048
  • Fix struct-type complex enum variant fields incorrectly exposing raw identifiers as r#ident in Python bindings. #5050

0.24.1 - 2025-03-31

Added

  • Add abi3-py313 feature. #4969
  • Add PyAnyMethods::getattr_opt. #4978
  • Add PyInt::new constructor for all supported number types (i32, u32, i64, u64, isize, usize). #4984
  • Add pyo3::sync::with_critical_section2. #4992
  • Implement PyCallArgs for Borrowed<'_, 'py, PyTuple>, &Bound<'py, PyTuple>, and &Py<PyTuple>. #5013

Fixed

  • Fix is_type_of for native types not using same specialized check as is_type_of_bound. #4981
  • Fix Probe class naming issue with #[pymethods]. #4988
  • Fix compile failure with required #[pyfunction] arguments taking Option<&str> and Option<&T> (for #[pyclass] types). #5002
  • Fix PyString::from_object causing of bounds reads with encoding and errors parameters which are not nul-terminated. #5008
  • Fix compile error when additional options follow after crate for #[pyfunction]. #5015

0.24.0 - 2025-03-09

Packaging

  • Add supported CPython/PyPy versions to cargo package metadata. #4756
  • Bump target-lexicon dependency to 0.13. #4822
  • Add optional jiff dependency to add conversions for jiff datetime types. #4823
  • Add optional uuid dependency to add conversions for uuid::Uuid. #4864
  • Bump minimum supported inventory version to 0.3.5. #4954

Added

  • Add PyIterator::send method to allow sending values into a python generator. #4746
  • Add PyCallArgs trait for passing arguments into the Python calling protocol. This enabled using a faster calling convention for certain types, improving performance. #4768
  • Add #[pyo3(default = ...'] option for #[derive(FromPyObject)] to set a default value for extracted fields of named structs. #4829
  • Add #[pyo3(into_py_with = ...)] option for #[derive(IntoPyObject, IntoPyObjectRef)]. #4850
  • Add FFI definitions PyThreadState_GetFrame and PyFrame_GetBack. #4866
  • Optimize last for BoundListIterator, BoundTupleIterator and BorrowedTupleIterator. #4878
  • Optimize Iterator::count() for PyDict, PyList, PyTuple & PySet. #4878
  • Optimize nth, nth_back, advance_by and advance_back_by for BoundTupleIterator #4897
  • Add support for types.GenericAlias as pyo3::types::PyGenericAlias. #4917
  • Add MutextExt trait to help avoid deadlocks with the GIL while locking a std::sync::Mutex. #4934
  • Add #[pyo3(rename_all = "...")] option for #[derive(FromPyObject)]. #4941

Changed

  • Optimize nth, nth_back, advance_by and advance_back_by for BoundListIterator. #4810
  • Use DerefToPyAny in blanket implementations of From<Py<T>> and From<Bound<'py, T>> for PyObject. #4593
  • Map io::ErrorKind::IsADirectory/NotADirectory to the corresponding Python exception on Rust 1.83+. #4747
  • PyAnyMethods::call and friends now require PyCallArgs for their positional arguments. #4768
  • Expose FFI definitions for PyObject_Vectorcall(Method) on the stable abi on 3.12+. #4853
  • #[pyo3(from_py_with = ...)] now take a path rather than a string literal #4860
  • Format Python traceback in impl Debug for PyErr. #4900
  • Convert PathBuf & Path into Python pathlib.Path instead of PyString. #4925
  • Relax parsing of exotic Python versions. #4949
  • PyO3 threads now hang instead of pthread_exit trying to acquire the GIL when the interpreter is shutting down. This mimics the Python 3.14 behavior and avoids undefined behavior and crashes. #4874

Removed

  • Remove implementations of Deref for PyAny and other “native” types. #4593
  • Remove implicit default of trailing optional arguments (see #2935) #4729
  • Remove the deprecated implicit eq fallback for simple enums. #4730

Fixed

  • Correct FFI definition of PyIter_Send to return a PySendResult. #4746
  • Fix a thread safety issue in the runtime borrow checker used by mutable pyclass instances on the free-threaded build. #4948

0.23.5 - 2025-02-22

Packaging

  • Add support for PyPy3.11 #4760

Fixed

  • Fix thread-unsafe implementation of freelist pyclasses on the free-threaded build. #4902
  • Re-enable a workaround for situations where CPython incorrectly does not add __builtins__ to __globals__ in code executed by Python::py_run (was removed in PyO3 0.23.0). #4921

0.23.4 - 2025-01-10

Added

  • Add PyList::locked_for_each, which uses a critical section to lock the list on the free-threaded build. #4789
  • Add pyo3_build_config::add_python_framework_link_args build script API to set rpath when using macOS system Python. #4833

Changed

  • Use datetime.fold to distinguish ambiguous datetimes when converting to and from chrono::DateTime<Tz> (rather than erroring). #4791
  • Optimize PyList iteration on the free-threaded build. #4789

Fixed

  • Fix unnecessary internal py.allow_threads GIL-switch when attempting to access contents of a PyErr which originated from Python (could lead to unintended deadlocks). #4766

  • Fix thread-unsafe access of dict internals in BoundDictIterator on the free-threaded build. #4788

  • Fix unnecessary critical sections in BoundDictIterator on the free-threaded build. #4788

  • Fix time-of-check to time-of-use issues with list iteration on the free-threaded build. #4789

  • Fix chrono::DateTime<Tz> to-Python conversion when Tz is chrono_tz::Tz. #4790

  • Fix #[pyclass] not being able to be named Probe. #4794

  • Fix not treating cross-compilation from x64 to aarch64 on Windows as a cross-compile. #4800

  • Fix missing struct fields on GraalPy when subclassing builtin classes. #4802

  • Fix generating import lib for PyPy when abi3 feature is enabled. #4806

  • Fix generating import lib for python3.13t when abi3 feature is enabled. #4808

  • Fix compile failure for raw identifiers like r#box in derive(FromPyObject). #4814

  • Fix compile failure for #[pyclass] enum variants with more than 12 fields. #4832

0.23.3 - 2024-12-03

Packaging

  • Bump optional python3-dll-a dependency to 0.2.11. #4749

Fixed

  • Fix unresolved symbol link failures on Windows when compiling for Python 3.13t with abi3 features enabled. #4733
  • Fix unresolved symbol link failures on Windows when compiling for Python 3.13t using the generate-import-lib feature. #4749
  • Fix compile-time regression in PyO3 0.23.0 where changing PYO3_CONFIG_FILE would not reconfigure PyO3 for the new interpreter. #4758

0.23.2 - 2024-11-25

Added

  • Add IntoPyObjectExt trait. #4708

Fixed

  • Fix compile failures when building for free-threaded Python when the abi3 or abi3-pyxx features are enabled. #4719
  • Fix ambiguous_associated_items lint error in #[pyclass] and #[derive(IntoPyObject)] macros. #4725

0.23.1 - 2024-11-16

Re-release of 0.23.0 with fixes to docs.rs build.

0.23.0 - 2024-11-15

Packaging

  • Drop support for PyPy 3.7 and 3.8. #4582
  • Extend range of supported versions of hashbrown optional dependency to include version 0.15. #4604
  • Bump minimum version of eyre optional dependency to 0.6.8. #4617
  • Bump minimum version of hashbrown optional dependency to 0.14.5. #4617
  • Bump minimum version of indexmap optional dependency to 2.5.0. #4617
  • Bump minimum version of num-complex optional dependency to 0.4.6. #4617
  • Bump minimum version of chrono-tz optional dependency to 0.10. #4617
  • Support free-threaded Python 3.13t. #4588

Added

  • Add IntoPyObject (fallible) conversion trait to convert from Rust to Python values. #4060
  • Add #[pyclass(str="<format string>")] option to generate __str__ based on a Display implementation or format string. #4233
  • Implement PartialEq for Bound<'py, PyInt> with u8, u16, u32, u64, u128, usize, i8, i16, i32, i64, i128 and isize. #4317
  • Implement PartialEq<f64> and PartialEq<f32> for Bound<'py, PyFloat>. #4348
  • Add as_super and into_super methods for Bound<T: PyClass>. #4351
  • Add FFI definitions PyCFunctionFast and PyCFunctionFastWithKeywords #4415
  • Add FFI definitions for PyMutex on Python 3.13 and newer. #4421
  • Add PyDict::locked_for_each to iterate efficiently on freethreaded Python. #4439
  • Add FFI definitions PyObject_GetOptionalAttr, PyObject_GetOptionalAttrString, PyObject_HasAttrWithError, PyObject_HasAttrStringWithError, Py_CONSTANT_* constants, Py_GetConstant, Py_GetConstantBorrowed, and PyType_GetModuleByDef on Python 3.13 and newer. #4447
  • Add FFI definitions for the Python critical section API available on Python 3.13 and newer. #4477
  • Add derive macro for IntoPyObject. #4495
  • Add Borrowed::as_ptr. #4520
  • Add FFI definition for PyImport_AddModuleRef. #4529
  • Add PyAnyMethods::try_iter. #4553
  • Add pyo3::sync::with_critical_section, a wrapper around the Python Critical Section API added in Python 3.13. #4587
  • Add #[pymodule(gil_used = false)] option to declare that a module supports the free-threaded build. #4588
  • Add PyModule::gil_used method to declare that a module supports the free-threaded build. #4588
  • Add FFI definition PyDateTime_CAPSULE_NAME. #4634
  • Add PyMappingProxy type to represent the mappingproxy Python class. #4644
  • Add FFI definitions PyList_Extend and PyList_Clear. #4667
  • Add derive macro for IntoPyObjectRef. #4674
  • Add pyo3::sync::OnceExt and pyo3::sync::OnceLockExt traits. #4676

Changed

  • Prefer IntoPyObject over IntoPy<Py<PyAny>>> for #[pyfunction] and #[pymethods] return types. #4060
  • Report multiple errors from #[pyclass] and #[pyo3(..)] attributes. #4243
  • Nested declarative #[pymodule] are automatically treated as submodules (no PyInit_ entrypoint is created). #4308
  • Deprecate PyAnyMethods::is_ellipsis (Py::is_ellipsis was deprecated in PyO3 0.20). #4322
  • Deprecate PyLong in favor of PyInt. #4347
  • Rename IntoPyDict::into_py_dict_bound to IntoPyDict::into_py_dict. #4388
  • PyModule::from_code now expects &CStr as arguments instead of &str. #4404
  • Use “fastcall” Python calling convention for #[pyfunction]s when compiling on abi3 for Python 3.10 and up. #4415
  • Remove Copy and Clone from PyObject struct FFI definition. #4434
  • Python::eval and Python::run now take a &CStr instead of &str. #4435
  • Deprecate IPowModulo, PyClassAttributeDef, PyGetterDef, PyMethodDef, PyMethodDefType, and PySetterDef from PyO3’s public API. #4441
  • IntoPyObject impls for Vec<u8>, &[u8], [u8; N], Cow<[u8]> and SmallVec<[u8; N]> now convert into Python bytes rather than a list of integers. #4442
  • Emit a compile-time error when attempting to subclass a class that doesn’t allow subclassing. #4453
  • IntoPyDict::into_py_dict is now fallible due to IntoPyObject migration. #4493
  • The abi3 feature will now override config files provided via PYO3_BUILD_CONFIG. #4497
  • Disable the GILProtected struct on free-threaded Python. #4504
  • Updated FFI definitions for functions and struct fields that have been deprecated or removed from CPython. #4534
  • Disable PyListMethods::get_item_unchecked on free-threaded Python. #4539
  • Add GILOnceCell::import. #4542
  • Deprecate PyAnyMethods::iter in favour of PyAnyMethods::try_iter. #4553
  • The #[pyclass] macro now requires a types to be Sync. (Except for #[pyclass(unsendable)] types). #4566
  • PyList::new and PyTuple::new are now fallible due to IntoPyObject migration. #4580
  • PyErr::matches is now fallible due to IntoPyObject migration. #4595
  • Deprecate ToPyObject in favour of IntoPyObject #4595
  • Deprecate PyWeakrefMethods::get_option. #4597
  • Seal PyWeakrefMethods trait. #4598
  • Remove PyNativeTypeInitializer and PyObjectInit from the PyO3 public API. #4611
  • Deprecate IntoPy in favor of IntoPyObject #4618
  • Eagerly normalize exceptions in PyErr::take() and PyErr::fetch() on Python 3.11 and older. #4655
  • Move IntoPy::type_output to IntoPyObject::type_output. #4657
  • Change return type of PyMapping::keys, PyMapping::values and PyMapping::items to Bound<'py, PyList> instead of Bound<'py, PySequence>. #4661
  • Complex enums now allow field types that either implement IntoPyObject by reference or by value together with Clone. This makes Py<T> available as field type. #4694

Removed

  • Remove all functionality deprecated in PyO3 0.20. #4322
  • Remove all functionality deprecated in PyO3 0.21. #4323
  • Deprecate PyUnicode in favour of PyString. #4370
  • Remove deprecated gil-refs feature. #4378
  • Remove private FFI definitions _Py_IMMORTAL_REFCNT, _Py_IsImmortal, _Py_TPFLAGS_STATIC_BUILTIN, _Py_Dealloc, _Py_IncRef, _Py_DecRef. #4447
  • Remove private FFI definitions _Py_c_sum, _Py_c_diff, _Py_c_neg, _Py_c_prod, _Py_c_quot, _Py_c_pow, _Py_c_abs. #4521
  • Remove _borrowed methods of PyWeakRef and PyWeakRefProxy. #4528
  • Removed private FFI definition _PyErr_ChainExceptions. #4534

Fixed

  • Fix invalid library search path lib_dir when cross-compiling. #4389
  • Fix FFI definition Py_Is for PyPy on 3.10 to call the function defined by PyPy. #4447
  • Fix compile failure when using #[cfg] attributes for simple enum variants. #4509
  • Fix compiler warning for non_snake_case method names inside #[pymethods] generated code. #4567
  • Fix compile error with #[derive(FromPyObject)] generic struct with trait bounds. #4645
  • Fix compile error for #[classmethod] and #[staticmethod] on magic methods. #4654
  • Fix compile warning for unsafe_op_in_unsafe_fn in generated macro code. #4674
  • Fix incorrect deprecation warning for #[pyclass] enums with custom __eq__ implementation. #4692
  • Fix non_upper_case_globals lint firing for generated __match_args__ on complex enums. #4705

0.22.5 - 2024-10-15

Fixed

  • Fix regression in 0.22.4 of naming collision in __clear__ slot and clear method generated code. #4619

0.22.4 - 2024-10-12

Added

  • Add FFI definition PyWeakref_GetRef and compat::PyWeakref_GetRef. #4528

Changed

  • Deprecate _borrowed methods on PyWeakRef and PyWeakrefProxy (just use the owning forms). #4590

Fixed

  • Revert removal of private FFI function _PyLong_NumBits on Python 3.13 and later. #4450
  • Fix __traverse__ functions for base classes not being called by subclasses created with #[pyclass(extends = ...)]. #4563
  • Fix regression in 0.22.3 failing compiles under #![forbid(unsafe_code)]. #4574
  • Fix create_exception macro triggering lint and compile errors due to interaction with gil-refs feature. #4589
  • Workaround possible use-after-free in _borrowed methods on PyWeakRef and PyWeakrefProxy by leaking their contents. #4590
  • Fix crash calling PyType_GetSlot on static types before Python 3.10. #4599

0.22.3 - 2024-09-15

Added

  • Add pyo3::ffi::compat namespace with compatibility shims for C API functions added in recent versions of Python.
  • Add FFI definition PyDict_GetItemRef on Python 3.13 and newer, and compat::PyDict_GetItemRef for all versions. #4355
  • Add FFI definition PyList_GetItemRef on Python 3.13 and newer, and pyo3_ffi::compat::PyList_GetItemRef for all versions. #4410
  • Add FFI definitions compat::Py_NewRef and compat::Py_XNewRef. #4445
  • Add FFI definitions compat::PyObject_CallNoArgs and compat::PyObject_CallMethodNoArgs. #4461
  • Add GilOnceCell<Py<T>>::clone_ref. #4511

Changed

  • Improve error messages for #[pyfunction] defined inside #[pymethods]. #4349
  • Improve performance of calls to Python by using the vectorcall calling convention where possible. #4456
  • Mention the type name in the exception message when trying to instantiate a class with no constructor defined. #4481

Removed

  • Remove private FFI definition _Py_PackageContext. #4420

Fixed

  • Fix compile failure in declarative #[pymodule] under presence of #![no_implicit_prelude]. #4328
  • Fix use of borrowed reference in PyDict::get_item (unsafe in free-threaded Python). #4355
  • Fix #[pyclass(eq)] macro hygiene issues for structs and enums. #4359
  • Fix hygiene/span issues of #[pyfunction] and #[pymethods] generated code which affected expansion in macro_rules context. #4382
  • Fix unsafe_code lint error in #[pyclass] generated code. #4396
  • Fix async functions returning a tuple only returning the first element to Python. #4407
  • Fix use of borrowed reference in PyList::get_item (unsafe in free-threaded Python). #4410
  • Correct FFI definition PyArg_ParseTupleAndKeywords to take *const *const c_char instead of *mut *mut c_char on Python 3.13 and up. #4420
  • Fix a soundness bug with PyClassInitializer: panic if adding subclass to existing instance via PyClassInitializer::from(Py<BaseClass>).add_subclass(SubClass). #4454
  • Fix illegal reference counting op inside implementation of __traverse__ handlers. #4479

0.22.2 - 2024-07-17

Packaging

  • Require opt-in to freethreaded Python using the UNSAFE_PYO3_BUILD_FREE_THREADED=1 environment variable (it is not yet supported by PyO3). #4327

Changed

  • Use FFI function calls for reference counting on all abi3 versions. #4324
  • #[pymodule(...)] now directly accepts all relevant #[pyo3(...)] options. #4330

Fixed

  • Fix compile failure in declarative #[pymodule] under presence of #![no_implicit_prelude]. #4328
  • Fix compile failure due to c-string literals on Rust < 1.79. #4353

0.22.1 - 2024-07-06

Added

  • Add #[pyo3(submodule)] option for declarative #[pymodule]s. #4301
  • Implement PartialEq<bool> for Bound<'py, PyBool>. #4305

Fixed

  • Return NotImplemented instead of raising TypeError from generated equality method when comparing different types. #4287
  • Handle full-path #[pyo3::prelude::pymodule] and similar for #[pyclass] and #[pyfunction] in declarative modules. #4288
  • Fix 128-bit int regression on big-endian platforms with Python <3.13. #4291
  • Stop generating code that will never be covered with declarative modules. #4297
  • Fix invalid deprecation warning for trailing optional on #[setter] function. #4304

0.22.0 - 2024-06-24

Packaging

  • Update heck dependency to 0.5. #3966
  • Extend range of supported versions of chrono-tz optional dependency to include version 0.10. #4061
  • Update MSRV to 1.63. #4129
  • Add optional num-rational feature to add conversions with Python’s fractions.Fraction. #4148
  • Support Python 3.13. #4184

Added

  • Add PyWeakref, PyWeakrefReference and PyWeakrefProxy. #3835
  • Support #[pyclass] on enums that have tuple variants. #4072
  • Add support for scientific notation in Decimal conversion. #4079
  • Add pyo3_disable_reference_pool conditional compilation flag to avoid the overhead of the global reference pool at the cost of known limitations as explained in the performance section of the guide. #4095
  • Add #[pyo3(constructor = (...))] to customize the generated constructors for complex enum variants. #4158
  • Add PyType::module, which always matches Python __module__. #4196
  • Add PyType::fully_qualified_name which matches the “fully qualified name” defined in PEP 737. #4196
  • Add PyTypeMethods::mro and PyTypeMethods::bases. #4197
  • Add #[pyclass(ord)] to implement ordering based on PartialOrd. #4202
  • Implement ToPyObject and IntoPy<PyObject> for PyBackedStr and PyBackedBytes. #4205
  • Add #[pyclass(hash)] option to implement __hash__ in terms of the Hash implementation #4206
  • Add #[pyclass(eq)] option to generate __eq__ based on PartialEq, and #[pyclass(eq_int)] for simple enums to implement equality based on their discriminants. #4210
  • Implement From<Bound<'py, T>> for PyClassInitializer<T>. #4214
  • Add as_super methods to PyRef and PyRefMut for accessing the base class by reference. #4219
  • Implement PartialEq<str> for Bound<'py, PyString>. #4245
  • Implement PyModuleMethods::filename on PyPy. #4249
  • Implement PartialEq<[u8]> for Bound<'py, PyBytes>. #4250
  • Add pyo3_ffi::c_str macro to create &'static CStr on Rust versions which don’t have 1.77’s c"" literals. #4255
  • Support bool conversion with numpy 2.0’s numpy.bool type #4258
  • Add PyAnyMethods::{bitnot, matmul, floor_div, rem, divmod}. #4264

Changed

  • Change the type of PySliceIndices::slicelength and the length parameter of PySlice::indices(). #3761
  • Deprecate implicit default for trailing optional arguments #4078
  • Cloneing pointers into the Python heap has been moved behind the py-clone feature, as it must panic without the GIL being held as a soundness fix. #4095
  • Add #[track_caller] to all Py<T>, Bound<'py, T> and Borrowed<'a, 'py, T> methods which can panic. #4098
  • Change PyAnyMethods::dir to be fallible and return PyResult<Bound<'py, PyList>> (and similar for PyAny::dir). #4100
  • The global reference pool (to track pending reference count decrements) is now initialized lazily to avoid the overhead of taking a mutex upon function entry when the functionality is not actually used. #4178
  • Emit error messages when using weakref or dict when compiling for abi3 for Python older than 3.9. #4194
  • Change PyType::name to always match Python __name__. #4196
  • Remove CPython internal ffi call for complex number including: add, sub, mul, div, neg, abs, pow. Added PyAnyMethods::{abs, pos, neg} #4201
  • Deprecate implicit integer comparison for simple enums in favor of #[pyclass(eq_int)]. #4210
  • Set the module= attribute of declarative modules’ child #[pymodule]s and #[pyclass]es. #4213
  • Set the module option for complex enum variants from the value set on the complex enum module. #4228
  • Respect the Python “limited API” when building for the abi3 feature on PyPy or GraalPy. #4237
  • Optimize code generated by #[pyo3(get)] on #[pyclass] fields. #4254
  • PyCFunction::new, PyCFunction::new_with_keywords and PyCFunction::new_closure now take &'static CStr name and doc arguments (previously was &'static str). #4255
  • The experimental-declarative-modules feature is now stabilized and available by default. #4257

Fixed

  • Fix panic when PYO3_CROSS_LIB_DIR is set to a missing path. #4043
  • Fix a compile error when exporting an exception created with create_exception! living in a different Rust module using the declarative-module feature. #4086
  • Fix FFI definitions of PY_VECTORCALL_ARGUMENTS_OFFSET and PyVectorcall_NARGS to fix a false-positive assertion. #4104
  • Disable PyUnicode_DATA on PyPy: not exposed by PyPy. #4116
  • Correctly handle #[pyo3(from_py_with = ...)] attribute on dunder (__magic__) method arguments instead of silently ignoring it. #4117
  • Fix a compile error when declaring a standalone function or class method with a Python name that is a Rust keyword. #4226
  • Fix declarative modules discarding doc comments on the mod node. #4236
  • Fix __dict__ attribute missing for #[pyclass(dict)] instances when building for abi3 on Python 3.9. #4251

0.21.2 - 2024-04-16

Changed

  • Deprecate the PySet::empty() gil-ref constructor. #4082

Fixed

  • Fix compile error for async fn in #[pymethods] with a &self receiver and more than one additional argument. #4035
  • Improve error message for wrong receiver type in __traverse__. #4045
  • Fix compile error when exporting a #[pyclass] living in a different Rust module using the experimental-declarative-modules feature. #4054
  • Fix missing_docs lint triggering on documented #[pymodule] functions. #4067
  • Fix undefined symbol errors for extension modules on AIX (by linking libpython). #4073

0.21.1 - 2024-04-01

Added

  • Implement Send and Sync for PyBackedStr and PyBackedBytes. #4007
  • Implement Clone, Debug, PartialEq, Eq, PartialOrd, Ord and Hash implementation for PyBackedBytes and PyBackedStr, and Display for PyBackedStr. #4020
  • Add import_exception_bound! macro to import exception types without generating GIL Ref functionality for them. #4027

Changed

  • Emit deprecation warning for uses of GIL Refs as #[setter] function arguments. #3998
  • Add #[inline] hints on many Bound and Borrowed methods. #4024

Fixed

  • Handle #[pyo3(from_py_with = "")] in #[setter] methods #3995
  • Allow extraction of &Bound in #[setter] methods. #3998
  • Fix some uncovered code blocks emitted by #[pymodule], #[pyfunction] and #[pyclass] macros. #4009
  • Fix typo in the panic message when a class referenced in pyo3::import_exception! does not exist. #4012
  • Fix compile error when using an async #[pymethod] with a receiver and additional arguments. #4015

0.21.0 - 2024-03-25

Added

  • Add support for GraalPy (24.0 and up). #3247
  • Add PyMemoryView type. #3514
  • Allow async fn in for #[pyfunction] and #[pymethods], with the experimental-async feature. #3540 #3588 #3599 #3931
  • Implement PyTypeInfo for PyEllipsis, PyNone and PyNotImplemented. #3577
  • Support #[pyclass] on enums that have non-unit variants. #3582
  • Support chrono feature with abi3 feature. #3664
  • FromPyObject, IntoPy<PyObject> and ToPyObject are implemented on std::duration::Duration #3670
  • Add PyString::to_cow. Add Py<PyString>::to_str, Py<PyString>::to_cow, and Py<PyString>::to_string_lossy, as ways to access Python string data safely beyond the GIL lifetime. #3677
  • Add Bound<T> and Borrowed<T> smart pointers as a new API for accessing Python objects. #3686
  • Add PyNativeType::as_borrowed to convert “GIL refs” to the new Bound smart pointer. #3692
  • Add FromPyObject::extract_bound method, to migrate FromPyObject implementations to the Bound API. #3706
  • Add gil-refs feature to allow continued use of the deprecated GIL Refs APIs. #3707
  • Add methods to PyAnyMethods for binary operators (add, sub, etc.) #3712
  • Add chrono-tz feature allowing conversion between chrono_tz::Tz and zoneinfo.ZoneInfo #3730
  • Add FFI definition PyType_GetModuleByDef. #3734
  • Conversion between std::time::SystemTime and datetime.datetime #3736
  • Add Py::as_any and Py::into_any. #3785
  • Add PyStringMethods::encode_utf8. #3801
  • Add PyBackedStr and PyBackedBytes, as alternatives to &str and &bytes where a Python object owns the data. #3802 #3991
  • Allow #[pymodule] macro on Rust mod blocks, with the experimental-declarative-modules feature. #3815
  • Implement ExactSizeIterator for set and frozenset iterators on abi3 feature. #3849
  • Add Py::drop_ref to explicitly drop a `Py`` and immediately decrease the Python reference count if the GIL is already held. #3871
  • Allow #[pymodule] macro on single argument functions that take &Bound<'_, PyModule>. #3905
  • Implement FromPyObject for Cow<str>. #3928
  • Implement Default for GILOnceCell. #3971
  • Add PyDictMethods::into_mapping, PyListMethods::into_sequence and PyTupleMethods::into_sequence. #3982

Changed

  • PyDict::from_sequence now takes a single argument of type &PyAny (previously took two arguments Python and PyObject). #3532
  • Deprecate Py::is_ellipsis and PyAny::is_ellipsis in favour of any.is(py.Ellipsis()). #3577
  • Split some PyTypeInfo functionality into new traits HasPyGilRef and PyTypeCheck. #3600
  • Deprecate PyTryFrom and PyTryInto traits in favor of any.downcast() via the PyTypeCheck and PyTypeInfo traits. #3601
  • Allow async methods to accept &self/&mut self #3609
  • FromPyObject for set types now also accept frozenset objects as input. #3632
  • FromPyObject for bool now also accepts NumPy’s bool_ as input. #3638
  • Add AsRefSource associated type to PyNativeType. #3653
  • Rename .is_true to .is_truthy on PyAny and Py<PyAny> to clarify that the test is not based on identity with or equality to the True singleton. #3657
  • PyType::name is now PyType::qualname whereas PyType::name efficiently accesses the full name which includes the module name. #3660
  • The Iter(A)NextOutput types are now deprecated and __(a)next__ can directly return anything which can be converted into Python objects, i.e. awaitables do not need to be wrapped into IterANextOutput or Option any more. Option can still be used as well and returning None will trigger the fast path for __next__, stopping iteration without having to raise a StopIteration exception. #3661
  • Implement FromPyObject on chrono::DateTime<Tz> for all Tz, not just FixedOffset and Utc. #3663
  • Add lifetime parameter to PyTzInfoAccess trait. For the deprecated gil-ref API, the trait is now implemented for &'py PyTime and &'py PyDateTime instead of PyTime and PyDate. #3679
  • Calls to __traverse__ become no-ops for unsendable pyclasses if on the wrong thread, thereby avoiding hard aborts at the cost of potential leakage. #3689
  • Include PyNativeType in pyo3::prelude. #3692
  • Improve performance of extract::<i64> (and other integer types) by avoiding call to __index__() converting the value to an integer for 3.10+. Gives performance improvement of around 30% for successful extraction. #3742
  • Relax bound of FromPyObject for Py<T> to just T: PyTypeCheck. #3776
  • PySet and PyFrozenSet iterators now always iterate the equivalent of iter(set). (A “fast path” with no noticeable performance benefit was removed.) #3849
  • Move implementations of FromPyObject for &str, Cow<str>, &[u8] and Cow<[u8]> onto a temporary trait FromPyObjectBound when gil-refs feature is deactivated. #3928
  • Deprecate GILPool, Python::with_pool, and Python::new_pool. #3947

Removed

  • Remove all functionality deprecated in PyO3 0.19. #3603

Fixed

  • Match PyPy 7.3.14 in removing PyPy-only symbol Py_MAX_NDIMS in favour of PyBUF_MAX_NDIM. #3757
  • Fix segmentation fault using datetime types when an invalid datetime module is on sys.path. #3818
  • Fix non_local_definitions lint warning triggered by many PyO3 macros. #3901
  • Disable PyCode and PyCode_Type on PyPy: PyCode_Type is not exposed by PyPy. #3934

0.21.0-beta.0 - 2024-03-10

Prerelease of PyO3 0.21. See the GitHub diff for what changed between 0.21.0-beta.0 and the final release.

0.20.3 - 2024-02-23

Packaging

  • Add portable-atomic dependency. #3619
  • Check maximum version of Python at build time and for versions not yet supported require opt-in to the abi3 stable ABI by the environment variable PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1. #3821

Fixed

  • Use portable-atomic to support platforms without 64-bit atomics. #3619
  • Fix compilation failure with either feature enabled without experimental-inspect enabled. #3834

0.20.2 - 2024-01-04

Packaging

  • Pin pyo3 and pyo3-ffi dependencies on pyo3-build-config to require the same patch version, i.e. pyo3 0.20.2 requires exactly pyo3-build-config 0.20.2. #3721

Fixed

  • Fix compile failure when building pyo3 0.20.0 with latest pyo3-build-config 0.20.X. #3724
  • Fix docs.rs build. #3722

0.20.1 - 2023-12-30

Added

  • Add optional either feature to add conversions for either::Either<L, R> sum type. #3456
  • Add optional smallvec feature to add conversions for smallvec::SmallVec. #3507
  • Add take and into_inner methods to GILOnceCell #3556
  • #[classmethod] methods can now also receive Py<PyType> as their first argument. #3587
  • #[pyfunction(pass_module)] can now also receive Py<PyModule> as their first argument. #3587
  • Add traverse method to GILProtected. #3616
  • Added abi3-py312 feature #3687

Fixed

  • Fix minimum version specification for optional chrono dependency. #3512
  • Silenced new clippy::unnecessary_fallible_conversions warning when using a Py<Self> self receiver. #3564

0.20.0 - 2023-10-11

Packaging

  • Dual-license PyO3 under either the Apache 2.0 OR the MIT license. This makes the project GPLv2 compatible. #3108
  • Update MSRV to Rust 1.56. #3208
  • Bump indoc dependency to 2.0 and unindent dependency to 0.2. #3237
  • Bump syn dependency to 2.0. #3239
  • Drop support for debug builds of Python 3.7. #3387
  • Bump chrono optional dependency to require 0.4.25 or newer. #3427
  • Support Python 3.12. #3488

Added

  • Support __lt__, __le__, __eq__, __ne__, __gt__ and __ge__ in #[pymethods]. #3203
  • Add FFI definition Py_GETENV. #3336
  • Add as_ptr and into_ptr inherent methods for Py, PyAny, PyRef, and PyRefMut. #3359
  • Implement DoubleEndedIterator for PyTupleIterator and PyListIterator. #3366
  • Add #[pyclass(rename_all = "...")] option: this allows renaming all getters and setters of a struct, or all variants of an enum. Available renaming rules are: "camelCase", "kebab-case", "lowercase", "PascalCase", "SCREAMING-KEBAB-CASE", "SCREAMING_SNAKE_CASE", "snake_case", "UPPERCASE". #3384
  • Add FFI definitions PyObject_GC_IsTracked and PyObject_GC_IsFinalized on Python 3.9 and up (PyPy 3.10 and up). #3403
  • Add types for None, Ellipsis, and NotImplemented. #3408
  • Add FFI definitions for the Py_mod_multiple_interpreters constant and its possible values. #3494
  • Add FFI definitions for PyInterpreterConfig struct, its constants and Py_NewInterpreterFromConfig. #3502

Changed

  • Change PySet::discard to return PyResult<bool> (previously returned nothing). #3281
  • Optimize implementation of IntoPy for Rust tuples to Python tuples. #3321
  • Change PyDict::get_item to no longer suppress arbitrary exceptions (the return type is now PyResult<Option<&PyAny>> instead of Option<&PyAny>), and deprecate PyDict::get_item_with_error. #3330
  • Deprecate FFI definitions which are deprecated in Python 3.12. #3336
  • AsPyPointer is now an unsafe trait. #3358
  • Accept all os.PathLike values in implementation of FromPyObject for PathBuf. #3374
  • Add __builtins__ to globals in py.run() and py.eval() if they’re missing. #3378
  • Optimize implementation of FromPyObject for BigInt and BigUint. #3379
  • PyIterator::from_object and PyByteArray::from now take a single argument of type &PyAny (previously took two arguments Python and AsPyPointer). #3389
  • Replace AsPyPointer with AsRef<PyAny> as a bound in the blanket implementation of From<&T> for PyObject. #3391
  • Replace blanket impl IntoPy<PyObject> for &T where T: AsPyPointer with implementations of impl IntoPy<PyObject> for &PyAny, &T where T: AsRef<PyAny>, and &Py<T>. #3393
  • Preserve std::io::Error kind in implementation of From<std::io::IntoInnerError> for PyErr #3396
  • Try to select a relevant ErrorKind in implementation of From<PyErr> for OSError subclass. #3397
  • Retrieve the original PyErr in implementation of From<std::io::Error> for PyErr if the std::io::Error has been built using a Python exception (previously would create a new exception wrapping the std::io::Error). #3402
  • #[pymodule] will now return the same module object on repeated import by the same Python interpreter, on Python 3.9 and up. #3446
  • Truncate leap-seconds and warn when converting chrono types to Python datetime types (datetime cannot represent leap-seconds). #3458
  • Err returned from #[pyfunction] will now have a non-None __context__ if called from inside a catch block. #3455
  • Deprecate undocumented #[__new__] form of #[new] attribute. #3505

Removed

  • Remove all functionality deprecated in PyO3 0.18, including #[args] attribute for #[pymethods]. #3232
  • Remove IntoPyPointer trait in favour of into_ptr inherent methods. #3385

Fixed

  • Handle exceptions properly in PySet::discard. #3281
  • The PyTupleIterator type returned by PyTuple::iter is now public and hence can be named by downstream crates. #3366
  • Linking of PyOS_FSPath on PyPy. #3374
  • Fix memory leak in PyTypeBuilder::build. #3401
  • Disable removed FFI definitions _Py_GetAllocatedBlocks, _PyObject_GC_Malloc, and _PyObject_GC_Calloc on Python 3.11 and up. #3403
  • Fix ResourceWarning and crashes related to GC when running with debug builds of CPython. #3404
  • Some-wrapping of Option<T> default arguments will no longer re-wrap Some(T) or expressions evaluating to None. #3461
  • Fix IterNextOutput::Return not returning a value on PyPy. #3471
  • Emit compile errors instead of ignoring macro invocations inside #[pymethods] blocks. #3491
  • Emit error on invalid arguments to #[new], #[classmethod], #[staticmethod], and #[classattr]. #3484
  • Disable PyMarshal_WriteObjectToString from PyMarshal_ReadObjectFromString with the abi3 feature. #3490
  • Fix FFI definitions for _PyFrameEvalFunction on Python 3.11 and up (it now receives a _PyInterpreterFrame opaque struct). #3500

0.19.2 - 2023-08-01

Added

  • Add FFI definitions PyState_AddModule, PyState_RemoveModule and PyState_FindModule for PyPy 3.9 and up. #3295
  • Add FFI definitions _PyObject_CallFunction_SizeT and _PyObject_CallMethod_SizeT. #3297
  • Add a “performance” section to the guide collecting performance-related tricks and problems. #3304
  • Add PyErr::Display for all Python versions, and FFI symbol PyErr_DisplayException for Python 3.12. #3334
  • Add FFI definition PyType_GetDict() for Python 3.12. #3339
  • Add PyAny::downcast_exact. #3346
  • Add PySlice::full() to construct a full slice (::). #3353

Changed

  • Update PyErr for 3.12 betas to avoid deprecated ffi methods. #3306
  • Update FFI definitions of object.h for Python 3.12.0b4. #3335
  • Update pyo3::ffi struct definitions to be compatible with 3.12.0b4. #3342
  • Optimize conversion of float to f64 (and PyFloat::value) on non-abi3 builds. #3345

Fixed

  • Fix timezone conversion bug for FixedOffset datetimes that were being incorrectly converted to and from UTC. #3269
  • Fix SystemError raised in PyUnicodeDecodeError_Create on PyPy 3.10. #3297
  • Correct FFI definition Py_EnterRecursiveCall to return c_int (was incorrectly returning ()). #3300
  • Fix case where PyErr::matches and PyErr::is_instance returned results inconsistent with PyErr::get_type. #3313
  • Fix loss of panic message in PanicException when unwinding after the exception was “normalized”. #3326
  • Fix PyErr::from_value and PyErr::into_value losing traceback on conversion. #3328
  • Fix reference counting of immortal objects on Python 3.12.0b4. #3335

0.19.1 - 2023-07-03

Packaging

  • Extend range of supported versions of hashbrown optional dependency to include version 0.14 #3258
  • Extend range of supported versions of indexmap optional dependency to include version 2. #3277
  • Support PyPy 3.10. #3289

Added

  • Add pyo3::types::PyFrozenSetBuilder to allow building a PyFrozenSet item by item. #3156
  • Add support for converting to and from Python’s ipaddress.IPv4Address/ipaddress.IPv6Address and std::net::IpAddr. #3197
  • Add support for num-bigint feature in combination with abi3. #3198
  • Add PyErr_GetRaisedException(), PyErr_SetRaisedException() to FFI definitions for Python 3.12 and later. #3248
  • Add Python::with_pool which is a safer but more limited alternative to Python::new_pool. #3263
  • Add PyDict::get_item_with_error on PyPy. #3270
  • Allow #[new] methods may to return Py<Self> in order to return existing instances. #3287

Fixed

  • Fix conversion of classes implementing __complex__ to Complex when using abi3 or PyPy. #3185
  • Stop suppressing unrelated exceptions in PyAny::hasattr. #3271
  • Fix memory leak when creating PySet or PyFrozenSet or returning types converted into these internally, e.g. HashSet or BTreeSet. #3286

0.19.0 - 2023-05-31

Packaging

  • Correct dependency on syn to version 1.0.85 instead of the incorrect version 1.0.56. #3152

Added

  • Accept text_signature option (and automatically generate signature) for #[new] in #[pymethods]. #2980
  • Add support for converting to and from Python’s decimal.Decimal and rust_decimal::Decimal. #3016
  • Add #[pyo3(from_item_all)] when deriving FromPyObject to specify get_item as getter for all fields. #3120
  • Add pyo3::exceptions::PyBaseExceptionGroup for Python 3.11, and corresponding FFI definition PyExc_BaseExceptionGroup. #3141
  • Accept #[new] with #[classmethod] to create a constructor which receives a (subtype’s) class/PyType as its first argument. #3157
  • Add PyClass::get and Py::get for GIL-independent access to classes with #[pyclass(frozen)]. #3158
  • Add PyAny::is_exact_instance and PyAny::is_exact_instance_of. #3161

Changed

  • PyAny::is_instance_of::<T>(obj) is now equivalent to T::is_type_of(obj), and now returns bool instead of PyResult<bool>. #2881
  • Deprecate text_signature option on #[pyclass] structs. #2980
  • No longer wrap anyhow::Error/eyre::Report containing a basic PyErr without a chain in a PyRuntimeError. #3004
    • Change #[getter] and #[setter] to use a common call “trampoline” to slightly reduce generated code size and compile times. #3029
  • Improve default values for str, numbers and bool in automatically-generated text_signature. #3050
  • Improve default value for None in automatically-generated text_signature. #3066
  • Rename PySequence::list and PySequence::tuple to PySequence::to_list and PySequence::to_tuple. (The old names continue to exist as deprecated forms.) #3111
  • Extend the lifetime of the GIL token returned by PyRef::py and PyRefMut::py to match the underlying borrow. #3131
  • Safe access to the GIL, for example via Python::with_gil, is now locked inside of implementations of the __traverse__ slot. #3168

Removed

  • Remove all functionality deprecated in PyO3 0.17, most prominently Python::acquire_gil is replaced by Python::with_gil. #2981

Fixed

  • Correct FFI definitions PyGetSetDef, PyMemberDef, PyStructSequence_Field and PyStructSequence_Desc to have *const c_char members for name and doc (not *mut c_char). #3036
  • Fix panic on fmt::Display, instead return "<unprintable object>" string and report error via sys.unraisablehook() #3062
  • Fix a compile error of “temporary value dropped while borrowed” when #[pyfunction]s take references into #[pyclass]es #3142
  • Fix crashes caused by PyO3 applying deferred reference count updates when entering a __traverse__ implementation. #3168
  • Forbid running the Drop implementations of unsendable classes on other threads. #3176
  • Fix a compile error when #[pymethods] items come from somewhere else (for example, as a macro argument) and a custom receiver like Py<Self> is used. #3178

0.18.3 - 2023-04-13

Added

  • Add GILProtected<T> to mediate concurrent access to a value using Python’s global interpreter lock (GIL). #2975
  • Support PyASCIIObject / PyUnicode and associated methods on big-endian architectures. #3015
  • Add FFI definition _PyDict_Contains_KnownHash() for CPython 3.10 and up. #3088

Fixed

  • Fix compile error for #[pymethods] and #[pyfunction] called “output”. #3022
  • Fix compile error in generated code for magic methods implemented as a #[staticmethod]. #3055
  • Fix is_instance for PyDateTime (would incorrectly check for a PyDate). #3071
  • Fix upstream deprecation of PyUnicode_InternImmortal since Python 3.10. #3071

0.18.2 - 2023-03-24

Packaging

  • Disable default features of chrono to avoid depending on time v0.1.x. #2939

Added

  • Implement IntoPy<PyObject>, ToPyObject and FromPyObject for Cow<[u8]> to efficiently handle both bytes and bytearray objects. #2899
  • Implement IntoPy<PyObject>, ToPyObject and FromPyObject for Cell<T>. #3014
  • Add PyList::to_tuple(), as a convenient and efficient conversion from lists to tuples. #3042
  • Add PyTuple::to_list(), as a convenient and efficient conversion from tuples to lists. #3044

Changed

  • Optimize PySequence conversion for list and tuple inputs. #2944
  • Improve exception raised when creating #[pyclass] type object fails during module import. #2947
  • Optimize PyMapping conversion for dict inputs. #2954
  • Allow create_exception! to take a dotted.module to place the exception in a submodule. #2979

Fixed

  • Fix a reference counting race condition affecting PyObjects cloned in allow_threads blocks. #2952
  • Fix clippy::redundant_closure lint on default arguments in #[pyo3(signature = (...))] annotations. #2990
  • Fix non_snake_case lint on generated code in #[pyfunction] macro. #2993
  • Fix some FFI definitions for the upcoming PyPy 3.10 release. #3031

0.18.1 - 2023-02-07

Added

  • Add PyErr::write_unraisable(). #2889
  • Add Python::Ellipsis() and PyAny::is_ellipsis() methods. #2911
  • Add PyDict::update() and PyDict::update_if_missing() methods. #2912

Changed

  • FFI definition PyIter_Check on CPython 3.7 is now implemented as hasattr(type(obj), "__next__"), which works correctly on all platforms and adds support for abi3. #2914
  • Warn about unknown config keys in PYO3_CONFIG_FILE instead of denying. #2926

Fixed

  • Send errors returned by __releasebuffer__ to sys.unraisablehook rather than causing SystemError. #2886
  • Fix downcast to PyIterator succeeding for Python classes which did not implement __next__. #2914
  • Fix segfault in __traverse__ when visiting None fields of Option<T: AsPyPointer>. #2921
  • Fix #[pymethods(crate = "...")] option being ignored. #2923
  • Link against pythonXY_d.dll for debug Python builds on Windows. #2937

0.18.0 - 2023-01-17

Packaging

  • Relax indexmap optional depecency to allow >= 1.6, < 2. #2849
  • Relax hashbrown optional dependency to allow >= 0.9, < 0.14. #2875
  • Update memoffset dependency to 0.8. #2875

Added

  • Add GILOnceCell::get_or_try_init for fallible GILOnceCell initialization. #2398
  • Add experimental feature experimental-inspect with type_input() and type_output() helpers to get the Python type of any Python-compatible object. #2490 #2882
  • The #[pyclass] macro can now take get_all and set_all to create getters and setters for every field. #2692
  • Add #[pyo3(signature = (...))] option for #[pyfunction] and #[pymethods]. #2702
  • pyo3-build-config: rebuild when PYO3_ENVIRONMENT_SIGNATURE environment variable value changes. #2727
  • Add conversions between non-zero int types in std::num and Python int. #2730
  • Add Py::downcast() as a companion to PyAny::downcast(), as well as downcast_unchecked() for both types. #2734
  • Add types for all built-in Warning classes as well as PyErr::warn_explicit. #2742
  • Add abi3-py311 feature. #2776
  • Add FFI definition _PyErr_ChainExceptions() for CPython. #2788
  • Add FFI definitions PyVectorcall_NARGS and PY_VECTORCALL_ARGUMENTS_OFFSET for PyPy 3.8 and up. #2811
  • Add PyList::get_item_unchecked for PyPy. #2827

Changed

  • PyO3’s macros now emit a much nicer error message if function return values don’t implement the required trait(s). #2664
  • Use a TypeError, rather than a ValueError, when refusing to treat a str as a Vec. #2685
  • Change PyCFunction::new_closure to take name and doc arguments. #2686
  • PyType::is_subclass, PyErr::is_instance and PyAny::is_instance now take &PyAny instead of &PyType arguments, so that they work with objects that pretend to be types using __subclasscheck__ and __instancecheck__. #2695
  • Deprecate #[args] attribute and passing “args” specification directly to #[pyfunction] in favor of the new #[pyo3(signature = (...))] option. #2702
  • Deprecate required arguments after Option<T> arguments to #[pyfunction] and #[pymethods] without also using #[pyo3(signature)] to specify whether the arguments should be required or have defaults. #2703
  • Change #[pyfunction] and #[pymethods] to use a common call “trampoline” to slightly reduce generated code size and compile times. #2705
  • PyAny::cast_as() and Py::cast_as() are now deprecated in favor of PyAny::downcast() and the new Py::downcast(). #2734
  • Relax lifetime bounds on PyAny::downcast(). #2734
  • Automatically generate __text_signature__ for all Python functions created using #[pyfunction] and #[pymethods]. #2784
  • Accept any iterator in PySet::new and PyFrozenSet::new. #2795
  • Mixing #[cfg(...)] and #[pyo3(...)] attributes on #[pyclass] struct fields will now work. #2796
  • Re-enable PyFunction on when building for abi3 or PyPy. #2838
  • Improve derive(FromPyObject) to use intern! when applicable for #[pyo3(item)]. #2879

Removed

  • Remove the deprecated pyproto feature, #[pyproto] macro, and all accompanying APIs. #2587
  • Remove all functionality deprecated in PyO3 0.16. #2843

Fixed

  • Disable PyModule::filename on PyPy. #2715
  • PyCodeObject is now once again defined with fields on Python 3.7. #2726
  • Raise a TypeError if #[new] pymethods with no arguments receive arguments when called from Python. #2749
  • Use the NOARGS argument calling convention for methods that have a single py: Python argument (as a performance optimization). #2760
  • Fix truncation of isize values to c_long in PySlice::new. #2769
  • Fix soundness issue with FFI definition PyUnicodeDecodeError_Create on PyPy leading to indeterminate behavior (typically a TypeError). #2772
  • Allow functions taking **kwargs to accept keyword arguments which share a name with a positional-only argument (as permitted by PEP 570). #2800
  • Fix unresolved symbol for PyObject_Vectorcall on PyPy 3.9 and up. #2811
  • Fix memory leak in PyCFunction::new_closure. #2842

0.17.3 - 2022-11-01

Packaging

  • Support Python 3.11. (Previous versions of PyO3 0.17 have been tested against Python 3.11 release candidates and are expected to be compatible, this is the first version tested against Python 3.11.0.) #2708

Added

  • Implemented ExactSizeIterator for PyListIterator, PyDictIterator, PySetIterator and PyFrozenSetIterator. #2676

Fixed

  • Fix regression of impl FromPyObject for [T; N] no longer accepting types passing PySequence_Check, e.g. NumPy arrays, since version 0.17.0. This the same fix that was applied impl FromPyObject for Vec<T> in version 0.17.1 extended to fixed-size arrays. #2675
  • Fix UB in FunctionDescription::extract_arguments_fastcall due to creating slices from a null pointer. #2687

0.17.2 - 2022-10-04

Packaging

  • Added optional chrono feature to convert chrono types into types in the datetime module. #2612

Added

  • Add support for num-bigint feature on PyPy. #2626

Fixed

  • Correctly implement __richcmp__ for enums, fixing __ne__ returning always returning True. #2622
  • Fix compile error since 0.17.0 with Option<&SomePyClass> argument with a default. #2630
  • Fix regression of impl FromPyObject for Vec<T> no longer accepting types passing PySequence_Check, e.g. NumPy arrays, since 0.17.0. #2631

0.17.1 - 2022-08-28

Fixed

  • Fix visibility of PyDictItems, PyDictKeys, and PyDictValues types added in PyO3 0.17.0.
  • Fix compile failure when using #[pyo3(from_py_with = "...")] attribute on an argument of type Option<T>. #2592
  • Fix clippy redundant-closure lint on **kwargs arguments for #[pyfunction] and #[pymethods]. #2595

0.17.0 - 2022-08-23

Packaging

  • Update inventory dependency to 0.3 (the multiple-pymethods feature now requires Rust 1.62 for correctness). #2492

Added

  • Add timezone_utc. #1588
  • Implement ToPyObject for [T; N]. #2313
  • Add PyDictKeys, PyDictValues and PyDictItems Rust types. #2358
  • Add append_to_inittab. #2377
  • Add FFI definition PyFrame_GetCode. #2406
  • Add PyCode and PyFrame high level objects. #2408
  • Add FFI definitions Py_fstring_input, sendfunc, and _PyErr_StackItem. #2423
  • Add PyDateTime::new_with_fold, PyTime::new_with_fold, PyTime::get_fold, and PyDateTime::get_fold for PyPy. #2428
  • Add #[pyclass(frozen)]. #2448
  • Accept #[pyo3(name)] on enum variants. #2457
  • Add CompareOp::matches to implement __richcmp__ as the result of a Rust std::cmp::Ordering comparison. #2460
  • Add PySuper type. #2486
  • Support PyPy on Windows with the generate-import-lib feature. #2506
  • Add FFI definitions Py_EnterRecursiveCall and Py_LeaveRecursiveCall. #2511
  • Add PyDict::get_item_with_error. #2536
  • Add #[pyclass(sequence)] option. #2567

Changed

  • Change datetime constructors taking a tzinfo to take Option<&PyTzInfo> instead of Option<&PyObject>: PyDateTime::new, PyDateTime::new_with_fold, PyTime::new, and PyTime::new_with_fold. #1588
  • Move PyTypeObject::type_object method to the PyTypeInfo trait, and deprecate the PyTypeObject trait. #2287
  • Methods of Py and PyAny now accept impl IntoPy<Py<PyString>> rather than just &str to allow use of the intern! macro. #2312
  • Change the deprecated pyproto feature to be opt-in instead of opt-out. #2322
  • Emit better error messages when #[pyfunction] return types do not implement IntoPy. #2326
  • Require T: IntoPy for impl<T, const N: usize> IntoPy<PyObject> for [T; N] instead of T: ToPyObject. #2326
  • Deprecate the ToBorrowedObject trait. #2333
  • Iterators over PySet and PyDict will now panic if the underlying collection is mutated during the iteration. #2380
  • Iterators over PySet and PyDict will now panic if the underlying collection is mutated during the iteration. #2380
  • Allow #[classattr] methods to be fallible. #2385
  • Prevent multiple #[pymethods] with the same name for a single #[pyclass]. #2399
  • Fixup lib_name when using PYO3_CONFIG_FILE. #2404
  • Add a message to the ValueError raised by the #[derive(FromPyObject)] implementation for a tuple struct. #2414
  • Allow #[classattr] methods to take Python argument. #2456
  • Rework PyCapsule type to resolve soundness issues: #2485
    • PyCapsule::new and PyCapsule::new_with_destructor now take name: Option<CString> instead of &CStr.
    • The destructor F in PyCapsule::new_with_destructor must now be Send.
    • PyCapsule::get_context deprecated in favor of PyCapsule::context which doesn’t take a py: Python<'_> argument.
    • PyCapsule::set_context no longer takes a py: Python<'_> argument.
    • PyCapsule::name now returns PyResult<Option<&CStr>> instead of &CStr.
  • FromPyObject::extract for Vec<T> no longer accepts Python str inputs. #2500
  • Ensure each #[pymodule] is only initialized once. #2523
  • pyo3_build_config::add_extension_module_link_args now also emits linker arguments for wasm32-unknown-emscripten. #2538
  • Type checks for PySequence and PyMapping now require inputs to inherit from (or register with) collections.abc.Sequence and collections.abc.Mapping respectively. #2477
  • Disable PyFunction on when building for abi3 or PyPy. #2542
  • Deprecate Python::acquire_gil. #2549

Removed

  • Remove all functionality deprecated in PyO3 0.15. #2283
  • Make the Dict, WeakRef and BaseNativeType members of the PyClass private implementation details. #2572

Fixed

  • Enable incorrectly disabled FFI definition PyThreadState_DeleteCurrent. #2357
  • Fix wrap_pymodule interactions with name resolution rules: it no longer “sees through” glob imports of use submodule::* when submodule::submodule is a #[pymodule]. #2363
  • Correct FFI definition PyEval_EvalCodeEx to take *const *mut PyObject array arguments instead of *mut *mut PyObject. #2368
  • Fix “raw-ident” structs (e.g. #[pyclass] struct r#RawName) incorrectly having r# at the start of the class name created in Python. #2395
  • Correct FFI definition Py_tracefunc to be unsafe extern "C" fn (was previously safe). #2407
  • Fix compile failure with #[pyo3(from_py_with = "...")] annotations on a field in a #[derive(FromPyObject)] struct. #2414
  • Fix FFI definitions _PyDateTime_BaseTime and _PyDateTime_BaseDateTime lacking leading underscores in their names. #2421
  • Remove FFI definition PyArena on Python 3.10 and up. #2421
  • Fix FFI definition PyCompilerFlags missing member cf_feature_version on Python 3.8 and up. #2423
  • Fix FFI definition PyAsyncMethods missing member am_send on Python 3.10 and up. #2423
  • Fix FFI definition PyGenObject having multiple incorrect members on various Python versions. #2423
  • Fix FFI definition PySyntaxErrorObject missing members end_lineno and end_offset on Python 3.10 and up. #2423
  • Fix FFI definition PyHeapTypeObject missing member ht_module on Python 3.9 and up. #2423
  • Fix FFI definition PyFrameObject having multiple incorrect members on various Python versions. #2424 #2434
  • Fix FFI definition PyTypeObject missing deprecated field tp_print on Python 3.8. #2428
  • Fix FFI definitions PyDateTime_CAPI. PyDateTime_Date, PyASCIIObject, PyBaseExceptionObject, PyListObject, and PyTypeObject on PyPy. #2428
  • Fix FFI definition _inittab field initfunc typo’d as initfun. #2431
  • Fix FFI definitions _PyDateTime_BaseTime and _PyDateTime_BaseDateTime incorrectly having fold member. #2432
  • Fix FFI definitions PyTypeObject. PyHeapTypeObject, and PyCFunctionObject having incorrect members on PyPy 3.9. #2433
  • Fix FFI definition PyGetSetDef to have *const c_char for doc member (not *mut c_char). #2439
  • Fix #[pyo3(from_py_with = "...")] being ignored for 1-element tuple structs and transparent structs. #2440
  • Use memoffset to avoid UB when computing PyCell layout. #2450
  • Fix incorrect enum names being returned by the generated repr for enums renamed by #[pyclass(name = "...")] #2457
  • Fix PyObject_CallNoArgs incorrectly being available when building for abi3 on Python 3.9. #2476
  • Fix several clippy warnings generated by #[pyfunction] arguments. #2503

0.16.6 - 2022-08-23

Changed

  • Fix soundness issues with PyCapsule type with select workarounds. Users are encourage to upgrade to PyO3 0.17 at their earliest convenience which contains API breakages which fix the issues in a long-term fashion. #2522
    • PyCapsule::new and PyCapsule::new_with_destructor now take ownership of a copy of the name to resolve a possible use-after-free.
    • PyCapsule::name now returns an empty CStr instead of dereferencing a null pointer if the capsule has no name.
    • The destructor F in PyCapsule::new_with_destructor will never be called if the capsule is deleted from a thread other than the one which the capsule was created in (a warning will be emitted).
  • Panics during drop of panic payload caught by PyO3 will now abort. #2544

0.16.5 - 2022-05-15

Added

  • Add an experimental generate-import-lib feature to support auto-generating non-abi3 python import libraries for Windows targets. #2364
  • Add FFI definition Py_ExitStatusException. #2374

Changed

  • Deprecate experimental generate-abi3-import-lib feature in favor of the new generate-import-lib feature. #2364

Fixed

  • Added missing warn_default_encoding field to PyConfig on 3.10+. The previously missing field could result in incorrect behavior or crashes. #2370
  • Fixed order of pathconfig_warnings and program_name fields of PyConfig on 3.10+. Previously, the order of the fields was swapped and this could lead to incorrect behavior or crashes. #2370

0.16.4 - 2022-04-14

Added

  • Add PyTzInfoAccess trait for safe access to time zone information. #2263
  • Add an experimental generate-abi3-import-lib feature to auto-generate python3.dll import libraries for Windows. #2282
  • Add FFI definitions for PyDateTime_BaseTime and PyDateTime_BaseDateTime. #2294

Changed

  • Improved performance of failing calls to FromPyObject::extract which is common when functions accept multiple distinct types. #2279
  • Default to “m” ABI tag when choosing libpython link name for CPython 3.7 on Unix. #2288
  • Allow to compile “abi3” extensions without a working build host Python interpreter. #2293

Fixed

  • Crates depending on PyO3 can collect code coverage via LLVM instrumentation using stable Rust. #2286
  • Fix segfault when calling FFI methods PyDateTime_DATE_GET_TZINFO or PyDateTime_TIME_GET_TZINFO on datetime or time without a tzinfo. #2289
  • Fix directory names starting with the letter n breaking serialization of the interpreter configuration on Windows since PyO3 0.16.3. #2299

0.16.3 - 2022-04-05

Packaging

  • Extend parking_lot dependency supported versions to include 0.12. #2239

Added

  • Add methods to pyo3_build_config::InterpreterConfig to run Python scripts using the configured executable. #2092
  • Add as_bytes method to Py<PyBytes>. #2235
  • Add FFI definitions for PyType_FromModuleAndSpec, PyType_GetModule, PyType_GetModuleState and PyModule_AddType. #2250
  • Add pyo3_build_config::cross_compiling_from_to as a helper to detect when PyO3 is cross-compiling. #2253
  • Add #[pyclass(mapping)] option to leave sequence slots empty in container implementations. #2265
  • Add PyString::intern to enable usage of the Python’s built-in string interning. #2268
  • Add intern! macro which can be used to amortize the cost of creating Python strings by storing them inside a GILOnceCell. #2269
  • Add PYO3_CROSS_PYTHON_IMPLEMENTATION environment variable for selecting the default cross Python implementation. #2272

Changed

  • Allow #[pyo3(crate = "...", text_signature = "...")] options to be used directly in #[pyclass(crate = "...", text_signature = "...")]. #2234
  • Make PYO3_CROSS_LIB_DIR environment variable optional when cross compiling. #2241
  • Mark METH_FASTCALL calling convention as limited API on Python 3.10. #2250
  • Deprecate pyo3_build_config::cross_compiling in favor of pyo3_build_config::cross_compiling_from_to. #2253

Fixed

  • Fix abi3-py310 feature: use Python 3.10 ABI when available instead of silently falling back to the 3.9 ABI. #2242
  • Use shared linking mode when cross compiling against a Framework bundle for macOS. #2233
  • Fix panic during compilation when PYO3_CROSS_LIB_DIR is set for some host/target combinations. #2232
  • Correct dependency version for syn to require minimal patch version 1.0.56. #2240

0.16.2 - 2022-03-15

Packaging

  • Warn when modules are imported on PyPy 3.7 versions older than PyPy 7.3.8, as they are known to have binary compatibility issues. #2217
  • Ensure build script of pyo3-ffi runs before that of pyo3 to fix cross compilation. #2224

0.16.1 - 2022-03-05

Packaging

  • Extend hashbrown optional dependency supported versions to include 0.12. #2197

Fixed

  • Fix incorrect platform detection for Windows in pyo3-build-config. #2198
  • Fix regression from 0.16 preventing cross compiling to aarch64 macOS. #2201

0.16.0 - 2022-02-27

Packaging

  • Update MSRV to Rust 1.48. #2004
  • Update indoc optional dependency to 1.0. #2004
  • Drop support for Python 3.6, remove abi3-py36 feature. #2006
  • pyo3-build-config no longer enables the resolve-config feature by default. #2008
  • Update inventory optional dependency to 0.2. #2019
  • Drop paste dependency. #2081
  • The bindings found in pyo3::ffi are now a re-export of a separate pyo3-ffi crate. #2126
  • Support PyPy 3.9. #2143

Added

  • Add PyCapsule type exposing the Capsule API. #1980
  • Add pyo3_build_config::Sysconfigdata and supporting APIs. #1996
  • Add Py::setattr method. #2009
  • Add #[pyo3(crate = "some::path")] option to all attribute macros (except the deprecated #[pyproto]). #2022
  • Enable create_exception! macro to take an optional docstring. #2027
  • Enable #[pyclass] for fieldless (aka C-like) enums. #2034
  • Add buffer magic methods __getbuffer__ and __releasebuffer__ to #[pymethods]. #2067
  • Add support for paths in wrap_pyfunction and wrap_pymodule. #2081
  • Enable wrap_pyfunction! to wrap a #[pyfunction] implemented in a different Rust module or crate. #2091
  • Add PyAny::contains method (in operator for PyAny). #2115
  • Add PyMapping::contains method (in operator for PyMapping). #2133
  • Add garbage collection magic magic methods __traverse__ and __clear__ to #[pymethods]. #2159
  • Add support for from_py_with on struct tuples and enums to override the default from-Python conversion. #2181
  • Add eq, ne, lt, le, gt, ge methods to PyAny that wrap rich_compare. #2175
  • Add Py::is and PyAny::is methods to check for object identity. #2183
  • Add support for the __getattribute__ magic method. #2187

Changed

  • PyType::is_subclass, PyErr::is_instance and PyAny::is_instance now operate run-time type object instead of a type known at compile-time. The old behavior is still available as PyType::is_subclass_of, PyErr::is_instance_of and PyAny::is_instance_of. #1985
  • Rename some methods on PyErr (the old names are just marked deprecated for now): #2026
    • pytype -> get_type
    • pvalue -> value (and deprecate equivalent instance)
    • ptraceback -> traceback
    • from_instance -> from_value
    • into_instance -> into_value
  • PyErr::new_type now takes an optional docstring and now returns PyResult<Py<PyType>> rather than a ffi::PyTypeObject pointer. #2027
  • Deprecate PyType::is_instance; it is inconsistent with other is_instance methods in PyO3. Instead of typ.is_instance(obj), use obj.is_instance(typ). #2031
  • __getitem__, __setitem__ and __delitem__ in #[pymethods] now implement both a Python mapping and sequence by default. #2065
  • Improve performance and error messages for #[derive(FromPyObject)] for enums. #2068
  • Reduce generated LLVM code size (to improve compile times) for:
  • Respect Rust privacy rules for items wrapped with wrap_pyfunction and wrap_pymodule. #2081
  • Add modulo argument to __ipow__ magic method. #2083
  • Fix FFI definition for _PyCFunctionFast. #2126
  • PyDateTimeAPI and PyDateTime_TimeZone_UTC are now unsafe functions instead of statics. #2126
  • PyDateTimeAPI does not implicitly call PyDateTime_IMPORT anymore to reflect the original Python API more closely. Before the first call to PyDateTime_IMPORT a null pointer is returned. Therefore before calling any of the following FFI functions PyDateTime_IMPORT must be called to avoid undefined behavior: #2126
    • PyDateTime_TimeZone_UTC
    • PyDate_Check
    • PyDate_CheckExact
    • PyDateTime_Check
    • PyDateTime_CheckExact
    • PyTime_Check
    • PyTime_CheckExact
    • PyDelta_Check
    • PyDelta_CheckExact
    • PyTZInfo_Check
    • PyTZInfo_CheckExact
    • PyDateTime_FromTimestamp
    • PyDate_FromTimestamp
  • Deprecate the gc option for pyclass (e.g. #[pyclass(gc)]). Just implement a __traverse__ #[pymethod]. #2159
  • The ml_meth field of PyMethodDef is now represented by the PyMethodDefPointer union. 2166
  • Deprecate the #[pyproto] traits. #2173

Removed

  • Remove all functionality deprecated in PyO3 0.14. #2007
  • Remove Default impl for PyMethodDef. #2166
  • Remove PartialEq impl for Py and PyAny (use the new is instead). #2183

Fixed

  • Fix undefined symbol for PyObject_HasAttr on PyPy. #2025
  • Fix memory leak in PyErr::into_value. #2026
  • Fix clippy warning needless-option-as-deref in code generated by #[pyfunction] and #[pymethods]. #2040
  • Fix undefined behavior in PySlice::indices. #2061
  • Fix the wrap_pymodule! macro using the wrong name for a #[pymodule] with a #[pyo3(name = "..")] attribute. #2081
  • Fix magic methods in #[pymethods] accepting implementations with the wrong number of arguments. #2083
  • Fix panic in #[pyfunction] generated code when a required argument following an Option was not provided. #2093
  • Fixed undefined behavior caused by incorrect ExactSizeIterator implementations. #2124
  • Fix missing FFI definition PyCMethod_New on Python 3.9 and up. #2143
  • Add missing FFI definitions _PyLong_NumBits and _PyLong_AsByteArray on PyPy. #2146
  • Fix memory leak in implementation of AsPyPointer for Option<T>. #2160
  • Fix FFI definition of _PyLong_NumBits to return size_t instead of c_int. #2161
  • Fix TypeError thrown when argument parsing failed missing the originating causes. 2177

0.15.2 - 2022-04-14

Packaging

  • Backport of PyPy 3.9 support from PyO3 0.16. #2262

0.15.1 - 2021-11-19

Added

  • Add implementations for Py::as_ref and Py::into_ref for Py<PySequence>, Py<PyIterator> and Py<PyMapping>. #1682
  • Add PyTraceback type to represent and format Python tracebacks. #1977

Changed

  • #[classattr] constants with a known magic method name (which is lowercase) no longer trigger lint warnings expecting constants to be uppercase. #1969

Fixed

  • Fix creating #[classattr] by functions with the name of a known magic method. #1969
  • Fix use of catch_unwind in allow_threads which can cause fatal crashes. #1989
  • Fix build failure on PyPy when abi3 features are activated. #1991
  • Fix mingw platform detection. #1993
  • Fix panic in __get__ implementation when accessing descriptor on type object. #1997

0.15.0 - 2021-11-03

Packaging

  • pyo3’s Cargo.toml now advertises links = "python" to inform Cargo that it links against libpython. #1819
  • Added optional anyhow feature to convert anyhow::Error into PyErr. #1822
  • Support Python 3.10. #1889
  • Added optional eyre feature to convert eyre::Report into PyErr. #1893
  • Support PyPy 3.8. #1948

Added

  • Add PyList::get_item_unchecked and PyTuple::get_item_unchecked to get items without bounds checks. #1733
  • Support #[doc = include_str!(...)] attributes on Rust 1.54 and up. #1746
  • Add PyAny::py as a convenience for PyNativeType::py. #1751
  • Add implementation of std::ops::Index<usize> for PyList, PyTuple and PySequence. #1825
  • Add range indexing implementations of std::ops::Index for PyList, PyTuple and PySequence. #1829
  • Add PyMapping type to represent the Python mapping protocol. #1844
  • Add commonly-used sequence methods to PyList and PyTuple. #1849
  • Add as_sequence methods to PyList and PyTuple. #1860
  • Add support for magic methods in #[pymethods], intended as a replacement for #[pyproto]. #1864
  • Add abi3-py310 feature. #1889
  • Add PyCFunction::new_closure to create a Python function from a Rust closure. #1901
  • Add support for positional-only arguments in #[pyfunction]. #1925
  • Add PyErr::take to attempt to fetch a Python exception if present. #1957

Changed

  • PyList, PyTuple and PySequence’s APIs now accepts only usize indices instead of isize. #1733, #1802, #1803
  • PyList::get_item and PyTuple::get_item now return PyResult<&PyAny> instead of panicking. #1733
  • PySequence::in_place_repeat and PySequence::in_place_concat now return PyResult<&PySequence> instead of PyResult<()>, which is needed in case of immutable sequences such as tuples. #1803
  • PySequence::get_slice now returns PyResult<&PySequence> instead of PyResult<&PyAny>. #1829
  • Deprecate PyTuple::split_from. #1804
  • Deprecate PyTuple::slice, new method PyTuple::get_slice added with usize indices. #1828
  • Deprecate FFI definitions PyParser_SimpleParseStringFlags, PyParser_SimpleParseStringFlagsFilename, PyParser_SimpleParseFileFlags when building for Python 3.9. #1830
  • Mark FFI definitions removed in Python 3.10 PyParser_ASTFromString, PyParser_ASTFromStringObject, PyParser_ASTFromFile, PyParser_ASTFromFileObject, PyParser_SimpleParseStringFlags, PyParser_SimpleParseStringFlagsFilename, PyParser_SimpleParseFileFlags, PyParser_SimpleParseString, PyParser_SimpleParseFile, Py_SymtableString, and Py_SymtableStringObject. #1830
  • #[pymethods] now handles magic methods similarly to #[pyproto]. In the future, #[pyproto] may be deprecated. #1864
  • Deprecate FFI definitions PySys_AddWarnOption, PySys_AddWarnOptionUnicode and PySys_HasWarnOptions. #1887
  • Deprecate #[call] attribute in favor of using fn __call__. #1929
  • Fix missing FFI definition _PyImport_FindExtensionObject on Python 3.10. #1942
  • Change PyErr::fetch to panic in debug mode if no exception is present. #1957

Fixed

  • Fix building with a conda environment on Windows. #1873
  • Fix panic on Python 3.6 when calling Python::with_gil with Python initialized but threading not initialized. #1874
  • Fix incorrect linking to version-specific DLL instead of python3.dll when cross-compiling to Windows with abi3. #1880
  • Fix FFI definition for PyTuple_ClearFreeList incorrectly being present for Python 3.9 and up. #1887
  • Fix panic in generated #[derive(FromPyObject)] for enums. #1888
  • Fix cross-compiling to Python 3.7 builds with the “m” abi flag. #1908
  • Fix __mod__ magic method fallback to __rmod__. #1934.
  • Fix missing FFI definition _PyImport_FindExtensionObject on Python 3.10. #1942

0.14.5 - 2021-09-05

Added

  • Make pyo3_build_config::InterpreterConfig and subfields public. #1848
  • Add resolve-config feature to the pyo3-build-config to control whether its build script does anything. #1856

Fixed

  • Fix 0.14.4 compile regression on s390x-unknown-linux-gnu target. #1850

0.14.4 - 2021-08-29

Changed

  • Mark PyString::data as unsafe and disable it and some supporting PyUnicode FFI APIs (which depend on a C bitfield) on big-endian targets. #1834

0.14.3 - 2021-08-22

Added

  • Add PyString::data to access the raw bytes stored in a Python string. #1794

Fixed

  • Raise AttributeError to avoid panic when calling del on a #[setter] defined class property. #1779
  • Restrict FFI definitions PyGILState_Check and Py_tracefunc to the unlimited API. #1787
  • Add missing _type field to PyStatus struct definition. #1791
  • Reduce lower bound num-complex optional dependency to support interop with rust-numpy and ndarray when building with the MSRV of 1.41 #1799
  • Fix memory leak in Python::run_code. #1806
  • Fix memory leak in PyModule::from_code. #1810
  • Remove use of pyo3:: in pyo3::types::datetime which broke builds using -Z avoid-dev-deps #1811

0.14.2 - 2021-08-09

Added

  • Add indexmap feature to add ToPyObject, IntoPy and FromPyObject implementations for indexmap::IndexMap. #1728
  • Add pyo3_build_config::add_extension_module_link_args to use in build scripts to set linker arguments (for macOS). #1755
  • Add Python::with_gil_unchecked unsafe variation of Python::with_gil to allow obtaining a Python in scenarios where Python::with_gil would fail. #1769

Changed

  • PyErr::new no longer acquires the Python GIL internally. #1724
  • Reverted PyO3 0.14.0’s use of cargo:rustc-cdylib-link-arg in its build script, as Cargo unintentionally allowed crates to pass linker args to downstream crates in this way. Projects supporting macOS may need to restore .cargo/config.toml files. #1755

Fixed

  • Fix regression in 0.14.0 rejecting usage of #[doc(hidden)] on structs and functions annotated with PyO3 macros. #1722
  • Fix regression in 0.14.0 leading to incorrect code coverage being computed for #[pyfunction]s. #1726
  • Fix incorrect FFI definition of Py_Buffer on PyPy. #1737
  • Fix incorrect calculation of dictoffset on 32-bit Windows. #1475
  • Fix regression in 0.13.2 leading to linking to incorrect Python library on Windows “gnu” targets. #1759
  • Fix compiler warning: deny trailing semicolons in expression macro. #1762
  • Fix incorrect FFI definition of Py_DecodeLocale. The 2nd argument is now *mut Py_ssize_t instead of Py_ssize_t. #1766

0.14.1 - 2021-07-04

Added

  • Implement IntoPy<PyObject> for &PathBuf and &OsString. #1712

Fixed

  • Fix crashes on PyPy due to incorrect definitions of PyList_SET_ITEM. #1713

0.14.0 - 2021-07-03

Packaging

  • Update num-bigint optional dependency to 0.4. #1481
  • Update num-complex optional dependency to 0.4. #1482
  • Extend hashbrown optional dependency supported versions to include 0.11. #1496
  • Support PyPy 3.7. #1538

Added

  • Extend conversions for [T; N] to all N using const generics (on Rust 1.51 and up). #1128
  • Add conversions between OsStr/ OsString and Python strings. #1379
  • Add conversions between Path/ PathBuf and Python strings (and pathlib.Path objects). #1379 #1654
  • Add a new set of #[pyo3(...)] attributes to control various PyO3 macro functionality:
    • #[pyo3(from_py_with = "...")] function arguments and struct fields to override the default from-Python conversion. #1411
    • #[pyo3(name = "...")] for setting Python names. #1567
    • #[pyo3(text_signature = "...")] for setting text signature. #1658
  • Add FFI definition PyCFunction_CheckExact for Python 3.9 and later. #1425
  • Add FFI definition Py_IS_TYPE. #1429
  • Add FFI definition _Py_InitializeMain. #1473
  • Add FFI definitions from cpython/import.h.#1475
  • Add tuple and unit struct support for #[pyclass] macro. #1504
  • Add FFI definition PyDateTime_TimeZone_UTC. #1572
  • Add support for #[pyclass(extends=Exception)]. #1591
  • Add PyErr::cause and PyErr::set_cause. #1679
  • Add FFI definitions from cpython/pystate.h. #1687
  • Add wrap_pyfunction! macro to pyo3::prelude. #1695

Changed

  • Allow only one #[pymethods] block per #[pyclass] by default, to remove the dependency on inventory. Add a multiple-pymethods feature to opt-in the original behavior and dependency on inventory. #1457
  • Change PyTimeAccess::get_fold to return a bool instead of a u8. #1397
  • Deprecate FFI definition PyCFunction_Call for Python 3.9 and up. #1425
  • Deprecate FFI definition PyModule_GetFilename. #1425
  • The auto-initialize feature is no longer enabled by default. #1443
  • Change PyCFunction::new and PyCFunction::new_with_keywords to take &'static str arguments rather than implicitly copying (and leaking) them. #1450
  • Deprecate PyModule::call, PyModule::call0, PyModule::call1 and PyModule::get. #1492
  • Add length information to PyBufferErrors raised from PyBuffer::copy_to_slice and PyBuffer::copy_from_slice. #1534
  • Automatically set -undefined and dynamic_lookup linker arguments on macOS with the extension-module feature. #1539
  • Deprecate #[pyproto] methods which are easier to implement as #[pymethods]: #1560
    • PyBasicProtocol::__bytes__ and PyBasicProtocol::__format__
    • PyContextProtocol::__enter__ and PyContextProtocol::__exit__
    • PyDescrProtocol::__delete__ and PyDescrProtocol::__set_name__
    • PyMappingProtocol::__reversed__
    • PyNumberProtocol::__complex__ and PyNumberProtocol::__round__
    • PyAsyncProtocol::__aenter__ and PyAsyncProtocol::__aexit__
  • Deprecate several attributes in favor of the new #[pyo3(...)] options:
    • #[name = "..."], replaced by #[pyo3(name = "...")] #1567
    • #[pyfn(m, "name")], replaced by #[pyfn(m)] #[pyo3(name = "...")]. #1610
    • #[pymodule(name)], replaced by #[pymodule] #[pyo3(name = "...")] #1650
    • #[text_signature = "..."], replaced by #[pyo3(text_signature = "...")]. #1658
  • Reduce LLVM line counts to improve compilation times. #1604
  • No longer call PyEval_InitThreads in #[pymodule] init code. #1630
  • Use METH_FASTCALL argument passing convention, when possible, to improve #[pyfunction] and method performance. #1619, #1660
  • Filter sysconfigdata candidates by architecture when cross-compiling. #1626

Removed

  • Remove deprecated exception names BaseException etc. #1426
  • Remove deprecated methods Python::is_instance, Python::is_subclass, Python::release, Python::xdecref, and Py::from_owned_ptr_or_panic. #1426
  • Remove many FFI definitions which never existed in the Python C-API:
    • (previously deprecated) PyGetSetDef_INIT, PyGetSetDef_DICT, PyCoro_Check, PyCoroWrapper_Check, and PyAsyncGen_Check #1426
    • PyMethodDef_INIT #1426
    • PyTypeObject_INIT #1429
    • PyObject_Check, PySuper_Check, and FreeFunc #1438
    • PyModuleDef_INIT #1630
  • Remove pyclass implementation details from PyTypeInfo:
    • Type, DESCRIPTION, and FLAGS #1456
    • BaseType, BaseLayout, Layout, Initializer #1596
  • Remove PYO3_CROSS_INCLUDE_DIR environment variable and the associated C header parsing functionality. #1521
  • Remove raw_pycfunction! macro. #1619
  • Remove PyClassAlloc trait. #1657
  • Remove PyList::get_parked_item. #1664

Fixed

  • Remove FFI definition PyCFunction_ClearFreeList for Python 3.9 and later. #1425
  • PYO3_CROSS_LIB_DIR environment variable no long required when compiling for x86-64 Python from macOS arm64 and reverse. #1428
  • Fix FFI definition _PyEval_RequestCodeExtraIndex, which took an argument of the wrong type. #1429
  • Fix FFI definition PyIndex_Check missing with the abi3 feature. #1436
  • Fix incorrect TypeError raised when keyword-only argument passed along with a positional argument in *args. #1440
  • Fix inability to use a named lifetime for &PyTuple of *args in #[pyfunction]. #1440
  • Fix use of Python argument for #[pymethods] inside macro expansions. #1505
  • No longer include __doc__ in __all__ generated for #[pymodule]. #1509
  • Always use cross-compiling configuration if any of the PYO3_CROSS family of environment variables are set. #1514
  • Support EnvironmentError, IOError, and WindowsError on PyPy. #1533
  • Fix unnecessary rebuilds when cycling between cargo check and cargo clippy in a Python virtualenv. #1557
  • Fix segfault when dereferencing ffi::PyDateTimeAPI without the GIL. #1563
  • Fix memory leak in FromPyObject implementations for u128 and i128. #1638
  • Fix #[pyclass(extends=PyDict)] leaking the dict contents on drop. #1657
  • Fix segfault when calling PyList::get_item with negative indices. #1668
  • Fix FFI definitions of PyEval_SetProfile/PyEval_SetTrace to take Option<Py_tracefunc> parameters. #1692
  • Fix ToPyObject impl for HashSet to accept non-default hashers. #1702

0.13.2 - 2021-02-12

Packaging

  • Lower minimum supported Rust version to 1.41. #1421

Added

  • Add unsafe API with_embedded_python_interpreter to initialize a Python interpreter, execute a closure, and finalize the interpreter. #1355
  • Add serde feature which provides implementations of Serialize and Deserialize for Py<T>. #1366
  • Add FFI definition _PyCFunctionFastWithKeywords on Python 3.7 and up. #1384
  • Add PyDateTime::new_with_fold method. #1398
  • Add size_hint impls for {PyDict,PyList,PySet,PyTuple}Iterators. #1699

Changed

  • prepare_freethreaded_python will no longer register an atexit handler to call Py_Finalize. This resolves a number of issues with incompatible C extensions causing crashes at finalization. #1355
  • Mark PyLayout::py_init, PyClassDict::clear_dict, and opt_to_pyobj safe, as they do not perform any unsafe operations. #1404

Fixed

  • Fix support for using r#raw_idents as argument names in pyfunctions. #1383
  • Fix typo in FFI definition for PyFunction_GetCode (was incorrectly PyFunction_Code). #1387
  • Fix FFI definitions PyMarshal_WriteObjectToString and PyMarshal_ReadObjectFromString as available in limited API. #1387
  • Fix FFI definitions PyListObject and those from funcobject.h as requiring non-limited API. #1387
  • Fix unqualified Result usage in pyobject_native_type_base. #1402
  • Fix build on systems where the default Python encoding is not UTF-8. #1405
  • Fix build on mingw / MSYS2. #1423

0.13.1 - 2021-01-10

Added

  • Add support for #[pyclass(dict)] and #[pyclass(weakref)] with the abi3 feature on Python 3.9 and up. #1342
  • Add FFI definitions PyOS_BeforeFork, PyOS_AfterFork_Parent, PyOS_AfterFork_Child for Python 3.7 and up. #1348
  • Add an auto-initialize feature to control whether PyO3 should automatically initialize an embedded Python interpreter. For compatibility this feature is enabled by default in PyO3 0.13.1, but is planned to become opt-in from PyO3 0.14.0. #1347
  • Add support for cross-compiling to Windows without needing PYO3_CROSS_INCLUDE_DIR. #1350

Deprecated

  • Deprecate FFI definitions PyEval_CallObjectWithKeywords, PyEval_CallObject, PyEval_CallFunction, PyEval_CallMethod when building for Python 3.9. #1338
  • Deprecate FFI definitions PyGetSetDef_DICT and PyGetSetDef_INIT which have never been in the Python API. #1341
  • Deprecate FFI definitions PyGen_NeedsFinalizing, PyImport_Cleanup (removed in 3.9), and PyOS_InitInterrupts (3.10). #1348
  • Deprecate FFI definition PyOS_AfterFork for Python 3.7 and up. #1348
  • Deprecate FFI definitions PyCoro_Check, PyAsyncGen_Check, and PyCoroWrapper_Check, which have never been in the Python API (for the first two, it is possible to use PyCoro_CheckExact and PyAsyncGen_CheckExact instead; these are the actual functions provided by the Python API). #1348
  • Deprecate FFI definitions for PyUnicode_FromUnicode, PyUnicode_AsUnicode and PyUnicode_AsUnicodeAndSize, which will be removed from 3.12 and up due to PEP 623. #1370

Removed

  • Remove FFI definition PyFrame_ClearFreeList when building for Python 3.9. #1341
  • Remove FFI definition _PyDict_Contains when building for Python 3.10. #1341
  • Remove FFI definitions PyGen_NeedsFinalizing and PyImport_Cleanup (for 3.9 and up), and PyOS_InitInterrupts (3.10). #1348

Fixed

  • Stop including Py_TRACE_REFS config setting automatically if Py_DEBUG is set on Python 3.8 and up. #1334
  • Remove #[deny(warnings)] attribute (and instead refuse warnings only in CI). #1340
  • Fix deprecation warning for missing __module__ with #[pyclass]. #1343
  • Correct return type of PyFrozenSet::empty to &PyFrozenSet (was incorrectly &PySet). #1351
  • Fix missing Py_INCREF on heap type objects on Python versions before 3.8. #1365

0.13.0 - 2020-12-22

Packaging

  • Drop support for Python 3.5 (as it is now end-of-life). #1250
  • Bump minimum supported Rust version to 1.45. #1272
  • Bump indoc dependency to 1.0. #1272
  • Bump paste dependency to 1.0. #1272
  • Rename internal crates pyo3cls and pyo3-derive-backend to pyo3-macros and pyo3-macros-backend respectively. #1317

Added

  • Add support for building for CPython limited API. Opting-in to the limited API enables a single extension wheel built with PyO3 to be installable on multiple Python versions. This required a few minor changes to runtime behavior of of PyO3 #[pyclass] types. See the migration guide for full details. #1152
    • Add feature flags abi3-py36, abi3-py37, abi3-py38 etc. to set the minimum Python version when using the limited API. #1263
  • Add argument names to TypeError messages generated by pymethod wrappers. #1212
  • Add FFI definitions for PEP 587 “Python Initialization Configuration”. #1247
  • Add FFI definitions for PyEval_SetProfile and PyEval_SetTrace. #1255
  • Add FFI definitions for context.h functions (PyContext_New, etc). #1259
  • Add PyAny::is_instance method. #1276
  • Add support for conversion between char and PyString. #1282
  • Add FFI definitions for PyBuffer_SizeFromFormat, PyObject_LengthHint, PyObject_CallNoArgs, PyObject_CallOneArg, PyObject_CallMethodNoArgs, PyObject_CallMethodOneArg, PyObject_VectorcallDict, and PyObject_VectorcallMethod. #1287
  • Add conversions between u128/i128 and PyLong for PyPy. #1310
  • Add Python::version and Python::version_info to get the running interpreter version. #1322
  • Add conversions for tuples of length 10, 11, and 12. #1454

Changed

  • Change return type of PyType::name from Cow<str> to PyResult<&str>. #1152
  • #[pyclass(subclass)] is now required for subclassing from Rust (was previously just required for subclassing from Python). #1152
  • Change PyIterator to be consistent with other native types: it is now used as &PyIterator instead of PyIterator<'a>. #1176
  • Change formatting of PyDowncastError messages to be closer to Python’s builtin error messages. #1212
  • Change Debug and Display impls for PyException to be consistent with PyAny. #1275
  • Change Debug impl of PyErr to output more helpful information (acquiring the GIL if necessary). #1275
  • Rename PyTypeInfo::is_instance and PyTypeInfo::is_exact_instance to PyTypeInfo::is_type_of and PyTypeInfo::is_exact_type_of. #1278
  • Optimize PyAny::call0, Py::call0 and PyAny::call_method0 and Py::call_method0 on Python 3.9 and up. #1287
  • Require double-quotes for pyclass name argument e.g #[pyclass(name = "MyClass")]. #1303

Deprecated

  • Deprecate Python::is_instance, Python::is_subclass, Python::release, and Python::xdecref. #1292

Removed

  • Remove deprecated ffi definitions PyUnicode_AsUnicodeCopy, PyUnicode_GetMax, _Py_CheckRecursionLimit, PyObject_AsCharBuffer, PyObject_AsReadBuffer, PyObject_CheckReadBuffer and PyObject_AsWriteBuffer, which will be removed in Python 3.10. #1217
  • Remove unused python3 feature. #1235

Fixed

  • Fix missing field in PyCodeObject struct (co_posonlyargcount) - caused invalid access to other fields in Python >3.7. #1260
  • Fix building for x86_64-unknown-linux-musl target from x86_64-unknown-linux-gnu host. #1267
  • Fix #[text_signature] interacting badly with rust r#raw_identifiers. #1286
  • Fix FFI definitions for PyObject_Vectorcall and PyVectorcall_Call. #1287
  • Fix building with Anaconda python inside a virtualenv. #1290
  • Fix definition of opaque FFI types. #1312
  • Fix using custom error type in pyclass #[new] methods. #1319

0.12.4 - 2020-11-28

Fixed

  • Fix reference count bug in implementation of From<Py<T>> for PyObject, a regression introduced in PyO3 0.12. #1297

0.12.3 - 2020-10-12

Fixed

  • Fix support for Rust versions 1.39 to 1.44, broken by an incorrect internal update to paste 1.0 which was done in PyO3 0.12.2. #1234

0.12.2 - 2020-10-12

Added

  • Add support for keyword-only arguments without default values in #[pyfunction]. #1209
  • Add Python::check_signals as a safe a wrapper for PyErr_CheckSignals. #1214

Fixed

  • Fix invalid document for protocol methods. #1169
  • Hide docs of PyO3 private implementation details in pyo3::class::methods. #1169
  • Fix unnecessary rebuild on PATH changes when the python interpreter is provided by PYO3_PYTHON. #1231

0.12.1 - 2020-09-16

Fixed

  • Fix building for a 32-bit Python on 64-bit Windows with a 64-bit Rust toolchain. #1179
  • Fix building on platforms where c_char is u8. #1182

0.12.0 - 2020-09-12

Added

  • Add FFI definitions Py_FinalizeEx, PyOS_getsig, and PyOS_setsig. #1021
  • Add PyString::to_str for accessing PyString as &str. #1023
  • Add Python::with_gil for executing a closure with the Python GIL. #1037
  • Add type information to failures in PyAny::downcast. #1050
  • Implement Debug for PyIterator. #1051
  • Add PyBytes::new_with and PyByteArray::new_with for initialising bytes and bytearray objects using a closure. #1074
  • Add #[derive(FromPyObject)] macro for enums and structs. #1065
  • Add Py::as_ref and Py::into_ref for converting Py<T> to &T. #1098
  • Add ability to return Result types other than PyResult from #[pyfunction], #[pymethod] and #[pyproto] functions. #1106.
  • Implement ToPyObject, IntoPy, and FromPyObject for hashbrown’s HashMap and HashSet types (requires the hashbrown feature). #1114
  • Add #[pyfunction(pass_module)] and #[pyfn(pass_module)] to pass the module object as the first function argument. #1143
  • Add PyModule::add_function and PyModule::add_submodule as typed alternatives to PyModule::add_wrapped. #1143
  • Add native PyCFunction and PyFunction types. #1163

Changed

  • Rework exception types: #1024 #1115
    • Rename exception types from e.g. RuntimeError to PyRuntimeError. The old names continue to exist but are deprecated.
    • Exception objects are now accessible as &T or Py<T>, just like other Python-native types.
    • Rename PyException::py_err to PyException::new_err.
    • Rename PyUnicodeDecodeErr::new_err to PyUnicodeDecodeErr::new.
    • Remove PyStopIteration::stop_iteration.
  • Require T: Send for the return value T of Python::allow_threads. #1036
  • Rename PYTHON_SYS_EXECUTABLE to PYO3_PYTHON. The old name will continue to work (undocumented) but will be removed in a future release. #1039
  • Remove unsafe from signature of PyType::as_type_ptr. #1047
  • Change return type of PyIterator::from_object to PyResult<PyIterator> (was Result<PyIterator, PyDowncastError>). #1051
  • IntoPy is no longer implied by FromPy. #1063
  • Change PyObject to be a type alias for Py<PyAny>. #1063
  • Rework PyErr to be compatible with the std::error::Error trait: #1067 #1115
    • Implement Display, Error, Send and Sync for PyErr and PyErrArguments.
    • Add PyErr::instance for accessing PyErr as &PyBaseException.
    • PyErr’s fields are now an implementation detail. The equivalent values can be accessed with PyErr::ptype, PyErr::pvalue and PyErr::ptraceback.
    • Change receiver of PyErr::print and PyErr::print_and_set_sys_last_vars to &self (was self).
    • Remove PyErrValue, PyErr::from_value, PyErr::into_normalized, and PyErr::normalize.
    • Remove PyException::into.
    • Remove Into<PyResult<T>> for PyErr and PyException.
  • Change methods generated by #[pyproto] to return NotImplemented if Python should try a reversed operation. #1072
  • Change argument to PyModule::add to impl IntoPy<PyObject> (was impl ToPyObject). #1124

Removed

  • Remove many exception and PyErr APIs; see the “changed” section above. #1024 #1067 #1115
  • Remove PyString::to_string (use new PyString::to_str). #1023
  • Remove PyString::as_bytes. #1023
  • Remove Python::register_any. #1023
  • Remove GILGuard::acquire from the public API. Use Python::acquire_gil or Python::with_gil. #1036
  • Remove the FromPy trait. #1063
  • Remove the AsPyRef trait. #1098

Fixed

  • Correct FFI definitions Py_SetProgramName and Py_SetPythonHome to take *const arguments (was *mut). #1021
  • Fix FromPyObject for num_bigint::BigInt for Python objects with an __index__ method. #1027
  • Correct FFI definition _PyLong_AsByteArray to take *mut c_uchar argument (was *const c_uchar). #1029
  • Fix segfault with #[pyclass(dict, unsendable)]. #1058 #1059
  • Fix using &Self as an argument type for functions in a #[pymethods] block. #1071
  • Fix best-effort build against PyPy 3.6. #1092
  • Fix many cases of lifetime elision in #[pyproto] implementations. #1093
  • Fix detection of Python build configuration when cross-compiling. #1095
  • Always link against libpython on android with the extension-module feature. #1095
  • Fix the + operator not trying __radd__ when both __add__ and __radd__ are defined in PyNumberProtocol (and similar for all other reversible operators). #1107
  • Fix building with Anaconda python. #1175

0.11.1 - 2020-06-30

Added

  • #[pyclass(unsendable)]. #1009

Changed

  • Update parking_lot dependency to 0.11. #1010

0.11.0 - 2020-06-28

Added

  • Support stable versions of Rust (>=1.39). #969
  • Add FFI definition PyObject_AsFileDescriptor. #938
  • Add PyByteArray::data, PyByteArray::as_bytes, and PyByteArray::as_bytes_mut. #967
  • Add GILOnceCell to use in situations where lazy_static or once_cell can deadlock. #975
  • Add Py::borrow, Py::borrow_mut, Py::try_borrow, and Py::try_borrow_mut for accessing #[pyclass] values. #976
  • Add IterNextOutput and IterANextOutput for returning from __next__ / __anext__. #997

Changed

  • Simplify internals of #[pyo3(get)] attribute. (Remove the hidden API GetPropertyValue.) #934
  • Call Py_Finalize at exit to flush buffers, etc. #943
  • Add type parameter to PyBuffer. #951
  • Require Send bound for #[pyclass]. #966
  • Add Python argument to most methods on PyObject and Py<T> to ensure GIL safety. #970
  • Change signature of PyTypeObject::type_object - now takes Python argument and returns &PyType. #970
  • Change return type of PyTuple::slice and PyTuple::split_from from Py<PyTuple> to &PyTuple. #970
  • Change return type of PyTuple::as_slice to &[&PyAny]. #971
  • Rename PyTypeInfo::type_object to type_object_raw, and add Python argument. #975
  • Update num-complex optional dependency from 0.2 to 0.3. #977
  • Update num-bigint optional dependency from 0.2 to 0.3. #978
  • #[pyproto] is re-implemented without specialization. #961
  • PyClassAlloc::alloc is renamed to PyClassAlloc::new. #990
  • #[pyproto] methods can now have return value T or PyResult<T> (previously only PyResult<T> was supported). #996
  • #[pyproto] methods can now skip annotating the return type if it is (). #998

Removed

  • Remove ManagedPyRef (unused, and needs specialization) #930

Fixed

  • Fix passing explicit None to Option<T> argument #[pyfunction] with a default value. #936
  • Fix PyClass.__new__’s not respecting subclasses when inherited by a Python class. #990
  • Fix returning Option<T> from #[pyproto] methods. #996
  • Fix accepting PyRef<Self> and PyRefMut<Self> to #[getter] and #[setter] methods. #999

0.10.1 - 2020-05-14

Fixed

  • Fix deadlock in Python::acquire_gil after dropping a PyObject or Py<T>. #924

0.10.0 - 2020-05-13

Added

  • Add FFI definition _PyDict_NewPresized. #849
  • Implement IntoPy<PyObject> for HashSet and BTreeSet. #864
  • Add PyAny::dir method. #886
  • Gate macros behind a macros feature (enabled by default). #897
  • Add ability to define class attributes using #[classattr] on functions in #[pymethods]. #905
  • Implement Clone for PyObject and Py<T>. #908
  • Implement Deref<Target = PyAny> for all builtin types. (PyList, PyTuple, PyDict etc.) #911
  • Implement Deref<Target = PyAny> for PyCell<T>. #911
  • Add #[classattr] support for associated constants in #[pymethods]. #914

Changed

  • Panics will now be raised as a Python PanicException. #797
  • Change PyObject and Py<T> reference counts to decrement immediately upon drop when the GIL is held. #851
  • Allow PyIterProtocol methods to use either PyRef or PyRefMut as the receiver type. #856
  • Change the implementation of FromPyObject for Py<T> to apply to a wider range of T, including all T: PyClass. #880
  • Move all methods from the ObjectProtocol trait to the PyAny struct. #911
  • Remove need for #![feature(specialization)] in crates depending on PyO3. #917

Removed

  • Remove PyMethodsProtocol trait. #889
  • Remove num-traits dependency. #895
  • Remove ObjectProtocol trait. #911
  • Remove PyAny::None. Users should use Python::None instead. #911
  • Remove all *ProtocolImpl traits. #917

Fixed

  • Fix support for __radd__ and other __r*__ methods as implementations for Python mathematical operators. #839
  • Fix panics during garbage collection when traversing objects that were already mutably borrowed. #855
  • Prevent &'static references to Python objects as arguments to #[pyfunction] and #[pymethods]. #869
  • Fix lifetime safety bug with AsPyRef::as_ref. #876
  • Fix #[pyo3(get)] attribute on Py<T> fields. #880
  • Fix segmentation faults caused by functions such as PyList::get_item returning borrowed objects when it was not safe to do so. #890
  • Fix segmentation faults caused by nested Python::acquire_gil calls creating dangling references. #893
  • Fix segmentatation faults when a panic occurs during a call to Python::allow_threads. #912

0.9.2 - 2020-04-09

Added

  • FromPyObject implementations for HashSet and BTreeSet. #842

Fixed

  • Correctly detect 32bit architecture. #830

0.9.1 - 2020-03-23

Fixed

  • Error messages for #[pyclass]. #826
  • FromPyObject implementation for PySequence. #827

0.9.0 - 2020-03-19

Added

  • PyCell, which has RefCell-like features. #770
  • PyClass, PyLayout, PyClassInitializer. #683
  • Implemented IntoIterator for PySet and PyFrozenSet. #716
  • FromPyObject is now automatically implemented for T: Clone pyclasses. #730
  • #[pyo3(get)] and #[pyo3(set)] will now use the Rust doc-comment from the field for the Python property. #755
  • #[setter] functions may now take an argument of Pyo3::Python. #760
  • PyTypeInfo::BaseLayout and PyClass::BaseNativeType. #770
  • PyDowncastImpl. #770
  • Implement FromPyObject and IntoPy<PyObject> traits for arrays (up to 32). #778
  • migration.md and types.md in the guide. #795, #802
  • ffi::{_PyBytes_Resize, _PyDict_Next, _PyDict_Contains, _PyDict_GetDictPtr}. #820

Changed

  • #[new] does not take PyRawObject and can return Self. #683
  • The blanket implementations for FromPyObject for &T and &mut T are no longer specializable. Implement PyTryFrom for your type to control the behavior of FromPyObject::extract for your types. #713
  • The implementation for IntoPy<U> for T where U: FromPy<T> is no longer specializable. Control the behavior of this via the implementation of FromPy. #713
  • Use parking_lot::Mutex instead of spin::Mutex. #734
  • Bumped minimum Rust version to 1.42.0-nightly 2020-01-21. #761
  • PyRef and PyRefMut are renewed for PyCell. #770
  • Some new FFI functions for Python 3.8. #784
  • PyAny is now on the top level module and prelude. #816

Removed

  • PyRawObject. #683
  • PyNoArgsFunction. #741
  • initialize_type. To set the module name for a #[pyclass], use the module argument to the macro. #751
  • AsPyRef::as_mut/with/with_mut/into_py/into_mut_py. #770
  • PyTryFrom::try_from_mut/try_from_mut_exact/try_from_mut_unchecked. #770
  • Python::mut_from_owned_ptr/mut_from_borrowed_ptr. #770
  • ObjectProtocol::get_base/get_mut_base. #770

Fixed

  • Fixed unsoundness of subclassing. #683.
  • Clear error indicator when the exception is handled on the Rust side. #719
  • Usage of raw identifiers with #[pyo3(set)]. #745
  • Usage of PyObject with #[pyo3(get)]. #760
  • #[pymethods] used in conjunction with #[cfg]. #769
  • "*" in a #[pyfunction()] argument list incorrectly accepting any number of positional arguments (use args = "*" when this behavior is desired). #792
  • PyModule::dict. #809
  • Fix the case where DESCRIPTION is not null-terminated. #822

0.8.5 - 2020-01-05

Added

  • Implemented FromPyObject for HashMap and BTreeMap
  • Support for #[name = "foo"] attribute for #[pyfunction] and in #[pymethods]. #692

0.8.4 - 2019-12-14

Added

  • Support for #[text_signature] attribute. #675

0.8.3 - 2019-11-23

Removed

  • #[init] is removed. #658

Fixed

  • Now all &Py~ types have !Send bound. #655
  • Fix a compile error raised by the stabilization of ! type. #672.

0.8.2 - 2019-10-27

Added

  • FFI compatibility for PEP 590 Vectorcall. #641

Fixed

  • Fix PySequenceProtocol::set_item. #624
  • Fix a corner case of BigInt::FromPyObject. #630
  • Fix index errors in parameter conversion. #631
  • Fix handling of invalid utf-8 sequences in PyString::as_bytes. #639 and PyString::to_string_lossy #642.
  • Remove __contains__ and __iter__ from PyMappingProtocol. #644
  • Fix proc-macro definition of PySetAttrProtocol. #645

0.8.1 - 2019-10-08

Added

Fixed

  • Make sure the right Python interpreter is used in OSX builds. #604
  • Patch specialization being broken by Rust 1.40. #614
  • Fix a segfault around PyErr. #597

0.8.0 - 2019-09-16

Added

  • module argument to pyclass macro. #499
  • py_run! macro #512
  • Use existing fields and methods before calling custom getattr. #505
  • PyBytes can now be indexed just like Vec<u8>
  • Implement IntoPy<PyObject> for PyRef and PyRefMut.

Changed

  • Implementing the Using the gc parameter for pyclass (e.g. #[pyclass(gc)]) without implementing the class::PyGCProtocol trait is now a compile-time error. Failing to implement this trait could lead to segfaults. #532
  • PyByteArray::data has been replaced with PyDataArray::to_vec because returning a &[u8] is unsound. (See this comment for a great write-up for why that was unsound)
  • Replace mashup with paste.
  • GILPool gained a Python marker to prevent it from being misused to release Python objects without the GIL held.

Removed

  • IntoPyObject was replaced with IntoPy<PyObject>
  • #[pyclass(subclass)] is hidden a unsound-subclass feature because it’s causing segmentation faults.

Fixed

  • More readable error message for generics in pyclass #503

0.7.0 - 2019-05-26

Added

  • PyPy support by omerbenamram in #393
  • Have PyModule generate an index of its members (__all__ list).
  • Allow slf: PyRef<T> for pyclass(#419)
  • Allow to use lifetime specifiers in pymethods
  • Add marshal module. #460

Changed

  • Python::run returns PyResult<()> instead of PyResult<&PyAny>.
  • Methods decorated with #[getter] and #[setter] can now omit wrapping the result type in PyResult if they don’t raise exceptions.

Fixed

  • type_object::PyTypeObject has been marked unsafe because breaking the contract type_object::PyTypeObject::init_type can lead to UB.
  • Fixed automatic derive of PySequenceProtocol implementation in #423.
  • Capitalization & better wording to README.md.
  • Docstrings of properties is now properly set using the doc of the #[getter] method.
  • Fixed issues with pymethods crashing on doc comments containing double quotes.
  • PySet::new and PyFrozenSet::new now return PyResult<&Py[Frozen]Set>; exceptions are raised if the items are not hashable.
  • Fixed building using venv on Windows.
  • PyTuple::new now returns &PyTuple instead of Py<PyTuple>.
  • Fixed several issues with argument parsing; notable, the *args and **kwargs tuple/dict now doesn’t contain arguments that are otherwise assigned to parameters.

0.6.0 - 2019-03-28

Regressions

  • Currently, #341 causes cargo test to fail with weird linking errors when the extension-module feature is activated. For now you can work around this by making the extension-module feature optional and running the tests with cargo test --no-default-features:
[dependencies.pyo3]
version = "0.6.0"

[features]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]

Added

  • Added a wrap_pymodule! macro similar to the existing wrap_pyfunction! macro. Only available on python 3
  • Added support for cross compiling (e.g. to arm v7) by mtp401 in #327. See the “Cross Compiling” section in the “Building and Distribution” chapter of the guide for more details.
  • The PyRef and PyRefMut types, which allow to differentiate between an instance of a rust struct on the rust heap and an instance that is embedded inside a python object. By kngwyu in #335
  • Added FromPy<T> and IntoPy<T> which are equivalent to From<T> and Into<T> except that they require a gil token.
  • Added ManagedPyRef, which should eventually replace ToBorrowedObject.

Changed

  • Renamed PyObjectRef to PyAny in #388
  • Renamed add_function to add_wrapped as it now also supports modules.
  • Renamed #[pymodinit] to #[pymodule]
  • py.init(|| value) becomes Py::new(value)
  • py.init_ref(|| value) becomes PyRef::new(value)
  • py.init_mut(|| value) becomes PyRefMut::new(value).
  • PyRawObject::init is now infallible, e.g. it returns () instead of PyResult<()>.
  • Renamed py_exception! to create_exception! and refactored the error macros.
  • Renamed wrap_function! to wrap_pyfunction!
  • Renamed #[prop(get, set)] to #[pyo3(get, set)]
  • #[pyfunction] now supports the same arguments as #[pyfn()]
  • Some macros now emit proper spanned errors instead of panics.
  • Migrated to the 2018 edition
  • crate::types::exceptions moved to crate::exceptions
  • Replace IntoPyTuple with IntoPy<Py<PyTuple>>.
  • IntoPyPointer and ToPyPointer moved into the crate root.
  • class::CompareOp moved into class::basic::CompareOp
  • PyTypeObject is now a direct subtrait PyTypeCreate, removing the old cyclical implementation in #350
  • Add PyList::{sort, reverse} by chr1sj0nes in #357 and #358
  • Renamed the typeob module to type_object

Removed

  • PyToken was removed due to unsoundness (See #94).
  • Removed the unnecessary type parameter from PyObjectAlloc
  • NoArgs. Just use an empty tuple
  • PyObjectWithGIL. PyNativeType is sufficient now that PyToken is removed.

Fixed

  • A soudness hole where every instances of a #[pyclass] struct was considered to be part of a python object, even though you can create instances that are not part of the python heap. This was fixed through PyRef and PyRefMut.
  • Fix kwargs support in #328.
  • Add full support for __dict__ in #403.

0.5.3 - 2019-01-04

Fixed

  • Fix memory leak in ArrayList by kngwyu #316

0.5.2 - 2018-11-25

Fixed

  • Fix indeterministic segfaults when creating many objects by kngwyu in #281

0.5.1 - 2018-11-24

Yanked

0.5.0 - 2018-11-11

Added

  • #[pyclass] objects can now be returned from rust functions
  • PyComplex by kngwyu in #226
  • PyDict::from_sequence, equivalent to dict([(key, val), ...])
  • Bindings for the datetime standard library types: PyDate, PyTime, PyDateTime, PyTzInfo, PyDelta with associated ffi types, by pganssle #200.
  • PyString, PyUnicode, and PyBytes now have an as_bytes method that returns &[u8].
  • PyObjectProtocol::get_type_ptr by ijl in #242

Changed

  • Removes the types from the root module and the prelude. They now live in pyo3::types instead.
  • All exceptions are constructed with py_err instead of new, as they return PyErr and not Self.
  • as_mut and friends take and &mut self instead of &self
  • ObjectProtocol::call now takes an Option<&PyDict> for the kwargs instead of an IntoPyDictPointer.
  • IntoPyDictPointer was replace by IntoPyDict which doesn’t convert PyDict itself anymore and returns a PyDict instead of *mut PyObject.
  • PyTuple::new now takes an IntoIterator instead of a slice
  • Updated to syn 0.15
  • Split PyTypeObject into PyTypeObject without the create method and PyTypeCreate with requires PyObjectAlloc<Self> + PyTypeInfo + Sized.
  • Ran cargo edition --fix which prefixed path with crate:: for rust 2018
  • Renamed async to pyasync as async will be a keyword in the 2018 edition.
  • Starting to use NonNull<*mut PyObject> for Py and PyObject by ijl #260

Removed

  • Removed most entries from the prelude. The new prelude is small and clear.
  • Slowly removing specialization uses
  • PyString, PyUnicode, and PyBytes no longer have a data method (replaced by as_bytes) and PyStringData has been removed.
  • The pyobject_extract macro

Fixed

  • Added an explanation that the GIL can temporarily be released even while holding a GILGuard.
  • Lots of clippy errors
  • Fix segfault on calling an unknown method on a PyObject
  • Work around a bug in the rust compiler by kngwyu #252
  • Fixed a segfault with subclassing pyo3 create classes and using __class__ by kngwyu #263

0.4.1 - 2018-08-20

Changed

  • PyTryFrom’s error is always to PyDowncastError

Fixed

  • Fixed compilation on nightly since use_extern_macros was stabilized

Removed

  • The pyobject_downcast macro

0.4.0 - 2018-07-30

Changed

  • Merged both examples into one
  • Rustfmt all the things :heavy_check_mark:
  • Switched to Keep a Changelog

Removed

0.3.2 - 2018-07-22

Changed

  • Replaced concat_idents with mashup

0.3.1 - 2018-07-18

Fixed

  • Fixed scoping bug in pyobject_native_type that would break rust-numpy

0.3.0 - 2018-07-18

Added

  • A few internal macros became part of the public api (#155, #186)
  • Always clone in getters. This allows using the get-annotation on all Clone-Types

Changed

  • Upgraded to syn 0.14 which means much better error messages :tada:
  • 128 bit integer support by kngwyu (#137)
  • proc_macro has been stabilized on nightly (rust-lang/rust#52081). This means that we can remove the proc_macro feature, but now we need the use_extern_macros from the 2018 edition instead.
  • All proc macro are now prefixed with py and live in the prelude. This means you can use #[pyclass], #[pymethods], #[pyproto], #[pyfunction] and #[pymodinit] directly, at least after a use pyo3::prelude::*. They were also moved into a module called proc_macro. You shouldn’t use #[pyo3::proc_macro::pyclass] or other longer paths in attributes because proc_macro_path_invoc isn’t going to be stabilized soon.
  • Renamed the base option in the pyclass macro to extends.
  • #[pymodinit] uses the function name as module name, unless the name is overridden with #[pymodinit(name)]
  • The guide is now properly versioned.

0.2.7 - 2018-05-18

Fixed

  • Fix nightly breakage with proc_macro_path

0.2.6 - 2018-04-03

Fixed

  • Fix compatibility with TryFrom trait #137

0.2.5 - 2018-02-21

Added

  • CPython 3.7 support

Fixed

  • Embedded CPython 3.7b1 crashes on initialization #110
  • Generated extension functions are weakly typed #108
  • call_method* crashes when the method does not exist #113
  • Allow importing exceptions from nested modules #116

0.2.4 - 2018-01-19

Added

  • Allow to get mutable ref from PyObject #106
  • Drop RefFromPyObject trait
  • Add Python::register_any method

Fixed

  • Fix impl FromPyObject for Py<T>
  • Mark method that work with raw pointers as unsafe #95

0.2.3 - 11-27-2017

Changed

  • Rustup to 1.23.0-nightly 2017-11-07

Fixed

  • Proper c_char usage #93

Removed

  • Remove use of now unneeded ‘AsciiExt’ trait

0.2.2 - 09-26-2017

Changed

  • Rustup to 1.22.0-nightly 2017-09-30

0.2.1 - 09-26-2017

Fixed

  • Fix rustc const_fn nightly breakage

0.2.0 - 08-12-2017

Added

  • Added inheritance support #15
  • Added weakref support #56
  • Added subclass support #64
  • Added self.__dict__ support #68
  • Added pyo3::prelude module #70
  • Better Iterator support for PyTuple, PyList, PyDict #75
  • Introduce IntoPyDictPointer similar to IntoPyTuple #69

Changed

  • Allow to add gc support without implementing PyGCProtocol #57
  • Refactor PyErr implementation. Drop py parameter from constructor.

0.1.0 - 07-23-2017

Added

  • Initial release

參與貢獻

感謝你有興趣貢獻 PyO3！歡迎所有人參與——請閱讀我們的行為準則以維持社群的正向與包容。

如果你正在尋找貢獻方向，請參考「開始參與貢獻」一節。如果你已找到想處理的特定議題並需要開發流程資訊，可參考「撰寫 Pull Request」一節。

若想熟悉程式碼庫，請參考 Architecture.md。

開始參與貢獻

歡迎從你感興趣的 PyO3 領域加入。我們使用 GitHub issues 記錄所有錯誤與想法。若你想處理某個議題，請隨時申請指派。

你可以在這裡瀏覽 PyO3 非公開部分的 API。

以下章節也提供一些適合開始貢獻 PyO3 的具體想法。

設定開發環境

要開發 PyO3，你需要在系統中安裝 Python 與 Rust。

  • 我們建議使用 rustup 以便依專案選擇特定工具鏈。
  • 也強烈建議使用 Pyenv 以便選擇特定 Python 版本。
  • virtualenv 也可搭配或不搭配 Pyenv 使用特定已安裝的 Python 版本。
  • nox 用於自動化多項 CI 任務。

使用 nox 進行測試、lint 等

Nox 用於自動化多項 CI 任務，也可在本地用來處理程式撰寫時的驗證工作。我們建議透過 nox 執行這些動作，以使用我們偏好的設定選項。你可以用 pip 安裝 nox 到全域 Python：pip install nox，或（建議）使用 pipx 安裝：pip install pipx、pipx install nox。

我們提供的主要 nox 指令如下：

  • nox -s test 會執行完整的 Rust 與 Python 測試套件（>10 分鐘）
  • nox -s test-rust -- skip-full 會執行較短的 Rust 測試套件（2-3 分鐘）
  • nox -s ruff 會檢查 Python lint 並套用標準格式規則
  • nox -s rustfmt 會檢查基本 Rust lint 並套用標準格式規則
  • nox -s rumdl 會檢查指南中的 Markdown
  • nox -s clippy 會執行 clippy 提供 Rust 風格建議
  • nox -s bench 會對 Rust 程式碼進行效能基準測試
  • nox -s codspeed 會執行我們的 Rust 與 Python 效能測試套件
  • nox -s coverage 會分析測試覆蓋率並輸出 coverage.json（或使用 nox -s coverage lcov 輸出 lcov.info）
  • nox -s check-guide 會使用 lychee 檢查指南與文件註解中的所有連結。

使用 nox -l 列出可執行的完整子指令。

UI 測試

PyO3 使用 trybuild 來開發 UI 測試，以擷取某些巨集功能的 Rust 編譯器錯誤訊息。

Rust 編譯器的錯誤輸出會因是否安裝 rust-src 元件而不同。PyO3 的 CI 已安裝 rust-src，所以若要讓本地 UI 測試輸出一致，你也需要安裝它：

rustup component add rust-src

由於這些 UI 測試有多種 feature 組合，當你要全部更新（例如因新版 Rust 編譯器）時，使用 update-ui-tests nox session 會很有幫助：

nox -s update-ui-tests

可協助的方式

協助使用者找出錯誤

PyO3 Discord 伺服器 非常活躍，許多使用者對 PyO3 甚至 Rust 都很新。協助他們除錯是熟悉 PyO3 程式碼庫的好方法。

協助他人常會暴露錯誤、文件不足與缺漏的 API。建議立刻在 GitHub 開 issue，讓解法得以規劃並實作！

實作可直接開發的議題

解法清楚且尚未有人處理的議題會使用 needs-implementer 標籤。

若你覺得解法不夠清楚也別擔心！PyO3 核心貢獻者很樂意針對你的任何疑問提供指導，幫助你完成解法。

協助撰寫優質文件

PyO3 有使用 mdbook 的使用者指南，以及一般的 Rust API 文件。我們的目標是讓兩者都詳盡、易懂且保持最新。歡迎提出拉取請求來修正錯字、調整措辭、補充範例等。

目前文件有一些需要協助的特定重點領域：

  • 與文件改進相關的議題會以 documentation 標籤追蹤。
  • 並非所有 API 在建立時都有文件或範例。我們的目標是讓所有 PyO3 API 都有文件（#306）。若你看到某個 API 缺少文件，請撰寫並開啟 PR！

若要建置文件（包含所有功能），請安裝 nox 並執行

nox -s docs -- open

文件測試

我們在文件中使用大量程式碼區塊。變更時請執行 cargo test --doc 以確認文件測試仍可正常運作，或執行 cargo test 來跑包含文件測試在內的所有 Rust 測試。文件測試指南請見 https://doc.rust-lang.org/rustdoc/documentation-tests.html。

建置指南

你可以使用 mdbook 在本地建置以預覽使用者指南。

首先安裝 mdbook、mdbook-tabs 外掛，以及 nox。然後執行

nox -s build-guide -- --open

若要檢查指南中所有連結是否有效，請另行安裝 lychee 並改用 check-guide session：

nox -s check-guide

協助設計下一代 PyO3

尚未有明確解法的議題會使用 needs-design 標籤。

如果這些議題有你感興趣的，請加入該議題的討論！所有意見都很重要；若你還願意透過草稿 PR 嘗試 API 設計，那就更好了！

審查拉取請求

歡迎所有人對開放中的 PR 提出意見。請協助確保新的 PyO3 API 安全、效能良好、整潔且易用！

撰寫拉取請求

以下是撰寫 PR 時需要注意的幾點。

測試與持續整合

PyO3 儲存庫使用 GitHub Actions。若 CI 未通過，PR 會被阻止合併。所有 Rust 與 Python 程式碼都會檢查格式化、lint 與測試（若格式化失敗，流水線會提早中止以節省資源）。此外，Rust 程式碼中的所有警告都不被允許（使用 RUSTFLAGS="-D warnings"）。

測試會在最新穩定版 Rust 編譯器與所有支援的 Python 版本上執行，並以最低支援的 Rust 版本測試 Python 3.9。

如果你新增了功能，請將其加入 Cargo.toml* 中的 full 功能集合，以便在 CI 中進行測試。

你可以用 nox 自行執行 CI 管線的各項元件，請見上方測試章節。

變更文件化

我們使用 towncrier 為每次發行生成變更日誌。

要將你的變更納入發行說明，請在 newsfragments 目錄中建立一個或多個新聞項目。有效的新聞項目應儲存為 <PR>.<CATEGORY>.md，其中 <PR> 為拉取請求編號，<CATEGORY> 為下列其中之一：

  • packaging - 用於相依性變更與 Python／Rust 版本相容性變更
  • added - 用於新增功能
  • changed - 用於既有功能的修改或棄用
  • removed - 用於移除的功能
  • fixed - 用於被歸類為錯誤修復的「變更」功能

僅文件的 PR 不需要新聞項目；將 PR 標題以 docs: 開頭即可略過檢查。

風格指南

泛型程式碼

PyO3 有許多泛型 API 以提升易用性，但代價可能是泛型程式碼膨脹。在合理的情況下，請嘗試實作泛型函式的具體子部分。這有兩種形式：

  • 若該具體子部分無法被其他函式重用，請命名為 inner 並保留為函式內部的區域項。
  • 若該具體子部分會被其他函式重用，建議命名為 _foo，並將其放在原泛型函式 foo 的下方。

FFI 呼叫

PyO3 使用原始指標對 Python 的 C API 進行大量 FFI 呼叫。能避免時請避免在運算式中使用指向暫時值的指標：

// 危險
pyo3::ffi::Something(name.to_object(py).as_ptr());

// 因為以下重構會造成 use-after-free 錯誤：
let name = name.to_object(py).as_ptr();
pyo3::ffi::Something(name)

改用先綁定安全擁有的 PyObject 包裝，再傳給 ffi 函式：

let name: PyObject = name.to_object(py);
pyo3::ffi::Something(name.as_ptr())
// name 在離開作用域後會自動釋放

Python 與 Rust 版本支援政策

PyO3 旨在維持足夠的相容性，讓以 PyO3 建置的 Python 擴充能在多數常見軟體包管理系統中可行。

為了讓軟體包維護者的工作更簡單，PyO3 會盡可能承諾僅在同一時間調整最低支援的 Rust 與 Python 版本。此升級僅會出現在 0.x 版本，約每年一次，並在最舊支援的 Python 版本到達生命週期終點後進行。（可參考 https://endoflife.date/python 了解時間表。）

以下為各語言的相容性指引，所有 PR 都應遵循。

Python

PyO3 支援所有官方支援的 Python 版本，以及最新的 PyPy3 版本。這些版本都會在 CI 中測試。

新增對新 CPython 版本的支援

如果你打算新增對 CPython 預發行版本的支援，以下是（非完整）檢查清單：

  • 等待最後一個 alpha 版本（通常為 alpha7），因為 ABI 直到第一個 beta 版才有保證
  • 在 .github/workflows/ci.yml 中新增 prerelease_ver-dev（例如 3.14-dev），並在 noxfile.py、pyo3-ffi/Cargo.toml 的 [package.metadata.cpython] 區段 max-version，以及 pyo3-ffi/build.rs 的 max 中提升版本
  • 為該版本新增 abi3-prerelease 功能（例如 abi3-py314）
    • 在 pyo3-build-config/Cargo.toml 中，將 abi3-most_current_stable 設為 [“abi3-prerelease”]，並將 abi3-prerelease 設為 [“abi3”]
    • 在 pyo3-ffi/Cargo.toml 中，將 abi3-most_current_stable 設為 [“abi3-prerelease”, “pyo3-build-config/abi3-most_current_stable”]，並將 abi3-prerelease 設為 [“abi3”, “pyo3-build-config/abi3-prerelease”]
    • 在 Cargo.toml 中，將 abi3-most_current_stable 設為 [“abi3-prerelease”, “pyo3-ffi/abi3-most_current_stable”]，並將 abi3-prerelease 設為 [“abi3”, “pyo3-ffi/abi3-prerelease”]
  • 使用 #[cfg(Py_prerelease])（例如 #[cfg(Py_3_14)]）與 #[cfg(not(Py_prerelease])) 來標示 CPython 穩定分支與預發行版之間的變更
  • 不要為 CPython 標頭中任何以 _ 開頭的函式、結構或全域變數新增 Rust 綁定
  • 如需協助請聯絡 @ngoldbaum 與 @davidhewitt

Rust

PyO3 旨在使用最新的 Rust 語言功能，以盡可能提高實作效率。

最低支援的 Rust 版本會在提升 Python 與 Rust 版本的發行時決定。當時設定的最低 Rust 版本不會高於當前 Debian、RHEL 與 Alpine Linux 發行版隨附的最低 Rust 版本。

CI 會測試最新穩定版 Rust 與最低支援 Rust 版本。由於 Rust 的穩定性保證，這足以確認中間所有版本的支援。

效能基準測試

PyO3 有兩組效能基準測試用於評估部分效能面向。目前測試套件還很小——若你願意協助擴充，歡迎新增基準測試並開 PR！

首先，在 pyo3-benches 子目錄中有以 Rust 為基礎的基準測試。你可以透過以下方式執行：

nox -s bench

其次，pytests 子目錄中包含一個以 Python 為基礎的基準測試。更多資訊請見此處。

程式碼覆蓋率

你可以檢視 PyO3 測試涵蓋與未涵蓋的程式碼。我們目標是 100% 覆蓋率——如果你發現覆蓋不足，請檢查覆蓋率並補上測試！

  • 首先，請確保已安裝 llvm-cov cargo 外掛。在與 nox 搭配使用前，你可能需要先透過 cargo 執行外掛一次。
cargo install cargo-llvm-cov
cargo llvm-cov
  • 接著用以下方式產生 lcov.info 檔案
nox -s coverage -- lcov

你可以安裝 IDE 外掛來檢視覆蓋率。例如使用 VSCode：

  • 安裝 coverage-gutters 外掛。
  • 在 VSCode 的 settings.json 加入以下設定：
{
    "coverage-gutters.coverageFileNames": [
        "lcov.info",
        "cov.xml",
        "coverage.xml",
    ],
    "coverage-gutters.showLineCoverage": true
}
  • 你現在應該可以看到綠色高亮代表已測試的程式碼，紅色高亮代表未測試的程式碼。

贊助此專案

目前沒有代表 PyO3 接受贊助的官方組織。若你想為 PyO3 生態系提供較大規模的資金支持，請透過 GitHub 或 Discord 與我們聯絡，我們可進一步討論。

此外，我們部分維護者有個人的 GitHub 贊助頁面，若你願意支持將非常感謝：