Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

為你的 Python 套件提供型別與 IDE 提示

PyO3 提供易用介面,可用 Rust 撰寫原生 Python 函式庫。搭配的 Maturin 讓你能將其建置並發布為套件。然而為了更好的使用體驗,Python 函式庫應為所有公開項目提供型別提示與文件,讓 IDE 能在開發時顯示,並讓 mypy 等型別分析工具正確驗證程式碼。

目前最佳的解法是手動維護 *.pyi 檔案,並與套件一同發布。

PyO3 正在進行自動產生的工作。請見型別 stub 產生文件以了解目前的自動產生狀態。

pyi 檔案介紹

pyi 檔案(Python Interface 的縮寫)在相關文件中多被稱為「stub 檔案」。對其定義可參考舊版 MyPy 文件

Stub 檔案只包含模組公開介面的描述,不包含任何實作。

也可參考官方 Python typing 文件中關於 type stubs 的完整說明

多數 Python 開發者可能在對內建型別使用 IDE 的「前往定義」功能時就已遇過它們。例如,幾個標準例外的定義如下:

class BaseException(object):
    args: Tuple[Any, ...]
    __cause__: BaseException | None
    __context__: BaseException | None
    __suppress_context__: bool
    __traceback__: TracebackType | None
    def __init__(self, *args: object) -> None: ...
    def __str__(self) -> str: ...
    def __repr__(self) -> str: ...
    def with_traceback(self: _TBE, tb: TracebackType | None) -> _TBE: ...

class SystemExit(BaseException):
    code: int

class Exception(BaseException): ...

class StopIteration(Exception):
    value: Any

如你所見,這些並非包含實作的完整定義,而只是介面描述。這通常已足以滿足函式庫使用者的需求。

PEP 有什麼說法?

在撰寫本文件時,pyi 檔案在四份 PEP 中被引用。

PEP8 - Python 程式碼風格指南 - #Function Annotations(最後一點)建議所有第三方函式庫作者提供 stub 檔案,作為型別檢查工具對套件的知識來源。

(……)預期第三方函式庫的使用者可能會對這些套件執行型別檢查。為此,PEP 484 建議使用 stub 檔案:型別檢查器會優先讀取 .pyi 檔案而不是對應的 .py 檔案。(……)

PEP484 - Type Hints - #Stub Files 對 stub 檔案的定義如下。

Stub 檔案是只提供給型別檢查器使用、而非執行階段使用的型別提示檔案。

其中包含對 stub 檔案的規範(強烈建議閱讀,因為至少有一項是一般 Python 程式碼不會用到的)以及關於存放位置的一般資訊。

PEP561 - Distributing and Packaging Type Information 詳細描述如何建置能啟用型別檢查的套件,特別是如何散布 stub 檔案以供型別檢查器使用。

PEP560 - Core support for typing module and generic types 描述了 Python 型別系統如何在內部支援泛型,包括執行階段行為與靜態型別檢查器的整合。

該怎麼做?

PEP561 提出三種散布型別資訊的方式:

  • inline - 型別直接放在原始碼(py)檔案中;
  • separate package with stub files - 型別放在 pyi 檔案中,並以獨立套件發布;
  • in-package stub files - 型別放在 pyi 檔案中,並與原始碼同套件發布。

第一種方式在 PyO3 中較困難,因為我們沒有 py 檔案。待完成研究與必要改動後,本文件將更新。

第二種方式很容易完成,而且可與主程式庫程式碼完全分離。帶有 stub 檔案的軟體包範例可在 PEP561 參考資料中找到:Stub 軟體包儲存庫

第三種方式如下所述。

在 PyO3/Maturin 建置套件中包含 pyi 檔案

當原始碼與 stub 檔案位於同一套件時,應該把它們放在彼此相鄰的位置。我們需要能用 Maturin 達成這點的方法。此外,為了標示套件具有型別資訊,我們需要在套件中新增一個名為 py.typed 的空檔案。

若你沒有其他 Python 檔案

若除了 pyi 之外不需要加入其他 Python 檔案,Maturin 提供了方法替你完成大部分工作。如同 Maturin 指南 所述,你只需在專案根目錄建立名為 <module_name>.pyi 的 stub 檔案,其餘由 Maturin 處理。

my-rust-project/
├── Cargo.toml
├── my_project.pyi  # <<< add type stubs for Rust functions in the my_project module here
├── pyproject.toml
└── src
    └── lib.rs

如需 pyi 範例檔案,請見 my_project.pyi 內容 一節。

If you need other Python files

If you need to add other Python files apart from pyi to the package, you can do it also, but that requires some more work. Maturin provides an easy way to add files to a package (documentation). You just need to create a folder with the name of your module next to the Cargo.toml file (for customization see documentation linked above).

The folder structure would be:

my-project
├── Cargo.toml
├── my_project
│   ├── __init__.py
│   ├── my_project.pyi
│   ├── other_python_file.py
│   └── py.typed
├── pyproject.toml
├── Readme.md
└── src
    └── lib.rs

Let’s go a little bit more into detail regarding the files inside the package folder.

__init__.py content

As we now specify our own package content, we have to provide the __init__.py file, so the folder is treated as a package and we can import things from it. We can always use the same content that Maturin creates for us if we do not specify a Python source folder. For PyO3 bindings it would be:

from .my_project import *

That way everything that is exposed by our native module can be imported directly from the package.

py.typed requirement

As stated in PEP561:

Package maintainers who wish to support type checking of their code MUST add a marker file named py.typed to their package supporting typing. This marker applies recursively: if a top-level package includes it, all its sub-packages MUST support type checking as well.

If we do not include that file, some IDEs might still use our pyi files to show hints, but the type checkers might not. MyPy will raise an error in this situation:

error: Skipping analyzing "my_project": found module but no type hints or library stubs

The file is just a marker file, so it should be empty.

my_project.pyi content

Our module stub file. This document does not aim at describing how to write them, since you can find a lot of documentation on it, starting from the already quoted PEP484.

The example can look like this:

class Car:
    """
    A class representing a car.

    :param body_type: the name of body type, e.g. hatchback, sedan
    :param horsepower: power of the engine in horsepower
    """
    def __init__(self, body_type: str, horsepower: int) -> None: ...

    @classmethod
    def from_unique_name(cls, name: str) -> 'Car':
        """
        Creates a Car based on unique name

        :param name: model name of a car to be created
        :return: a Car instance with default data
        """

    def best_color(self) -> str:
        """
        Gets the best color for the car.

        :return: the name of the color our great algorithm thinks is the best for this car
        """

Supporting Generics

Type annotations can also be made generic in Python. They are useful for working with different types while maintaining type safety. Usually, generic classes inherit from the typing.Generic metaclass.

Take for example the following .pyi file that specifies a Car that can accept multiple types of wheels:

from typing import Generic, TypeVar

W = TypeVar('W')

class Car(Generic[W]):
    def __init__(self, wheels: list[W]) -> None: ...

    def get_wheels(self) -> list[W]: ...

    def change_wheel(self, wheel_number: int, wheel: W) -> None: ...

This way, the end-user can specify the type with variables such as truck: Car[SteelWheel] = ... and f1_car: Car[AlloyWheel] = ....

There is also a special syntax for specifying generic types in Python 3.12+:

class Car[W]:
    def __init__(self, wheels: list[W]) -> None: ...

    def get_wheels(self) -> list[W]: ...

Runtime Behaviour

Stub files (pyi) are only useful for static type checkers and ignored at runtime. Therefore, PyO3 classes do not inherit from typing.Generic even if specified in the stub files.

This can cause some runtime issues, as annotating a variable like f1_car: Car[AlloyWheel] = ... can make Python call magic methods that are not defined.

To overcome this limitation, implementers can pass the generic parameter to pyclass in Rust:

#[pyclass(generic)]

Advanced Users

#[pyclass(generic)] implements a very simple runtime behavior that accepts any generic argument. Advanced users can opt to manually implement __class_getitem__ for the generic class to have more control.

impl MyClass {
    #[classmethod]
    #[pyo3(signature = (key, /))]
    pub fn __class_getitem__(
        cls: &Bound<'_, PyType>,
        key: &Bound<'_, PyAny>,
    ) -> PyResult<Py<PyAny>> {
        /* implementation details */
    }
}

Note that pyo3::types::PyGenericAlias can be helpful when implementing __class_getitem__ as it can create types.GenericAlias objects from Rust.