Buffer Objects

Python objects implemented in C can export a group of functions called the 「buffer interface.」 These functions can be used by an object to expose its data in a raw, byte-oriented format. Clients of the object can use the buffer interface to access the object data directly, without needing to copy it first.

Two examples of objects that support the buffer interface are strings and arrays. The string object exposes the character contents in the buffer interface’s byte-oriented form. An array can also expose its contents, but it should be noted that array elements may be multi-byte values.

An example user of the buffer interface is the file object’s write() method. Any object that can export a series of bytes through the buffer interface can be written to a file. There are a number of format codes to :cfunc:`PyArg_ParseTuple` that operate against an object’s buffer interface, returning data from the target object.

Starting from version 1.6, Python has been providing Python-level buffer objects and a C-level buffer API so that any built-in or used-defined type can expose its characteristics. Both, however, have been deprecated because of various shortcomings, and have been officially removed in Python 3.0 in favour of a new C-level buffer API and a new Python-level object named memoryview.

The new buffer API has been backported to Python 2.6, and the memoryview object has been backported to Python 2.7. It is strongly advised to use them rather than the old APIs, unless you are blocked from doing so for compatibility reasons.

The new-style Py_buffer struct

Buffer related functions

Old-style buffer objects

More information on the buffer interface is provided in the section Buffer Object Structures, under the description for :ctype:`PyBufferProcs`.

A 「buffer object」 is defined in the bufferobject.h header (included by Python.h). These objects look very similar to string objects at the Python programming level: they support slicing, indexing, concatenation, and some other standard string operations. However, their data can come from one of two sources: from a block of memory, or from another object which exports the buffer interface.

Buffer objects are useful as a way to expose the data from another object’s buffer interface to the Python programmer. They can also be used as a zero-copy slicing mechanism. Using their ability to reference a block of memory, it is possible to expose any data to the Python programmer quite easily. The memory could be a large, constant array in a C extension, it could be a raw block of memory for manipulation before passing to an operating system library, or it could be used to pass around structured data in its native, in-memory format.