A brief conceptual sketch related to NEP 18 (a NumPy Enhancement Proposal) . Triggered by a discussion on the NumPy mailing list.

1. __array_function__ conceptual
design & related concepts
Ralf Gommers

2. Context
These slides contain some sketches about
what __array_function__ does and is for, as
well as thoughts on how interoperability,
ndarray subclasses and “duck arrays”
relate to it.
2

3. NumPy API and semantics
3
Function body
(the “implementation”, defines
semantics)
Function signature (the API)
Example for one function - this applies to ufuncs
and other functions. Of course there are also other
objects in the NumPy API - let’s ignore those for simplicity.

4. __array_function__ concept
4
Function body
Function signature (the API)
tl;dr use the NumPy API, bring your own implementation
Function body
Function signature
Function signature (the API)
if input arg has __array_function__:
execute other_function
Function body
Function signature
==

5. “array_like” vs “duck typing”
Most NumPy functions accept “array-like” input, which is:
“An array, any object exposing the array interface, an object whose __array__
method returns an array, or any (nested) sequence.”
Duck typing:
"If it walks like a duck and it quacks like a duck, then it must be a duck”
Lists and tuples clearly aren’t the same animal as ndarray. Hence
the NumPy API does not use duck typing. Instead, it coerces
inputs to ndarray.
It’s unclear what an ndarray “duck” even means - MaskedArray
could be some sort of duck, or it could be a swan ….
5

6. Reusing NumPy implementations of the functions in its API?
6
Reusing part of the implementation of NumPy functions is an
order of magnitude harder than reusing the API. doable …??
Function body
Function signature
Function signature (the API)
if input arg has __some_attribute__:
customize implementation==
array_like “duck array” or
ndarray subclass
Function body
….
np.asarray
….
rest of function body
Function signature
Function signature (the API)
if input arg has __some_attribute__:
customize implementation==
vs.

7. NEP 18 goals
◎ Separate NumPy API from NumPy “execution engine”
◎ Allow other array libraries (Dask, CuPy, pydata/sparse,
PyTorch, …) to reuse the NumPy API
◎ In the bigger picture: reduce or avoid ecosystem
fragmentation (e.g. we don’t want to see a reimplementation of SciPy for
PyTorch, SciPy for Tensorflow, etc.).
Note: NEP 18 increases interoperability in a fully backwards compatible way.
Duck typing and handling “duck arrays” or ndarray subclasses differently are
specific kinds of interoperability that NEP 18 does not directly deal with.
7