Subclass of numpy ndarray doesnt work as expected

2024/9/20 21:38:45

`Hello, everyone.

I found there is a strange behavior when subclassing a ndarray.

import numpy as npclass fooarray(np.ndarray):def __new__(cls, input_array, *args, **kwargs):obj = np.asarray(input_array).view(cls)return objdef __init__(self, *args, **kwargs):returndef __array_finalize__(self, obj):returna=fooarray(np.random.randn(3,5))
b=np.random.randn(3,5)a_sum=np.sum(a,axis=0,keepdims=True)
b_sum=np.sum(b,axis=0, keepdims=True)print a_sum.ndim #1
print b_sum.ndim #2

As you have seen, the keepdims argument doesn't work for my subclass fooarray. It lost one of its axis. How can't I avoid this problem? Or more generally, how can I subclass numpy ndarray correctly?

Answer

np.sum can accept a variety of objects as input: not only ndarrays, but also lists, generators, np.matrixs, for instance. The keepdims parameter obviously does not make sense for lists or generators. It is also not appropriate for np.matrix instances either, since np.matrixs always have 2 dimensions. If you look at the call signature for np.matrix.sum you see that its sum method has no keepdims parameter:

Definition: np.matrix.sum(self, axis=None, dtype=None, out=None)

So some subclasses of ndarray may have sum methods which do not have a keepdims parameter. This is an unfortunate violation of the Liskov substitution principle and the origin of the pitfall you encountered.

Now if you look at the source code for np.sum, you see that it is a delegating function which tries to determine what to do based on the type of the first argument.

If the type of the first argument is not ndarray, it drops the keepdims parameter. It does this because passing the keepdims parameter to np.matrix.sum would raise an exception.

So because np.sum is trying to do the delegation in the most general way, not making any assumption about what arguments a subclass of ndarray may take, it drops the keepdims parameter when passed a fooarray.

The workaround is to not use np.sum, but call a.sum instead. This is more direct anyway, since np.sum is merely a delegating function.

import numpy as npclass fooarray(np.ndarray):def __new__(cls, input_array, *args, **kwargs):obj = np.asarray(input_array, *args, **kwargs).view(cls)return obja = fooarray(np.random.randn(3, 5))
b = np.random.randn(3, 5)a_sum = a.sum(axis=0, keepdims=True)
b_sum = np.sum(b, axis=0, keepdims=True)print(a_sum.ndim)  # 2
print(b_sum.ndim)  # 2
https://en.xdnf.cn/q/72452.html

Related Q&A

How to correctly load images asynchronously in PyQt5?

Im trying to figure out how to accomplish an async image load correctly, in PyQt Qlistview.My main widget consists of a Qlistview and a QLineEdit textbox. I have a database of actors which I query usin…

How to print results of Python ThreadPoolExecutor.map immediately?

I am running a function for several sets of iterables, returning a list of all results as soon as all processes are finished.def fct(variable1, variable2):# do an operation that does not necessarily ta…

Python dir equivalent in perl?

The dir command in Python 2.7.x lists all accessible symbols from a module. Is there an equivalent in Perl 5.x to list all accessible symbols from a package?

Entire JSON into One SQLite Field with Python

I have what is likely an easy question. Im trying to pull a JSON from an online source, and store it in a SQLite table. In addition to storing the data in a rich table, corresponding to the many fiel…

Python scipy module import error due to missing ._ufuncs dll

I have some troubles with sub-module integrate from scipy in python. I have a 64 bits architecture, and it seems, according to the first lines of the python interpreter (see below) that I am also using…

How can I call python program from VBA?

Just as the title goes.I have a python program which processes some data file I downloaded from email.I am writing a vba script which can download the email attachments and execute the python program t…

Embedding CPython: how do you constuct Python callables to wrap C callback pointers?

Suppose I am embedding the CPython interpreter into a larger program, written in C. The C component of the program occasionally needs to call functions written in Python, supplying callback functions …

python - beautifulsoup - TypeError: sequence item 0: expected string, Tag found

Im using beautifulsoup to extract images and links from a html string. It all works perfectly fine, however with some links that have a tag in the link contents it is throwing an error.Example Link:<…

Python evdev detect device unplugged

Im using the great "evdev" library to listen to a USB barcode reader input and I need to detect if the device suddenly gets unplugged/unresponsive because otherwise the python script reading …

python: urllib2 using different network interface

I have the following code:f = urllib2.urlopen(url) data = f.read() f.close()Its running on a machine with two network interfaces. Id like to specify which interface I want the code to use. Specifically…