How to apply linregress in Pandas bygroup

2024/10/1 21:29:41

I would like to apply a scipy.stats.linregress within Pandas ByGroup. I had looked through the documentation but all I could see was how to apply something to a single column like

grouped.agg(np.sum)

or a function like

grouped.agg('D' : lambda x: np.std(x, ddof=1)) 

But how do I apply a linregress which has TWO inputs X and Y?

Answer

The linregress function, as well as many other scipy/numpy functions, accepts "array-like" X and Y, both Series and DataFrame could qualify.

For example:

from scipy.stats import linregress
X = pd.Series(np.arange(10))
Y = pd.Series(np.arange(10))In [4]: linregress(X, Y)
Out[4]: (1.0, 0.0, 1.0, 4.3749999999999517e-80, 0.0)

In fact, being able to use scipy (and numpy) functions is one of pandas killer features!

So if you have a DataFrame you can use linregress on its columns (which are Series):

linregress(df['col_X'], df['col_Y'])

and if using a groupby you can similarly apply (to each group):

grouped.apply(lambda x: linregress(x['col_X'], x['col_Y']))
https://en.xdnf.cn/q/70920.html

Related Q&A

Python Shared Memory Array, no attribute get_obj()

I am working on manipulating numpy arrays using the multiprocessing module and am running into an issue trying out some of the code I have run across here. Specifically, I am creating a ctypes array f…

What is a qualified/unqualified name in Python?

In Python: what is a "qualified name" or "unqualified name"?Ive seen it mentioned a couple of times, but no explanation as to what it is.

Python code explanation for stationary distribution of a Markov chain

I have got this code: import numpy as np from scipy.linalg import eig transition_mat = np.matrix([[.95, .05, 0., 0.],\[0., 0.9, 0.09, 0.01],\[0., 0.05, 0.9, 0.05],\[0.8, 0., 0.05, 0.15]])S, U = eig(tr…

Detecting insertion/removal of USB input devices on Windows 10

I already have some working Python code to detect the insertion of some USB device types (from here).import wmiraw_wql = "SELECT * FROM __InstanceCreationEvent WITHIN 2 WHERE TargetInstance ISA \W…

FastAPI as a Windows service

I am trying to run FastAPI as a windows service.Couldnt find any documentation or any article to run Uvicorn as a Windows service. I tried using NSSM as well but my windows service stops.

Why not use runserver for production at Django?

Everywhere i see that uWSGI and Gunicorn are recommended for production mode from everyone. However, there is a lot more suffering to operate with it, the python manage.py runserver is more faster, sim…

How to create decorator for lazy initialization of a property

I want to create a decorator that works like a property, only it calls the decorated function only once, and on subsequent calls always return the result of the first call. An example:def SomeClass(obj…

How to convert a 24-bit wav file to 16 or 32 bit files in python3

I am trying to make spectrograms of a bunch of .wav files so I can further analyze them(in python 3.6), however, I keep getting this nasty errorValueError: Unsupported bit depth: the wav file has 24-bi…

Get consistent Key error: \n [duplicate]

This question already has answers here:How do I escape curly-brace ({}) characters in a string while using .format (or an f-string)?(23 answers)Closed 8 years ago.When trying to run a script containin…

Cannot take the length of Shape with unknown rank

I have a neural network, from a tf.data data generator and a tf.keras model, as follows (a simplified version-because it would be too long):dataset = ...A tf.data.Dataset object that with the next_x me…