Interpolating one time series onto another in pandas

2024/11/10 22:40:53

I have one set of values measured at regular times. Say:

import pandas as pd
import numpy as np
rng = pd.date_range('2013-01-01', periods=12, freq='H')
data = pd.Series(np.random.randn(len(rng)), index=rng)

And another set of more arbitrary times, for example, (in reality these times are not a regular sequence)

ts_rng = pd.date_range('2013-01-01 01:11:21', periods=7, freq='87Min')
ts = pd.Series(index=ts_rng)

I want to know the value of data interpolated at the times in ts.
I can do this in numpy:

x = np.asarray(ts_rng,dtype=np.float64)
xp = np.asarray(data.index,dtype=np.float64)
fp = np.asarray(data)
ts[:] = np.interp(x,xp,fp)

But I feel pandas has this functionality somewhere in resample, reindex etc. but I can't quite get it.

Answer

You can concatenate the two time series and sort by index. Since the values in the second series are NaN you can interpolate and the just select out the values that represent the points from the second series:

 pd.concat([data, ts]).sort_index().interpolate().reindex(ts.index)

or

 pd.concat([data, ts]).sort_index().interpolate()[ts.index]
https://en.xdnf.cn/q/72108.html

Related Q&A

Reference class variable in a comprehension of another class variable

This may be a simple question, but Im having trouble making a unique search for it. I have a class that defines a static dictionary, then attempts to define a subset of that dictionary, also statically…

Pyspark module not found

Im trying to execute a simple Pyspark job in Yarn. This is the code:from pyspark import SparkConf, SparkContextconf = (SparkConf().setMaster("yarn-client").setAppName("HDFS Filter")…

Multiple windows in PyQt4?

Ive just begun using pyqt4. I followed a tutorial (http://zetcode.com/tutorials/pyqt4/) One thing that puzzles me is this part:def main():app = QtGui.QApplication(sys.argv)ex = GUI()sys.exit(app.exec()…

Fill missing timeseries data using pandas or numpy

I have a list of dictionaries which looks like this :L=[ { "timeline": "2014-10", "total_prescriptions": 17 }, { "timeline": "2014-11", "total_…

Can Biopython perform Seq.find() accounting for ambiguity codes

I want to be able to search a Seq object for a subsequnce Seq object accounting for ambiguity codes. For example, the following should be true:from Bio.Seq import Seq from Bio.Alphabet.IUPAC import IUP…

MySQL and lock a table, read, and then truncate

I am using mysqldb in python.I need to do the following for a table.1) Lock 2) Read 3) Truncate the table 4) UnlockWhen I run the below code, I get the below error. So, I am rather unsure on how to lo…

Train and predict on variable length sequences

Sensors (of the same type) scattered on my site are manually reporting on irregular intervals to my backend. Between reports the sensors aggregate events and report them as a batch. The following datas…

What should a Python project structure look like for Travis CI to find and run tests?

I currently have a project with the following .travis.yml file:language: python install: "pip install tox" script: "tox"Locally, tox properly executes and runs 35 tests, but on Trav…

Having trouble building a Dns Packet in Python

Im trying to build a dns packet to send over a socket. I dont want to use any libraries because I want direct access to the socket variable that sends it. Whenever I send the DNS packet, wireshark says…

Element wise comparison between 1D and 2D array

Want to perform an element wise comparison between an 1D and 2D array. Each element of the 1D array need to be compared (e.g. greater) against the corresponding row of 2D and a mask will be created. He…