Suppose I wish to re-index, with linear interpolation, a time series to a pre-defined index, where none of the index values are shared between old and new index. For example

# index is all precise timestamps e.g. 2018-10-08 05:23:07
series = pandas.Series(data,index) # I want rounded date-times
desired_index = pandas.date_range("2010-10-08",periods=10,freq="30min") 

Tutorials/API suggest the way to do this is to reindex then fill NaN values using interpolate. But, as there is no overlap of datetimes between the old and new index, reindex outputs all NaN:

# The following outputs all NaN as no date times match old to new index
series.reindex(desired_index)

I do not want to fill nearest values during reindex as that will lose precision, so I came up with the following; concatenate the reindexed series with the original before interpolating:

pandas.concat([series,series.reindex(desired_index)]).sort_index().interpolate(method="linear")

This seems very inefficient, concatenating and then sorting the two series. Is there a better way?

The only (simple) way I can see of doing this is to use resample to upsample to your time resolution (say 1 second), then reindex.

Get an example DataFrame:

import numpy as np
import pandas as pdnp.random.seed(2)df = (pd.DataFrame().assign(SampleTime=pd.date_range(start='2018-10-01', end='2018-10-08', freq='30T')+ pd.to_timedelta(np.random.randint(-5, 5, size=337), unit='s'),Value=np.random.randn(337)).set_index(['SampleTime'])
)

Let's see what the data looks like:

df.head()Value
SampleTime
2018-10-01 00:00:03     0.033171
2018-10-01 00:30:03     0.481966
2018-10-01 01:00:01     -0.495496

Get the desired index:

desired_index = pd.date_range('2018-10-01', periods=10, freq='30T')

Now, reindex the data with the union of the desired and existing indices, interpolate based on the time, and reindex again using only the desired index:

(df.reindex(df.index.union(desired_index)).interpolate(method='time').reindex(desired_index)
)Value
2018-10-01 00:00:00     NaN
2018-10-01 00:30:00     0.481218
2018-10-01 01:00:00     -0.494952
2018-10-01 01:30:00     -0.103270

As you can see, you still have an issue with the first timestamp because it's outside the range of the original index; there are number of ways to deal with this (pad, for example).