The data set is made of a list dfList containing pandas DataFrames, each DataFrame consisting of the column Y and an identical index column. I am trying to plot all the DataFrames as a 2D plot with pixel color representing the Y values.

Example of the Style of Plot Needed

Problem: However, using scipy.interpolate.griddata with matplotlib.pyplot.imshow produces a blank plot! What might be the problem?

I have added a link to the pickle.dump of dfList for reproducing the problem. Any help appreciated!!

Matploblib Image

Code

import scipy# Meshgrid
xgrid = dfList[0].index.tolist()
ygrid = np.linspace(266, 1, 532)
Xgrid, Ygrid = np.meshgrid(xgrid, ygrid)# Points
xo = dfList[0].index.tolist()
yo = [266, 300, 350, 400, 450, 500, 532]    # one for each DataFrame
points = [ [x, y] for y in yo for x in xo]
points = np.array(points)# Values
values = []
for df in dfList:values.extend(df['Y'].real)
# values = [ item for item in df['Y'].real for df in dfList]    # faster way of collapsing list
values = np.array(values)# Griddata
resampled = scipy.interpolate.griddata(points, values, (Xgrid, Ygrid), method='cubic')plt.imshow(resampled.T, extent=[365,1099,266,532], origin='lower')

dfList: Pickle Dump

https://gist.githubusercontent.com/anonymous/06076ecda9afcacfffd92b965996fe3e/raw/658e6157388ddedfe8882c2ad6c8f89af1eee5ac/dfList%2520(pickle%2520dump)

To make this answer somehow useful for other people, find here first a general explanation. Below there is a more concrete solution to the question.

The general explanation, np.meshgrid vs. np.mgrid in the use with scipy.interpolate.griddata.

I here provide an example which compares the use of np.meshgrid with np.mgrid when it comes to interpolation with scipy.interpolate.griddata. Gnerally speaking, the returns of np.meshgrid are the transposed returns of np.mgrid for the same grid.

import numpy as np; np.random.seed(0)
import scipy.interpolate
import matplotlib.pyplot as plt# np. meshgrid
xgrid = np.arange(21)[::2]
ygrid = np.linspace(0,5,6)
Xgrid, Ygrid = np.meshgrid(xgrid, ygrid)# np. mgrid
Xgrid2, Ygrid2 = np.mgrid[0:20:11j,0:5:6j]# points for interpolation
points = np.random.rand(200, 2)
points[:,0] *= 20 
points[:,1] *= 5# values
f = lambda x,y: np.sin(x)+ y
values = f(points[:,0], points[:,1])# initerpolation using grid defined with np.meshgrid
resampled = scipy.interpolate.griddata(points, values, (Xgrid2, Ygrid2), method='cubic')# interpolation using grid defined with np.mgrid
resampled2 = scipy.interpolate.griddata(points, values, (Xgrid.T, Ygrid.T), method='cubic')fig, (ax1, ax2, ax3) = plt.subplots(3,1)
kws = dict( extent=[-1,21,-0.5,5.5], vmin=-1, vmax=6, origin="lower")
ax1.set_title("function evaluated on grid")
ax1.imshow(f(Xgrid, Ygrid), **kws)ax2.set_title("interpolation using grid defined with np.meshgrid")
ax2.imshow(resampled.T, **kws)ax3.set_title("interpolation using grid defined with np.mgrid")
ax3.imshow(resampled2.T, **kws)for ax in (ax1, ax2, ax3):ax.set_yticks(range(6))ax.set_xticks(range(21)[::2])plt.tight_layout()
plt.show()

Step 1. Create a MCVE

(can be omitted, since more experienced users create those themselves when asking a question)

import numpy as np; np.random.seed(0)
import scipy.interpolate
import matplotlib.pyplot as plt
import pandas as pda = np.random.rand(532, 7)
dfList = [pd.DataFrame(a[:,i], columns=["Y"]) for i in range(7)]# Meshgrid
xgrid = dfList[0].index.tolist()
ygrid = np.linspace(266, 1, 532)
Xgrid, Ygrid = np.meshgrid(xgrid, ygrid)# Points
xo = dfList[0].index.tolist()
yo = [266, 300, 350, 400, 450, 500, 532]    # one for each DataFrame
points = [ [x, y] for y in yo for x in xo]
points = np.array(points)# Values
values = []
for df in dfList:values.extend(df['Y'].real)values = np.array(values)# Griddata
resampled = scipy.interpolate.griddata(points, values, (Xgrid, Ygrid), method='cubic')plt.imshow(resampled.T, extent=[365,1099,266,532], origin='lower')
plt.show()

creates

Step 2. The Problem.

We see a blank plot with only a small line of dots in the left handside of the image, while we would expect the complete graph to be filled with an image of shape (266, 532).

Step 3. The solution.

Using scipy.interpolate.griddata we need to supply the grids to the xi argument as a tuple (Xgrid.T, Ygrid.T), where the grids are generated via numpy.meshgrid: Xgrid, Ygrid = np.meshgrid(xgrid, ygrid). Note that meshgrid is different from numpy.mgrid.

There are some other inconsistencies with the points of the meshgrid compared to the sample points, so here I assume that you want to have the values between 266 and 532 being interpolated.

import numpy as np; np.random.seed(0)
import scipy.interpolate
import matplotlib.pyplot as plt
import pandas as pda = np.random.rand(532, 7)
dfList = [pd.DataFrame(a[:,i], columns=["Y"]) for i in range(7)]# Meshgrid
xgrid = dfList[0].index.values
ygrid = np.arange(266,532)
Xgrid, Ygrid = np.meshgrid(xgrid, ygrid)# Points
xo = dfList[0].index.tolist()
yo = [266, 300, 350, 400, 450, 500, 532]    # one for each DataFrame
points = [ [x, y] for y in yo for x in xo]
points = np.array(points)
print points.shape# Values
values = []
for df in dfList:values.extend(df['Y'].real)
values = np.array(values)# Griddata
resampled = scipy.interpolate.griddata(points, values, (Xgrid.T, Ygrid.T), method='cubic')
print resampled.T.shape
plt.imshow(resampled.T, extent=[365,1099,266,532], origin='lower') #, plt.show()