I have following data frame in which i want to get missing date along with its keys in pandas.

     size     number     key      date
0  153.2 K    12345     Hello     20181001
1  153.2 K    12345     No        20181001
2  153.2 K    12345     Hello     20181003
3  153.2 K    12345     No        20181003
4  153.2 K    12345     Hello     20181004
5  153.2 K    12345     No        20181004

I want the output as,

  key      dateHello     20181002No        20181002

thanks in advance

You can do this with some fancy reshaping like this:

(df.pivot('date', 'key').reindex(np.arange(df['date'].min(), df['date'].max()+1)).stack('key', dropna=False).loc[lambda x: x['size'].isna()].index.to_frame(index=False))

Output:

       date    key
0  20181002  Hello
1  20181002     No

How?

• Reshape the dataframe such that you have a single date per row

• Next, reindex the dataframe to fill in missing dates

• Reshape the dataframe stacking key but keeping NaN values

• Filter dataframe to only missing values using isna

• Convert the index to a dataframe with to_frame

Update address date concern mentioned by @Cimbali below

df['date'] = pd.to_datetime(df['date'], format='%Y%m%d')
(df.pivot('date', 'key').reindex(pd.date_range(df['date'].min(), df['date'].max(), freq='D')).stack('key', dropna=False).loc[lambda x: x['size'].isna()].index.to_frame(index=False))

Output:

           0    key
0 2018-10-02  Hello
1 2018-10-02     No