Missing values in Pandas Pivot table?

2024/9/23 15:30:31

I have a data set that looks like the following:

student     question                        answer   number
Bob         How many donuts in a dozen?       A        1
Sally       How many donuts in a dozen?       C        1
Edward      How many donuts in a dozen?       A        1
....
Edward      What colour is the sky?           C        1
Marvin      What colour is the sky?           D        1

From which I wrote some code that generates a pivot table to total up the results of a test, like so:

data = pd.pivot_table(df,index=['question'],columns = ['answer'],aggfunc='count',fill_value = 0)numberanswer     A    B   C   Dquestion
How many donuts in a dozen?     1    4   3   2
What colour is the sky?         1    9   0   0

From there I'm creating a heatmap from the pivot table for visualization purposes. Generally this works. However, if for some reason there are no students in the selected set who have chosen one of the answers (say, no one selected "D" for any questions) then that column doesn't show up in the heatmap; the column is left off.

How can I ensure that all the required columns display in the heatmap, even if no one selected that answer?

Answer

I think an even simpler approach would be to add 'dropna = False' to the pivot table parameters, default behavior is set to 'True'. This worked for me in a similar situation with time series data that contained large swaths of days with NaNs.

pd.pivot_table(dropna = False)
https://en.xdnf.cn/q/71810.html

Related Q&A

Selecting Element followed by text with Selenium WebDriver

I am using Selenium WebDriver and the Python bindings to automate some monotonous WordPress tasks, and it has been pretty straightforward up until this point. I am trying to select a checkbox, but the …

AttributeError: module keras.backend has no attribute image_dim_ordering

I tried to execute some tutorial transfer learning project. But Ive got attribute error.I checked my tensorflow and keras version.tensorflow : 1.14.0 keras : 2.2.5and python 3.6.9 version.the code is h…

Python Interpreter String Pooling Optimization [duplicate]

This question already has answers here:What determines which strings are interned and when? [duplicate](3 answers)Closed 6 years ago.After seeing this question and its duplicate a question still remai…

Flattening an array in pandas

One of the columns in DataFrame is an array. How do I flatten it? column1 column2 column3 var1 var11 [1, 2, 3, 4] var2 var22 [1, 2, 3, 4, -2, 12] var3 var33 [1, 2, 3, 4, 33, 544]Afte…

Difficulty in using sympy solver in python

Please run the following codefrom sympy.solvers import solvefrom sympy import Symbolx = Symbol(x)R2 = solve(-109*x**5/3870720+4157*x**4/1935360-3607*x**3/69120+23069*x**2/60480+5491*x/2520+38-67,x)prin…

Add custom html between two model fields in Django admins change_form

Lets say Ive two models:class Book(models.Model):name = models.CharField(max_length=50)library = models.ForeignKeyField(Library)class Library(models.Model):name = models.CharField(max_length=50) addr…

Plotly: How to add a horizontal scrollbar to a plotly express figure?

Im beginning to learn more about plotly and pandas and have a multivariate time series I wish to plot and interact with using plotly.express features. I also want my plot to a horizontal scrollbar so t…

How to run script in Pyspark and drop into IPython shell when done?

I want to run a spark script and drop into an IPython shell to interactively examine data. Running both:$ IPYTHON=1 pyspark --master local[2] myscript.pyand$ IPYTHON=1 spark-submit --master local[2] my…

Finding Min/Max Date with List Comprehension in Python

So I have this list:snapshots = [2014-04-05,2014-04-06,2014-04-07,2014-04-08,2014-04-09]I would like to find the earliest date using a list comprehension.Heres what I have now, earliest_date = snapshot…

plotting single 3D point on top of plot_surface in python matplotlib

I have some code to plot 3D surfaces in Python using matplotlib:import math import numpy as np import matplotlib.pyplot as plt from pylab import meshgrid,cm,imshow,contour,clabel,colorbar,axis from mpl…