I'm trying to do something like

df.query("'column' == 'a'").count()

but with

df.query("'column' == False").count()

What is the right way of using query with a bool column?

It's simply 'column == False'.

>>> df = pd.DataFrame([[False, 1], [True, 2], [False, 3]], columns=['column', 'another_column'])                       
>>> df                                                                                                                 column  another_column
0   False               1
1    True               2
2   False               3
>>> df.query('column == False')                                                                                        column  another_column
0   False               1
2   False               3
>>> df.query('column == False').count()                                                                                
column            2
another_column    2
dtype: int64

Personally, I prefer boolean indexing (if applicable to your situation).

>>> df[~df['column']]                                                                                                  column  another_column
0   False               1
2   False               3
>>> df[~df['column']].count()                                                                                          
column            2
another_column    2
dtype: int64