I want to concatenate two dataframes with category-type columns, by first adding the missing categories to each column.
df = pd.DataFrame({"a": pd.Categorical(["foo", "foo", "bar"]), "b": [1, 2, 1]})
df2 = pd.DataFrame({"a": pd.Categorical(["baz"]), "b": [1]})df["a"] = df["a"].cat.add_categories("baz")
df2["a"] = df2["a"].cat.add_categories(["foo", "bar"])
In theory the categories for both "a"
columns are the same:
In [33]: df.a.cat.categories
Out[33]: Index(['bar', 'foo', 'baz'], dtype='object')In [34]: df2.a.cat.categories
Out[34]: Index(['baz', 'foo', 'bar'], dtype='object')
However, when concatenating the two dataframes, I get an object
-type "a"
column:
In [35]: pd.concat([df, df2]).info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 4 entries, 0 to 0
Data columns (total 2 columns):
a 4 non-null object
b 4 non-null int64
dtypes: int64(1), object(1)
memory usage: 96.0+ bytes
In the documentation it says that when categories are the same, it should result in a category
-type column. Does the order of the categories matter even though the category is unordered? I am using pandas-0.20.3
.