ZetCode

Pandas Categorical Data

last modified November 17, 2026 Python 3.15

A categorical dtype is a column whose values are drawn from a fixed set of categories rather than being arbitrary strings or numbers. A column of shirt sizes, for instance, may only ever contain small, medium or large. Pandas stores such a column as a list of the allowed categories plus a compact array of codes pointing into that list.

This representation saves memory when values repeat, gives meaningful order to labels that are not numbers, and lets Pandas treat declared-but-unused categories explicitly. All examples below were run with pandas 3.1.1.

Creating a categorical with astype

The quickest way to convert an ordinary column is astype('category'). Pandas discovers the categories from the values present and, on this version, lists them in sorted order. The dtype of the result is category, not object or str.

cat_astype.py
import pandas as pd

s = pd.Series(['low', 'high', 'low', 'medium', 'high'])
c = s.astype('category')
print(c)
print()
print(c.dtype)

The printed Series still shows the original labels, but the final line reports the new dtype and the three categories Pandas found: high, low and medium. The values themselves were not rewritten.

$ python cat_astype.py
0       low
1      high
2       low
3    medium
4      high
dtype: category
Categories (3, str): ['high', 'low', 'medium']

category

Choosing the categories explicitly

To control the set of allowed values, build a Categorical object and pass a categories list. Values outside that list become missing values (NaN). The categories attribute shows the declared set, while codes shows the integer position of each value inside it; -1 marks a missing value.

cat_explicit.py
import pandas as pd

cat = pd.Categorical(['low', 'high', 'low'], categories=['low', 'medium', 'high'])
print(cat)
print()
print(cat.categories)
print()
print(cat.codes)

Here the categories appear in the order we supplied them, and the codes 0, 2, 0 refer to positions in that list: low is at index 0 and high at index 2. The declared category medium is present even though no value uses it.

$ python cat_explicit.py
['low', 'high', 'low']
Categories (3, str): ['low', 'medium', 'high']

Index(['low', 'medium', 'high'], dtype='str')

[0 2 0]

Ordered categories and comparisons

By default categories have no order, so comparing two of them is not meaningful. Setting ordered=True declares that the listed order is real. Comparisons such as cat > 'medium' then work, and sorting follows the declared order instead of alphabetical order.

cat_ordered.py
import pandas as pd

cat = pd.Categorical(
    ['low', 'high', 'medium'],
    categories=['low', 'medium', 'high'],
    ordered=True
)
print(cat > 'medium')
print()
print(cat.sort_values())

Only the value high is above medium, so the boolean array is False, True, False. The sorted result follows low, medium, high; the dtype line uses < signs to show that the ordering is defined.

$ python cat_ordered.py
[False  True False]

['low', 'medium', 'high']
Categories (3, str): ['low' < 'medium' < 'high']

Saving memory

The memory_usage method reports how many bytes a column occupies. Passing deep=True counts the actual string data for text columns; without it, Pandas only counts the references. Comparing the two forms on a repeated-value column shows why categoricals are attractive.

cat_memory.py
import pandas as pd

s = pd.Series(['small', 'medium', 'large'] * 1000)
obj = s.memory_usage(deep=True)
cat = s.astype('category').memory_usage(deep=True)
print('object:  ', obj, 'bytes')
print('category:', cat, 'bytes')

On this run the text column needs 40132 bytes while the categorical needs only 3173 bytes: the three distinct strings are stored once and reused via one-byte codes. Exact figures depend on the platform, the values and the Pandas version.

$ python cat_memory.py
object:   40132 bytes
category: 3173 bytes

Grouping and unused categories

A groupby on a categorical column has a choice about categories that appear in the dtype but not in the data. With observed=False every declared category gets a group, including empty ones; with observed=True only categories that actually occur are returned.

cat_group.py
import pandas as pd

df = pd.DataFrame({
    'size': pd.Categorical(['small', 'large'], categories=['small', 'medium', 'large']),
    'price': [10, 20]
})
print(df.groupby('size', observed=False)['price'].sum())
print()
print(df.groupby('size', observed=True)['price'].sum())

The first result contains a medium row with a sum of 0 even though no row has that size. The second result omits it. Use observed=True when you want only groups backed by real data, and the default when a stable set of groups matters. See the grouping data tutorial for more on grouping.

$ python cat_group.py
size
small     10
medium     0
large     20
Name: price, dtype: int64

size
small    10
large    20
Name: price, dtype: int64

Binning with cut and qcut

Binning means converting continuous numbers into discrete intervals, which produces an ordered categorical result. cut divides the range into equal-width bins, while qcut divides the data into bins of equal size (quantiles). Both return a Series with a category dtype whose categories are intervals.

cat_cut.py
import pandas as pd

s = pd.Series([1, 7, 12, 19, 25])
print(pd.cut(s, bins=3))
print()
print(pd.qcut(s, q=3))

The first block has three equal-width intervals; the second uses quantiles, splitting the values into three groups of roughly equal size. The parenthesised and square brackets describe which endpoints are included. A fuller treatment is in the Pandas binning tutorial.

$ python cat_cut.py
0    (0.976, 9.0]
1    (0.976, 9.0]
2     (9.0, 17.0]
3    (17.0, 25.0]
4    (17.0, 25.0]
dtype: category
Categories (3, interval[float64, right]): [(0.976, 9.0] < (9.0, 17.0] <
                                           (17.0, 25.0]]

0     (0.999, 8.667]
1     (0.999, 8.667]
2    (8.667, 16.667]
3     (16.667, 25.0]
4     (16.667, 25.0]
dtype: category
Categories (3, interval[float64, right]): [(0.999, 8.667] < (8.667, 16.667] <
                                           (16.667, 25.0]]

Source

Pandas Categorical Data Documentation, Pandas Categorical Reference, Pandas cut Documentation, and Pandas qcut Documentation

In this article we have converted columns to the category dtype, declared explicit and ordered categories, measured the memory saved, grouped with unused categories, and glimpsed binning. For the memory figures see the Pandas memory usage tutorial, and for the interval details see Pandas binning.

Author

I am Jan Bodnar, a programmer and technical writer. I have been writing programming tutorials at ZetCode since 2007 and have published eight e-books. See about the author.

List all Pandas tutorials.