ZetCode

Pandas and Polars

last modified November 17, 2026 Python 3.15

Pandas and Polars are two DataFrame libraries for Python. Pandas is the older, widely used library built around a row index; Polars is a newer library whose API is built around expressions and that offers a lazy mode. This article is a migration-oriented comparison: each operation is shown in both libraries with its real output, so you can map what you already know onto the other API.

Two terms are needed up front. Eager execution means a call is carried out immediately and returns a result. Lazy execution means a call records a step in a plan and nothing runs until the plan is explicitly collected. Pandas is eager; Polars supports both an eager DataFrame and a lazy LazyFrame. No performance benchmark is claimed here — the examples illustrate syntax and output only.

Reading data and printed output

Both libraries read a CSV file with a function named read_csv, so migration of the loading step is straightforward. What differs immediately is the printed representation: Pandas shows a numeric index on the left and packs columns into fixed-width text, while Polars prints a box-drawing table with a shape line and an explicit data-type row, and has no index. The data itself is the same.

compare_read.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf)
print()
print('polars:')
print(pldf)

Both outputs contain four rows and four columns. Pandas labels the rows 0 to 3 and leaves the data types out of the printed view; Polars reports shape: (4, 4) and shows that id and score are i64 while name and group are str. The two libraries also differ in how they infer string columns by default, so check dtypes when moving code between them.

$ python compare_read.py
pandas:
   id     name  score group
0   1    Alice     10     A
1   2      Bob     20     B
2   3  Charlie     30     A
3   4    Diana     40     B

polars:
shape: (4, 4)
┌─────┬─────────┬───────┬───────┐
│ id  ┆ name    ┆ score ┆ group │
│ --- ┆ ---     ┆ ---   ┆ ---   │
│ i64 ┆ str     ┆ i64   ┆ str   │
╞═════╪═════════╪═══════╪═══════╡
│ 1   ┆ Alice   ┆ 10    ┆ A     │
│ 2   ┆ Bob     ┆ 20    ┆ B     │
│ 3   ┆ Charlie ┆ 30    ┆ A     │
│ 4   ┆ Diana   ┆ 40    ┆ B     │
└─────┴─────────┴───────┴───────┘

Selecting columns

In Pandas a subset of columns is taken with bracket indexing, for example df[['name', 'score']]. Polars uses the select method and takes the column names as separate arguments. Both return a new table containing just the requested columns and all four rows.

compare_select.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf[['name', 'score']])
print()
print('polars:')
print(pldf.select('name', 'score'))

The values are identical, but the shapes of the printed output follow the same pattern as before: Pandas keeps the index and Polars shows a shaped table without one. Note that Polars also accepts expressions inside select, so a column can be renamed or computed while it is being selected.

$ python compare_select.py
pandas:
      name  score
0    Alice     10
1      Bob     20
2  Charlie     30
3    Diana     40

polars:
shape: (4, 2)
┌─────────┬───────┐
│ name    ┆ score │
│ ---     ┆ ---   │
│ str     ┆ i64   │
╞═════════╪═══════╡
│ Alice   ┆ 10    │
│ Bob     ┆ 20    │
│ Charlie ┆ 30    │
│ Diana   ┆ 40    │
└─────────┴───────┘

Filtering rows

Pandas filters rows with a Boolean Series inside brackets; Polars uses the filter method with an expression built from pl.col. In both cases three rows survive. See the Polars filtering data tutorial for more ways to express conditions.

compare_filter.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf[pdf['score'] >= 20])
print()
print('polars:')
print(pldf.filter(pl.col('score') >= 20))

Pandas retains the original row labels 1, 2 and 3, which is why its index is not consecutive; Polars simply prints the three rows it kept. This is a visible consequence of the row index that Pandas carries and Polars does not. More on index-based selection is in the Pandas boolean indexing tutorial.

$ python compare_filter.py
pandas:
   id     name  score group
1   2      Bob     20     B
2   3  Charlie     30     A
3   4    Diana     40     B

polars:
shape: (3, 4)
┌─────┬─────────┬───────┬───────┐
│ id  ┆ name    ┆ score ┆ group │
│ --- ┆ ---     ┆ ---   ┆ ---   │
│ i64 ┆ str     ┆ i64   ┆ str   │
╞═════╪═════════╪═══════╪═══════╡
│ 2   ┆ Bob     ┆ 20    ┆ B     │
│ 3   ┆ Charlie ┆ 30    ┆ A     │
│ 4   ┆ Diana   ┆ 40    ┆ B     │
└─────┴─────────┴───────┴───────┘

Expressions versus index-based operations

The deeper difference behind these examples is how each library describes a calculation. Pandas works largely through the index: you label rows and columns, select with loc or iloc, and align results by their index labels. Polars works through expressions, instructions such as pl.col('score') * 2 that describe an operation on a column; those expressions are passed to methods like select, filter and with_columns.

Neither style is universally better. Index-based code feels natural when the row label has meaning and when operations must line up by that label. Expression-based code is compact for column arithmetic and composes well into long pipelines. Migration usually means replacing a chain of temporary Pandas Series with a few expressions inside one method call, and remembering that a Polars expression is a description of work, not the computed values themselves.

Grouping and aggregating

Both libraries group rows by a column and compute a summary per group. Pandas uses groupby followed by an aggregation; Polars uses group_by and an agg call containing expressions. The Polars groupby tutorial goes into the expression form in detail.

compare_group.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf.groupby('group')['score'].sum())
print()
print('polars:')
print(pldf.group_by('group', maintain_order=True).agg(pl.col('score').sum()))

Group A sums to 40 and group B to 60 in both results. The printed forms are quite different: Pandas returns a Series indexed by the group names, with the name score and the dtype shown underneath; Polars returns an ordinary two-column table with group and score as columns. Pandas sorts group keys by default, while Polars does not guarantee an order, so maintain_order=True is used here to keep the groups in first-seen order and make the printed output repeatable. Code that expects the group labels in the index needs adjusting after a move to Polars.

$ python compare_group.py
pandas:
group
A    40
B    60
Name: score, dtype: int64

polars:
shape: (2, 2)
┌───────┬───────┐
│ group ┆ score │
│ ---   ┆ ---   │
│ str   ┆ i64   │
╞═══════╪═══════╡
│ A     ┆ 40    │
│ B     ┆ 60    │
└───────┴───────┘

Adding columns: assign and with_columns

To add a derived column, Pandas offers assign and Polars offers with_columns. The Pandas keyword argument names the new column and its value is computed from existing columns; the Polars expression names the new column explicitly with alias. Both leave the original table unchanged.

compare_columns.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf.assign(bonus=pdf['score'] * 2))
print()
print('polars:')
print(pldf.with_columns((pl.col('score') * 2).alias('bonus')))

A bonus column equal to twice score appears in both outputs, with values 20, 40, 60 and 80. The Polars version needs no reference to the whole DataFrame: the expression describes which column to read and what to do with it. For more Pandas-side patterns see the Pandas method chaining tutorial.

$ python compare_columns.py
pandas:
   id     name  score group  bonus
0   1    Alice     10     A     20
1   2      Bob     20     B     40
2   3  Charlie     30     A     60
3   4    Diana     40     B     80

polars:
shape: (4, 5)
┌─────┬─────────┬───────┬───────┬───────┐
│ id  ┆ name    ┆ score ┆ group ┆ bonus │
│ --- ┆ ---     ┆ ---   ┆ ---   ┆ ---   │
│ i64 ┆ str     ┆ i64   ┆ str   ┆ i64   │
╞═════╪═════════╪═══════╪═══════╪═══════╡
│ 1   ┆ Alice   ┆ 10    ┆ A     ┆ 20    │
│ 2   ┆ Bob     ┆ 20    ┆ B     ┆ 40    │
│ 3   ┆ Charlie ┆ 30    ┆ A     ┆ 60    │
│ 4   ┆ Diana   ┆ 40    ┆ B     ┆ 80    │
└─────┴─────────┴───────┴───────┴───────┘

Joining tables

Pandas performs a join with merge; Polars uses join. Here both tables share an id column, and each library matches the rows on it and attaches the bonus column. The Polars join tutorial covers the other join types.

compare_join.py
import pandas as pd
import polars as pl

pdf_l = pd.DataFrame({'id': [1, 2, 3], 'score': [10, 20, 30]})
pdf_r = pd.DataFrame({'id': [1, 2, 3], 'bonus': [5, 6, 7]})
print('pandas:')
print(pdf_l.merge(pdf_r, on='id'))
print()
pldf_l = pl.DataFrame({'id': [1, 2, 3], 'score': [10, 20, 30]})
pldf_r = pl.DataFrame({'id': [1, 2, 3], 'bonus': [5, 6, 7]})
print('polars:')
print(pldf_l.join(pldf_r, on='id'))

Both results have three rows with matching id, score and bonus values. Again Pandas shows its default integer index while Polars prints a shaped table. The Pandas merge tutorial is linked under merging data if you want to compare the join options in detail.

$ python compare_join.py
pandas:
   id  score  bonus
0   1     10      5
1   2     20      6
2   3     30      7

polars:
shape: (3, 3)
┌─────┬───────┬───────┐
│ id  ┆ score ┆ bonus │
│ --- ┆ ---   ┆ ---   │
│ i64 ┆ i64   ┆ i64   │
╞═════╪═══════╪═══════╡
│ 1   ┆ 10    ┆ 5     │
│ 2   ┆ 20    ┆ 6     │
│ 3   ┆ 30    ┆ 7     │
└─────┴───────┴───────┘

Eager and lazy execution

The Pandas calls above all run at once. Polars can do the same, but it can also build a LazyFrame with scan_csv: the filter and select steps below are recorded into a plan, and only collect carries them out. The plan can be inspected with explain before it runs. Both libraries produce the same three names here.

compare_lazy.py
import pandas as pd
import polars as pl

pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pandas_result = pdf[pdf['score'] >= 20]['name'].tolist()

lazy = (pl.scan_csv('/tmp/opencode/agentH_data.csv')
        .filter(pl.col('score') >= 20)
        .select('name'))
polars_result = lazy.collect()['name'].to_list()

print('pandas (eager):', pandas_result)
print('polars (lazy): ', polars_result)

The two lists match. The difference is when the work happens: the Pandas filter is evaluated on the line that contains it, while the Polars chain does no reading or filtering until collect is called. This lets Polars look at the whole plan before running it, for example to avoid reading columns that are never used.

As for choosing between them, Pandas is a comfortable fit for small and medium tables, exploratory work, and code that relies on a meaningful row index or on the many third-party tools built around it. Polars is a natural fit for pipelines that can be expressed as expressions, for data that is larger than memory, and for workflows that benefit from deferring work until a plan is collected. These are differences of design, not a claim that one is always faster; measure your own workload before migrating for performance reasons.

$ python compare_lazy.py
pandas (eager): ['Bob', 'Charlie', 'Diana']
polars (lazy):  ['Bob', 'Charlie', 'Diana']

Source

Pandas read_csv Documentation, Pandas DataFrame.groupby Documentation, Polars LazyFrame Documentation, and Polars Expressions Documentation

In this article we have compared Pandas and Polars on reading CSV files, selecting and filtering data, grouping, adding columns, joining and lazy execution, running both libraries for every example. To start with Polars from the beginning see the Polars introduction and the Polars DataFrame tutorial; for the Pandas side see the Pandas introduction and the Pandas query tutorial.

Author

I am Jan Bodnar, a programmer and technical writer. I have been writing programming tutorials at ZetCode since 2007 and have published eight e-books. See about the author.

List all Pandas tutorials.