Pandas and Polars
last modified November 17, 2026 Python 3.15
Pandas and Polars are two DataFrame libraries for Python. Pandas is the older, widely used library built around a row index; Polars is a newer library whose API is built around expressions and that offers a lazy mode. This article is a migration-oriented comparison: each operation is shown in both libraries with its real output, so you can map what you already know onto the other API.
Two terms are needed up front. Eager execution means a call is
carried out immediately and returns a result. Lazy execution means a
call records a step in a plan and nothing runs until the plan is explicitly
collected. Pandas is eager; Polars supports both an eager DataFrame and a lazy
LazyFrame. No performance benchmark is claimed here — the examples
illustrate syntax and output only.
Reading data and printed output
Both libraries read a CSV file with a function named read_csv, so
migration of the loading step is straightforward. What differs immediately is
the printed representation: Pandas shows a numeric index on the left and packs
columns into fixed-width text, while Polars prints a box-drawing table with a
shape line and an explicit data-type row, and has no index. The data itself is
the same.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf)
print()
print('polars:')
print(pldf)
Both outputs contain four rows and four columns. Pandas labels the rows 0 to 3
and leaves the data types out of the printed view; Polars reports
shape: (4, 4) and shows that id and
score are i64 while name and group are
str. The two libraries also differ in how they infer string columns by
default, so check dtypes when moving code between them.
$ python compare_read.py pandas: id name score group 0 1 Alice 10 A 1 2 Bob 20 B 2 3 Charlie 30 A 3 4 Diana 40 B polars: shape: (4, 4) ┌─────┬─────────┬───────┬───────┐ │ id ┆ name ┆ score ┆ group │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ i64 ┆ str │ ╞═════╪═════════╪═══════╪═══════╡ │ 1 ┆ Alice ┆ 10 ┆ A │ │ 2 ┆ Bob ┆ 20 ┆ B │ │ 3 ┆ Charlie ┆ 30 ┆ A │ │ 4 ┆ Diana ┆ 40 ┆ B │ └─────┴─────────┴───────┴───────┘
Selecting columns
In Pandas a subset of columns is taken with bracket indexing, for example
df[['name', 'score']]. Polars uses the
select method and takes the column names
as separate arguments. Both return a new table containing just the requested
columns and all four rows.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf[['name', 'score']])
print()
print('polars:')
print(pldf.select('name', 'score'))
The values are identical, but the shapes of the printed output follow the
same pattern as before: Pandas keeps the index and Polars shows a shaped table
without one. Note that Polars also accepts expressions inside
select, so a column can be renamed or computed while it is being
selected.
$ python compare_select.py
pandas:
name score
0 Alice 10
1 Bob 20
2 Charlie 30
3 Diana 40
polars:
shape: (4, 2)
┌─────────┬───────┐
│ name ┆ score │
│ --- ┆ --- │
│ str ┆ i64 │
╞═════════╪═══════╡
│ Alice ┆ 10 │
│ Bob ┆ 20 │
│ Charlie ┆ 30 │
│ Diana ┆ 40 │
└─────────┴───────┘
Filtering rows
Pandas filters rows with a Boolean Series inside brackets; Polars uses the
filter method with an expression built from
pl.col. In both cases three rows survive. See the
Polars filtering data tutorial for more
ways to express conditions.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf[pdf['score'] >= 20])
print()
print('polars:')
print(pldf.filter(pl.col('score') >= 20))
Pandas retains the original row labels 1, 2 and 3, which is why its index is not consecutive; Polars simply prints the three rows it kept. This is a visible consequence of the row index that Pandas carries and Polars does not. More on index-based selection is in the Pandas boolean indexing tutorial.
$ python compare_filter.py pandas: id name score group 1 2 Bob 20 B 2 3 Charlie 30 A 3 4 Diana 40 B polars: shape: (3, 4) ┌─────┬─────────┬───────┬───────┐ │ id ┆ name ┆ score ┆ group │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ i64 ┆ str │ ╞═════╪═════════╪═══════╪═══════╡ │ 2 ┆ Bob ┆ 20 ┆ B │ │ 3 ┆ Charlie ┆ 30 ┆ A │ │ 4 ┆ Diana ┆ 40 ┆ B │ └─────┴─────────┴───────┴───────┘
Expressions versus index-based operations
The deeper difference behind these examples is how each library describes a
calculation. Pandas works largely through the index: you label rows and
columns, select with loc or iloc, and align results by
their index labels. Polars works through expressions, instructions
such as pl.col('score') * 2 that describe an operation on a
column; those expressions are passed to methods like select,
filter and with_columns.
Neither style is universally better. Index-based code feels natural when the row label has meaning and when operations must line up by that label. Expression-based code is compact for column arithmetic and composes well into long pipelines. Migration usually means replacing a chain of temporary Pandas Series with a few expressions inside one method call, and remembering that a Polars expression is a description of work, not the computed values themselves.
Grouping and aggregating
Both libraries group rows by a column and compute a summary per group. Pandas
uses groupby followed by an aggregation; Polars uses
group_by and an agg call containing expressions. The
Polars groupby tutorial goes into the
expression form in detail.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf.groupby('group')['score'].sum())
print()
print('polars:')
print(pldf.group_by('group', maintain_order=True).agg(pl.col('score').sum()))
Group A sums to 40 and group B to 60 in both results. The printed forms are
quite different: Pandas returns a Series indexed by the group names, with the
name score and the dtype shown underneath; Polars returns an
ordinary two-column table with group and score as
columns. Pandas sorts group keys by default, while Polars does not guarantee
an order, so maintain_order=True is used here to keep the groups
in first-seen order and make the printed output repeatable. Code that expects
the group labels in the index needs adjusting after a move to Polars.
$ python compare_group.py pandas: group A 40 B 60 Name: score, dtype: int64 polars: shape: (2, 2) ┌───────┬───────┐ │ group ┆ score │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═══════╪═══════╡ │ A ┆ 40 │ │ B ┆ 60 │ └───────┴───────┘
Adding columns: assign and with_columns
To add a derived column, Pandas offers assign and Polars offers
with_columns. The Pandas keyword argument names the new column and
its value is computed from existing columns; the Polars expression names the
new column explicitly with alias. Both leave the original table
unchanged.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pldf = pl.read_csv('/tmp/opencode/agentH_data.csv')
print('pandas:')
print(pdf.assign(bonus=pdf['score'] * 2))
print()
print('polars:')
print(pldf.with_columns((pl.col('score') * 2).alias('bonus')))
A bonus column equal to twice score appears in both
outputs, with values 20, 40, 60 and 80. The Polars version needs no reference
to the whole DataFrame: the expression describes which column to read and what
to do with it. For more Pandas-side patterns see the
Pandas method chaining tutorial.
$ python compare_columns.py pandas: id name score group bonus 0 1 Alice 10 A 20 1 2 Bob 20 B 40 2 3 Charlie 30 A 60 3 4 Diana 40 B 80 polars: shape: (4, 5) ┌─────┬─────────┬───────┬───────┬───────┐ │ id ┆ name ┆ score ┆ group ┆ bonus │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ i64 ┆ str ┆ i64 │ ╞═════╪═════════╪═══════╪═══════╪═══════╡ │ 1 ┆ Alice ┆ 10 ┆ A ┆ 20 │ │ 2 ┆ Bob ┆ 20 ┆ B ┆ 40 │ │ 3 ┆ Charlie ┆ 30 ┆ A ┆ 60 │ │ 4 ┆ Diana ┆ 40 ┆ B ┆ 80 │ └─────┴─────────┴───────┴───────┴───────┘
Joining tables
Pandas performs a join with merge; Polars uses
join. Here both tables share an id column, and each
library matches the rows on it and attaches the bonus column. The
Polars join tutorial covers the other
join types.
import pandas as pd
import polars as pl
pdf_l = pd.DataFrame({'id': [1, 2, 3], 'score': [10, 20, 30]})
pdf_r = pd.DataFrame({'id': [1, 2, 3], 'bonus': [5, 6, 7]})
print('pandas:')
print(pdf_l.merge(pdf_r, on='id'))
print()
pldf_l = pl.DataFrame({'id': [1, 2, 3], 'score': [10, 20, 30]})
pldf_r = pl.DataFrame({'id': [1, 2, 3], 'bonus': [5, 6, 7]})
print('polars:')
print(pldf_l.join(pldf_r, on='id'))
Both results have three rows with matching id, score
and bonus values. Again Pandas shows its default integer index
while Polars prints a shaped table. The Pandas merge tutorial is linked under
merging data if you want to compare the
join options in detail.
$ python compare_join.py pandas: id score bonus 0 1 10 5 1 2 20 6 2 3 30 7 polars: shape: (3, 3) ┌─────┬───────┬───────┐ │ id ┆ score ┆ bonus │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═══════╪═══════╡ │ 1 ┆ 10 ┆ 5 │ │ 2 ┆ 20 ┆ 6 │ │ 3 ┆ 30 ┆ 7 │ └─────┴───────┴───────┘
Eager and lazy execution
The Pandas calls above all run at once. Polars can do the same, but it can also
build a LazyFrame with scan_csv: the filter and
select steps below are recorded into a plan, and only collect
carries them out. The plan can be inspected with explain before it
runs. Both libraries produce the same three names here.
import pandas as pd
import polars as pl
pdf = pd.read_csv('/tmp/opencode/agentH_data.csv')
pandas_result = pdf[pdf['score'] >= 20]['name'].tolist()
lazy = (pl.scan_csv('/tmp/opencode/agentH_data.csv')
.filter(pl.col('score') >= 20)
.select('name'))
polars_result = lazy.collect()['name'].to_list()
print('pandas (eager):', pandas_result)
print('polars (lazy): ', polars_result)
The two lists match. The difference is when the work happens: the Pandas
filter is evaluated on the line that contains it, while the Polars chain does
no reading or filtering until collect is called. This lets Polars
look at the whole plan before running it, for example to avoid reading columns
that are never used.
As for choosing between them, Pandas is a comfortable fit for small and medium tables, exploratory work, and code that relies on a meaningful row index or on the many third-party tools built around it. Polars is a natural fit for pipelines that can be expressed as expressions, for data that is larger than memory, and for workflows that benefit from deferring work until a plan is collected. These are differences of design, not a claim that one is always faster; measure your own workload before migrating for performance reasons.
$ python compare_lazy.py pandas (eager): ['Bob', 'Charlie', 'Diana'] polars (lazy): ['Bob', 'Charlie', 'Diana']
Source
Pandas read_csv Documentation, Pandas DataFrame.groupby Documentation, Polars LazyFrame Documentation, and Polars Expressions Documentation
In this article we have compared Pandas and Polars on reading CSV files, selecting and filtering data, grouping, adding columns, joining and lazy execution, running both libraries for every example. To start with Polars from the beginning see the Polars introduction and the Polars DataFrame tutorial; for the Pandas side see the Pandas introduction and the Pandas query tutorial.
Author
List all Pandas tutorials.