ZetCode

Pandas Reading and Writing CSV

last modified November 17, 2026 Python 3.15

CSV (comma-separated values) is a plain-text format for tabular data: each line is a row and the fields on a line are separated by a delimiter. It is one of the most common ways to exchange data, and Pandas reads and writes it with read_csv and to_csv.

To keep every example self-contained, we use io.StringIO to turn a short CSV string into a file-like object. In real code you pass a file path or a URL to read_csv in exactly the same way. The examples were verified with Pandas 3.1.1.

Reading a CSV file

The simplest call to pd.read_csv takes a path, a URL, or a file-like object. By default the first line is treated as a header row and its fields become the column names. The result is a DataFrame.

csv_basic.py
import pandas as pd
from io import StringIO

data = """name,age,city
Alice,30,Prague
Bob,25,Bratislava
Charlie,35,Vienna
"""

df = pd.read_csv(StringIO(data))
print(df)

The three fields on the first line, name, age, and city, become the column names. Pandas guesses the type of each column, so age is read as an integer column and the others as text, and it adds a numerical row index starting at zero.

$ python csv_basic.py
      name  age        city
0    Alice   30      Prague
1      Bob   25  Bratislava
2  Charlie   35      Vienna

Separator and header

The sep parameter sets the field delimiter; the default is a comma. Semicolons are common in CSV files produced by European spreadsheet software. The header parameter tells Pandas which row contains the column names; header=None means the file has no header, so Pandas creates the integer column names 0, 1, and 2.

csv_sep_header.py
import pandas as pd
from io import StringIO

data = """name;age;city
Alice;30;Prague
Bob;25;Bratislava
"""

df = pd.read_csv(StringIO(data), sep=';')
print('sep=";":')
print(df)
print()

data2 = """Alice,30,Prague
Bob,25,Bratislava
"""
df2 = pd.read_csv(StringIO(data2), header=None)
print('header=None:')
print(df2)

With sep=';' the semicolon-separated file is parsed into three columns just like a comma-separated one. In the second example every line is a data row, so the first line is no longer lost as a header; the columns are named 0, 1, and 2 instead.

$ python csv_sep_header.py
sep=";":
    name  age        city
0  Alice   30      Prague
1    Bob   25  Bratislava

header=None:
       0   1           2
0  Alice  30      Prague
1    Bob  25  Bratislava

Supplying column names with names

When a file has no header row, the names parameter provides the column names directly. It is also useful for replacing an existing header when the one in the file is unusable. In that case combine names with header=0 to overwrite the first line.

csv_names.py
import pandas as pd
from io import StringIO

data = """Alice,30,Prague
Bob,25,Bratislava
"""

df = pd.read_csv(StringIO(data), names=['name', 'age', 'city'])
print(df)

The resulting DataFrame has meaningful column names even though the CSV data contains no header line. The number of names must match the number of fields in each row; if there are fewer names, Pandas uses the remaining fields as an index.

$ python csv_names.py
    name  age        city
0  Alice   30      Prague
1    Bob   25  Bratislava

Data types and missing markers

Pandas infers a data type for each column, but the guess is not always what we want. The dtype parameter forces a type for chosen columns, which matters for values with leading zeros. The na_values parameter lists strings that should be interpreted as missing data and turned into NaN.

csv_dtype_na.py
import pandas as pd
from io import StringIO

data = """zip,temp,status
00123,3.5,ok
00456,NA,missing
00789,7.2,
"""

df = pd.read_csv(StringIO(data), dtype={'zip': 'string'}, na_values=['NA', 'missing'])
print(df)
print()
print(df.dtypes)

The zip column keeps its leading zeros because we read it as a string; read as a number, 00123 would become 123. The strings NA and missing are converted to NaN, and the empty field in the last row is recognised as missing automatically. The printed dtypes confirm that zip is a string column and temp is numeric.

$ python csv_dtype_na.py
     zip  temp status
0  00123   3.5     ok
1  00456   NaN    NaN
2  00789   7.2    NaN

zip        string
temp      float64
status        str
dtype: object

Reading only selected columns

Large files often contain many columns we do not need. The usecols parameter reads only the listed columns, which saves memory and speeds up parsing because the other fields are skipped.

csv_usecols.py
import pandas as pd
from io import StringIO

data = """name,age,city,salary
Alice,30,Prague,50000
Bob,25,Bratislava,45000
"""

df = pd.read_csv(StringIO(data), usecols=['name', 'city'])
print(df)

Only name and city appear in the result; the age and salary columns are never loaded. The usecols value can also be a list of column positions or a function that decides column by column.

$ python csv_usecols.py
    name        city
0  Alice      Prague
1    Bob  Bratislava

Writing a CSV file

The to_csv method writes a DataFrame back to CSV. By default it includes the row index as an extra unnamed first column, so index=False is commonly used to omit it. The quoting parameter controls when fields are wrapped in double quotes; csv.QUOTE_ALL quotes every field, including the header. A comma inside a value is always quoted, whatever the setting, so the file stays valid CSV.

csv_tocsv.py
import pandas as pd
import csv
from io import StringIO

df = pd.DataFrame({'name': ['Alice', 'Bob'], 'note': ['hello, world', 'plain']})

out = StringIO()
df.to_csv(out, index=False)
print('default:')
print(out.getvalue(), end='')

out2 = StringIO()
df.to_csv(out2, index=False, quoting=csv.QUOTE_ALL)
print('quoted:')
print(out2.getvalue(), end='')

The first output starts with a header and contains no index column because of index=False. The value hello, world is automatically wrapped in quotes because it contains a delimiter. With csv.QUOTE_ALL, every field is quoted, which some applications prefer. To save to disk, pass a filename instead of a StringIO object.

$ python csv_tocsv.py
default:
name,note
Alice,"hello, world"
Bob,plain
quoted:
"name","note"
"Alice","hello, world"
"Bob","plain"

Reading large files in chunks

A huge CSV file need not be loaded into memory all at once. Setting chunksize makes read_csv return an iterator that yields one DataFrame of chunksize rows at a time. This lets us process a file of any size with a fixed amount of memory.

csv_chunksize.py
import pandas as pd
from io import StringIO

data = """name,age
Alice,30
Bob,25
Charlie,35
Diana,28
"""

total = 0
for chunk in pd.read_csv(StringIO(data), chunksize=2):
    total += len(chunk)
    print(f'chunk of {len(chunk)} rows, ages: {list(chunk["age"])}')

print('total rows:', total)

The four data rows arrive as two chunks of two rows each. Each chunk is an ordinary DataFrame, so we can filter, aggregate, or write it before moving on to the next one. The running total shows that all four rows were processed even though no full DataFrame ever held them together.

$ python csv_chunksize.py
chunk of 2 rows, ages: [30, 25]
chunk of 2 rows, ages: [35, 28]
total rows: 4

Source

Pandas documentation - read_csv
Pandas documentation - to_csv
Python documentation - csv module

In this article we have read CSV data with read_csv, controlling the separator, header, column names, data types, missing markers, and the set of columns loaded, and we have written CSV with to_csv, including quoting and index handling. Finally we processed a file in chunks. See also the Pandas DataFrame, reading and writing Excel, and dropping missing values articles.

Author

I am Jan Bodnar, a programmer and technical writer. I have been writing programming tutorials at ZetCode since 2007 and have published eight e-books. See about the author.

List all Pandas tutorials.