ZetCode

Python read text

last modified November 18, 2026 Python 3.15

In this article we show how to read text data in Python.

A text file is a sequence of bytes that represent characters in some encoding. When Python reads a file in text mode, it decodes the bytes into a str and translates the platform's line endings to \n. Reading text correctly is therefore mostly about the right encoding and the right handling of newlines. General file operations such as read(size), seek, and binary files are covered in Python read file.

We read text with the built-in open function or with the Path.read_text method of the pathlib module.

open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, ...)

The default mode 'r' means reading in text mode (same as 'rt'). The encoding parameter names the encoding used to decode the file, errors says what to do with bytes that cannot be decoded, and newline controls the translation of line endings.

The text files

Most examples use the following two files. The first contains English words, the second Slovak sentences with non-ASCII characters. Both are saved in UTF-8.

words.txt
falcon
sky
book
sum
cup
cloud
water
win

The words.txt file has eight lines; the last line ends with a newline, as is customary for text files.

towns.txt
Bratislava leží na Dunaji.
Košice sú druhé najväčšie mesto.
Žilina je na severe.

The towns.txt file contains characters such as ž, š, and ä, which take two bytes each in UTF-8.

Reading the whole file

The read method without arguments reads the file to the end and returns a single string.

read_all.py
with open('words.txt', encoding='utf-8') as f:
    contents = f.read()

print(type(contents), len(contents))
print(contents, end='')

The program reads the whole file and prints its contents.

with open('words.txt', encoding='utf-8') as f:

We open the file in the default text mode for reading. The with statement closes the file when the block ends, even if an exception is raised. We name the encoding explicitly; this makes the program behave the same on every system and Python version.

print(contents, end='')

The contents already end with a newline, so we suppress the one added by print.

$ python read_all.py
<class 'str'> 40
falcon
sky
book
sum
cup
cloud
water
win

Path.read_text

The Path.read_text method opens the file, reads it into a string, and closes it. It is the shortest way to read a small text file. For large files, reading line by line is better because the whole file is not held in memory.

read_path.py
from pathlib import Path

path = Path('towns.txt')
text = path.read_text(encoding='utf-8')

print(text, end='')
print(len(text), path.stat().st_size)

We read the file with Slovak text and compare its length in characters with its size in bytes.

print(len(text), path.stat().st_size)

The string has 81 characters but the file has 90 bytes, because nine of the characters are encoded in two bytes.

$ python read_path.py
Bratislava leží na Dunaji.
Košice sú druhé najväčšie mesto.
Žilina je na severe.
81 90

Reading line by line

A file object is an iterator over its lines. Each line keeps its trailing newline character. Iterating over the file reads it lazily, one line at a time.

read_lines.py
with open('words.txt', encoding='utf-8') as f:
    for line in f:
        print(repr(line), end=' ')
print()

with open('words.txt', encoding='utf-8') as f:
    words = [line.rstrip('\n') for line in f]

print(words)

We first print the raw lines with repr to show the newline characters, then build a list of words without them.

words = [line.rstrip('\n') for line in f]

rstrip('\n') removes only the newline. A plain rstrip or strip would also remove other whitespace, which is often what we want for word lists but not for data where spaces matter.

$ python read_lines.py
'falcon\n' 'sky\n' 'book\n' 'sum\n' 'cup\n' 'cloud\n' 'water\n' 'win\n' 
['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win']

Splitting text into lines

When the text is already in a string, there are several ways to split it into lines, and they differ in how they treat the final newline.

splitlines.py
from pathlib import Path

text = Path('words.txt').read_text(encoding='utf-8')

print(text.split('\n'))
print(text.splitlines())

with open('words.txt', encoding='utf-8') as f:
    print(f.readlines())

We split the text with split and splitlines and compare the result with readlines.

print(text.split('\n'))

Because the file ends with a newline, split('\n') produces an empty string at the end.

print(text.splitlines())

The splitlines method does not return an empty last item and also recognizes \r\n, \r, and other Unicode line boundaries. The readlines method returns a list of lines with their newlines.

$ python splitlines.py
['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win', '']
['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win']
['falcon\n', 'sky\n', 'book\n', 'sum\n', 'cup\n', 'cloud\n', 'water\n', 'win\n']

The default encoding

When no encoding is given, open uses a default. Up to Python 3.14 it was the locale encoding, so the same program could read a file correctly on one machine and produce garbage on another. Python 3.15 enables the UTF-8 mode by default (PEP 686): open, Path.read_text, and the standard streams use UTF-8 regardless of the locale.

default_encoding.py
import locale
import sys

print(sys.version.split()[0], 'UTF-8 mode:', sys.flags.utf8_mode)
print('locale encoding:', locale.getencoding())

with open('towns.txt') as f:
    print('file encoding:', f.encoding)
    print(f.readline(), end='')

The program prints the interpreter version, the UTF-8 mode flag, the locale encoding, and the encoding chosen for a file opened without the encoding argument. Then it reads the first line of the Slovak file.

We run the program with a legacy ISO-8859-2 locale, first with Python 3.15 and then with Python 3.14. The 3.14 run decodes the UTF-8 file as ISO-8859-2 and turns each two-byte character into two wrong ones.

$ LC_ALL=sk_SK.ISO-8859-2 python default_encoding.py
3.15.2 UTF-8 mode: 1
locale encoding: ISO-8859-2
file encoding: utf-8
Bratislava leží na Dunaji.
$ LC_ALL=sk_SK.ISO-8859-2 python3.14 default_encoding.py
3.14.0 UTF-8 mode: 0
locale encoding: ISO-8859-2
file encoding: ISO-8859-2
Bratislava leŞí na Dunaji.

EncodingWarning

To find places that rely on the default encoding, we run Python with the -X warn_default_encoding option. It emits an EncodingWarning for every open call without an explicit encoding.

Code that should run on older versions too must keep passing encoding='utf-8'. Code that intentionally wants the locale encoding should pass encoding='locale'.

$ python -X warn_default_encoding default_encoding.py
3.15.2 UTF-8 mode: 1
locale encoding: UTF-8
/home/user/python/default_encoding.py:7: EncodingWarning: 'encoding' argument not specified
  with open('towns.txt') as f:
file encoding: utf-8
Bratislava leží na Dunaji.

Decoding errors

If the file contains bytes that are not valid in the chosen encoding, reading raises UnicodeDecodeError. Some encodings, such as cp1250, can decode almost any byte and silently produce wrong characters instead. The errors parameter selects how invalid bytes are handled.

decode_errors.py
for enc in ['ascii', 'cp1250']:
    try:
        with open('towns.txt', encoding=enc) as f:
            print(f.readline(), end='')
    except UnicodeDecodeError as e:
        print(f'{enc}: {e}')

for handler in ['replace', 'ignore', 'backslashreplace']:
    with open('towns.txt', encoding='ascii', errors=handler) as f:
        print(f'{handler:16} {f.readline()}', end='')

We read the UTF-8 file with two wrong encodings and with three error handlers.

for handler in ['replace', 'ignore', 'backslashreplace']:

The replace handler puts the replacement character U+FFFD in place of each invalid byte, ignore drops the bytes, and backslashreplace shows them as escape sequences. The default is strict, which raises the exception.

$ python decode_errors.py
ascii: 'ascii' codec can't decode byte 0xc5 in position 13: ordinal not in range(128)
Bratislava leží na Dunaji.
replace          Bratislava le���� na Dunaji.
ignore           Bratislava le na Dunaji.
backslashreplace Bratislava le\xc5\xbe\xc3\xad na Dunaji.

Legacy encodings

Older files are often saved in a legacy single-byte encoding, such as Windows-1250 for Central European languages. To read them, we must know and specify the encoding.

legacy.py
from pathlib import Path

path = Path('towns-cp1250.txt')
path.write_text('Košice, Žilina, Prešov\n', encoding='cp1250')

print(path.read_bytes())
print(path.read_text(encoding='cp1250'), end='')
print(repr(path.read_text(encoding='latin-1')))

We write a short text in cp1250 and read it back with the right and a wrong encoding.

print(path.read_bytes())

Each Slovak character takes one byte in cp1250.

print(repr(path.read_text(encoding='latin-1')))

Latin-1 maps every byte to a character, so it never fails, but bytes 0x9a and 0x8e become invisible control characters. We print the repr to make them visible.

$ python legacy.py
b'Ko\x9aice, \x8eilina, Pre\x9aov\n'
Košice, Žilina, Prešov
'Ko\x9aice, \x8eilina, Pre\x9aov\n'

The byte order mark

Some Windows programs start UTF-8 files with a byte order mark (BOM), the bytes EF BB BF. The utf-8 codec keeps it as the \ufeff character at the start of the text. The utf-8-sig codec removes it if present.

bom.py
from pathlib import Path

path = Path('bom.txt')
path.write_bytes(b'\xef\xbb\xbfname,age\nPeter,34\n')

with open(path, encoding='utf-8') as f:
    print(repr(f.readline()))

with open(path, encoding='utf-8-sig') as f:
    print(repr(f.readline()))

We create a CSV file with a BOM and read its first line with both codecs. With utf-8, the first column name would be '\ufeffname' instead of 'name'.

$ python bom.py
'\ufeffname,age\n'
'name,age\n'

Newlines

In text mode, Python uses universal newlines by default: \r\n (Windows) and \r (old Mac) line endings are translated to \n when reading. Passing newline='' turns the translation off.

newlines.py
from pathlib import Path

path = Path('dos.txt')
path.write_bytes(b'first\r\nsecond\r\nthird\n')

with open(path, encoding='utf-8') as f:
    print(f.readlines(), f.newlines)

with open(path, encoding='utf-8', newline='') as f:
    print(f.readlines())

We create a file with mixed line endings and read it in two ways.

print(f.readlines(), f.newlines)

After reading, the newlines attribute lists the kinds of line endings that were found.

with open(path, encoding='utf-8', newline='') as f:

With newline='', the lines keep their original endings. The csv module requires files opened this way.

$ python newlines.py
['first\n', 'second\n', 'third\n'] ('\n', '\r\n')
['first\r\n', 'second\r\n', 'third\n']

Processing large files

Iterating over the file object processes one line at a time, so memory use does not depend on the file size. In this example we count words in the following text.

the-raven.txt
Once upon a midnight dreary, while I pondered, weak and weary,
Over many a quaint and curious volume of forgotten lore—
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
"'Tis some visitor," I muttered, "tapping at my chamber door—
            Only this and nothing more."

These are the first lines of Edgar Allan Poe's poem The Raven.

count_words.py
import re
from collections import Counter

counter = Counter()

with open('the-raven.txt', encoding='utf-8') as f:
    for line in f:
        counter.update(re.findall(r"[a-z]+(?:'[a-z]+)?", line.lower()))

print(counter.total(), 'words,', len(counter), 'unique')

for word, n in counter.most_common(5):
    print(f'{word:8} {n}')

We update a Counter with the words of each line and print the five most common words.

counter.update(re.findall(r"[a-z]+(?:'[a-z]+)?", line.lower()))

The regular expression matches words, optionally with an apostrophe inside, and skips punctuation such as quotes and dashes.

$ python count_words.py
57 words, 42 unique
a        3
i        3
and      3
while    2
of       2

Reading standard input

The sys.stdin object is a text stream like an open file. A program that reads it can process redirected files and the output of other commands.

upper.py
import sys

for nr, line in enumerate(sys.stdin, 1):
    print(nr, line.rstrip('\n').upper())

The program prints the lines of its input in uppercase with line numbers.

for nr, line in enumerate(sys.stdin, 1):

We iterate over standard input exactly like over a file. In Python 3.15, sys.stdin uses UTF-8 too, so the Slovak characters are decoded correctly.

$ python upper.py < words.txt
1 FALCON
2 SKY
3 BOOK
4 SUM
5 CUP
6 CLOUD
7 WATER
8 WIN
$ head -3 towns.txt | python upper.py
1 BRATISLAVA LEŽÍ NA DUNAJI.
2 KOŠICE SÚ DRUHÉ NAJVÄČŠIE MESTO.
3 ŽILINA JE NA SEVERE.

Reading several files

The fileinput module iterates over the lines of several files as one stream. It keeps track of the current file name and line numbers.

multi_files.py
import fileinput

with fileinput.input(files=('words.txt', 'towns.txt'), encoding='utf-8') as f:
    for line in f:
        if f.isfirstline():
            print(f'--- {f.filename()}')
        if len(line.rstrip()) > 4:
            print(f.filelineno(), line, end='')

We print the lines longer than four characters from both files.

if f.isfirstline():
    print(f'--- {f.filename()}')

The isfirstline method tells us when a new file starts. The filelineno method returns the line number within the current file, while lineno would count across all files.

$ python multi_files.py
--- words.txt
1 falcon
6 cloud
7 water
--- towns.txt
1 Bratislava leží na Dunaji.
2 Košice sú druhé najväčšie mesto.
3 Žilina je na severe.

Text in memory

The io.StringIO class is an in-memory text stream. It has the same interface as a file opened in text mode, which is handy for tests and for passing strings to functions that expect files.

string_io.py
import csv
import io

data = 'name,city\nPeter,Bratislava\nLucia,Košice\n'

with io.StringIO(data) as f:
    print(f.readline(), end='')
    for row in csv.reader(f):
        print(row)

We read the header line of CSV data with readline and pass the rest of the stream to csv.reader.

$ python string_io.py
name,city
['Peter', 'Bratislava']
['Lucia', 'Košice']

Source

open – Python built-in functions

Text I/O – io module

PEP 686 – Make UTF-8 mode default

In this article we have read text data in Python. See also Python read file, Python encode and decode, Python pathlib, and Python CSV.

Author

I am Jan Bodnar, a programmer and technical writer. I have been writing programming tutorials at ZetCode since 2007 and have published eight e-books. See about the author.

List all Python tutorials.