Python read text
last modified November 18, 2026 Python 3.15
In this article we show how to read text data in Python.
A text file is a sequence of bytes that represent characters in some
encoding. When Python reads a file in text mode, it decodes the bytes into a
str and translates the platform's line endings to
\n. Reading text correctly is therefore mostly about the right
encoding and the right handling of newlines. General file operations such as
read(size), seek, and binary files are covered in
Python read file.
We read text with the built-in open function or with the
Path.read_text method of the pathlib module.
open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, ...)
The default mode 'r' means reading in text mode (same as
'rt'). The encoding parameter names the encoding
used to decode the file, errors says what to do with bytes that
cannot be decoded, and newline controls the translation of line
endings.
The text files
Most examples use the following two files. The first contains English words, the second Slovak sentences with non-ASCII characters. Both are saved in UTF-8.
falcon sky book sum cup cloud water win
The words.txt file has eight lines; the last line ends with a
newline, as is customary for text files.
Bratislava leží na Dunaji. Košice sú druhé najväčšie mesto. Žilina je na severe.
The towns.txt file contains characters such as
ž, š, and ä, which take two bytes
each in UTF-8.
Reading the whole file
The read method without arguments reads the file to the end
and returns a single string.
with open('words.txt', encoding='utf-8') as f:
contents = f.read()
print(type(contents), len(contents))
print(contents, end='')
The program reads the whole file and prints its contents.
with open('words.txt', encoding='utf-8') as f:
We open the file in the default text mode for reading. The
with statement closes the file when the block ends, even if an
exception is raised. We name the encoding explicitly; this makes the program
behave the same on every system and Python version.
print(contents, end='')
The contents already end with a newline, so we suppress the one added by
print.
$ python read_all.py <class 'str'> 40 falcon sky book sum cup cloud water win
Path.read_text
The Path.read_text method opens the file, reads it into a
string, and closes it. It is the shortest way to read a small text file.
For large files, reading line by line is better because the whole file is
not held in memory.
from pathlib import Path
path = Path('towns.txt')
text = path.read_text(encoding='utf-8')
print(text, end='')
print(len(text), path.stat().st_size)
We read the file with Slovak text and compare its length in characters with its size in bytes.
print(len(text), path.stat().st_size)
The string has 81 characters but the file has 90 bytes, because nine of the characters are encoded in two bytes.
$ python read_path.py Bratislava leží na Dunaji. Košice sú druhé najväčšie mesto. Žilina je na severe. 81 90
Reading line by line
A file object is an iterator over its lines. Each line keeps its trailing newline character. Iterating over the file reads it lazily, one line at a time.
with open('words.txt', encoding='utf-8') as f:
for line in f:
print(repr(line), end=' ')
print()
with open('words.txt', encoding='utf-8') as f:
words = [line.rstrip('\n') for line in f]
print(words)
We first print the raw lines with repr to show the newline
characters, then build a list of words without them.
words = [line.rstrip('\n') for line in f]
rstrip('\n') removes only the newline. A plain
rstrip or strip would also remove other
whitespace, which is often what we want for word lists but not for data
where spaces matter.
$ python read_lines.py 'falcon\n' 'sky\n' 'book\n' 'sum\n' 'cup\n' 'cloud\n' 'water\n' 'win\n' ['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win']
Splitting text into lines
When the text is already in a string, there are several ways to split it into lines, and they differ in how they treat the final newline.
from pathlib import Path
text = Path('words.txt').read_text(encoding='utf-8')
print(text.split('\n'))
print(text.splitlines())
with open('words.txt', encoding='utf-8') as f:
print(f.readlines())
We split the text with split and splitlines and
compare the result with readlines.
print(text.split('\n'))
Because the file ends with a newline, split('\n') produces an
empty string at the end.
print(text.splitlines())
The splitlines method does not return an empty last item and
also recognizes \r\n, \r, and other Unicode line
boundaries. The readlines method returns a list of lines with
their newlines.
$ python splitlines.py ['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win', ''] ['falcon', 'sky', 'book', 'sum', 'cup', 'cloud', 'water', 'win'] ['falcon\n', 'sky\n', 'book\n', 'sum\n', 'cup\n', 'cloud\n', 'water\n', 'win\n']
The default encoding
When no encoding is given, open uses a default. Up to Python
3.14 it was the locale encoding, so the same program could read a file
correctly on one machine and produce garbage on another. Python 3.15
enables the UTF-8 mode by default (PEP 686): open,
Path.read_text, and the standard streams use UTF-8 regardless
of the locale.
import locale
import sys
print(sys.version.split()[0], 'UTF-8 mode:', sys.flags.utf8_mode)
print('locale encoding:', locale.getencoding())
with open('towns.txt') as f:
print('file encoding:', f.encoding)
print(f.readline(), end='')
The program prints the interpreter version, the UTF-8 mode flag, the locale
encoding, and the encoding chosen for a file opened without the
encoding argument. Then it reads the first line of the Slovak
file.
We run the program with a legacy ISO-8859-2 locale, first with Python 3.15 and then with Python 3.14. The 3.14 run decodes the UTF-8 file as ISO-8859-2 and turns each two-byte character into two wrong ones.
$ LC_ALL=sk_SK.ISO-8859-2 python default_encoding.py 3.15.2 UTF-8 mode: 1 locale encoding: ISO-8859-2 file encoding: utf-8 Bratislava leží na Dunaji. $ LC_ALL=sk_SK.ISO-8859-2 python3.14 default_encoding.py 3.14.0 UTF-8 mode: 0 locale encoding: ISO-8859-2 file encoding: ISO-8859-2 Bratislava leŞà na Dunaji.
EncodingWarning
To find places that rely on the default encoding, we run Python with the
-X warn_default_encoding option. It emits an
EncodingWarning for every open call without an
explicit encoding.
Code that should run on older versions too must keep passing
encoding='utf-8'. Code that intentionally wants the locale
encoding should pass encoding='locale'.
$ python -X warn_default_encoding default_encoding.py
3.15.2 UTF-8 mode: 1
locale encoding: UTF-8
/home/user/python/default_encoding.py:7: EncodingWarning: 'encoding' argument not specified
with open('towns.txt') as f:
file encoding: utf-8
Bratislava leží na Dunaji.
Decoding errors
If the file contains bytes that are not valid in the chosen encoding,
reading raises UnicodeDecodeError. Some encodings, such as
cp1250, can decode almost any byte and silently produce wrong
characters instead. The errors parameter selects how invalid
bytes are handled.
for enc in ['ascii', 'cp1250']:
try:
with open('towns.txt', encoding=enc) as f:
print(f.readline(), end='')
except UnicodeDecodeError as e:
print(f'{enc}: {e}')
for handler in ['replace', 'ignore', 'backslashreplace']:
with open('towns.txt', encoding='ascii', errors=handler) as f:
print(f'{handler:16} {f.readline()}', end='')
We read the UTF-8 file with two wrong encodings and with three error handlers.
for handler in ['replace', 'ignore', 'backslashreplace']:
The replace handler puts the replacement character U+FFFD in
place of each invalid byte, ignore drops the bytes, and
backslashreplace shows them as escape sequences. The default is
strict, which raises the exception.
$ python decode_errors.py ascii: 'ascii' codec can't decode byte 0xc5 in position 13: ordinal not in range(128) Bratislava ležà na Dunaji. replace Bratislava le���� na Dunaji. ignore Bratislava le na Dunaji. backslashreplace Bratislava le\xc5\xbe\xc3\xad na Dunaji.
Legacy encodings
Older files are often saved in a legacy single-byte encoding, such as Windows-1250 for Central European languages. To read them, we must know and specify the encoding.
from pathlib import Path
path = Path('towns-cp1250.txt')
path.write_text('Košice, Žilina, Prešov\n', encoding='cp1250')
print(path.read_bytes())
print(path.read_text(encoding='cp1250'), end='')
print(repr(path.read_text(encoding='latin-1')))
We write a short text in cp1250 and read it back with the right
and a wrong encoding.
print(path.read_bytes())
Each Slovak character takes one byte in cp1250.
print(repr(path.read_text(encoding='latin-1')))
Latin-1 maps every byte to a character, so it never fails, but bytes 0x9a and
0x8e become invisible control characters. We print the repr to
make them visible.
$ python legacy.py b'Ko\x9aice, \x8eilina, Pre\x9aov\n' Košice, Žilina, Prešov 'Ko\x9aice, \x8eilina, Pre\x9aov\n'
The byte order mark
Some Windows programs start UTF-8 files with a byte order mark (BOM), the
bytes EF BB BF. The utf-8 codec keeps it as the
\ufeff character at the start of the text. The
utf-8-sig codec removes it if present.
from pathlib import Path
path = Path('bom.txt')
path.write_bytes(b'\xef\xbb\xbfname,age\nPeter,34\n')
with open(path, encoding='utf-8') as f:
print(repr(f.readline()))
with open(path, encoding='utf-8-sig') as f:
print(repr(f.readline()))
We create a CSV file with a BOM and read its first line with both codecs.
With utf-8, the first column name would be
'\ufeffname' instead of 'name'.
$ python bom.py '\ufeffname,age\n' 'name,age\n'
Newlines
In text mode, Python uses universal newlines by default: \r\n
(Windows) and \r (old Mac) line endings are translated to
\n when reading. Passing newline='' turns the
translation off.
from pathlib import Path
path = Path('dos.txt')
path.write_bytes(b'first\r\nsecond\r\nthird\n')
with open(path, encoding='utf-8') as f:
print(f.readlines(), f.newlines)
with open(path, encoding='utf-8', newline='') as f:
print(f.readlines())
We create a file with mixed line endings and read it in two ways.
print(f.readlines(), f.newlines)
After reading, the newlines attribute lists the kinds of line
endings that were found.
with open(path, encoding='utf-8', newline='') as f:
With newline='', the lines keep their original endings. The
csv module requires files opened this way.
$ python newlines.py
['first\n', 'second\n', 'third\n'] ('\n', '\r\n')
['first\r\n', 'second\r\n', 'third\n']
Processing large files
Iterating over the file object processes one line at a time, so memory use does not depend on the file size. In this example we count words in the following text.
Once upon a midnight dreary, while I pondered, weak and weary,
Over many a quaint and curious volume of forgotten lore—
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
"'Tis some visitor," I muttered, "tapping at my chamber door—
Only this and nothing more."
These are the first lines of Edgar Allan Poe's poem The Raven.
import re
from collections import Counter
counter = Counter()
with open('the-raven.txt', encoding='utf-8') as f:
for line in f:
counter.update(re.findall(r"[a-z]+(?:'[a-z]+)?", line.lower()))
print(counter.total(), 'words,', len(counter), 'unique')
for word, n in counter.most_common(5):
print(f'{word:8} {n}')
We update a Counter with the words of each line and print the
five most common words.
counter.update(re.findall(r"[a-z]+(?:'[a-z]+)?", line.lower()))
The regular expression matches words, optionally with an apostrophe inside, and skips punctuation such as quotes and dashes.
$ python count_words.py 57 words, 42 unique a 3 i 3 and 3 while 2 of 2
Reading standard input
The sys.stdin object is a text stream like an open file. A
program that reads it can process redirected files and the output of other
commands.
import sys
for nr, line in enumerate(sys.stdin, 1):
print(nr, line.rstrip('\n').upper())
The program prints the lines of its input in uppercase with line numbers.
for nr, line in enumerate(sys.stdin, 1):
We iterate over standard input exactly like over a file. In Python 3.15,
sys.stdin uses UTF-8 too, so the Slovak characters are decoded
correctly.
$ python upper.py < words.txt 1 FALCON 2 SKY 3 BOOK 4 SUM 5 CUP 6 CLOUD 7 WATER 8 WIN $ head -3 towns.txt | python upper.py 1 BRATISLAVA LEŽÍ NA DUNAJI. 2 KOŠICE SÚ DRUHÉ NAJVÄČŠIE MESTO. 3 ŽILINA JE NA SEVERE.
Reading several files
The fileinput module iterates over the lines of several files
as one stream. It keeps track of the current file name and line numbers.
import fileinput
with fileinput.input(files=('words.txt', 'towns.txt'), encoding='utf-8') as f:
for line in f:
if f.isfirstline():
print(f'--- {f.filename()}')
if len(line.rstrip()) > 4:
print(f.filelineno(), line, end='')
We print the lines longer than four characters from both files.
if f.isfirstline():
print(f'--- {f.filename()}')
The isfirstline method tells us when a new file starts. The
filelineno method returns the line number within the current
file, while lineno would count across all files.
$ python multi_files.py --- words.txt 1 falcon 6 cloud 7 water --- towns.txt 1 Bratislava leží na Dunaji. 2 Košice sú druhé najväčšie mesto. 3 Žilina je na severe.
Text in memory
The io.StringIO class is an in-memory text stream. It has the
same interface as a file opened in text mode, which is handy for tests and
for passing strings to functions that expect files.
import csv
import io
data = 'name,city\nPeter,Bratislava\nLucia,Košice\n'
with io.StringIO(data) as f:
print(f.readline(), end='')
for row in csv.reader(f):
print(row)
We read the header line of CSV data with readline and pass the
rest of the stream to csv.reader.
$ python string_io.py name,city ['Peter', 'Bratislava'] ['Lucia', 'Košice']
Source
open – Python built-in functions
PEP 686 – Make UTF-8 mode default
In this article we have read text data in Python. See also Python read file, Python encode and decode, Python pathlib, and Python CSV.
Author
List all Python tutorials.