Python BeautifulSoup
last modified October 30, 2026 Python 3.15
In this article we show how to use the BeautifulSoup Python library. The examples find tags, traverse the document tree, modify the document, and scrape web pages.
BeautifulSoup
BeautifulSoup is a Python library for parsing HTML and XML documents. It is often used for web scraping. BeautifulSoup transforms a complex HTML document into a tree of Python objects, such as tags, navigable strings, and comments.
Installing BeautifulSoup
BeautifulSoup is installed from the beautifulsoup4 package; it
is imported as bs4. We also install the lxml
parser, which BeautifulSoup uses in our examples, and the
requests library for downloading web pages.
$ pip install beautifulsoup4 lxml requests
Parsers
BeautifulSoup does not parse the document itself; it uses a parser. The
parser is chosen with the second argument of the BeautifulSoup
constructor.
html.parser– built into Python, no installation neededlxml– very fast, needs thelxmlpackagehtml5lib– parses the document like a web browser, slow, needs thehtml5libpackagexml– the lxml XML parser, for XML documents
For valid documents, the parsers produce the same tree. They differ in how they fix broken HTML.
from bs4 import BeautifulSoup
broken = '<p>An <b>unclosed tag<li>item</p>'
for parser in ['html.parser', 'lxml', 'html5lib']:
try:
soup = BeautifulSoup(broken, parser)
print(f'{parser:12} {soup}')
except Exception as e:
print(f'{parser:12} {type(e).__name__}: {e}')
We parse a fragment with an unclosed b tag and an
li element outside a list. The lxml parser adds
the html and body elements; html5lib
builds the same tree as a browser would. It is a good idea to always name
the parser explicitly, so that the program behaves the same on every
computer.
$ python parsers.py html.parser <p>An <b>unclosed tag<li>item</li></b></p> lxml <html><body><p>An <b>unclosed tag<li>item</li></b></p></body></html> html5lib <html><head></head><body><p>An <b>unclosed tag</b></p><li><b>item<p></p></b></li></body></html>
The HTML file
In most examples, we will use the following HTML file:
<!DOCTYPE html>
<html>
<head>
<title>Header</title>
<meta charset="utf-8">
</head>
<body>
<h2>Operating systems</h2>
<ul id="mylist" style="width:150px">
<li>Solaris</li>
<li>FreeBSD</li>
<li>Debian</li>
<li>NetBSD</li>
<li>Windows</li>
</ul>
<p>
FreeBSD is an advanced computer operating system used to
power modern servers, desktops, and embedded platforms.
</p>
<p>
Debian is a Unix-like computer operating system that is
composed entirely of free software.
</p>
</body>
</html>
The document has a heading, a list with an id, and two paragraphs.
Python BeautifulSoup simple example
In the first example, we use the BeautifulSoup module to get three tags.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
contents = f.read()
soup = BeautifulSoup(contents, 'lxml')
print(soup.h2)
print(soup.head)
print(soup.li)
The code example prints the HTML code of three tags.
from bs4 import BeautifulSoup
We import the BeautifulSoup class from the bs4
module. The BeautifulSoup is the main class for doing work.
with open('index.html', encoding='utf-8') as f:
contents = f.read()
We open the index.html file and read its contents with the
read method.
soup = BeautifulSoup(contents, 'lxml')
A BeautifulSoup object is created; the HTML data is passed to the
constructor. The second argument specifies the parser.
print(soup.h2) print(soup.head)
Here we print the HTML code of two tags: h2 and
head. Accessing a tag name as an attribute returns the first
tag of that name.
print(soup.li)
There are multiple li elements; the line prints the first one.
$ python simple.py <h2>Operating systems</h2> <head> <title>Header</title> <meta charset="utf-8"/> </head> <li>Solaris</li>
BeautifulSoup tags, name, text
The name attribute of a tag gives its name and the
text attribute its text content.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
print(f'HTML: {soup.h2}, name: {soup.h2.name}, text: {soup.h2.text}')
print(type(soup.h2))
print(type(soup.h2.string))
The code example prints the HTML code, name, and text of the h2
tag. The constructor also accepts an open file, so we do not have to read it
first.
print(type(soup.h2)) print(type(soup.h2.string))
Tags are Tag objects. The text inside a tag is a
NavigableString, a subclass of str.
$ python tags_names.py HTML: <h2>Operating systems</h2>, name: h2, text: Operating systems <class 'bs4.element.Tag'> <class 'bs4.element.NavigableString'>
BeautifulSoup traverse tags
With the descendants attribute of the soup object, we traverse
the whole HTML document.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
for el in soup.descendants:
if el.name:
print(el.name)
The example goes through the document tree and prints the names of all HTML
tags. Strings have no name, so the if condition skips them. The
older recursiveChildGenerator method does the same, but it is
deprecated.
$ python traverse_tree.py html head title meta body h2 ul li li li li li p p
BeautifulSoup element children
With the children attribute, we can get the direct children of
a tag.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
root = soup.html
print([e.name for e in root.children])
print([e.name for e in root.children if e.name is not None])
print([e.name for e in soup.ul.find_all(recursive=False)])
The example retrieves the children of the html tag.
print([e.name for e in root.children])
The children attribute also returns the whitespace strings
between the tags; their name is None.
print([e.name for e in root.children if e.name is not None])
We add a condition to include only the tags.
print([e.name for e in soup.ul.find_all(recursive=False)])
Alternatively, find_all with recursive=False
returns only the child tags.
$ python get_children.py [None, 'head', None, 'body', None] ['head', 'body'] ['li', 'li', 'li', 'li', 'li']
BeautifulSoup element descendants
With the descendants attribute we get all descendants (children
of all levels) of a tag.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
root = soup.body
print([e.name for e in root.descendants if e.name is not None])
The example retrieves all descendants of the body tag.
$ python get_descendants.py ['h2', 'ul', 'li', 'li', 'li', 'li', 'li', 'p', 'p']
BeautifulSoup parents and siblings
We can also move up the tree and sideways. The parent and
parents attributes return the enclosing tags; the
find_next_sibling and find_previous_sibling
methods find tags at the same level.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
debian = soup.find('li', string='Debian')
print(debian.parent.name, debian.parent['id'])
print([p.name for p in debian.parents])
print(debian.find_previous_sibling('li'))
print(debian.find_next_sibling('li'))
print([li.text for li in debian.find_next_siblings('li')])
print(soup.ul.find_next_sibling('p').text.split()[0])
We start from the li tag that contains Debian.
print([p.name for p in debian.parents])
The parents generator goes up to the soup object itself, whose
name is [document].
print(soup.ul.find_next_sibling('p').text.split()[0])
The first paragraph after the list is its next sibling of the
p type. We print its first word.
$ python navigate.py ul mylist ['ul', 'body', 'html', '[document]'] <li>FreeBSD</li> <li>NetBSD</li> ['NetBSD', 'Windows'] FreeBSD
BeautifulSoup attributes
The attributes of a tag are accessed like dictionary items. For the next examples, we use a second HTML file with a table and links.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Products</title>
</head>
<body>
<h1>Products</h1>
<table id="products">
<thead>
<tr><th>Name</th><th>Price</th><th>Stock</th></tr>
</thead>
<tbody>
<tr class="product"><td><a href="/products/1">Keyboard</a></td><td>49.90</td><td>12</td></tr>
<tr class="product sale"><td><a href="/products/2">Mouse</a></td><td>19.50</td><td>0</td></tr>
<tr class="product"><td><a href="/products/3">Monitor</a></td><td>189.00</td><td>4</td></tr>
</tbody>
</table>
<p>
See also <a href="https://www.example.com/catalog" title="Full catalog">the full catalog</a>
and <a href="docs/shipping.html">shipping information</a>.
</p>
</body>
</html>
The table has an id, the rows have classes, and one link has a title.
from bs4 import BeautifulSoup
with open('products.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
table = soup.table
print(table['id'])
print(table.attrs)
row = soup.find('tr', class_='sale')
print(row['class'])
link = soup.find('a', title=True)
print(link.attrs)
print(link.get('title'))
print(link.get('target'))
print(link.get('target', '_self'))
try:
print(link['target'])
except KeyError as e:
print(f'KeyError: {e}')
We read the attributes of several tags.
row = soup.find('tr', class_='sale')
print(row['class'])
Since class is a Python keyword, the argument is called
class_. The class attribute can have multiple values,
so BeautifulSoup returns it as a list.
link = soup.find('a', title=True)
print(link.attrs)
The title=True argument finds a tag that has the attribute with
any value. The attrs dictionary contains all attributes.
print(link.get('target'))
print(link.get('target', '_self'))
The get method returns None or a default value for
a missing attribute, while the square brackets raise a
KeyError.
$ python attributes.py
products
{'id': 'products'}
['product', 'sale']
{'href': 'https://www.example.com/catalog', 'title': 'Full catalog'}
Full catalog
None
_self
KeyError: 'target'
BeautifulSoup get text
The text attribute returns all the text inside a tag, including
the whitespace from the HTML source. The get_text method and the
stripped_strings generator give us more control.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
ul = soup.ul
print(repr(ul.text))
print(ul.get_text(', ', strip=True))
print(list(ul.stripped_strings))
p = soup.p
print(' '.join(p.text.split()))
We get the text of the list in several ways.
print(ul.get_text(', ', strip=True))
The get_text method joins the strings with the given separator;
strip=True removes the surrounding whitespace from each string
and drops the empty ones.
print(' '.join(p.text.split()))
To normalize the whitespace inside a paragraph, we split the text into words and join them with single spaces.
$ python get_text.py '\nSolaris\nFreeBSD\nDebian\nNetBSD\nWindows\n' Solaris, FreeBSD, Debian, NetBSD, Windows ['Solaris', 'FreeBSD', 'Debian', 'NetBSD', 'Windows'] FreeBSD is an advanced computer operating system used to power modern servers, desktops, and embedded platforms.
BeautifulSoup find elements by id
With the find method we can find elements by various means
including element id. It returns the first match or None.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
print(soup.find('ul', id='mylist'))
print(soup.find('ul', attrs={'id': 'mylist'}) is soup.find(id='mylist'))
print(soup.find('ol'))
The code example finds the ul tag that has the
mylist id.
print(soup.find('ul', attrs={'id': 'mylist'}) is soup.find(id='mylist'))
Attributes can be given as keyword arguments or in the attrs
dictionary; the latter works for attributes whose names are not valid Python
identifiers, such as data-id. The tag name is optional.
print(soup.find('ol'))
There is no ol tag in the document, so find returns
None.
$ python find_by_id.py <ul id="mylist" style="width:150px"> <li>Solaris</li> <li>FreeBSD</li> <li>Debian</li> <li>NetBSD</li> <li>Windows</li> </ul> True None
BeautifulSoup find all tags
With the find_all method we can find all elements that meet
some criteria.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
for tag in soup.find_all('li'):
print(f'{tag.name}: {tag.text}')
print(soup.find_all('li', limit=2))
The code example finds and prints all li tags. The
limit argument stops the search after the given number of
matches.
$ python find_all.py li: Solaris li: FreeBSD li: Debian li: NetBSD li: Windows [<li>Solaris</li>, <li>FreeBSD</li>]
Finding multiple tag types
The find_all method can take a list of tag names to search for.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
tags = soup.find_all(['h2', 'p'])
for tag in tags:
print(f"{tag.name}: {' '.join(tag.text.split())}")
The example finds all h2 and p elements and prints
their text. The elements are returned in document order.
$ python find_all2.py h2: Operating systems p: FreeBSD is an advanced computer operating system used to power modern servers, desktops, and embedded platforms. p: Debian is a Unix-like computer operating system that is composed entirely of free software.
Finding tags with a function
The find_all method can also take a function which determines
which elements should be returned. The function receives a tag and returns a
boolean value.
from bs4 import BeautifulSoup
def is_bsd_item(tag):
return tag.name == 'li' and tag.text.endswith('BSD')
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
print(soup.find_all(is_bsd_item))
print(soup.find_all(lambda tag: tag.is_empty_element))
The is_bsd_item function selects list items that end with
BSD.
print(soup.find_all(lambda tag: tag.is_empty_element))
A lambda selects empty elements. The only empty element in the document is
meta.
$ python find_by_fun.py [<li>FreeBSD</li>, <li>NetBSD</li>] [<meta charset="utf-8"/>]
Finding with regular expressions
It is also possible to find elements and strings by using regular expressions.
import re
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
strings = soup.find_all(string=re.compile('BSD'))
for txt in strings:
print(' '.join(txt.split()))
print([tag.name for tag in soup.find_all(re.compile('^h[1-6]$|^ul$'))])
With the string argument, find_all searches the
text of the document and returns the matching strings. We print the strings
that contain BSD.
print([tag.name for tag in soup.find_all(re.compile('^h[1-6]$|^ul$'))])
A regular expression in place of the tag name matches tag names: here, headings and unordered lists.
$ python regex.py FreeBSD NetBSD FreeBSD is an advanced computer operating system used to power modern servers, desktops, and embedded platforms. ['h2', 'ul']
BeautifulSoup CSS selectors
With the select and select_one methods, we can use
CSS selectors to find elements. The select method returns a list
of all matches, select_one the first match or
None.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
print(soup.select('li:nth-of-type(3)'))
print(soup.select_one('li:nth-of-type(3)'))
print(soup.select_one('#mylist').name)
print([li.text for li in soup.select('ul#mylist > li:nth-child(odd)')])
print([li.text for li in soup.select('li:-soup-contains("BSD")')])
print(len(soup.select('body p')))
This example uses several CSS selectors.
print(soup.select('li:nth-of-type(3)'))
print(soup.select_one('li:nth-of-type(3)'))
Both lines find the third li element; the first one returns it in
a list.
print(soup.select_one('#mylist').name)
The # character is used in CSS to select tags by their id
attribute.
print([li.text for li in soup.select('ul#mylist > li:nth-child(odd)')])
print([li.text for li in soup.select('li:-soup-contains("BSD")')])
The > combinator selects direct children, and
:nth-child(odd) every other item. The
:-soup-contains pseudo-class is a non-standard extension that
matches elements containing the given text.
$ python css_selectors.py [<li>Debian</li>] <li>Debian</li> ul ['Solaris', 'Debian', 'Windows'] ['FreeBSD', 'NetBSD'] 2
Selecting by class and attribute
CSS selectors work well with classes and attributes. We use the
products.html file.
from bs4 import BeautifulSoup
with open('products.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
print([a.text for a in soup.select('tr.product a')])
print([a.text for a in soup.select('tr.sale a')])
print([a['href'] for a in soup.select('a[href^="http"]')])
print(soup.select_one('tbody tr:last-child td:nth-of-type(2)').text)
The tr.product a selector finds links in rows with the
product class, and tr.sale a those in rows with the
sale class.
print([a['href'] for a in soup.select('a[href^="http"]')])
The ^= attribute selector matches values with the given prefix,
so we get the absolute links.
print(soup.select_one('tbody tr:last-child td:nth-of-type(2)').text)
We get the price, the second cell of the last row.
$ python select_products.py ['Keyboard', 'Mouse', 'Monitor'] ['Mouse'] ['https://www.example.com/catalog'] 189.00
Extracting table data
A common scraping task is to turn an HTML table into Python data.
from bs4 import BeautifulSoup
with open('products.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
table = soup.find('table', id='products')
headers = [th.text.lower() for th in table.thead.find_all('th')]
products = []
for row in table.tbody.find_all('tr'):
values = [td.get_text(strip=True) for td in row.find_all('td')]
product = dict(zip(headers, values))
product['price'] = float(product['price'])
product['stock'] = int(product['stock'])
products.append(product)
for product in products:
print(product)
in_stock = sum(p['price'] * p['stock'] for p in products)
print(f'stock value: {in_stock:.2f}')
We read the table into a list of dictionaries.
headers = [th.text.lower() for th in table.thead.find_all('th')]
The column names come from the th cells of the table head.
values = [td.get_text(strip=True) for td in row.find_all('td')]
product = dict(zip(headers, values))
For each row of the table body, we pair the cell texts with the column names. The text of a cell includes the text of its link.
product['price'] = float(product['price']) product['stock'] = int(product['stock'])
All values from HTML are strings; we convert the numbers so that we can compute the stock value.
$ python table_data.py
{'name': 'Keyboard', 'price': 49.9, 'stock': 12}
{'name': 'Mouse', 'price': 19.5, 'stock': 0}
{'name': 'Monitor', 'price': 189.0, 'stock': 4}
stock value: 1354.80
Extracting links
Links in a page are often relative. The urljoin function from
urllib.parse resolves them against the URL of the page.
from urllib.parse import urljoin
from bs4 import BeautifulSoup
base = 'https://shop.example.com/store/'
with open('products.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
for a in soup.find_all('a', href=True):
print(f'{a.text:24} {urljoin(base, a["href"])}')
The href=True argument finds only a tags that have
an href attribute.
print(f'{a.text:24} {urljoin(base, a["href"])}')
Links starting with / are resolved against the host, other
relative links against the directory of the page; absolute links are kept.
$ python links.py Keyboard https://shop.example.com/products/1 Mouse https://shop.example.com/products/2 Monitor https://shop.example.com/products/3 the full catalog https://www.example.com/catalog shipping information https://shop.example.com/store/docs/shipping.html
BeautifulSoup append element
The append method appends a new tag to the HTML document.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
newtag = soup.new_tag('li')
newtag.string = 'OpenBSD'
ultag = soup.ul
ultag.append(newtag)
print(ultag.prettify())
The example appends a new li tag.
newtag = soup.new_tag('li')
newtag.string = 'OpenBSD'
First, we create a new tag with the new_tag method and set its
text.
ultag = soup.ul ultag.append(newtag)
We get the reference to the ul tag and append the newly created
tag to it.
print(ultag.prettify())
We print the ul tag in a neat format.
$ python append_tag.py <ul id="mylist" style="width:150px"> <li> Solaris </li> <li> FreeBSD </li> <li> Debian </li> <li> NetBSD </li> <li> Windows </li> <li> OpenBSD </li> </ul>
BeautifulSoup insert element
The insert method inserts a tag at the specified position.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
newtag = soup.new_tag('li', attrs={'class': 'new'})
newtag.string = 'OpenBSD'
ultag = soup.ul
ultag.insert(2, newtag)
print([li.text for li in ultag.find_all('li')])
print(newtag.find_previous_sibling('li'))
The example creates a li tag with a class attribute and inserts
it into the ul tag.
ultag.insert(2, newtag)
The position counts all children, including the whitespace strings between
the tags. Position 2 is after the first li element and the
newline that follows it, so the new item becomes the second one.
$ python insert_tag.py ['Solaris', 'OpenBSD', 'FreeBSD', 'Debian', 'NetBSD', 'Windows'] <li>Solaris</li>
BeautifulSoup replace text
The replace_with method replaces an element or a string with
another one.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
tag = soup.find(string='Windows')
tag.replace_with('OpenBSD')
h2 = soup.h2
h2.string = 'Free operating systems'
print(h2)
print(soup.ul.prettify())
The example finds a specific string with the find method and
replaces it with the replace_with method.
h2.string = 'Free operating systems'
Assigning to the string attribute replaces the whole content of
a tag.
$ python replace_text.py <h2>Free operating systems</h2> <ul id="mylist" style="width:150px"> <li> Solaris </li> <li> FreeBSD </li> <li> Debian </li> <li> NetBSD </li> <li> OpenBSD </li> </ul>
BeautifulSoup remove element
The decompose method removes a tag from the tree and destroys it.
from bs4 import BeautifulSoup
with open('index.html', encoding='utf-8') as f:
soup = BeautifulSoup(f, 'lxml')
ptag2 = soup.select_one('p:nth-of-type(2)')
ptag2.decompose()
for li in soup.find_all('li', string=['Windows', 'Solaris']):
li.decompose()
del soup.ul['style']
print(soup.body.prettify())
The example removes the second p element.
for li in soup.find_all('li', string=['Windows', 'Solaris']):
li.decompose()
We also remove two list items. The string argument accepts a
list of strings.
del soup.ul['style']
Attributes are removed with the del statement.
$ python decompose_tag.py
<body>
<h2>
Operating systems
</h2>
<ul id="mylist">
<li>
FreeBSD
</li>
<li>
Debian
</li>
<li>
NetBSD
</li>
</ul>
<p>
FreeBSD is an advanced computer operating system used to
power modern servers, desktops, and embedded platforms.
</p>
</body>
BeautifulSoup prettify code
With the prettify method, we can make the HTML code look better.
from bs4 import BeautifulSoup html = '<html><head><title>Hello</title></head><body><p>Today is a <b>beautiful</b> day.<p>Hello there.</body></html>' soup = BeautifulSoup(html, 'lxml') print(soup.prettify())
We prettify the HTML code of a small document written on a single line. Each tag and string is placed on its own line, and the second paragraph is closed by the parser.
$ python prettify.py
<html>
<head>
<title>
Hello
</title>
</head>
<body>
<p>
Today is a
<b>
beautiful
</b>
day.
</p>
<p>
Hello there.
</p>
</body>
</html>
BeautifulSoup web scraping
To scrape a web page, we first download it. Requests is a simple Python HTTP library. It provides methods for accessing web resources via HTTP.
import requests
from bs4 import BeautifulSoup
resp = requests.get('https://example.com', timeout=10)
resp.raise_for_status()
soup = BeautifulSoup(resp.text, 'lxml')
print(soup.title)
print(soup.title.text)
print(soup.title.parent.name)
print(soup.p.text)
The example retrieves the title of a simple web page and the name of the title's parent tag. It also prints the text of the first paragraph.
resp = requests.get('https://example.com', timeout=10)
resp.raise_for_status()
soup = BeautifulSoup(resp.text, 'lxml')
We get the HTML data of the page. The raise_for_status method
raises an exception if the server returns an error status, so we do not try
to parse an error page.
$ python scraping.py <title>Example Domain</title> Example Domain head This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.
BeautifulSoup scraping with built-in web server
We can also serve HTML pages with the built-in HTTP server of Python, which is useful for testing scrapers. See Python http.server.
import requests
from bs4 import BeautifulSoup
resp = requests.get('http://localhost:8000/', timeout=10)
soup = BeautifulSoup(resp.text, 'lxml')
print(soup.title.text)
print([li.text for li in soup.select('#mylist li')])
The program gets the document from the locally running server and prints its title and list items.
We create a public directory, copy the index.html
file there, and start the server in the background. The server logs the
request of the scraper. Finally, we stop the server with Ctrl+C.
$ mkdir public $ cp index.html public/ $ python -m http.server --bind 127.0.0.1 --directory public 8000 & Serving HTTP on 127.0.0.1 port 8000 (http://127.0.0.1:8000/) ... $ python scraping_local.py Header ['Solaris', 'FreeBSD', 'Debian', 'NetBSD', 'Windows'] 127.0.0.1 - - [30/Oct/2026 18:48:31] "GET / HTTP/1.1" 200 - Keyboard interrupt received, exiting.
Source
In this article we have worked with the Python BeautifulSoup library.
Author
List all Python tutorials.