Welcome

LIVE Classes

Courses

Practice Platforms

Leaderboard

Rewards

Referral

Profile

Finish

Welcome LIVE Classes Courses Practice Platforms Leaderboard Rewards Referral Profile Finish

Welcome to HCL GUVI

Hey there! Welcome to HCL GUVI—Grab Your Vernacular Imprint—where tech learning is easy, fun, and curated specially for you. Incubated by IIT Madras & IIM Ahmedabad in 2014 and now part of HCL Group, we're making quality tech education accessible to all.

Join 3M+ learners breaking barriers and upskilling for a brighter future. We're here to guide you every step of the way! 🚀

LIVE Classes

Zen Classes are HCL GUVI's most refined and flagship product—live, expert-led tech programs for beginners and pros. With IITM Pravartak affiliations, master Full-Stack, Data Science, DevOps, UI/UX, and more in multiple languages!

Explore More

Courses

Looking for flexibility? HCL GUVI's 200+ self-paced courses let you learn anytime, anywhere! From free lessons to IIT-M & Autodesk-certified programs, gain in-demand skills in your preferred language.

Explore More

Practice Platforms

Enhance your coding skills with HCL GUVI's Practice Platforms—interactive, structured, and designed to help you master programming effortlessly.

CodeKata:

A structured coding practice platform with 1500+ coding problems designed by industry experts. Ideal for beginners and professionals preparing for tech interviews with real-world coding challenges.

Try Now >

WebKata:

An interactive platform to master HTML, CSS, JavaScript, and Bootstrap with a live coding environment. Perfect for hands-on web development practice without any setup.

Try Now >

SQLKata:

A practice ground for mastering SQL queries used in real-world applications. Write, optimize, and refine your queries to build strong database skills.

Try Now >

Debugging:

Hone your bug-fixing skills with real-world debugging challenges in Python, C++, JavaScript, and Golang. More languages coming soon!

Try Now >

IDE:

A free online compiler supporting 20+ programming languages with auto-complete, debugging, and AI-powered code generation—all in the cloud!

Try Now >

Leaderboard

Climb the leaderboard as you earn Geekoins by learning and practicing! The top scorers get featured, making learning competitive and rewarding. Keep going—you could be next!

Explore More

Rewards

Earn Geekoins by watching videos and practicing problems, then redeem them for exciting rewards. The more you engage, the more you win!

Explore More

Referral

Love learning with HCL GUVI? Share it with friends! Invite them using your unique link or code and unlock exciting rewards—Amazon vouchers, iPhones, and more. A Win-Win.

Explore More

Profile

Your HCL GUVI profile is your digital portfolio! Track progress, showcase skills, add projects, and build a resume. Keep it updated—opportunities await!

Explore More

That's It! You Are Ready!

You're all set to dive into your learning journey with HCL GUVI. Explore, upskill, and make each step count—exciting possibilities awaits!

Home
Python 3
What is BeautifulSoup Module?

Introduction to BeautifulSoup Module

Introduction to BeautifulSoup Module
BeautifulSoup: Prettify Content
import modules
importing the beautifulsoup module
send a request and receive the information from https://www.google.com
creating BeautifulSoup object
using 'prettify' method to print the content
BeautifulSoup: Accessing HTML Tags
import modules
importing the beautifulsoup module
send a request and receive the information from https://www.google.com
creating BeautifulSoup object
getting 'title' tag from the google BeautifulSoup -> 'soup'
import modules
importing the beautifulsoup module
send a request and receive the information from https://www.google.com
creating BeautifulSoup object
getting 'title' tag from the google BeautifulSoup -> 'soup'
import modules
importing the beautifulsoup module
send a request and receive the information from https://www.google.com
creating BeautifulSoup object
getting 'head' tag from the google BeautifulSoup -> 'soup'
getting 'title' tag from the google BeautifulSoup -> 'soup'
getting 'title' tag from the google BeautifulSoup -> 'soup'

Introduction to BeautifulSoup Module

In this tutorial we will learn how we can use the BeautifulSoup module of python to parse the source code of webpage(which we can get using the requests module) and find various useful information from the source code like all the HTML table headings, or all the links on the webpage etc.

BeautifulSoup can search and return all occurences of an HTML tag, if we provide all the information to it about the HTML tag.

Before we jump into searching HTML tags and accessing information from a webpage, let's see how we can format the HTTP response content received to make it more readable.

BeautifulSoup: Prettify Content

The method prettify available in BeautifulSOup module can be used to format the HTTP response received using the requests module.

Below we have the code example, extending teh example from last tutorial:

import modules

import requests from fake_useragent import UserAgent

importing the beautifulsoup module

import bs4

send a request and receive the information from https://www.google.com

response = requests.get("https://www.google.com")

creating BeautifulSoup object

soup = bs4.BeautifulSoup(response.content, "html.parser")

using 'prettify' method to print the content

print(soup.prettify())


In the code above we did the following:

*   Imported the modules: **requests**, **fake\_useragent** and **bs4**.
*   Get teh response from any URL you like.
*   Create a **BeautifulSoup** object using the `BeautifulSoup` class.
*   Print the response using the `prettify` method using the BeautifulSoup object.

If you are coming here after reading the previous tutorial, you must have seen how the response from the GET request made using the `requests` module looked like.

When we format that response using the `prettify` method, it looks like **this**(click on _this_ to download the file).

Now that the response is formatted, let's learn how can we use BeautifulSoup to access various HTML tags and related information from the HTTP response(source code).

BeautifulSoup: Accessing HTML Tags

Using the BeautifulSoup module we can easily find and access the content of various HTML tags like head, title, div, p, h1 etc. Let's see a simple example where we will print the title tag of the webpage.

import modules

import requests from fake_useragent import UserAgent

importing the beautifulsoup module

import bs4

send a request and receive the information from https://www.google.com

response = requests.get("https://www.google.com")

creating BeautifulSoup object

soup = bs4.BeautifulSoup(response.content, "html.parser")

getting 'title' tag from the google BeautifulSoup -> 'soup'

title_tag = soup.title print(title_tag)


\<title>Google\</title>

We can also get only the text enclosed within the opening and closing **title** tag:

```python

import modules

import requests from fake_useragent import UserAgent

importing the beautifulsoup module

import bs4

send a request and receive the information from https://www.google.com

response = requests.get("https://www.google.com")

creating BeautifulSoup object

soup = bs4.BeautifulSoup(response.content, "html.parser")

getting 'title' tag from the google BeautifulSoup -> 'soup'

title_text = soup.title.text print(title_text)


**Output:**

Google

This is standard for all the HTML tags, for example to get the **head** tag, we can use `soup.head` like this,

```python

import modules

import requests from fake_useragent import UserAgent

importing the beautifulsoup module

import bs4

send a request and receive the information from https://www.google.com

response = requests.get("https://www.google.com")

creating BeautifulSoup object

soup = bs4.BeautifulSoup(response.content, "html.parser")

getting 'head' tag from the google BeautifulSoup -> 'soup'

print(soup.head)


This will return the complete **head** tag from the page's source code.

\<head> \<meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>\< meta content="/images/branding/googleg/1x/googleg\_standard\_color\_128dp.png" itemprop="image"/> \<title>Google\</title >\<script nonce="GWwjLi7M0YGkyNTLDmVPsQ=="> ... \<style> ... \</style> ... \</head>

We have not added the complete code in the output as it is huge. But as you can see that the **title** tag is inside the **head** tag and there is **style** tag too in there.

We can also get the **title** tag content via the **head** tag:

```python

getting 'title' tag from the google BeautifulSoup -> 'soup'

print(soup.head.title.text)


**Output:**

Google

This is just to show you that as the BeautifulSoup follows the **tree traversal** technique to parse the HTML code, we can also access the tags by following their heirarchy.

Similarly let's access the **style** tag:

```python

getting 'title' tag from the google BeautifulSoup -> 'soup'

print(soup.head.style.text)


Up until now we have covered basic HTML parsing and accessing the tags. In the next tutorial we will see some more methods of the BeautifulSoup module and some more ways of navigating through the HTML source code of any webpage to collect useful data.

Recommended Handbooks

4.7

C++ Handbook

Level up your programming skills with our C++ Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

6254

3 Hrs

4.7

Python Basics Handbook

Level up your programming skills with our Python Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

7323

3.5 Hrs

4.6

Javascript Handbook

Level up your programming skills with our JavaScript Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

6188

2 Hrs

ReactJS Projects Handbook

Learn ReactJS by building projects that mirror real-world applications. Strengthen your skills with step-by-step guidance and hands-on coding experience.

English

2.5 Hrs

Computer Networks Tutorial

A complete guide to computer networking, from fundamentals to protocols, routing, addressing, and real-world data communication.

English

1.5 Hrs

Operating System Tutorial

Your complete guide to Operating Systems, from fundamentals to advanced topics like memory management, scheduling, threads, and deadlock handling.

English

1 Hr

DBMS and SQL Tutorial

A complete handbook to guide you through DBMS fundamentals and SQL mastery, perfect for building data-driven applications, managing data systems, or preparing for database roles.

English

0.5 Hr

Java Tutorial

Beginner-friendly Java handbook covering core concepts, OOP principles, and hands-on programming examples.

English

2 Hrs

C Language Tutorial

A step-by-step C programming handbook for beginners. Understand C syntax, logic, memory, and hands-on coding to build solid programming foundations.

English

0.5 Hr

PHP Tutorial

Step-by-step PHP handbook for web developers. Master server-side scripting with practical code and concepts.

English

0.5 Hr

Android Tutorial

Beginner-friendly Android handbook covering app fundamentals, UI design, and hands-on development concepts.

English

1 Hr

Linux Guide Tutorial

A practical Linux handbook covering command-line basics, file management, and system operations.

English

2.5 Hrs

Data Structures and Algorithms Tutorial

Learn core data structures and algorithms with practical examples to improve coding efficiency and problem-solving skills.

English

0.5 Hr

Computer Architecture

A beginner-friendly guide to computer architecture covering processors, memory, and system-level concepts.

English

0.5 Hr

HTML 5 References Tutorial

A handy HTML5 reference guide covering modern tags, attributes, and semantic elements.

English

1.5 Hrs

Docker Tutorial

A hands-on Docker handbook covering containers, images, and modern application deployment basics.

English

0 Hr

GIT (Using Github) Tutorial

A hands-on Git and GitHub handbook for managing code, tracking changes, and collaborating on projects.

English

0.5 Hr

Go Language Tutorial

A beginner-friendly Go handbook covering core language concepts and modern backend programming.

English

0.5 Hr

GIT Guide

A practical Git guide covering version control basics, branching, and real project workflows.

English

1 Hr

CSS Tutorial

A beginner-friendly CSS handbook covering page styling, layouts, and responsive design basics.

English

1 Hr

Advanced Data Structures

A focused handbook covering advanced data structures for efficient and scalable problem solving.

English

0 Hr

Spring Framework Tutorial

A hands-on Spring Framework handbook covering core concepts and backend development fundamentals.

English

1 Hr

Spring Boot Tutorial

A practical Spring Boot handbook focused on building and running modern Java backend applications.

English

0.5 Hr

Kotlin Tutorial

A beginner-friendly Kotlin handbook covering modern language features and real-world development concepts.

English

1 Hr

Apache Cordova Tutorial

A hands-on Apache Cordova handbook for building cross-platform mobile apps with web technologies.

English

0 Hr

Python Tutorial

A beginner-friendly Python handbook covering core concepts and practical programming examples.

English

1.5 Hrs

SASS-SCSS Tutorial

A hands-on SASS / SCSS handbook for writing clean, reusable, and scalable stylesheets.

English

0.5 Hr

MongoDB Tutorial

A hands-on MongoDB handbook covering NoSQL concepts and modern database operations.

English

0.5 Hr

Numpy Tutorial

A hands-on NumPy handbook for fast numerical computation and data manipulation using Python.

English

1.5 Hrs

PL-SQL Tutorial

A hands-on PL/SQL handbook for writing procedural database programs and business logic.

English

0.5 Hr

Python Built-in Functions Tutorial

A handy reference guide to Python’s built-in functions for cleaner and faster coding.

English

0.5 Hr

Pandas Tutorial

A hands-on Pandas handbook for data manipulation, cleaning, and analysis using Python.

English

2.5 Hrs

Elasticsearch Tutorial

A hands-on Elasticsearch handbook covering indexing, searching, and data analysis concepts.

English

0 Hr

Matplotlib Tutorial

A hands-on Matplotlib handbook for creating charts and visualizing data using Python.

English

0.5 Hr

Networking with Python

A hands-on handbook for building network-enabled applications using Python.

English

0.5 Hr

Tkinter Tutorial

A hands-on Tkinter handbook for building desktop applications with Python.

English

0.5 Hr

Java Programs Tutorial

A hands-on Java programs handbook for practicing core concepts and problem-solving in Java.

English

2 Hrs

Introduction to BeautifulSoup Module

Contents

Introduction to BeautifulSoup Module

BeautifulSoup: Prettify Content

import modules

importing the beautifulsoup module

send a request and receive the information from https://www.google.com

creating BeautifulSoup object

using 'prettify' method to print the content

BeautifulSoup: Accessing HTML Tags

import modules

importing the beautifulsoup module

send a request and receive the information from https://www.google.com

creating BeautifulSoup object

getting 'title' tag from the google BeautifulSoup -> 'soup'

import modules

importing the beautifulsoup module

send a request and receive the information from https://www.google.com

creating BeautifulSoup object

getting 'title' tag from the google BeautifulSoup -> 'soup'

import modules

importing the beautifulsoup module

send a request and receive the information from https://www.google.com

creating BeautifulSoup object

getting 'head' tag from the google BeautifulSoup -> 'soup'

getting 'title' tag from the google BeautifulSoup -> 'soup'

getting 'title' tag from the google BeautifulSoup -> 'soup'

Web Scraping Tutorial

Recommended Handbooks

C++ Handbook

Python Basics Handbook

Javascript Handbook

ReactJS Projects Handbook

Computer Networks Tutorial

Operating System Tutorial

DBMS and SQL Tutorial

Java Tutorial

C Language Tutorial

PHP Tutorial

Android Tutorial

Linux Guide Tutorial

Data Structures and Algorithms Tutorial

Computer Architecture

HTML 5 References Tutorial

Docker Tutorial

GIT (Using Github) Tutorial

Go Language Tutorial

GIT Guide

CSS Tutorial

Advanced Data Structures

Spring Framework Tutorial

Spring Boot Tutorial

Kotlin Tutorial

Apache Cordova Tutorial

Python Tutorial

SASS-SCSS Tutorial

MongoDB Tutorial

Numpy Tutorial

PL-SQL Tutorial

Python Built-in Functions Tutorial

Pandas Tutorial

Elasticsearch Tutorial

Matplotlib Tutorial

Networking with Python

Tkinter Tutorial

Java Programs Tutorial