Welcome

LIVE Classes

Courses

Practice Platforms

Leaderboard

Rewards

Referral

Profile

Finish

Welcome LIVE Classes Courses Practice Platforms Leaderboard Rewards Referral Profile Finish

Welcome to HCL GUVI

Hey there! Welcome to HCL GUVI—Grab Your Vernacular Imprint—where tech learning is easy, fun, and curated specially for you. Incubated by IIT Madras & IIM Ahmedabad in 2014 and now part of HCL Group, we're making quality tech education accessible to all.

Join 3M+ learners breaking barriers and upskilling for a brighter future. We're here to guide you every step of the way! 🚀

LIVE Classes

Zen Classes are HCL GUVI's most refined and flagship product—live, expert-led tech programs for beginners and pros. With IITM Pravartak affiliations, master Full-Stack, Data Science, DevOps, UI/UX, and more in multiple languages!

Explore More

Courses

Looking for flexibility? HCL GUVI's 200+ self-paced courses let you learn anytime, anywhere! From free lessons to IIT-M & Autodesk-certified programs, gain in-demand skills in your preferred language.

Explore More

Practice Platforms

Enhance your coding skills with HCL GUVI's Practice Platforms—interactive, structured, and designed to help you master programming effortlessly.

CodeKata:

A structured coding practice platform with 1500+ coding problems designed by industry experts. Ideal for beginners and professionals preparing for tech interviews with real-world coding challenges.

Try Now >

WebKata:

An interactive platform to master HTML, CSS, JavaScript, and Bootstrap with a live coding environment. Perfect for hands-on web development practice without any setup.

Try Now >

SQLKata:

A practice ground for mastering SQL queries used in real-world applications. Write, optimize, and refine your queries to build strong database skills.

Try Now >

Debugging:

Hone your bug-fixing skills with real-world debugging challenges in Python, C++, JavaScript, and Golang. More languages coming soon!

Try Now >

IDE:

A free online compiler supporting 20+ programming languages with auto-complete, debugging, and AI-powered code generation—all in the cloud!

Try Now >

Leaderboard

Climb the leaderboard as you earn Geekoins by learning and practicing! The top scorers get featured, making learning competitive and rewarding. Keep going—you could be next!

Explore More

Rewards

Earn Geekoins by watching videos and practicing problems, then redeem them for exciting rewards. The more you engage, the more you win!

Explore More

Referral

Love learning with HCL GUVI? Share it with friends! Invite them using your unique link or code and unlock exciting rewards—Amazon vouchers, iPhones, and more. A Win-Win.

Explore More

Profile

Your HCL GUVI profile is your digital portfolio! Track progress, showcase skills, add projects, and build a resume. Keep it updated—opportunities await!

Explore More

That's It! You Are Ready!

You're all set to dive into your learning journey with HCL GUVI. Explore, upskill, and make each step count—exciting possibilities awaits!

Home
Python 3
Exploring BeautifulSoup Methods

Understanding Exploring BeautifulSoup Methods

Exploring BeautifulSoup Methods
BeautifulSoup: Accessing HTML Tags
reading content from the file
reading content from the file
creating a BeautifulSoup object
reading content from the file
creating a BeautifulSoup object
getting anchor tag
printing the 'href' attribute of anchor tag
getting all the children of 'body' using 'contents'
printing all the children using for loop
we can also convert iterator into list using the 'list(iterator)'
getting child tags of 'body' tag using 'descendants' method

Exploring BeautifulSoup Methods

In this tutorial we will learn various different ways to access HTML tags using different methods of the BeautifulSoup module. For a basic introduction to the BeautifulSoup module, start from the previous tutorial.

BeautifulSoup: Accessing HTML Tags

The methods that we will cover in this section are used to traverse through different HTML tags considering HTML code as a tree.

Create a file sample_webpage.html and copy the following HTML code in it:

<!DOCTYPE html>
<html>
    
    <head>
        <title> Sample HTML Page</title>
        <style>
            * {
                margin: 0;
                padding: 0;
            }

            div {
                width: 95%;
                height: 75px;
                margin: 10px 2.5%;
                border: 1px dotted grey;
                text-align: center;
            }
              
            p {
                font-family: sans-serif;
                font-size: 18px;
                color: #000;
                line-height: 75px;
            }

            a {
                position: relative;
                top: 25px;
            }
        </style>
    </head>
    
    <body>
        <div id="first-div">
            <p class="first">First Paragraph</p>
        </div>

        <div id="second-div">
            <p class="second">Second Paragraph</p>
        </div>

        <div id="third-div">
            <a href="https://www.studytonight.com">Studytonight</a>
            <p class="third">Third Paragraph</p>        
        </div>

        <div id="fourth-div">
            <p class="fourth">Fourth Paragraph</p>        
        </div>

        <div id="fifth-div">
            <p class="fifth">Fifth Paragraph</p>        
        </div>
    </body>
</html>

Now to read the content of the above HTML file, use the following python code to store the content into a variable:

reading content from the file

with open("sample_webpage.html") as html_file: html = html_file.read()


Now we will use different methods of the BeautifulSoup module and see how they work.

For warmup, let's start with using the `prettify` method.

```python
import bs4

reading content from the file

with open("sample_webpage.html") as html_file: html = html_file.read()

creating a BeautifulSoup object

soup = bs4.BeautifulSoup(html, "html.parser")

print(soup.prettify)


### **BeautifulSoup: Accessing HTML Tag Attributes**

We can retrieve the attributes of any HTML tag using the following syntax:

```html
TagName["AttributeName"]

Let's extract the href attribute from the anchor tag in our HTML code.

import bs4

reading content from the file

with open("sample_webpage.html") as html_file: html = html_file.read()

creating a BeautifulSoup object

soup = bs4.BeautifulSoup(html, "html.parser")

getting anchor tag

link = soup.a

printing the 'href' attribute of anchor tag

print(link["href"])


### **BeautifulSoup:** `contents` **method**

`contents` method is used to list out all the tags that are present in the parent tag. Let's list all the children HTML tags of the **body** tag using the `contents` method.

```python
body = soup.body

getting all the children of 'body' using 'contents'

content_list = body.contents

printing all the children using for loop

for tag in content_list: if tag != "\n": print(tag) print("\n")


### **BeautifulSoup:** `children` **method**

`children` method is similar to the `contents` method, but `children` method returns an **iterator** while the `contents` method returns a **list** of all the children. Let's see an example:

```python
body = soup.body

we can also convert iterator into list using the 'list(iterator)'

for tag in body.children: if tag != "\n": print(tag) print("\n")


### **BeautifulSoup:** `descendants` **method**

`descendants` method helps to retrieve all the child tags of a parent tag. You must be wondering that is what the two methods above also did. Well this method is different from `contents` and `children` method as this method extracts all the child tags and content up until the end. In simple words if we use it to extract the **body** tag then it will print the first **div** tag, then it will print the child of the **div** tag and then their child until it reaches the end, then it will move on to the next **div** tag and so on.

This method returns a **generator**. Let's see an example:

```python
body = soup.body

getting child tags of 'body' tag using 'descendants' method

for tag in body.descendants: if tag != "\n": print(tag) print("\n")


Now you are familiar with most of the methods that are used in web scraping. In the following tutorial, we will learn how to find a specific tag from a bunch of similar tags.

Recommended Handbooks

4.7

C++ Handbook

Level up your programming skills with our C++ Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

6254

3 Hrs

4.7

Python Basics Handbook

Level up your programming skills with our Python Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

7323

3.5 Hrs

4.6

Javascript Handbook

Level up your programming skills with our JavaScript Tutorials hub, guiding you from beginner to advanced. Start your journey now!

English

6188

2 Hrs

ReactJS Projects Handbook

Learn ReactJS by building projects that mirror real-world applications. Strengthen your skills with step-by-step guidance and hands-on coding experience.

English

2.5 Hrs

Computer Networks Tutorial

A complete guide to computer networking, from fundamentals to protocols, routing, addressing, and real-world data communication.

English

1.5 Hrs

Operating System Tutorial

Your complete guide to Operating Systems, from fundamentals to advanced topics like memory management, scheduling, threads, and deadlock handling.

English

1 Hr

DBMS and SQL Tutorial

A complete handbook to guide you through DBMS fundamentals and SQL mastery, perfect for building data-driven applications, managing data systems, or preparing for database roles.

English

0.5 Hr

Java Tutorial

Beginner-friendly Java handbook covering core concepts, OOP principles, and hands-on programming examples.

English

2 Hrs

C Language Tutorial

A step-by-step C programming handbook for beginners. Understand C syntax, logic, memory, and hands-on coding to build solid programming foundations.

English

0.5 Hr

PHP Tutorial

Step-by-step PHP handbook for web developers. Master server-side scripting with practical code and concepts.

English

0.5 Hr

Android Tutorial

Beginner-friendly Android handbook covering app fundamentals, UI design, and hands-on development concepts.

English

1 Hr

Linux Guide Tutorial

A practical Linux handbook covering command-line basics, file management, and system operations.

English

2.5 Hrs

Data Structures and Algorithms Tutorial

Learn core data structures and algorithms with practical examples to improve coding efficiency and problem-solving skills.

English

0.5 Hr

Computer Architecture

A beginner-friendly guide to computer architecture covering processors, memory, and system-level concepts.

English

0.5 Hr

HTML 5 References Tutorial

A handy HTML5 reference guide covering modern tags, attributes, and semantic elements.

English

1.5 Hrs

Docker Tutorial

A hands-on Docker handbook covering containers, images, and modern application deployment basics.

English

0 Hr

GIT (Using Github) Tutorial

A hands-on Git and GitHub handbook for managing code, tracking changes, and collaborating on projects.

English

0.5 Hr

Go Language Tutorial

A beginner-friendly Go handbook covering core language concepts and modern backend programming.

English

0.5 Hr

GIT Guide

A practical Git guide covering version control basics, branching, and real project workflows.

English

1 Hr

CSS Tutorial

A beginner-friendly CSS handbook covering page styling, layouts, and responsive design basics.

English

1 Hr

Advanced Data Structures

A focused handbook covering advanced data structures for efficient and scalable problem solving.

English

0 Hr

Spring Framework Tutorial

A hands-on Spring Framework handbook covering core concepts and backend development fundamentals.

English

1 Hr

Spring Boot Tutorial

A practical Spring Boot handbook focused on building and running modern Java backend applications.

English

0.5 Hr

Kotlin Tutorial

A beginner-friendly Kotlin handbook covering modern language features and real-world development concepts.

English

1 Hr

Apache Cordova Tutorial

A hands-on Apache Cordova handbook for building cross-platform mobile apps with web technologies.

English

0 Hr

Python Tutorial

A beginner-friendly Python handbook covering core concepts and practical programming examples.

English

1.5 Hrs

SASS-SCSS Tutorial

A hands-on SASS / SCSS handbook for writing clean, reusable, and scalable stylesheets.

English

0.5 Hr

MongoDB Tutorial

A hands-on MongoDB handbook covering NoSQL concepts and modern database operations.

English

0.5 Hr

Numpy Tutorial

A hands-on NumPy handbook for fast numerical computation and data manipulation using Python.

English

1.5 Hrs

PL-SQL Tutorial

A hands-on PL/SQL handbook for writing procedural database programs and business logic.

English

0.5 Hr

Python Built-in Functions Tutorial

A handy reference guide to Python’s built-in functions for cleaner and faster coding.

English

0.5 Hr

Pandas Tutorial

A hands-on Pandas handbook for data manipulation, cleaning, and analysis using Python.

English

2.5 Hrs

Elasticsearch Tutorial

A hands-on Elasticsearch handbook covering indexing, searching, and data analysis concepts.

English

0 Hr

Matplotlib Tutorial

A hands-on Matplotlib handbook for creating charts and visualizing data using Python.

English

0.5 Hr

Networking with Python

A hands-on handbook for building network-enabled applications using Python.

English

0.5 Hr

Tkinter Tutorial

A hands-on Tkinter handbook for building desktop applications with Python.

English

0.5 Hr

Java Programs Tutorial

A hands-on Java programs handbook for practicing core concepts and problem-solving in Java.

English

2 Hrs

Understanding Exploring BeautifulSoup Methods

Contents

Exploring BeautifulSoup Methods

BeautifulSoup: Accessing HTML Tags

reading content from the file

reading content from the file

creating a BeautifulSoup object

reading content from the file

creating a BeautifulSoup object

getting anchor tag

printing the 'href' attribute of anchor tag

getting all the children of 'body' using 'contents'

printing all the children using for loop

we can also convert iterator into list using the 'list(iterator)'

getting child tags of 'body' tag using 'descendants' method

Web Scraping Tutorial

Recommended Handbooks

C++ Handbook

Python Basics Handbook

Javascript Handbook

ReactJS Projects Handbook

Computer Networks Tutorial

Operating System Tutorial

DBMS and SQL Tutorial

Java Tutorial

C Language Tutorial

PHP Tutorial

Android Tutorial

Linux Guide Tutorial

Data Structures and Algorithms Tutorial

Computer Architecture

HTML 5 References Tutorial

Docker Tutorial

GIT (Using Github) Tutorial

Go Language Tutorial

GIT Guide

CSS Tutorial

Advanced Data Structures

Spring Framework Tutorial

Spring Boot Tutorial

Kotlin Tutorial

Apache Cordova Tutorial

Python Tutorial

SASS-SCSS Tutorial

MongoDB Tutorial

Numpy Tutorial

PL-SQL Tutorial

Python Built-in Functions Tutorial

Pandas Tutorial

Elasticsearch Tutorial

Matplotlib Tutorial

Networking with Python

Tkinter Tutorial

Java Programs Tutorial