Mastering XPath in Selenium: How to Locate Elements

By Titus Kamunya and edited by Narendra Mohan Mittal

Mastering XPath in Selenium How to Locate Elements

Test engineers are essential in the software development lifecycle as they ensure a team delivers bug-free and working applications. These engineers take the applications through different tests before they can be shipped or declared ready for users.

Testors should be skilled and able to locate and interact with web elements. Selenium is one of the most-used test automation tools by modern development teams. This tool has four components; Selenium Grid, Selenium WebDriver, Selenium IDE, and Selenium RC.

Our focus today will be on Selenium WebDriver, as it contains XPath. This article will define XPath, discuss the basic XPath syntax and illustrate how to XPath in Selenium.

What is XPath

XPath( XML Path Language) is a query language for selecting and navigating attributes and elements within XML documents. XPath defines a path expression, offering a way to address specific parts of an XML document and retrieve info from it.

Its syntax resembles a file system path. It also features functions and symbols that make selecting elements based on their attributes and hierarchy easy. You can use XPath with technologies like XML, HTML, and XSLT to extract and manipulate data.

Why use XPath?

It is flexible: Unlike CSS selectors that can only locate elements using tag name, ID, or class, XPath allows you to locate elements using other attributes.
Reusable: You can store your XPath in variables and reuse them in your code.
Precise node selection: XPath provides a standardized way to target specific elements in a web document.

Basic Syntax of XPath

XPath allows you to find any element on a webpage using the DOM. However, before we check the syntax, we need to understand the following XPath expressions;

Expression	Description
nodename/ tagname	Selects all nodes the name ‘nodename’/ ‘tagname’
/	Selects from the root node
//	Selects nodes in the current document from the current node matching the selection, irrespective of where they are
@	Selects attributes
..	Selects the parent of the current node
.	Selects the current node

The standard syntax for XPath is;

XPath=//tagname[@attribute='value']

As you can see, the syntax starts with a double slash (//), which starts with the current node as defined by the tag/node name.

Absolute XPath vs. Relative XPath

We have two paths when dealing with XPath; Absolute XPath and Relative XPath;

Absolute XPath

An Absolute XPath is a direct path from the root to the desired element. You start with the root node and end with the target node.

You can have an HTML document with the following content;

<!DOCTYPE html>
<html>
<head>
    <title>Geekflare</title>
</head>
<body>
    <div>
        <h1>Welcome to Geekflare</h1>
    </div>
</body>
</html>

If we want to locate the element with the content “Welcome to Geekflare”, you will follow this path;

/html/body/div/h1

In the above document, we have;

The html as the root node: /html
The body is a parent node: /html/body
The div as a child of the body node: /html/body/div
The h1 as a child of the div node: /html/body/div/h1

To get the innermost element, you have to follow that path.

When to use Absolute XPath

Absolute XPath follows a ‘specific’ path. It will thus be a perfect fit when you have multiple elements with similar attributes on a page, ensuring you are targeting the exact elements on a document.

However, XPath is over-sensitive to changes in the HTML document’s structure. As such, a simple change might break your Absolute XPath.

Relative XPath

Relative XPath starts from any node and ends with the target node. This path is not affected by changes in the document, making it preferable in most cases. With Relative XPath, you can locate elements from any part of the document. The Relative XPath expression starts with double slashes ‘//’.

If we use the HTML document, we can locate our H1 that says “Welcome to Geekflare”;

<!DOCTYPE html>
<html>
<head>
    <title>Geekflare</title>
</head>
<body>
    <div>
        <h1>Welcome to Geekflare</h1>
    </div>
</body>
</html>

Our Relative XPath to h1 will be;

//body/div/h1

When to use Relative XPath

You should use Relative XPath when looking for a balance between flexibility and specificity. This path is resilient to changes in the HTML document, given that the relationship between the elements remains specific.

Locate Elements using XPath in Selenium

Selenium is an open-source framework that allows users to automate web browsers. The framework has a collection of libraries and tools that helps testers interact with web elements automatically and systematically.

Assuming we have a web document that contains a list of songs as follows;

<!DOCTYPE html>
<html>
<head>
    <title>Song Library</title>
</head>
<body>
    <h1>Song Library</h1>
    <ul class="song-list">
        <li class="song" title="Song Title 1">Song 1 - Artist 1</li>
        <li class="song" title="Song Title 2">Song 2 - Artist 2</li>
        <li class="song" title="Song Title 3">Song 3 - Artist 1</li>
        <li class="song" title="Song Title 4">Song 4 - Artist 3</li>
    </ul>
</body>
</html>

Our root node is <html>.
We have <body> as our parent node.
We have <h1> as the child of <body>.
We have <ul> as a child of <body>.
We have <li> as a child of <ul>.

We can use different XPath locators in the above HTML document. For instance, we can locate elements by ID, name, class name, contains, texts, ends-with, and starts-with, among many other locators. You can use Selenium with different programming languages. We will use Python to demonstrate.

Locate by index

Assuming we want to locate song number 3, we can have this code;

third_song = driver.find_element_by_xpath("//li[@class='song'][3]")
print("Third Song:", third_song.text)

We have used Relative XPath and start with the ‘li’ node. When Selenium locates the third song on our list, it will print its text.

Locate by Attribute

We can have an XPath that looks for all the songs from ‘Artist 1” and prints out their titles. Our code can be as follows;

songs_by_artist1 = driver.find_elements_by_xpath("//li[contains(@class, 'song') and contains(text(), 'Artist 1')]")
print("Songs by Artist 1:")
for song in songs_by_artist1:
    print(song.text)

Locate by Text

This locator helps you find elements with specific text. We can look for a song with “Song 4” text and prints out its text. We can represent this locator using this code;

song_with_text = driver.find_element_by_xpath("//li[contains(text(), 'Song 4')]")
print("Song with Text:", song_with_text.text)

XPath Axes

The approaches we have discussed so far work perfectly with simple web pages. However, there are instances where XPath element search methods like by Text, ID, Class name, and Name will not work.

XPath Axes are used for dynamic content where normal locators don’t work. Here, you locate elements based on their relationship with other elements. These are some of the common XPath Axes locators;

Ancestor

The Ancestor Axis method is perfect for dealing with XML documents with highly nested elements. You can select all ancestor elements like grandparent and parent of the current node from the closest to the furthest.

We can have the following code;

<bookstore>
  <book>
    <title>The Great Gatsby</title>
    <author>F. Scott Fitzgerald</author>
    <genre>Fiction</genre>
  </book>
  <book>
    <title>The Biggest Dilemma</title>
    <author>George Orwell</author>
    <genre>Dystopian</genre>
  </book>
</bookstore>

If we want to select all the ancestors of the element ‘title’ for the book “The Biggest Dilemma”, we can have this Ancestor Axis method;

//title[text() = '1984']/ancestor::*

Following

The Following Axis method locates all nodes post the current node’s closing tag. This method does not consider the hierarchy or location of its target nodes. For instance, if you have an XML document or web page with multiple sections, you can identify an element that appears after a particular section without navigating the entire tree structure.

Parent

The Parent Axis method in XPath selects the parent of the current node. You can use the following path to locate the parent node;

//tag[@attribute='value']/parent::tagName

This approach works when the child elements in the current node have unique attributes that you can easily locate, and you want to check with the parent.

Child

The Child Axis method in XPath selects all children of the current node. It remains one of the most popular XPath Axis methods as it helps select child nodes of a particular element.

Consider this piece of code;

<section id='text'>
    <p>Paragraph one</p>
    <p>Paragraph two</p>
    <p>Paragraph three</p>
    <p>Paragraph four</p>
</section>

We can locate all the ‘p’ elements in our code using this Axis;

//section[@id='text']/child::p

FAQs

Why use XPath instead of CSS selectors?

CSS selectors can only find elements based on their ID, tag name, and class. On the other hand, you can use XPath to locate elements based on their location, text content, and other attributes in the HTML structure. You can also store XPath expressions in variables and reuse them in your application.

Which languages does XPath in Selenium support?

You can use XPath with any language that supports Selenium. You can write your scripts in JavaScript, Java, Python, Ruby, C#, and PHP.

What are XPath’s alternatives?

You can use CSS Selectors, Image Recognition, or Selenium’s built-in locators as an alternative to XPath. CSS Selectors are the most common; you can find elements based on their tag name, ID, or class. Image Recognition allows you to locate elements based on their images. The Selenium’s built-in locators are designed to be easy to use.

Conclusion

You can now define XPath in Selenium, differentiate between Absolute and Relative XPath, and locate elements using different XPath locators. The choice of locator will depend on the nature of the content and your end goals.

Check out our article on Selenium interview questions if you want to ace your next interview.

Titus Kamunya

Author

Titus is a Software Engineer and Technical Writer. He develops web apps and writes on SaaS, React, HTML, CSS, JavaScript, Ruby and Ruby on Rails read more
Narendra Mohan Mittal

Editor

Narendra Mohan Mittal is a versatile and experienced digital branding strategist and content editor with over 12 years of experience. He is a Gold Medalist in M-Tech and B-Tech in Computer Science & Engineering.

Currently,… read more