AI Extraction Comes to the Geekflare Web Scraping API

We are excited to announce AI Extraction, a new feature in the Geekflare Web Scraping API.

Scraping a webpage is only the first step. Many applications still need to extract structured information, summarize content, or answer questions before the data becomes useful. AI Extraction brings these capabilities directly into Geekflare’s Web Scraping API.

Let’s explore how it works and what you can build with it.

How AI Extraction Works

AI Extraction extends the Geekflare Web Scraping API with AI-powered data extraction. Simply provide a webpage URL and select the type of information you want to extract. The API handles the processing and returns the extracted result along with the scraped webpage content.

Behind the scenes, your request looks like this:

{
  "url": "https://example.com/products/wireless-headphones",
  "aiPrompt": {
    "type": "prompt | schema | product | listing | summary | contact | sentiment | keywords",
    "...": "mode-specific fields"
  }
}

Here is a simplified view of how the extraction works

After the request is processed, the extracted information is available in the aiResult field. The response also includes the scraped webpage content in the data field and additional request details in the meta field.

Supported Extraction Types

AI Extraction offers eight extraction types for common data extraction tasks. Choose an extraction type based on the information you want to extract.

See AI Extraction Types in Action

All examples use the same Web Scraping API endpoint. Simply change the extraction type based on your use case.

Open-ended Question (Prompt)

The Prompt extraction type lets you ask questions about a webpage using natural language. Simply enter your question, and AI Extraction analyzes the page and returns an answer based on its content.

For this example, we’ll use a Python tutorial page and ask a simple question.

URL

https://docs.python.org/3/tutorial/

Question

What does this tutorial teach?

Response:

Custom JSON Schema (Schema)

The Schema extraction type lets you define the exact information you want to extract from a webpage. Provide a JSON schema, and AI Extraction returns the data in the same structure.

For this example, we’ll extract basic company information from the OpenAI About page.

URL

https://openai.com/about/

Schema

{
  "company_name": "string",
  "mission": "string",
  "founded": "string",
}

Response:

Product Extraction(Product)

The Product extraction type extracts product information from ecommerce pages. It identifies details such as the product name, description, price, availability, images, specifications, and more.

For this example, we’ll extract product details from an ecommerce product page.

URL

https://frame.work/products/laptop13

Response

Listing Extraction (Listing)

The Listing extraction type extracts multiple items from a webpage, such as products, articles, job postings, or search results. It returns the extracted items as a structured list. This is useful for category pages, directories, and search results.

For this example, we’ll extract product listings from an ecommerce category page.

URL

https://books.toscrape.com/

Schema:

{
  "books": [
    {
      "title": "string",
    }
  ]
}

Response

Contact Info (Contact)

The Contact extraction type extracts contact information from a webpage. It identifies details such as names, email addresses, phone numbers, physical addresses, social media links, and other contact details.

For this example, we’ll extract contact information from a company’s contact page.

URL

https://automattic.com/contact/

Response

Summary (Article)

The Article extraction type extracts information from news articles, blogs, documentation, and other content pages. It identifies details such as the title, author, publication date, summary, tags, and the main content.

It also supports three summary styles such as paragraph, TL;DR, and bullet points. You can also specify a Focus Area to generate a summary around a particular topic and set a Max Length to control the summary size.

For this example, we’ll extract information from a blog article.

URL

https://geekflare.com/guides/geekflare-api-credit-pack/

Response

Sentiment Analysis (Sentiment)

The Sentiment extraction type analyzes the overall sentiment of a webpage. It classifies the content as positive, negative, or neutral and provides a confidence score for the detected sentiment. You can also specify Aspects to analyze sentiment for the topics that matter to your use case.

For example, if you’re analyzing reviews from the URL below, you can specify aspects such as:

• Story
• Acting
• Visual effects
• Direction
• Cinematography
• Soundtrack
• Pacing
• Ending

URL

https://www.rottentomatoes.com/m/inception/reviews

Response

Keywords & Entities (Keyword)

The Keyword extraction type identifies the most relevant keywords and key phrases from a webpage. It helps you understand the main topics of the content. This is useful for SEO analysis, content categorization, topic discovery, and search applications.

You can also specify the maximum keyword count to control how many keywords are returned.

URL

https://en.wikipedia.org/wiki/Artificial_intelligence

Response

Use AI Extraction in Your Applications

AI Extraction helps you retrieve useful information from webpages in a single API request. It simplifies common data extraction tasks and reduces the amount of post-processing in your application.

Here are a few examples:

• Ecommerce: Extract product details, pricing, availability, and ratings from online stores.
• Market Research: Generate summaries and compare information across multiple webpages.
• Lead Generation: Collect contact details from company websites.
• Content Analysis: Extract keywords and analyze sentiment from articles, blogs, and reviews.
• AI Applications: Retrieve structured information from webpages for AI agents and RAG applications.

Get Started with AI Extraction

Ready to try AI extraction? Visit the Geekflare Playground to explore the feature and test different extraction types using your own webpages.

When you’re ready to integrate AI Extraction into your application, refer to the AI Extraction documentation for implementation guides, API references, and SDK examples.