Oxylabs Web Scraping MCP Server

Data Science ToolsPython

Access web data seamlessly with Oxylabs' web scraping capabilities

Available Tools

scrape_serp

Scrape search engine results pages (SERPs) from Google, Bing, Yandex, and other search engines

querysearch_enginecountrylanguagedevice_type

scrape_ecommerce

Scrape product data from e-commerce websites like Amazon, eBay, Walmart, etc.

urlcountryparseuser_agent_type

scrape_universal

Universal web scraper that can extract data from any website

urlcountryuser_agent_typegeolocation

Oxylabs MCP provides a powerful interface to access Oxylabs' web scraping infrastructure through the Model Context Protocol. It enables AI models to retrieve structured and unstructured data from websites with advanced proxy management, CAPTCHA solving, and anti-bot detection capabilities. This integration allows for reliable data extraction from various sources while handling complex web scraping challenges automatically.

Introduction

Oxylabs MCP Server provides a seamless way to integrate Oxylabs' powerful web scraping capabilities with AI models through the Model Context Protocol. This integration allows you to extract data from websites reliably, even those with anti-bot protections, CAPTCHAs, and other scraping challenges.

Installation

Using pip

The simplest way to install the Oxylabs MCP server is via pip:

pip install oxylabs-mcp

From Source

You can also install from the source repository:

git clone https://github.com/oxylabs/oxylabs-mcp.git
cd oxylabs-mcp
pip install -e .

Configuration

Before using the Oxylabs MCP server, you need to set up your Oxylabs credentials. You can do this by setting environment variables:

export OXYLABS_USERNAME="your_username"
export OXYLABS_PASSWORD="your_password"

Alternatively, you can provide these credentials when starting the server.

Running the Server

To start the Oxylabs MCP server:

oxylabs-mcp --username your_username --password your_password

By default, the server runs on port 8000. You can specify a different port using the --port option:

oxylabs-mcp --port 8080

Using with AI Models

To use Oxylabs MCP with AI models like Claude or other MCP-compatible systems, you'll need to add the server configuration to your client setup. The server provides various tools for web scraping that can be accessed through the MCP protocol.

Docker Support

Oxylabs MCP can also be run as a Docker container:

docker build -t oxylabs-mcp .
docker run -e OXYLABS_USERNAME=your_username -e OXYLABS_PASSWORD=your_password -p 8000:8000 oxylabs-mcp

Advanced Usage

The Oxylabs MCP server provides several advanced features:

Proxy Rotation: Automatically rotates through different proxies to avoid IP blocking
CAPTCHA Solving: Handles CAPTCHAs and other anti-bot challenges
Geolocation Targeting: Allows scraping from specific geographic locations
User-Agent Management: Customizes browser fingerprints for more reliable scraping

Refer to the tool documentation for specific parameters and options available for each scraping method.