RAG Web Browser MCP Server

Browser AutomationTypeScript

A web browser for LLMs that enables web search and content extraction

Available Tools

search

Query Google Search, scrape the top N URLs from the results, and return their cleaned content as Markdown

querymaxResultsscrapingTooloutputFormatsrequestTimeoutSecs

The RAG Web Browser provides AI assistants with the ability to search the web and extract content from websites. It acts as a bridge between large language models and the internet, similar to web search capabilities in ChatGPT. This tool enables AI agents to perform Google searches, scrape content from multiple URLs, and return cleaned content in Markdown format for easy consumption by LLMs.

Overview

The RAG Web Browser MCP server connects AI assistants to the web, allowing them to search for information and extract content from websites. It works by communicating with the RAG Web Browser Actor on the Apify platform, which runs in standby mode to provide fast responses.

Installation

To use the RAG Web Browser MCP server, you'll need to:

Clone the repository:

git clone https://github.com/apify/mcp-server-rag-web-browser.git
cd mcp-server-rag-web-browser

Install dependencies:
```
npm install
```
Create a .env file based on the .env.example template:
```
APIFY_TOKEN=your_apify_token
```
You'll need an Apify token, which you can obtain by creating an account at apify.com.
Start the server:
```
npm start
```
Add the server to your MCP-compatible client (like Claude Desktop) using the following configuration:

"mcpServers": {
  "rag-web-browser": {
    "command": "npm",
    "args": [
      "--prefix",
      "PATH_TO_YOUR_PROJECT_DIRECTORY",
      "start"
    ]
  }
}

Replace PATH_TO_YOUR_PROJECT_DIRECTORY with the actual path to where you cloned the repository.

Usage

Once installed, you can use the RAG Web Browser through your AI assistant. The primary functionality is accessed through the search tool, which allows you to:

Perform web searches via Google and extract content from the top results
Directly fetch and extract content from a specific URL

Example Prompts

To search the web:

"Search for the latest news about artificial intelligence"
"Find information about climate change solutions"

To fetch a specific URL:

"Get the content from https://example.com/article"

Configuration Options

When using the search tool, you can customize its behavior with these parameters:

maxResults: Control how many search results to process (default: 1)
scrapingTool: Choose between 'browser-playwright' for JavaScript-heavy sites or 'raw-http' for faster, simpler sites
outputFormats: Select the format for returned content (markdown, text, or html)
requestTimeoutSecs: Set a timeout for requests to prevent long-running operations

Troubleshooting

If you encounter issues:

Verify your Apify token is correct and has sufficient permissions
Check that the RAG Web Browser Actor is accessible with your account
Ensure your network allows the server to connect to Apify's API
Review server logs for specific error messages

For more detailed information, visit the repository on GitHub.