Documentation
Fetch your first web page in two minutes.
One web scraping API endpoint for public page content. Copy a request, keep Auto mode on, and use the returned content in your app, automation, or AI agent.
Quick Start
Create an API key, save it as WEB_BYPASS_API_KEY on your server, then run one of the examples below. Replace https://example.com with the public page you need.
curl
curl -X POST https://web-bypass.dev/v1/fetch \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","mode":"auto"}'Python
import json
import urllib.request
request = urllib.request.Request(
"https://web-bypass.dev/v1/fetch",
data=json.dumps({"url": "https://example.com", "mode": "auto"}).encode(),
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
method="POST",
)
with urllib.request.urlopen(request) as response:
print(response.read().decode())JavaScript
const response = await fetch("https://web-bypass.dev/v1/fetch", {
method: "POST",
headers: {
Authorization: "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ url: "https://example.com", mode: "auto" }),
});
console.log(await response.json());Coding Agent Setup
Prefer AI coding? Store your real key in the server environment, then paste this stack-neutral task into Codex, Claude Code, Cursor, or another coding agent. The prompt includes the API contract and safety requirements, but never includes your key.
Connect with AI coding
Let your coding agent build the integration.
Save the key in your server environment, then paste this ready-made task into Codex, Claude Code, Cursor, or another coding agent.
Preview prompt
Please integrate the Web Bypass API into this project.
Goal: create a server-only fetchPublicPage(url, mode = "auto") function for retrieving publicly accessible web content.
API contract:
- Endpoint: POST https://web-bypass.dev/v1/fetch
- Header: Authorization: Bearer <WEB_BYPASS_API_KEY>
- Header: Content-Type: application/json
- Body: {"url":"https://example.com","mode":"auto"}
- mode accepts auto, easy, or hard. Use auto by default.
Requirements:
1. Inspect this project's language, framework, HTTP client, and environment-variable conventions before changing code. Follow the existing style and do not add an unnecessary SDK or framework.
2. Add a server-side environment variable named WEB_BYPASS_API_KEY. Put only an empty placeholder in the example environment file. Never put the real key in source code, tests, logs, browser code, or Git.
3. Add the smallest practical client module for the endpoint above. Read the API key only on the server.
4. Handle non-2xx responses, success=false, timeouts, and invalid JSON. Preserve request_id for debugging, but never log the key or full page content.
5. Do not automatically retry a billable request. Any retry must be an explicit caller decision.
6. Add unit tests with mocked HTTP. Automated tests must not call the live API.
7. Add one short usage example, then run the project's lint, typecheck, and tests.
Important success fields:
- success
- request_id
- final_url
- status_code
- content_type
- content
- access.used_mode
- billing.charge_usd
- billing.balance_remaining_usd
When finished, report the files changed, environment-variable name, usage example, and test results. Do not print the real API key.Python + Beautiful Soup Guide
Need a complete scraper instead of a raw API response? Follow the Python web scraping guide to fetch a page, parse its HTML with Beautiful Soup, resolve relative links, and handle API errors.
Authentication
Authenticate requests to /v1/fetch with:
Authorization: Bearer YOUR_API_KEYThis is a server-side credential. Never place it in browser code, a mobile app bundle, public repository, screenshot, or client-side environment variable.
Fetch API
Send the target page URL to POST /v1/fetch. The request body is intentionally small:
{
"url": "https://example.com",
"mode": "auto"
}urlis a public HTTP or HTTPS URL.modeis optional and defaults toauto.- Accepted modes are
auto,easy, andhard.
Auto Mode
Use Auto unless you have a specific reason not to. It tries the fast, lower-cost Easy path first and escalates to Hard only when the page needs browser-level access. You are charged once for the successful path; a final failure costs $0.
Easy Mode
Easy is the fast, lowest-cost choice for standard public pages that respond to ordinary web requests. Choose it directly only when you know the target does not need JavaScript rendering or browser-level access.
Hard Mode
Hard is the browser-capable path for JavaScript-heavy pages and sites that reject basic HTTP clients. It is more expensive than Easy and still cannot guarantee access to every website.
Billing
You spend balance only when a request successfully returns page content:
- Successful Easy requests use the Easy rate.
- Successful Hard requests use the Hard rate.
- Auto with Easy success uses the Easy rate.
- Auto with Easy failure and Hard success uses the Hard rate only.
- A final failure costs $0.
There is no subscription or monthly minimum. Add USDT balance from $1; the payment page shows the exact amount, supported network, address, QR code, and live status before you send funds.
Response
A successful response gives your application the page content plus the metadata needed to store, parse, monitor, or pass it to an AI workflow:
{
"success": true,
"request_id": "req_...",
"url": "https://example.com",
"final_url": "https://example.com/",
"status_code": 200,
"content_type": "text/html",
"content_encoding": "utf8",
"content": "...",
"access": {
"requested_mode": "auto",
"used_mode": "easy",
"easy_attempted": true,
"hard_attempted": false
},
"billing": {
"charged": true,
"charge_usd": "0.000100",
"balance_remaining_usd": "4.999900"
}
}content_encoding is utf8 for valid text and base64 when binary or non-text content cannot safely be represented as text.
Errors
Failures use stable error codes so your application can respond predictably. A final failed fetch includes a zero-charge billing result:
| Code | Meaning |
|---|---|
| INVALID_REQUEST | The request is invalid. |
| INVALID_API_KEY | The API key is invalid. |
| INSUFFICIENT_BALANCE | Insufficient balance. |
| INSUFFICIENT_BALANCE_FOR_HARD | Insufficient balance for Hard mode. |
| INVALID_URL | The URL is invalid. |
| TARGET_NOT_ALLOWED | The target URL is not allowed. |
| TARGET_NOT_FOUND | The target could not be found. |
| RATE_LIMITED | Too many concurrent requests. |
| FETCH_FAILED | The page could not be retrieved. |
| FETCH_TIMEOUT | The page retrieval timed out. |
| INTERNAL_ERROR | An internal error occurred. |
Rate Limits
Each API key can run up to 10 requests at the same time by default. Queue additional work in your application instead of launching unbounded parallel requests.
Security
- Call the API from your server over HTTPS and keep API keys out of browser code.
- Target URLs must be public HTTP or HTTPS resources.
- Local, private, internal, and metadata-network targets are rejected.
- Do not send passwords, cookies, or private authorization data in a target URL.
- Rotate the API key immediately if you believe it has been exposed.