Python Proxy Headers

Extensions for Python HTTP libraries to support sending and receiving custom proxy headers during HTTPS CONNECT tunneling.

The Problem

When making HTTPS requests through a proxy, the connection is established via a CONNECT tunnel. During this process:

1. Sending headers to the proxy - Most Python HTTP libraries don't provide an easy way to send custom headers (like X-ProxyMesh-Country) to the proxy server during the CONNECT handshake.

2. Receiving headers from the proxy - The proxy's response headers from the CONNECT request are typically discarded, making it impossible to read custom headers (like X-ProxyMesh-IP) that the proxy sends back.

This library solves both problems for popular Python HTTP libraries.

Supported Libraries

Library	Module	Use Case
urllib3	urllib3_proxy_manager	Low-level HTTP client
requests	requests_adapter	Simple HTTP requests
aiohttp	aiohttp_proxy	Async HTTP client
httpx	httpx_proxy	Modern HTTP client
pycurl	pycurl_proxy	libcurl bindings
cloudscraper	cloudscraper_proxy	Cloudflare bypass
autoscraper	autoscraper_proxy	Automatic web scraping

Installation

pip install python-proxy-headers

Or install the client extra you need. Each extra pins a range that CI tests:

pip install 'python-proxy-headers[requests]'
pip install 'python-proxy-headers[urllib3]'
pip install 'python-proxy-headers[httpx]'
pip install 'python-proxy-headers[aiohttp]'
pip install 'python-proxy-headers[pycurl]'
pip install 'python-proxy-headers[cloudscraper]'
pip install 'python-proxy-headers[autoscraper]'

The httpx extra is httpx>=0.28,<1. Importing python_proxy_headers.httpx_proxy on httpx 1.x raises ImportError. See the httpx docs. Proxy header names are specific to your provider; ProxyMesh documents theirs at docs.proxymesh.com.

Note: This package has no required dependencies. Install only the extra you need.

Quick Start

requests

from python_proxy_headers.requests_adapter import ProxySession

with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
    session.proxies = {'https': 'http://user:[email protected]:8080'}
    response = session.get('https://httpbin.org/ip')
    
    # Proxy headers are merged into response.headers
    print(response.headers.get('X-ProxyMesh-IP'))

httpx

from python_proxy_headers.httpx_proxy import get

response = get(
    'https://httpbin.org/ip',
    proxy='http://user:[email protected]:8080'
)

# Proxy CONNECT response headers are merged into response.headers
print(response.headers.get('X-ProxyMesh-IP'))

aiohttp

import asyncio
from python_proxy_headers.aiohttp_proxy import ProxyClientSession

async def main():
    async with ProxyClientSession() as session:
        async with session.get(
            'https://httpbin.org/ip',
            proxy='http://user:[email protected]:8080'
        ) as response:
            # Proxy headers merged into response.headers
            print(response.headers.get('X-ProxyMesh-IP'))

asyncio.run(main())

pycurl (low-level)

import pycurl
from python_proxy_headers.pycurl_proxy import set_proxy_headers, HeaderCapture

c = pycurl.Curl()
c.setopt(pycurl.URL, 'https://httpbin.org/ip')
c.setopt(pycurl.PROXY, 'http://proxy.example.com:8080')

# Add these two lines to any existing pycurl code
set_proxy_headers(c, {'X-ProxyMesh-Country': 'US'})
capture = HeaderCapture(c)

c.perform()

print(capture.proxy_headers)  # Headers from proxy CONNECT response
c.close()

cloudscraper

from python_proxy_headers.cloudscraper_proxy import create_scraper

# Drop-in replacement for cloudscraper.create_scraper()
scraper = create_scraper(proxy_headers={'X-ProxyMesh-Country': 'US'})
scraper.proxies = {'https': 'http://proxy.example.com:8080'}

response = scraper.get('https://example.com')
# All CloudScraper features (Cloudflare bypass) preserved

Testing

A test harness is included to verify proxy header functionality:

# Header checks and private-hook checks (no proxy credentials)
python test_header_security.py
python test_httpx_guard.py
python test_client_internals.py

# Set your proxy
export PROXY_URL='http://user:[email protected]:8080'

# Test all modules
python test_proxy_headers.py

# Test specific modules
python test_proxy_headers.py requests httpx

# Verbose output (show header values)
python test_proxy_headers.py -v

Documentation

For detailed documentation, API reference, and more examples:

• Full Documentation: python-proxy-headers.readthedocs.io
• Example Code: proxy-examples for Python

Other languages

• javascript-proxy-headers - Send and read custom proxy headers on HTTPS CONNECT for Node.js HTTP clients
• ruby-proxy-headers - Send and read custom proxy headers on HTTPS CONNECT for Ruby HTTP clients
• scrapy-proxy-headers - Proxy header support for Scrapy

About

Created by ProxyMesh to help our customers use custom headers to control proxy behavior. Works with any proxy that supports custom headers.

License

MIT License