Make HTTP requests in Python using HTTPX - a modern HTTP client with sync/async APIs, HTTP/2 support, and requests-compatible interface. Install, configure, and use HTTPX for API calls, web scraping, and HTTP operations.
A skill for working with HTTPX, the next-generation HTTP client library for Python. HTTPX provides a modern, fully-featured HTTP client with both synchronous and asynchronous APIs, HTTP/2 support, and a requests-compatible interface.
This skill helps you install, configure, and use HTTPX for making HTTP requests in Python. It covers basic usage, advanced features, async operations, HTTP/2, and CLI usage.
Install HTTPX based on the features needed:
**Basic installation:**
```bash
pip install httpx
```
**With HTTP/2 support:**
```bash
pip install httpx[http2]
```
**With CLI tool:**
```bash
pip install httpx[cli]
```
**With all extras:**
```bash
pip install httpx[http2,cli,brotli,zstd,socks]
```
For simple HTTP requests using the synchronous API:
```python
import httpx
response = httpx.get('https://api.example.com/data')
print(response.status_code)
print(response.headers['content-type'])
print(response.text)
print(response.json())
data = {'key': 'value'}
response = httpx.post('https://api.example.com/create', json=data)
form_data = {'username': 'user', 'password': 'pass'}
response = httpx.post('https://api.example.com/login', data=form_data)
headers = {'Authorization': 'Bearer token123'}
response = httpx.get('https://api.example.com/protected', headers=headers)
params = {'page': 1, 'limit': 10}
response = httpx.get('https://api.example.com/items', params=params)
```
For making multiple requests with shared configuration (connection pooling, cookies, headers):
```python
import httpx
with httpx.Client() as client:
# Shared configuration
client.headers.update({'User-Agent': 'my-app/1.0'})
# Multiple requests reuse connection
r1 = client.get('https://api.example.com/users')
r2 = client.get('https://api.example.com/posts')
r3 = client.post('https://api.example.com/data', json={'key': 'value'})
# Cookies persist across requests
client.cookies.set('session', 'abc123')
```
For concurrent requests or async applications:
```python
import httpx
import asyncio
async def fetch_data():
async with httpx.AsyncClient() as client:
response = await client.get('https://api.example.com/data')
return response.json()
async def fetch_multiple():
async with httpx.AsyncClient() as client:
tasks = [
client.get('https://api.example.com/endpoint1'),
client.get('https://api.example.com/endpoint2'),
client.get('https://api.example.com/endpoint3'),
]
responses = await asyncio.gather(*tasks)
return [r.json() for r in responses]
data = asyncio.run(fetch_data())
```
**Timeouts:**
```python
response = httpx.get('https://api.example.com/slow', timeout=10.0)
timeout = httpx.Timeout(connect=5.0, read=10.0, write=5.0, pool=5.0)
response = httpx.get('https://api.example.com/data', timeout=timeout)
```
**Authentication:**
```python
response = httpx.get('https://api.example.com/protected',
auth=('username', 'password'))
from httpx import DigestAuth
response = httpx.get('https://api.example.com/protected',
auth=DigestAuth('username', 'password'))
```
**File uploads:**
```python
files = {'file': open('document.pdf', 'rb')}
response = httpx.post('https://api.example.com/upload', files=files)
files = {
'file1': ('report.pdf', open('report.pdf', 'rb'), 'application/pdf'),
'file2': ('image.png', open('image.png', 'rb'), 'image/png'),
}
response = httpx.post('https://api.example.com/upload', files=files)
```
**Streaming responses:**
```python
with httpx.stream('GET', 'https://example.com/large-file.zip') as response:
for chunk in response.iter_bytes():
process_chunk(chunk)
```
**HTTP/2:**
```python
client = httpx.Client(http2=True)
response = client.get('https://http2.example.com/')
```
**Proxies:**
```python
proxies = {
'http://': 'http://proxy.example.com:8080',
'https://': 'http://proxy.example.com:8080',
}
response = httpx.get('https://api.example.com/data', proxies=proxies)
```
Use HTTPX from the command line:
```bash
httpx https://api.example.com/data
httpx POST https://api.example.com/create key=value
httpx https://api.example.com/protected Authorization:"Bearer token123"
httpx --follow-redirects https://example.com
httpx https://example.com/file.pdf > output.pdf
httpx -v https://api.example.com/data
```
Handle common HTTP errors and exceptions:
```python
import httpx
try:
response = httpx.get('https://api.example.com/data', timeout=5.0)
response.raise_for_status() # Raise exception for 4xx/5xx
data = response.json()
except httpx.TimeoutException:
print("Request timed out")
except httpx.HTTPStatusError as e:
print(f"HTTP error: {e.response.status_code}")
except httpx.RequestError as e:
print(f"Request failed: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
```
Use HTTPX transports for testing without network calls:
```python
import httpx
from myapp import app # Your ASGI/WSGI app
async with httpx.AsyncClient(app=app, base_url='http://testserver') as client:
response = await client.get('/api/endpoint')
assert response.status_code == 200
with httpx.Client(app=app, base_url='http://testserver') as client:
response = client.get('/api/endpoint')
assert response.status_code == 200
```
**API client wrapper:**
```python
import httpx
class APIClient:
def __init__(self, base_url: str, api_key: str):
self.client = httpx.Client(
base_url=base_url,
headers={'Authorization': f'Bearer {api_key}'},
timeout=10.0
)
def get_users(self):
response = self.client.get('/users')
response.raise_for_status()
return response.json()
def create_user(self, data: dict):
response = self.client.post('/users', json=data)
response.raise_for_status()
return response.json()
def __enter__(self):
return self
def __exit__(self, *args):
self.client.close()
```
**Async web scraper:**
```python
import httpx
import asyncio
async def scrape_pages(urls: list[str]):
async with httpx.AsyncClient(timeout=30.0) as client:
tasks = [client.get(url) for url in urls]
responses = await asyncio.gather(*tasks, return_exceptions=True)
results = []
for url, response in zip(urls, responses):
if isinstance(response, Exception):
results.append({'url': url, 'error': str(response)})
else:
results.append({'url': url, 'status': response.status_code, 'content': response.text})
return results
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/httpx-next-gen-python-http-client/raw