---
name: wikimedia-pageviews-ingestion
description: "How to get Wikimedia Pageviews data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load Wikimedia page views into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/wikimedia-pageviews
Provider: https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/
Checked against the provider's documentation: 2026-10-29

## Access
- No account or key.
- Send a User-Agent header with a client name and contact details; clients without one may be blocked without notice.

## Request
```bash
curl -A 'my-research-bot/0.1 (your.email@example.com)' https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/user/Bitcoin/daily/20261001/20261028
```

## Format
JSON: items, one per day or month, with project, article, granularity, timestamp (YYYYMMDDHH), access, agent and views.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""Wikimedia Analytics API: daily page views for a few articles -> pandas -> Parquet. No key; User-Agent required."""
from urllib.parse import quote
import requests
import pandas as pd

HEADERS = {"User-Agent": "my-research-bot/0.1 (your.email@example.com) python-requests"}
BASE = "https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article"
frames = []
for title in ["Bitcoin", "Federal_Reserve", "S&P_500"]:          # titles use underscores for spaces
    url = f"{BASE}/en.wikipedia/all-access/user/{quote(title, safe='')}/daily/20250101/20261028"
    r = requests.get(url, headers=HEADERS, timeout=30)          # one request at a time, as the policy asks
    r.raise_for_status()
    frames.append(pd.DataFrame(r.json()["items"]))

df = pd.concat(frames, ignore_index=True)
df["date"] = pd.to_datetime(df["timestamp"], format="%Y%m%d%H")  # daily timestamps look like 2026102800
df = df[["article", "date", "views", "access", "agent"]]
df.to_parquet("wikimedia_pageviews.parquet", index=False)
print(len(df), "rows;", df.groupby("article")["views"].sum().to_dict())
```
Run by us on 2026-10-29 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 1,998 daily rows for three English Wikipedia articles, 2025-01-01 to 2026-10-28.

## Limits
- Rate limit: Not stated for the Analytics API. The access policy asks you to wait for each request to finish before sending another.

## Quirks
- Article titles use underscores for spaces and must be URL-encoded (S&P_500 becomes S%26P_500).
- Per-article data is daily or monthly only and starts on 2015-07-01.
- agent=user leaves out spiders and automated traffic; all-agents includes them.
- Bulk files for the same data are on dumps.wikimedia.org.

## Terms and attribution
- The data is CC0 1.0. Using the API also means agreeing to Wikimedia's terms of use and privacy policy.
- Terms page: https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/documentation/access-policy.html

## Sources
- https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/documentation/access-policy.html: "Clients making requests without a User-Agent header may be blocked without notice."
- https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/documentation/access-policy.html: "To stay within a safe request rate, wait for each request to finish before sending another request."
- https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/documentation/access-policy.html: "Data provided by the API is available under the CC0 1.0 license"
- https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/reference/page-views.html: "These endpoints serve data starting on July 1, 2015."

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.