---
name: sec-edgar-apis-ingestion
description: "How to get SEC EDGAR APIs data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load SEC EDGAR XBRL company facts into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/sec-edgar-apis
Provider: https://www.sec.gov/search-filings/edgar-application-programming-interfaces
Checked against the provider's documentation: 2026-10-29

## Access
- No account or API key.
- Send a User-Agent header that names you and gives a contact email (SEC asks automated tools to declare one).

## Request
```bash
curl -H 'User-Agent: Your Name your.email@example.com' https://data.sec.gov/api/xbrl/companyfacts/CIK0000320193.json
```

## Format
JSON: facts → taxonomy (us-gaap, dei, …) → concept → units (USD, shares, …) → a list of reported values with end, val, accn, fy, fp, form, filed, and start and frame when present.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""SEC EDGAR XBRL company facts -> long pandas table -> Parquet. No key; declare a User-Agent."""
import requests
import pandas as pd

HEADERS = {"User-Agent": "Your Name your.email@example.com"}  # SEC asks you to declare who you are
CIK = "0000320193"  # 10-digit CIK with leading zeros (this one is Apple Inc.)
r = requests.get(f"https://data.sec.gov/api/xbrl/companyfacts/CIK{CIK}.json", headers=HEADERS, timeout=60)
r.raise_for_status()
facts = r.json()["facts"]

rows = []
for taxonomy, concepts in facts.items():          # e.g. "us-gaap", "dei"
    for concept, body in concepts.items():
        for unit, points in body["units"].items():  # e.g. "USD", "shares"
            for p in points:
                rows.append({"taxonomy": taxonomy, "concept": concept, "unit": unit, **p})

df = pd.DataFrame(rows)
for c in ("start", "end", "filed"):
    if c in df:
        df[c] = pd.to_datetime(df[c])
df.to_parquet("sec_companyfacts_320193.parquet", index=False)
print(len(df), "facts;", df["concept"].nunique(), "concepts; forms:", sorted(df["form"].dropna().unique())[:6])
```
Run by us on 2026-10-29 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 25,135 reported values across 505 concepts for CIK 0000320193 (Apple Inc.).

## Limits
- Rate limit: at most 10 requests per second in total, however many machines you use; above that, requests may be limited for a brief period.
- For many companies, use the nightly bulk file companyfacts.zip instead of one request per company.

## Quirks
- The CIK in the URL must be 10 digits with leading zeros.
- Later filings repeat earlier values and can restate them, so one concept, unit, start and end can carry different values (Observed 2026-10-29: 426 such periods for Apple, for example AccruedLiabilitiesCurrent at 2008-09-27 is 3.719B in the 10-K and 4.224B in the later 10-K/A). To keep the latest reported value per period, sort by filed and keep the last row for each concept, unit, start and end, and keep accn so each value traces back to its filing.
- Companies report the same idea under different concepts (revenue has several us-gaap tags), so check which concept a company uses.
- Observed 2026-10-24: requests without a declared User-Agent returned HTTP 403.
- data.sec.gov does not support CORS, so call it from a script or server, not from browser JavaScript.

## Terms and attribution
- SEC asks for appropriate citation of the SEC as the source. EDGAR data is public information; the EDGAR name and logos are restricted.
- Terms page: https://www.sec.gov/about/privacy-information#dissemination

## Sources
- https://www.sec.gov/search-filings/edgar-application-programming-interfaces: "These APIs do not require any authentication or API keys to access."
- https://www.sec.gov/search-filings/edgar-application-programming-interfaces: "Where the ########## is the entity’s 10-digit central index key (CIK), including leading zeros."
- https://www.sec.gov/search-filings/edgar-application-programming-interfaces: "The most efficient means to fetch large amounts of API data is the bulk archive ZIP files, which are recompiled nightly."
- https://www.sec.gov/privacy.htm: "Current guidelines limit users to a total of no more than 10 requests per second, regardless of the number of machines used to submit requests."
- https://www.sec.gov/about/webmaster-frequently-asked-questions: "Please declare your user agent in request headers"

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.