Awesome Datasets

Curated public data for building and evaluating software, with license, size and access terms recorded for every entry.

Two catalogs. The global one covers domains software teams hit constantly — source code, defects, vulnerabilities, network traffic, logs, search, recommendations, forecasting. The country one splits those same domains by where the data comes from, because "car sales data" means something different in Delhi than in Detroit.

Every entry is a dataset you can actually obtain. The link resolves, the provider is named, and the license and access requirements are stated up front so you can tell before downloading whether you are allowed to use it. This list links and describes; it redistributes no data.

⚠️ Read this before you use anything here

Browsing this list is safe. Using what it points at is where the obligations start.

This repository contains no data — only links and factual metadata about them. Nothing you do here binds you to anyone's terms. The moment you download a dataset, however, that dataset's license applies to you in full: how you may use it, whether you may redistribute it, and whether you may build a commercial product on it.

Of the 489 datasets listed here, 382 permit commercial use, 28 are non-commercial only, and 79 have mixed or unclear terms that you have to check yourself. 409 are a direct download, 78 need a free account, and 2 need a signed agreement.

So:

• Read the license at the source before you build on a dataset, not before you bookmark it. Terms change without notice and this list can lag behind them.
• Treat commercial_use: check as "no" until you have read the terms. It means the dataset bundles third-party content under its original license, which is common and easy to get wrong.
• Non-commercial means non-commercial. Several widely used entries — MovieLens, MS MARCO, Instacart, Olist among them — are research-only. Training a production model on them is a licensing decision, not a technical one.
• Attribution is usually mandatory, not polite. Most open government licenses and every CC BY variant require it. The cite field on each entry records what the publisher asks for.

The license and commercial_use fields are a starting point for your own diligence, not legal advice, and they come with no warranty — see license. If an entry is wrong, tell us; corrections are as welcome as additions.

Contents

Global datasets - not tied to one country.

• Source Code & Software Repositories
• Software Defects & Testing
• Security & Vulnerabilities
• Network Traffic & Intrusion Detection
• Logs & Observability
• Databases & Query Benchmarking
• Web Crawl & Internet-Scale Text
• Natural Language Processing
• Computer Vision
• Speech & Audio
• Recommender Systems
• Search & Information Retrieval
• Tabular Machine Learning
• Time Series & Forecasting
• Fraud & Anomaly Detection
• Graphs & Networks
• Geospatial & Mapping
• Finance & Economics
• E-commerce & Retail
• Transportation & Mobility

By country - the same domains split by where the data comes from.

Global datasets

Source Code & Software Repositories

Corpora of source code and repository activity used to train code models and study how software is built.

• CodeSearchNet Corpus - Two million functions from open-source projects in six languages, each paired with its natural-language docstring, for code search and code-summarisation models.
  _{GitHub · JSONL, GZIP · ~20 GB · 2,000,000 functions · MIT (tooling); individual functions retain their upstream repository licenses · Direct download · Commercial use: check terms}
• GH Archive - Every public GitHub event since 2011, archived hourly and queryable in BigQuery, covering pushes, pull requests, issues and stars.
  _{GH Archive (Ilya Grigorik) · JSON, GZIP, BigQuery · Multi-TB, grows continuously · Billions of events · MIT (project); event data derived from the GitHub public timeline · Direct download · Commercial use: check terms}

Software Defects & Testing

Reproducible bugs, regressions and failing builds used for automated program repair, fault localisation and test-generation research.

• Defects4J - A reproducible collection of real Java bugs, each with a buggy and fixed revision plus the tests that expose the failure, and a harness that checks out and builds any of them on demand.
  _{René Just et al. · Git, Java, Shell · ~2 GB after setup · 800+ bugs across 17 open-source Java projects · MIT · Direct download · Commercial use permitted}
• BugSwarm - Fail-pass pairs of real continuous-integration builds packaged as Docker images, so a historical build failure and its fix can be reproduced exactly years later.
  _{UC Davis DECAL Lab · Docker images, JSON · Varies per artifact · 3,000+ reproducible artifacts · MIT (client tooling); artifacts retain upstream project licenses · Direct download · Commercial use: check terms}

Security & Vulnerabilities

Authoritative feeds of publicly disclosed software vulnerabilities and their severity metadata.

• NVD Vulnerability Data Feeds - The US National Vulnerability Database's enriched view of every CVE, adding CVSS severity scores, CWE weakness classes and CPE product identifiers.
  _{NIST · JSON, API · ~10 GB uncompressed · 290,000+ CVE records · US Government public domain · Direct download · Commercial use permitted}
• CVE List V5 - The authoritative CVE Program record set in JSON 5.0 format, published directly to Git so every vulnerability disclosure has a full commit history.
  _{CVE Program / MITRE · JSON, Git · ~5 GB · 290,000+ CVE records · CVE Program Terms of Use · Direct download · Commercial use permitted}

Network Traffic & Intrusion Detection

Labelled packet captures and flow records used to build and benchmark intrusion detection systems.

• CIC-IDS2017 - Five days of realistic background traffic interleaved with labelled brute-force, DoS, web-attack, infiltration and botnet scenarios, published as both raw PCAPs and extracted flow features.
  _{Canadian Institute for Cybersecurity, University of New Brunswick · PCAP, CSV · ~50 GB · 2,800,000+ labelled flows · Free for research use with citation · Free account required · Non-commercial only}
• UNSW-NB15 - Synthetic-but-realistic network traffic covering nine attack families, with 49 engineered flow features and a fixed train/test split that has become a standard IDS benchmark.
  _{UNSW Canberra Cyber · PCAP, CSV · ~100 GB raw, ~500 MB feature CSVs · 2,540,044 flow records · Free for academic research with citation · Direct download · Non-commercial only}

Logs & Observability

Production system logs and workload traces for log parsing, anomaly detection and performance benchmarking.

• Loghub - System logs from distributed systems, supercomputers, operating systems, mobile devices and servers, with hand-labelled templates for a subset, aimed at log parsing and anomaly detection.
  _{LogPAI · Plain text, CSV · ~77 GB · 16 log collections, billions of lines · Freely available for research; individual collections retain source terms · Direct download · Commercial use: check terms}
• Elasticsearch Rally Tracks - The standard benchmarking workloads for Elasticsearch, bundling real document corpora such as HTTP server logs, geonames and NYC taxi rides with the query mixes run against them.
  _{Elastic · JSON, Bzip2 · Up to ~200 GB per track · Varies by track (millions to billions of documents) · No single license; chosen per track, typically matching the terms of the source data · Direct download · Commercial use: check terms}

Databases & Query Benchmarking

Standard schemas, data generators and query workloads for comparing analytical and transactional database engines.

• TPC-H - The decision-support benchmark that defines a fixed eight-table schema, a scalable data generator and 22 analytical queries, used to compare query engines at scale factors from 1 GB to 100 TB.
  _{Transaction Processing Performance Council · Data generator (C), SQL · Configurable (1 GB to 100 TB) · ~8.7M rows at scale factor 1 · TPC End User License Agreement (free download) · Direct download · Commercial use: check terms}
• ClickBench - A single wide table of anonymised web analytics hits plus 43 typical aggregation queries, designed to compare analytical databases on realistic clickstream workloads.
  _{ClickHouse · Parquet, CSV, TSV · ~15 GB Parquet (~75 GB raw) · 99,997,497 rows · Apache-2.0 · Direct download · Commercial use permitted}

Web Crawl & Internet-Scale Text

Petabyte-scale crawls of the public web and the cleaned text corpora derived from them.

• Common Crawl - A monthly crawl of the public web hosted free on S3, published as raw WARC responses plus extracted plain text and link graphs, and the upstream source for most open LLM pretraining corpora.
  _{Common Crawl Foundation · WARC, WET, WAT, Parquet · ~100 TB per monthly crawl · ~3 billion pages per crawl · Common Crawl Terms of Use · Direct download · Commercial use: check terms}
• C4 (Colossal Clean Crawled Corpus) - A heavily filtered English subset of Common Crawl created for training T5, with deduplication, boilerplate removal and language detection already applied.
  _{Google / AllenAI · JSON, GZIP, Parquet · ~305 GB (en) · ~365 million documents (en) · ODC-BY 1.0; underlying text subject to Common Crawl terms · Direct download · Commercial use: check terms}

Natural Language Processing

Annotated text corpora for classification, question answering and language-understanding benchmarks.

• Large Movie Review Dataset (IMDB) - Polar movie reviews split evenly into positive and negative classes with a balanced train/test partition, the default smoke test for text classification pipelines.
  _{Stanford AI Lab · Plain text, TAR.GZ · ~80 MB · 50,000 labelled reviews plus 50,000 unlabelled · Research use; see dataset README · Direct download · Commercial use: check terms}
• SQuAD 2.0 - Crowd-written questions over Wikipedia paragraphs, combining answerable questions with adversarially written unanswerable ones so models must learn to abstain.
  _{Stanford NLP Group · JSON · ~45 MB · 150,000 questions (100,000 answerable, 50,000 unanswerable) · CC BY-SA 4.0 · Direct download · Commercial use permitted}

Computer Vision

Labelled image collections for classification, detection and segmentation.

• COCO (Common Objects in Context) - Everyday scenes annotated with object instance masks, keypoints, panoptic segments and five captions per image, the reference benchmark for detection and segmentation.
  _{COCO Consortium · JPEG, JSON · ~25 GB (2017 splits) · 330,000 images, 1.5 million object instances, 80 categories · CC BY 4.0 (annotations); images subject to their original Flickr terms · Direct download · Commercial use: check terms}
• CIFAR-10 / CIFAR-100 - Small 32x32 colour images across 10 or 100 balanced classes, small enough to iterate on in minutes and still the standard sanity check for new architectures.
  _{University of Toronto · Pickle, Binary, TAR.GZ · ~170 MB each · 60,000 images per variant · Research use; attribution requested · Direct download · Commercial use: check terms}

Speech & Audio

Transcribed speech corpora for automatic speech recognition and text-to-speech.

• Mozilla Common Voice - Crowd-donated and crowd-validated read speech released into the public domain, covering more than a hundred languages including many with little other ASR data.
  _{Mozilla Foundation · MP3, TSV · Varies by language (tens of GB for English) · 30,000+ validated hours across 130+ languages · CC0 1.0 · Free account required · Commercial use permitted}
• LibriSpeech - Read English audiobook speech from LibriVox, force-aligned to its Project Gutenberg source text and split into clean and noisy subsets that define the standard ASR difficulty tiers.
  _{Johns Hopkins University / OpenSLR · FLAC, TXT, TAR.GZ · ~60 GB (all training subsets) · 1,000 hours of 16 kHz speech · CC BY 4.0 · Direct download · Commercial use permitted}

Recommender Systems

User-item interaction logs and reviews for collaborative filtering and ranking research.

• MovieLens 25M - Star ratings and free-text tags collected from the MovieLens service over 24 years, the most widely used benchmark for collaborative filtering.
  _{GroupLens Research, University of Minnesota · CSV, ZIP · ~250 MB · 25,000,095 ratings from 162,541 users on 62,423 movies · GroupLens usage license: research use only, no redistribution, no commercial use without permission · Direct download · Non-commercial only}
• Amazon Reviews 2023 - Product reviews spanning 1996 to 2023 with ratings, review text, images and rich item metadata, organised into 33 product categories for large-scale recommendation research.
  _{McAuley Lab, UC San Diego · JSONL, Parquet · ~200 GB · 571.54 million reviews, 48.19 million items · Research use; cite the source papers · Direct download · Non-commercial only}

Search & Information Retrieval

Query-document relevance judgements for training and evaluating retrieval and reranking systems.

• MS MARCO - Real anonymised Bing queries paired with human-judged passages and documents, the dataset that made neural ranking models practical to train at scale.
  _{Microsoft · TSV, JSON · ~20 GB · 1,010,916 queries, 8.8 million passages · MS MARCO non-commercial research license · Direct download · Non-commercial only}
• BEIR - A heterogeneous zero-shot retrieval benchmark that wraps 18 existing IR datasets behind one loader and evaluation protocol, so a retriever can be scored across domains without per-dataset plumbing.
  _{UKP Lab, TU Darmstadt · JSONL, TSV · ~50 GB for all subsets · 18 datasets across 9 retrieval tasks · Apache-2.0 (toolkit); each subset retains its own license · Direct download · Commercial use: check terms}

Tabular Machine Learning

Curated repositories of small-to-medium tabular datasets used as standard classification and regression benchmarks.

• UCI Machine Learning Repository - The longest-running public collection of machine learning datasets, hosting the small tabular classics such as Iris, Adult, Wine Quality and Covertype behind a consistent Python API.
  _{University of California, Irvine · CSV, ARFF, Python API · Mostly under 100 MB per dataset · 670+ datasets · CC BY 4.0 for most datasets; check per dataset · Direct download · Commercial use: check terms}
• OpenML - Datasets, tasks, flows and millions of recorded experiment runs behind one REST API, including the curated CC-18 and AutoML benchmark suites for reproducible model comparison.
  _{OpenML Foundation · ARFF, Parquet, REST API · Varies by dataset · 5,000+ curated datasets, 100M+ recorded runs · Varies per dataset; mostly CC BY or public domain · Direct download · Commercial use: check terms}

Time Series & Forecasting

Large collections of time series with held-out horizons for reproducible forecasting evaluation.

• M4 Competition Dataset - One hundred thousand real series at yearly through hourly frequencies drawn from demographics, finance, industry and macro sources, with held-out horizons and every competitor's submitted forecasts.
  _{Makridakis Open Forecasting Center · CSV · ~500 MB · 100,000 time series · Open for research and benchmarking; see repository · Direct download · Commercial use: check terms}
• Monash Time Series Forecasting Archive - Thirty forecasting datasets normalised into a single .tsf format with baseline results for statistical and deep models, covering tourism, traffic, electricity, weather and web traffic.
  _{Monash University · TSF, CSV, Zenodo archives · ~10 GB · 30 datasets, 100,000+ series · CC BY 4.0 · Direct download · Commercial use permitted}

Fraud & Anomaly Detection

Heavily imbalanced transaction datasets for fraud scoring and rare-event detection.

• Credit Card Fraud Detection - Two days of European cardholder transactions with PCA-anonymised features and a 0.172% positive rate, the canonical teaching example for extreme class imbalance.
  _{Machine Learning Group, ULB / Worldline · CSV · ~150 MB · 284,807 transactions, 492 frauds · Database Contents License (DbCL) v1.0 · Free account required · Commercial use: check terms}
• IEEE-CIS Fraud Detection - Real e-commerce transactions from Vesta with 400+ engineered features spanning device, card, address and identity signals, plus a separately joined identity table.
  _{IEEE Computational Intelligence Society / Vesta Corporation · CSV · ~1.2 GB · 590,540 training transactions · Competition rules; non-commercial research use · Free account required · Non-commercial only}

Graphs & Networks

Social, citation and web graphs for network analysis and graph neural network benchmarks.

• SNAP Large Network Dataset Collection - Social, citation, collaboration, web, road and communication networks in a uniform edge-list format, ranging from thousands to billions of edges.
  _{Stanford Network Analysis Project · Edge list, TXT.GZ · KB to hundreds of GB per network · 100+ network datasets · BSD-3-Clause (library); datasets vary, cite the source paper · Direct download · Commercial use: check terms}
• Open Graph Benchmark - Large realistic graph benchmarks for node, link and graph property prediction, shipping with standardised non-random splits and an automated evaluator to stop leaderboard drift.
  _{Stanford University · Python package, NPZ, CSV · MB to ~100 GB (OGB-LSC) · 15+ datasets up to 100M+ nodes · MIT (package); datasets carry individual licenses such as ODC-BY and CC BY · Direct download · Commercial use: check terms}

Geospatial & Mapping

Open map data and cartographic base layers for geocoding, routing and visualisation.

• OpenStreetMap Planet - The complete crowd-mapped world as a single weekly export of nodes, ways and relations, plus daily and minutely diffs for keeping a local mirror current.
  _{OpenStreetMap Foundation · PBF, XML.BZ2 · ~80 GB (PBF) · 9 billion+ nodes · ODbL 1.0 (share-alike; attribution required) · Direct download · Commercial use permitted}
• Natural Earth - Public-domain vector and raster base maps at three cartographic scales, with boundaries, coastlines, rivers, cities and shaded relief built to line up cleanly across layers.
  _{Natural Earth / NACIS · Shapefile, GeoPackage, GeoTIFF · ~1 GB for the full set · Vector and raster layers at 1:10m, 1:50m and 1:110m · Public domain · Direct download · Commercial use permitted}

Finance & Economics

Official macroeconomic and development indicators published by central banks and international bodies.

• FRED (Federal Reserve Economic Data) - Macroeconomic time series aggregated from more than a hundred official sources, covering rates, employment, prices and output, with a well-documented REST API and vintage-aware ALFRED archive.
  _{Federal Reserve Bank of St. Louis · CSV, XLS, REST API · Varies by series · 800,000+ time series · FRED Terms of Use; many series carry upstream copyright restrictions · Direct download · Commercial use: check terms}
• World Bank Open Data - Development indicators for every country from 1960 onward, covering GDP, population, health, education, energy and trade, all available in bulk and through a free API.
  _{The World Bank · CSV, XML, REST API · ~200 MB for the full WDI archive · 1,400+ indicators, 217 economies · CC BY 4.0 · Direct download · Commercial use permitted}

E-commerce & Retail

Transaction and basket data for demand forecasting, market basket analysis and customer segmentation.

• Online Retail II - Every transaction of a UK online gift retailer over two years, with invoice, product, quantity, price, customer and country columns, widely used for RFM segmentation and basket analysis.
  _{UCI Machine Learning Repository · XLSX, CSV · ~45 MB · 1,067,371 transactions (Dec 2009 - Dec 2011) · CC BY 4.0 · Direct download · Commercial use permitted}
• Instacart Market Basket Analysis - Anonymised grocery orders with per-order product sequences, reorder flags, aisle and department taxonomies, and day-of-week and hour-of-day signals for next-basket prediction.
  _{Instacart · CSV · ~700 MB · 3.4 million orders from 200,000+ users · Instacart Data Sharing Agreement; non-commercial use · Free account required · Non-commercial only}

Transportation & Mobility

Trip records and transit schedules for routing, demand modelling and urban analytics.

• NYC TLC Trip Record Data - Every yellow taxi, green taxi, for-hire and high-volume for-hire trip in New York City since 2009, with pickup and dropoff timestamps, zones, distances and fare breakdowns.
  _{NYC Taxi and Limousine Commission · Parquet · ~50 GB for the full history · Billions of trips since 2009 · NYC Open Data Terms of Use · Direct download · Commercial use permitted}
• Mobility Database - A catalogue of public transit feeds worldwide in GTFS and GTFS-Realtime, with a stable API and validation reports so schedule and vehicle-position data can be pulled without hunting agency websites.
  _{MobilityData · GTFS, GTFS-RT, REST API · Varies by feed · 2,000+ transit feeds worldwide · Catalogue CC0; individual feeds carry their agency licenses · Free account required · Commercial use: check terms}

By country

449 datasets across 30 topics and 12 countries, two countries per topic. Each topic links to its own page.

Topic	Countries	Datasets
Addresses & Geocoding	Germany, France, United States	15
Agriculture & Crops	Brazil, India, United States	15
Air Quality & Pollution	European Union, India, United States	15
Aviation & Flights	Australia, European Union, United States	14
Banking & Payments	Brazil, Germany, United States	15
Census & Demographics	United Kingdom, India, United States	15
Company Registries	Germany, United Kingdom, India	15
Courts & Legal	United Kingdom, India, United States	15
Crime & Policing	United Kingdom, India, United States	15
Earthquakes & Natural Hazards	European Union, Japan, United States	15
E-commerce & Retail	Brazil, United Kingdom, United States	15
Education & Schools	United Kingdom, India, United States	15
Elections & Voting	United Kingdom, India, United States	15
Energy & Electricity	Germany, France, United Kingdom	15
Financial Markets	United Kingdom, India, United States	15
Government Spending & Budgets	United Kingdom, India, United States	15
Labour & Employment	Canada, United Kingdom, United States	15
Patents & Trademarks	European Union, Japan, United States	15
Public Health	United Kingdom, India, United States	15
Public Transit	Germany, France, United States	15
Real Estate & Housing	Australia, United Kingdom, United States	15
Road Safety & Accidents	France, United Kingdom, United States	15
Satellite & Earth Observation	European Union, Japan, United States	15
Taxi & Ride-Hailing	United Kingdom, Singapore, United States	15
Telecom & Broadband	United Kingdom, India, United States	15
Tourism & Hospitality	Spain, Japan, United States	15
Trade & Customs	European Union, India, United States	15
Vehicles & Car Sales	United Kingdom, India, United States	15
Water & Sanitation	United Kingdom, India, United States	15
Weather & Climate	Australia, United Kingdom, United States	15

How this repo is laid out

data/datasets.json                        global datasets (source of truth)
data/countries.json                       country code -> display name
data/schema.json                          JSON Schema for both catalogs
topics/<topic>/topic.json                 topic title and description
topics/<topic>/<country>/dataset-NN.json  one file per country-specific dataset
topics/<topic>/README.md                  generated
data/*.csv                                generated

One dataset per file is deliberate: two people adding datasets to the same country never touch the same file, so their pull requests do not conflict.

Using the catalog

python3 scripts/validate.py           # structure, referential integrity, style rules
python3 scripts/validate.py --links   # also verify every URL resolves
python3 scripts/build.py              # regenerate READMEs and CSVs
python3 scripts/new_dataset.py --help # scaffold a new entry

Load everything in a few lines:

import json, pathlib

catalog = json.loads(pathlib.Path("data/datasets.json").read_text())
country = [json.loads(p.read_text())
           for p in pathlib.Path("topics").glob("*/*/dataset-*.json")]

# Everything you can use commercially without an account.
usable = [d for d in catalog["datasets"] + country
          if d["access"] == "open" and d["commercial_use"] == "yes"]
print(len(usable), "datasets are open access and commercially usable")

Or skip the JSON and open data/datasets.csv and data/country-datasets.csv in anything.

Field meanings are documented in data/schema.json. Two are worth calling out:

• access — open (direct download), registration (free account), agreement (signed terms), credentialed (identity or ethics vetting).
• commercial_use — yes, no, or check when terms are mixed across components, which is common for corpora that bundle third-party content under its original license.

Contributing

Contributions are welcome, and corrections are as valuable as additions. Read contributing.md — entries are added as JSON files, never to the generated lists directly. New contributors should also read the code of conduct.

Sources and credits

Every dataset here is hosted and maintained by the organisation named in its entry. Full credit for collection, curation and hosting belongs to them.

National statistical offices and government agencies — the single largest source in this list. Among them: the US Census Bureau, Bureau of Labor Statistics, NOAA, USGS, NASA, EPA, NIST, NHTSA, FCC, SEC, USPTO and USDA in the United States; the Office for National Statistics, HM Land Registry, Companies House, Ofcom, NHS, Ordnance Survey and the Home Office in the United Kingdom; the Ministry of Statistics and Programme Implementation, Reserve Bank of India, Election Commission of India, Central Pollution Control Board and the National Informatics Centre in India; and their counterparts in Germany, France, Spain, Brazil, Canada, Australia, Japan and Singapore.

International bodies — Eurostat, the European Space Agency and Copernicus programme, the European Central Bank, EUROCONTROL, the European Patent Office, EUIPO, The World Bank, the FAO and the UN Statistics Division.

Universities and research groups — Stanford (SNAP, OGB, SQuAD, IMDB reviews, SEDA), Minnesota (GroupLens, IPUMS), UC Irvine (UCI ML Repository), UC San Diego (McAuley Lab), UC Davis (BugSwarm), Toronto (CIFAR), Johns Hopkins (LibriSpeech), TU Darmstadt (BEIR), Monash (Forecasting Archive), Michigan (ICPSR), Harvard (Dataverse, Caselaw Access Project), MIT (Election Lab), Oxford (OpenPrescribing), Imperial College (UK-DALE), Ashoka (Lok Dhaba), New Brunswick (CIC), UNSW (UNSW-NB15), ICRISAT, NIED, Development Data Lab, Defects4J and LogPAI.

Companies, foundations and community projects — GitHub, Microsoft, Google and AllenAI, Elastic, ClickHouse, Mozilla, Deutsche Bahn, Ookla, Olist, Instacart, Vesta and IEEE-CIS, Worldline and the ULB Machine Learning Group, Common Crawl, OpenStreetMap, Overture Maps, OpenML, MobilityData, Natural Earth and NACIS, GeoNames, OpenAQ, PurpleAir, Measurement Lab, RIPE NCC, OpenSky Network, Open Food Facts, Inside Airbnb, OpenCorporates, OpenSecrets, Free Law Project, Association for Democratic Reforms, ASER Centre, the COCO Consortium, the Makridakis Open Forecasting Center, Open Power System Data, Fraunhofer ISE and GH Archive.

Hosting platforms — Kaggle, Hugging Face, Zenodo, OpenSLR, Harvard Dataverse and AWS Open Data distribute several of these on behalf of their creators. Where a dataset is reachable through more than one host, the entry links to the host whose terms are clearest.

The cite field on each entry gives the publication its authors ask you to reference. Cite the dataset, not this list.

Related lists

• awesome-public-datasets - Broad topic-first index of open datasets across the sciences.
• awesome-json-datasets - Public JSON endpoints and API-accessible data.
• awesome-machine-learning - Machine learning frameworks and libraries by language.
• awesome-opensource-data-engineering - Open-source tooling for moving and processing the data above.