Datasets

Every dataset below is openly available, has a citable source, and has enough documentation that you can tell what the columns mean. They are grouped by flavor rather than by method, because the useful question is “what kind of data is this?” before it is “what model should I fit?”

Scale and license are listed wherever the source states them. Check both before you plan around a dataset — a benchmark licensed for evaluation only, or a 1.7 GB multi-format download, changes what the work looks like.

Note

None of these are used in the chapters of this book, and none are derived from it. That is deliberate: working on data you have not already seen analyzed is most of the point.

A few entries are published in repositories that carry no license file. They are included because the data is genuinely useful and its provenance is clear, but the absence is called out in the table, and it means you should ask the authors before redistributing the data or releasing anything derived from it.

The Data column records how each dataset was produced: Measured for instrument, plant, or laboratory data; Simulated for the output of a model, whether a process simulator, a quantum-chemistry calculation, or a building-energy model; Mixed where a single collection holds both; and See source where the source does not say plainly enough to label it. The distinction matters more than it looks. Simulated data has no sensor dropout, no drift, and no unexplained outliers, so a method that works on it has not yet met the thing that makes real data hard.

The scale, license, and provenance details below are summarized from each source and have not all been independently verified — see how this section was compiled before relying on them.

Process data and time series

These are the datasets that look most like industrial practice: many correlated sensors, autocorrelation in time, batch or run structure, missing values, and faults. They are also the ones where a careless random train/test split will quietly invalidate your results.

Dataset	Data	Good for	Scale and notes	Source
Combined Cycle Power Plant	Measured	Multivariate regression; a clean first target with real measurements	9,568 rows; 4 inputs (ambient temperature, ambient pressure, relative humidity, exhaust vacuum), 1 output (net hourly electrical energy output)	UCI 294 · 10.1016/j.ijepes.2014.02.027
Tennessee Eastman process simulation	Simulated	Fault detection and classification, PCA, anomaly detection, time-aware validation	500 runs, 52 variables, 20 fault types. Simulated, so it is clean by the standards of real plant data; split by run and by time	10.7910/DVN/6C3JR1
PRONTO multiphase flow benchmark	Measured	Process monitoring and fault detection with heterogeneous data types	About 1.7 GB across several data types — meaningfully more setup than a CSV	Zenodo 1341583
Fluid catalytic cracking operating data	See source	Data-driven modeling and control of a refinery unit	Released with the paper; see its data availability statement	10.1016/j.compchemeng.2022.107900
IndPenSim penicillin fed-batch	Simulated	Bioprocess modeling, batch-wise validation, control and optimization studies	Industrial-scale simulated fed-batch fermentation, released as multiple batches	Mendeley Data · simulator paper 10.1016/j.jbiotec.2014.10.029
SECOM semiconductor manufacturing	Measured	Imbalanced classification, missing data, aggressive feature selection	1,567 rows × 591 sensor features. Many missing values and very few failure cases — the class imbalance is the problem. CC BY 4.0	UCI 179
NASA C-MAPSS turbofan degradation	Simulated	Remaining useful life, predictive maintenance, distribution shift between train and test	708 training and 707 test engine trajectories across four sub-datasets (FD001–FD004) with different fault modes and operating conditions. The catalog does not state an explicit license	data.nasa.gov · Saxena et al. 2008
Paracetamol batch crystallization under control	Measured	Batch-structured time series; soft-sensor regression; data from a closed control loop	Five experimental controlled runs, 243–772 rows × 19 columns each, plus separate instrument training sets (1,054 rows). Every run is a batch, so a split that mixes runs leaks. Small enough to open directly in pandas. MIT	GitHub · 10.1021/acs.iecr.5c03894
Nuclear waste slurry fault detection	Measured	Fault detection on measured rather than simulated data; fusing three instruments	Raman, ATR-FTIR, and particle-size measurements from simulant slurry processing runs, distributed as one 7 MB archive alongside the analysis scripts. A measured counterpart to Tennessee Eastman, and a rare thing: real process data with real faults in it. MIT	GitHub · 10.1002/aic.70234

Spectroscopy and soft sensors

Spectra are the most common process analytical measurement in chemical engineering, and they have a shape almost nothing else in this catalog has: far more features than samples. A few dozen spectra against sixteen hundred Raman shifts is ordinary. You cannot fit ordinary least squares to data like that at all, which makes these the natural companions to High-dimensional Data, High-dimensional Regression, and Dimensionality Reduction.

The recurring application is the soft sensor: an instrument measures something fast and easy — a spectrum, a chord length distribution — and a calibration model converts it into the quantity you actually care about, such as a concentration or a crystal size distribution. The model is the sensor, which means how you validate it is a process decision and not merely a statistic.

Most entries below come from the Grover lab at Georgia Tech, whose data repositories are collected at github.com/magrover.

Dataset	Data	Good for	Scale and notes	Source
Paracetamol ATR-FTIR in mixed solvent	Measured	Calibration model development; PLS against PCA + neural network; in-run calibration strategy	247 spectra × 464 wavenumber channels, each labeled with ethanol weight fraction, temperature, and paracetamol concentration. Spans ethanol/water mixtures and deliberately includes probe-to-probe variability, which is what makes the calibration hard. CC BY 4.0	GitHub · 10.1021/acs.oprd.5c00338
Nuclear waste simulant FTIR	Measured	Quantification when unmodeled species are present; blind source separation, MCR-ALS, PLS	3,902 spectra × 155 wavenumbers from a 2 L non-radioactive process run, plus 8 training spectra, 5 single-species references, and ion-chromatography concentrations for nitrate and nitrite. The README documents every file and column — rarer than it should be. No license file	GitHub · 10.3389/fnuen.2023.1295995
Multicomponent slurry quantification	Measured	Comparing two instruments on one problem; multi-species quantification in a dense slurry	48 ATR-FTIR spectra × 243 wavenumbers and 66 Raman spectra × 1,601 shifts, each paired with gravimetric concentrations for five species. The two instruments have different sample sets, so they cannot simply be concatenated. No license file	GitHub · 10.1021/acs.iecr.3c01249

Images and microscopy

The only image data in the chapters of this book is the 8 × 8 handwritten digit set built into scikit-learn, which stands in for image data rather than being an example of it. The datasets below are the real thing, and they come in two flavors: still images of crystals captured through an in-line process probe, and video of nanoparticles moving in a liquid cell. Both connect most directly to Neural Network Architectures.

The four still-image sets are annotated for a specific task, and they carry two access conditions worth settling before you plan around them: downloading requires a free Kaggle account, and all four are CC BY-SA 4.0, whose ShareAlike clause constrains what you may do with a derived dataset. All four come from OpenCrystalData, a maintained open-access database with Georgia Tech co-authors. Start with the EasyViewer set — it is an order of magnitude smaller than the largest.

Dataset	Data	Good for	Scale and notes	Source
EasyViewer in-line images	Measured	A first look at image-based process monitoring, at a size you can actually download	120 in-situ images of wollastonite and L-glutamic acid, with size and chord length information. 0.45 GB — the smallest of the four and the place to start. CC BY-SA 4.0	Kaggle
Crystallization impurity detection	Measured	Anomaly and morphology detection — a classification problem where the classes are shapes	400 raw and 6,000 cropped in-situ images of a cephalexin monohydrate slurry, captured with an EasyViewer-100 probe, with phenylglycine impurity crystals present in some. 1.47 GB. CC BY-SA 4.0	Kaggle
Standard polystyrene microspheres	Measured	Object detection and segmentation against particle-level ground truth	2,300 in-situ images of standard spheres with particle-level annotations. Known, uniform geometry is what makes this the fair test case before you attempt real crystals. 1.72 GB. CC BY-SA 4.0	Kaggle
Ag-Crystal needle images	Measured	Estimating a particle size distribution from images and checking it against offline measurement	3,888 in-situ images of needle-like crystals from an industrial agrochemical process, intended for models whose PSD output is comparable with offline PSD. Needles are far harder than spheres. 5.31 GB — the largest of the four. CC BY-SA 4.0	Kaggle

All four are described together in 10.1016/j.dche.2024.100150, which is the paper to cite and the best guide to what each one contains. Note that the per-dataset DOIs printed in that paper did not resolve when this section was last checked — use the Kaggle links above.

The video set below is a different problem. A still image asks what is in this frame; a video asks that plus which object here is the same object as in the last one, and that second question is where the interesting failures live.

Dataset	Data	Good for	Scale and notes	Source
SAM-EM liquid-phase TEM videos	Mixed	Instance segmentation and single-particle tracking, against ground truth that carries identity across frames	1,000 simulated liquid-cell TEM videos: 50,000 grayscale frames (1024 × 1024, 0.25 nm per pixel) with 50,000 matching instance masks, spanning liquid layer thickness 5–170 nm and electron flux 12.5–187.5. Plus a 3,600-frame held-out set over 12 layer and flux combinations, and two real experimental videos (237 and 75 frames) that carry no masks. 102 GB in total, but every piece downloads separately and the useful ones are small. No license file	10.57967/hf/9700

Download one piece, not the repository. That 102 GB is the whole release, most of it a 64 GB ablation training set and a 2.7 GB model checkpoint. Individual files can be downloaded on their own, and the two to start with are holdoutValidationSet-4ParticlesOverlapping (30 MB: one 50-frame video of four overlapping particles, with masks — note that it is a zip archive despite having no file extension) and experimentalAggregation.zip (3 MB: 237 frames of a real aggregation event, without masks). The full labeled validation set, holdoutValidationSet.zip, is 2.3 GB.

The annotations are instance masks, not binary masks. Each is a PNG the size of its frame: black background, one distinct color per particle, and the color assigned to a particle holds across every frame of its video. Two consequences worth planning around. You can recover per-particle trajectories from the labels alone, with no model involved, which makes this a mean-squared-displacement problem as much as a segmentation one (Time Series Basics). And a model can be scored on whether it kept each particle’s identity, not just on how well its outlines overlap — which is the harder question, and the one this dataset exists to ask.

The labeled videos are simulated and the real videos are unlabeled. Every mask here belongs to a simulated video; the two experimental videos ship as raw frames. So a supervised result is trained and validated entirely on simulator output, and the experimental frames are where you find out whether it transferred. The caution in the note at the top of this page applies with unusual force, because here the simulator is what defines the noise you are learning to see through.

The paper is 10.1039/d6dd00211k (Digital Discovery, 2026, from a Georgia Tech group). The analysis code and a desktop application are at JamaliLab/SAM-EM under Apache 2.0, and that repository carries a 50-frame worked example, so you can see the file layout before downloading anything. The data itself has no license file and no dataset card — its documentation lives in the paper and the code repository rather than alongside the data — so ask the authors before redistributing it or releasing anything derived from it.

Materials and molecules

Structure-to-property problems, where the interesting question is usually how you represent the input rather than which regressor you use. These connect most directly to High-dimensional Data and Neural Network Architectures.

Dataset	Data	Good for	Scale and notes	Source
QM9	Simulated	Structure-to-property mapping; comparing molecular representations	About 134,000 small organic molecules with properties computed by DFT — atomization energy, internal energy, enthalpy, band gap, dipole moment	10.1038/sdata.2014.22 · quantum-machine.org
Matbench	Mixed	Materials regression and classification, and fair model comparison	13 tasks spanning roughly 300 to 132,000 samples, most of them computed; three are measured — experimental band gap, experimental metallicity, and steel yield strength. Check the terms of each underlying source dataset	Matbench
Open Reaction Database	Measured	Reaction outcome prediction, experiment planning, working with a real schema	More than 2 million reactions. Data CC BY-SA 4.0, software Apache 2.0. Preparing it is more work than reading a CSV	open-reaction-database.org
Polymerization property data	See source	Predictive models for polymer properties and polymerization processes	See the paper’s data availability statement	10.1039/D4PY00995A
Li-ion battery cycle life	Measured	Predicting long-run behavior from early-cycle features; high-dimensional regression at small sample size	Commercial cells cycled to failure, with early-cycle features used to predict eventual cycle life. Few samples, many candidate features	10.1038/s41560-019-0356-8 · related battery-parameter-spaces
PET stabilizer additives	Mixed	Small-data classification and feature selection where the features vastly outnumber the samples	59 additives × 1,875 computed molecular descriptors, labeled with six measured stabilizer performance columns (dry at 1–3 hours, wet at 3–15 days). The extreme shape — thirty times more descriptors than samples — is the problem, and a companion file adds MACCS fingerprints for 10,000 candidate molecules. No license file	GitHub · 10.1021/acsapm.0c00921

Thermophysical properties

Dataset	Data	Good for	Scale and notes	Source
NIST ThermoML Archive	Measured	Property prediction, and practice with structured XML, units, uncertainty, and source tracking	A large XML archive. Availability and reuse terms depend on the original publisher	ThermoML
UCI Energy Efficiency	Simulated	Regression, model comparison, residual diagnostics	768 rows, 8 inputs, 2 targets, produced by building-energy simulation of 768 candidate building shapes. Small and simple by design — a good warm-up, not a process dataset. CC BY 4.0	UCI 242
Henry’s law constants	Mixed	Regression against temperature across a very wide range of solutes	A large curated compilation drawn from the literature — predominantly measured values, alongside estimated and calculated ones	10.5194/acp-23-10901-2023

Inverse problems and generative modeling

Dataset	Data	Good for	Scale and notes	Source
Light-color inverse problem	Measured	Demonstrating and evaluating generative models on an inverse problem with a known forward map	Measured light intensity as a function of red, green, and blue input settings. Small, physically interpretable, and the forward direction is easy — which is what makes the inverse direction a fair test	10.1039/D5DD00137D

Benchmarks for evaluating models

Dataset	Data	Good for	Scale and notes	Source
ChemBench	—	Testing what a language model actually knows about chemistry; comparing tool use against no tool use	More than 2,700 questions. Intended for evaluation, not for training	ChemBench

Finding your own data

The catalog above is a starting point, not a boundary. Two habits make independent searching far more productive.

Read the data availability statement first. Most journals now require one, and it is the fastest way to tell whether a paper’s data is genuinely obtainable. It will point you at a repository record, a supporting information file, or — often enough — a sentence saying the data are available on request, which in practice means budget for a delay. Check this before you get attached to the idea.

Prefer repositories that give you a stable identifier. A DOI or a numbered record means the data you download today is the data someone else downloads next year.

General-purpose and chemistry-specific places to look:

Source	What you will find
Awesome Industrial Datasets (browsable version)	The best single index for this domain: around 190 industrial datasets, each tagged with domain, modality, task, access route, size, year, and license. Roughly twenty are chemical and process datasets, and several of the entries above are catalogued there with more metadata than their original hosts provide
UCI Machine Learning Repository	Several hundred curated, well-documented datasets with stated licenses; several of the entries above live here
Zenodo	General-purpose research data with DOIs, including a great deal of supplementary data from papers
Harvard Dataverse	Institutional repository with DOIs; hosts the Tennessee Eastman record above
Materials Project	Computed properties for a very large number of inorganic materials, with a documented API
NIST Standard Reference Data and WebBook	Authoritative thermophysical and spectroscopic data, with uncertainties
PubChem	Chemical structures, properties, and bioassay data, with a REST API

Several of these are exercised directly in Online Data Access and Accessing Data with AI Tools, so the mechanics of pulling from an API and parsing the response are already covered.