Molecular property prediction

Predict a molecule's properties from its atom–bond graph with a GNN and a permutation-invariant readout; benchmarks are MoleculeNet, QM9 and OGB, and scaffold splits test generalisation to unseen chemotypes, which random splits overstate.

8 min read789 wordsadvanced0 viewsupdated 2026-10-21

Definition

Molecular property prediction is the supervised task of mapping a molecule to a measured or computed property: a quantum-chemical energy, solubility, lipophilicity, binding to a protein target, or toxicity. With graph neural networks the molecule is a graph of atoms (nodes) and bonds (edges), and the prediction is made at graph level. The standard benchmark suites are MoleculeNet (Wu et al., Chemical Science 2018) and the molecular datasets of the Open Graph Benchmark (Hu et al., NeurIPS 2020).

The input usually arrives as a SMILES string such as Oc1ccccc1 (phenol). A toolkit like RDKit parses it into heavy atoms and bonds. Hydrogens are normally kept only as a per-atom count. The output is one number per task: a regression value, or a logit for each of several binary assays.

The task is hard for four reasons. Labelled sets are small: hundreds to tens of thousands of molecules, because every label is an experiment or a DFT calculation. Multi-task assay data has missing labels. Tiny structural edits can flip activity ("activity cliffs"). And the molecules a chemist wants scored next usually come from new chemical series, not the ones already measured.

Before GNNs the standard approach was a fixed fingerprint fed to a random forest or SVM. The most used is ECFP (extended-connectivity fingerprints, Rogers and Hahn, 2010), which hashes circular atom neighbourhoods of radius 2 or 3 into a bit vector. ECFP plus a random forest is still the baseline that every new model must beat.

From molecule to prediction

Each atom gets a feature vector: element (one-hot), degree, formal charge, number of hydrogens, hybridisation, aromaticity, ring membership and chirality. Each bond gets its type (single, double, triple, aromatic), conjugation, ring membership and stereo. A GNN runs TT rounds of message passing (see Message Passing Neural Networks (MPNN)), a readout pools the atoms, and an MLP head outputs one value per task:

y^=MLP(∑v∈GhvT)∈RK,L=1∑i,kmik∑i∑k=1Kmik BCE(yik, σ(y^ik))\hat{\mathbf y}=\mathrm{MLP}\Big(\sum_{v\in G} h_v^T\Big)\in\mathbb R^{K},\qquad \mathcal L=\frac{1}{\sum_{i,k} m_{ik}}\sum_{i}\sum_{k=1}^{K} m_{ik}\,\mathrm{BCE}\big(y_{ik},\,\sigma(\hat y_{ik})\big)
hvTh_v^T
state of atom vv after TT message-passing rounds; the sum may be replaced by a mean or set2set
KK
number of tasks, e.g. 12 assays in Tox21
mikm_{ik}
1 if molecule ii has a label for task kk, else 0; missing labels contribute no loss
molecule → graph → GNN → readout → per-task predictionSMILES Oc1ccccc1(phenol)CCCCCCOC1C2scaffold: benzeneatom and bond features (RDKit)x_O = [O, deg 1, H 1, arom 0, ring 0]x_C1 = [C, deg 3, H 0, arom 1, ring 1]x_C2 = [C, deg 2, H 1, arom 1, ring 1]e_O–C1 = [single, ring 0]e_C1–C2 = [aromatic, ring 1]7 heavy atoms, 7 bonds; H kept as a countGNN encoderT rounds of message passingreadout h_GΣ_v h_v, mean or set2sethead: MLP → ŷone output per taskregression: MAE / RMSEclassification: masked BCE, ROC-AUCscaffold split: group by Bemis–Murcko scaffold, largest groups fill train firsttrain (80%)benzene: phenol, toluene, aniline, benzoic acidfuran: furfuralindole: tryptophanvalid (10%)pyridine: 3-methylpyridinetest (10%)cyclohexane: cyclohexanola random split would put toluene in train and phenol in test: same scaffold on both sides
fig.Top: phenol (SMILES Oc1ccccc1) becomes a graph of 7 heavy atoms and 7 bonds. Each atom and bond carries a feature vector. A GNN encoder, a permutation-invariant readout and an MLP head give one output per task. Bottom: the scaffold split groups molecules by Bemis–Murcko scaffold and assigns whole groups, largest first, to train, then validation, then test. No test scaffold is ever seen in training.

Model choices are covered in their own articles. Message Passing Neural Networks (MPNN) were designed for QM9. The directed MPNN of Chemprop (Yang et al., 2019) passes messages along directed bonds and never sends a message straight back along the bond it arrived on. Graph Isomorphism Network (GIN) is the usual backbone for pre-training. On QM9, 3D models such as SchNet (Schütt et al., 2017) add interatomic distances. Energies depend on geometry, so this helps a lot there.

Splits and metrics

A random split puts close analogues of each test molecule in the training set. It measures interpolation within known chemical series. A scaffold split removes side chains to get each molecule's Bemis–Murcko scaffold (Bemis and Murcko, 1996): its ring systems plus the linkers between them. It then sorts the scaffold groups by size and fills train (80%) with the largest groups, then validation (10%), then test (10%). The test set therefore holds rare, unseen scaffolds. This is harder and closer to prospective use.

MAE=1n∑i∣y^i−yi∣,AUC=1∣P∣ ∣N∣∑i∈P∑j∈N(1[si>sj]+12 1[si=sj])\mathrm{MAE}=\frac1n\sum_i|\hat y_i-y_i|,\qquad \mathrm{AUC}=\frac{1}{|P|\,|N|}\sum_{i\in P}\sum_{j\in N}\Big(\mathbb 1[s_i>s_j]+\tfrac12\,\mathbb 1[s_i=s_j]\Big)
P, NP,\ N
labelled positives and negatives of one task; AUC is computed per task and then averaged
sis_i
predicted score of molecule ii

For energies, "chemical accuracy" means an error below 1 kcal/mol (about 0.043 eV). For sparse, highly imbalanced assays such as PCBA, average precision is reported instead of ROC-AUC, because a ROC-AUC near 1 can hide poor precision on the rare actives.

DatasetMoleculesTasksTargetMetricSplit
QM9133,88512DFT quantum propertiesMAErandom
ESOL1,1281aqueous solubilityRMSErandom
FreeSolv6421hydration free energyRMSErandom
Lipophilicity4,2001octanol/water distributionRMSErandom
Tox217,83112toxicity assaysROC-AUCrandom
HIV41,1271inhibits HIV replicationROC-AUCscaffold
BACE1,5131BACE-1 inhibitionROC-AUCscaffold
ogbg-molpcba (OGB)437,929128bioassaysAPscaffold
ZINC subset12,0001constrained solubilityMAEfixed 10k/1k/1k
Common benchmarks (MoleculeNet unless noted) with their standard metric and split
random split, 80 train / 20 test · 100 molecules in 20 scaffold familiesone cell per molecule, grouped by scaffold (largest family left)traintest-2-2002244true y (test molecules)predicted ŷ (3-NN, Tanimoto)test MAE, 3-NN0.70vs 1.50 for ŷ = train meanmax Tanimoto to train, mean0.66test scaffold also in train100%20 test molecules from 11 scaffoldsscaffold seen in trainingunseen scaffoldy = scaffold effect + Σ substituenteffects + noise; fingerprint = 8 corebits ∪ 3 substituent bits (of 64)
split
fig.A synthetic library of 100 molecules in 20 scaffold families, scored by a 3-nearest-neighbour regressor on Tanimoto similarity. Default (random split, seed 1): all 20 test molecules have their scaffold in training, the mean similarity to the nearest training molecule is 0.66, and the test MAE is 0.70, against 1.50 for predicting the training mean. Over seeds 1–10 the random-split MAE ranges from 0.49 to 1.11. Switch to the scaffold split: the 20 test molecules come from 10 unseen scaffolds, the similarity drops to 0.25, and the MAE rises to 1.70. That is worse than the 0.94 of the training mean.

Worked example: masked loss and per-task AUC

Moleculey_Ap_ABCE_Ay_Bp_BBCE_B
110.900.105–0.30–
200.200.22310.800.223
310.600.51100.400.511
400.701.20400.200.223
5–0.50–10.351.050
600.100.105–0.60–
Six molecules, two assays (A, B); a dash marks a missing label
  1. 1Masked loss: 9 of 12 entries are labelled. Their BCE terms sum to 4.155, so L=4.155/9=0.462\mathcal L=4.155/9=0.462. Predictions 0.50, 0.30 and 0.60 get no gradient.
  2. 2AUC, task A: positives {0.90, 0.60}, negatives {0.20, 0.70, 0.10}. Of the 6 pairs, 5 rank the positive higher; only 0.60 < 0.70 fails. AUC = 5/6 = 0.833.
  3. 3AUC, task B: positives {0.80, 0.35}, negatives {0.40, 0.20}. 3 of the 4 pairs are ordered correctly, since 0.35 < 0.40. AUC = 3/4 = 0.750.
  4. 4The reported score is the mean over tasks: (0.833 + 0.750)/2 = 0.792.
python
1from collections import defaultdict
2from rdkit.Chem.Scaffolds import MurckoScaffold
3
4def scaffold_split(smiles, frac_train=0.8, frac_valid=0.1):
5 groups = defaultdict(list)
6 for i, s in enumerate(smiles):
7 groups[MurckoScaffold.MurckoScaffoldSmiles(smiles=s, includeChirality=False)].append(i)
8 fams = sorted(groups.values(), key=lambda g: (len(g), g[0]), reverse=True) # largest first
9 n = len(smiles)
10 train, valid, test = [], [], []
11 for g in fams:
12 if len(train) + len(g) <= frac_train * n:
13 train += g
14 elif len(train) + len(valid) + len(g) <= (frac_train + frac_valid) * n:
15 valid += g
16 else:
17 test += g
18 return train, valid, test
19
20smiles = ["Oc1ccccc1", "Cc1ccccc1", "Nc1ccccc1", "OC(=O)c1ccccc1", # benzene scaffold
21 "OC1CCCCC1", "Cc1cccnc1", "NC(Cc1c[nH]c2ccccc12)C(=O)O", "O=Cc1ccco1"]
22print(scaffold_split(smiles)) # ([0, 1, 2, 3, 7, 6], [5], [4])
A Bemis–Murcko scaffold split with RDKit, using the same ordering rule as DeepChem's ScaffoldSplitter. Not run here, since RDKit was not available; the split logic was checked separately on these scaffold groups.

Use when

  • +GNNs learn task-specific substructure features instead of a fixed hash.
  • +One encoder can serve many tasks through a multi-task head with masked loss.
  • +Self-supervised pre-training on unlabelled molecules (Hu et al., ICLR 2020) helps on small datasets.

Avoid when

  • −On small datasets, ECFP plus a random forest is often as good or better.
  • −2D graphs ignore conformation, and 3D models need reliable geometries.
  • −Out-of-distribution generalisation to new scaffolds remains the main gap.

Where it's used

Virtual screening

rank millions of purchasable compounds for activity before any are made

ADMET filters

solubility, permeability, toxicity and metabolic stability flags in lead optimisation

DFT surrogates

QM9-style models predict energies and orbital gaps far faster than the DFT calculation

Check yourself

Why does a scaffold split usually give a lower score than a random split on the same dataset?

Check yourself

In multi-task Tox21 training, how should missing assay labels be handled?

Remember

  • ✓Molecule → RDKit graph (atom and bond features) → GNN → permutation-invariant readout → one output per task.
  • ✓Multi-task assays use a masked BCE: ∑mik BCE/∑mik\sum m_{ik}\,\mathrm{BCE}/\sum m_{ik}. Regression uses MAE or RMSE; classification uses ROC-AUC, or AP when actives are rare.
  • ✓Benchmarks: QM9 (133,885 molecules), ESOL, FreeSolv, Lipophilicity, Tox21, HIV, BACE, OGB molhiv and molpcba, ZINC.
  • ✓A scaffold split groups by Bemis–Murcko scaffold and fills train with the largest groups, so test scaffolds are unseen.
  • ✓Random splits overstate accuracy for new chemistry. Average over seeds on small sets.
  • ✓ECFP plus a random forest is a strong baseline. 3D models such as SchNet matter for quantum targets.
← back to Deep Learning roadmap