Definition
Message Passing Neural Network (MPNN) is a framework, introduced by Gilmer, Schoenholz, Riley, Vinyals and Dahl in "Neural Message Passing for Quantum Chemistry" (ICML 2017). It describes a graph neural network by three learned functions: a message function computed on every edge, an update function for every node, and a readout that maps the final node states to a graph-level output.
The input is a graph with node features and edge features . For a molecule these are atoms (element, charge, hybridisation, …) and bonds (single, double, aromatic, distance). The output here is one vector per graph: 13 quantum-chemical properties of the molecule.
Three constraints shape the model. Graphs vary in size, so the weights cannot depend on the number of nodes. There is no grid, so a node has no "left neighbour" for a convolution kernel to align with. And the atoms of a molecule have no canonical order, so the output must not change when the nodes are renumbered (permutation invariance).
Before 2017, several models met these constraints in different notation: molecular fingerprints by Duvenaud et al. (2015), Gated Graph Neural Networks (GG-NN, Li et al., 2016), Interaction Networks (Battaglia et al., 2016), Kearnes et al.'s molecular graph convolutions, Deep Tensor Neural Networks (Schütt et al., 2017), and spectral GCNs. Gilmer et al. showed they are all instances of one scheme. They then searched within that scheme for the best variant on QM9.
Message, update, readout
Node states start from the features, . For rounds, every node gathers messages from its neighbours and updates its state:
- neighbours of node
- message function, applied to every edge with shared weights
- update function, shared by all nodes
- the aggregator; any symmetric function (sum, mean, max) keeps the result independent of neighbour order
After rounds a readout turns the set of final node states into the prediction:
- must be invariant to permutations of the nodes, e.g. a sum, a mean or set2set
- number of message-passing rounds; depends only on nodes within hops of
Because and are shared across edges and nodes, the parameter count does not depend on graph size. Because the aggregator and are symmetric, relabelling the nodes permutes the but leaves unchanged. Node-level tasks skip and read directly.
The variant that won on QM9
Gilmer et al. compared several message and readout functions. The best used an edge network: a small MLP maps the edge feature vector to a matrix that multiplies the neighbour's state. Updates used a Gated recurrent unit (GRU) whose weights are shared across all rounds, as in GG-NN. The readout was set2set (Vinyals, Bengio and Kudlur, 2016), an LSTM-driven attention over the node set:
- MLP output reshaped to ; a continuous edge feature such as bond length changes the message smoothly
- gated update: the message is the GRU input, the hidden state
Two further ideas from the paper: a virtual master node connected to every atom, which lets information cross the molecule in two hops, and towers, which split into chunks of size with separate message functions to cut the cost of the edge network.
Setting and results (Gilmer et al., 2017)
- Dataset
- QM9: about 134k (133,885) small organic molecules, up to 9 heavy atoms (C, N, O, F); properties computed with DFT
- Targets
- 13 per molecule, including dipole moment, HOMO, LUMO, gap, zero-point energy, internal energy U₀ and heat capacity
- Split
- 10,000 validation and 10,000 test molecules chosen at random; the rest for training
- Metric
- error ratio = MAE / chemical accuracy for each target; below 1 means within chemical accuracy
- Best model
- edge network + GRU + set2set: chemical accuracy on 11 of the 13 targets
Worked example: two rounds by hand
Take 4 nodes with scalar states and edges A–B, B–D, C–D (weight 1) and B–C (weight 0.5). Use , and a sum readout. The initial readout is .
| Node | h⁰ | messages, round 1 | m¹ | h¹ | messages, round 2 | m² | h² |
|---|---|---|---|---|---|---|---|
| A | 1 | 1·2 | 2 | 1.5 | 1·2 | 2 | 1.75 |
| B | 2 | 1·1 + 0.5·0 + 1·1 | 2 | 2 | 1·1.5 + 0.5·1 + 1·1.5 | 3.5 | 2.75 |
| C | 0 | 0.5·2 + 1·1 | 2 | 1 | 0.5·2 + 1·1.5 | 2.5 | 1.75 |
| D | 1 | 1·2 + 1·0 | 2 | 1.5 | 1·2 + 1·1 | 3 | 2.25 |
| Σ (readout) | 4 | 6 | 8.5 |
In round 1 every node happens to receive , yet the states differ because also sees . Renumbering the nodes would reorder the rows but leave the readouts 6 and 8.5 unchanged.
1import torch2from torch import nn3from torch_geometric.datasets import QM94from torch_geometric.loader import DataLoader5from torch_geometric.nn import NNConv, Set2Set67class MPNN(nn.Module):8 def __init__(self, d_node=11, d_edge=4, d=64, T=3):9 super().__init__()10 self.T = T11 self.embed = nn.Linear(d_node, d)12 A = nn.Sequential(nn.Linear(d_edge, 128), nn.ReLU(), nn.Linear(128, d * d)) # e_vw -> d×d13 self.msg = NNConv(d, d, A, aggr="add") # m_v = Σ_w A(e_vw) h_w14 self.gru = nn.GRU(d, d) # U, shared across the T rounds15 self.readout = Set2Set(d, processing_steps=3) # R, permutation-invariant16 self.head = nn.Sequential(nn.Linear(2 * d, d), nn.ReLU(), nn.Linear(d, 1))1718 def forward(self, g):19 h = torch.relu(self.embed(g.x))20 state = h.unsqueeze(0)21 for _ in range(self.T):22 m = torch.relu(self.msg(h, g.edge_index, g.edge_attr))23 h, state = self.gru(m.unsqueeze(0), state)24 h = h.squeeze(0)25 return self.head(self.readout(h, g.batch)).view(-1)2627ds = QM9("data/QM9").shuffle()28test, val, train = ds[:10000], ds[10000:20000], ds[20000:]29model = MPNN()30opt = torch.optim.Adam(model.parameters(), lr=1e-3)31for g in DataLoader(train, batch_size=128, shuffle=True):32 loss = nn.functional.l1_loss(model(g), g.y[:, 0]) # MAE on target 0 (dipole moment)33 opt.zero_grad(); loss.backward(); opt.step()What the framework implies
Graph Convolutional Network (GCN) is the case with and . Graph Attention Network (GAT) makes the weight of each message a learned attention coefficient. GraphSAGE swaps the sum for mean, max-pool or LSTM aggregators. Graph Isomorphism Network (GIN) keeps the sum and adds .
The framework also has a ceiling. Xu et al. (2019) and Morris et al. (2019) proved that an MPNN distinguishes two graphs at most as well as the 1-WL colour-refinement test. Mean and max aggregators can be weaker still, because they lose neighbour counts. Battaglia et al. (2018) generalised the scheme to graph networks, which also update edge states and a global state.
Check yourself
Why must the MPNN readout R be a symmetric function such as sum, mean or set2set?
Check yourself
In the interactive figure, why does atom O0's state only start to depend on O5 at t = 4?
Remember
- ✓MPNN (Gilmer et al., ICML 2017): , , .
- ✓Shared M and U handle graphs of any size. A symmetric aggregator and readout give permutation invariance.
- ✓GCN, GG-NN, Interaction Networks, molecular fingerprints and DTNN are all special cases.
- ✓The best QM9 model: edge network , a GRU update shared over rounds, and a set2set readout. It reached chemical accuracy on 11 of 13 targets.
- ✓T rounds give a T-hop receptive field. Deep stacks over-smooth and over-squash.
- ✓Expressiveness is bounded by the 1-WL test. GIN reaches that bound with sum aggregation.