Heterogeneous graph neural networks

Heterogeneous GNNs handle typed nodes and edges: HAN (2019) attends within metapath neighbourhoods, then across metapaths with learned weights; HGT (2020) uses type-specific Q/K/V and relation matrices, so no metapaths are hand-picked.

8 min read778 wordsadvanced0 viewsupdated 2026-10-21

Definition

Heterogeneous graph neural network is a GNN for a graph G=(V,E)G=(V,E) with a node-type map τ:V→A\tau:V\to\mathcal A and an edge-type map ϕ:E→R\phi:E\to\mathcal R, ∣A∣+∣R∣>2|\mathcal A|+|\mathcal R|>2, whose parameters depend on those types. Two reference models: HAN, "Heterogeneous Graph Attention Network" (Wang et al., WWW 2019), and HGT, "Heterogeneous Graph Transformer" (Hu, Dong, Wang and Sun, WWW 2020).

Typical inputs are academic graphs (authors, papers, venues, fields), e-commerce graphs (users, items, shops) and knowledge graphs. The task is usually node classification on one target type, such as a paper's venue or an author's area, or link prediction between types, such as user–item edges. The network schema lists the types and which pairs can be linked: author–writes–paper, paper–published in–venue.

A homogeneous GNN treats all of this as one graph, and that causes two problems. Node types have different feature spaces, of different dimension or with no features at all, so one weight matrix cannot read them all. Relations have different meanings: being cited by a paper is not the same as being written by an author. Summing all neighbours with one weight blurs them together. The earliest fix is Relational GCN, which gives each relation its own weight matrix. Heterogeneous GNNs add type-aware attention on top.

Metapaths

A metapath is a type sequence through the schema, such as A→writesP→written byAA\xrightarrow{\text{writes}}P\xrightarrow{\text{written by}}A (APA, co-authors) or APVPA (authors who published at the same venue). It turns the heterogeneous graph into one homogeneous graph per metapath, with adjacency given by a product of bipartite adjacency matrices. Metapaths predate GNNs: PathSim (Sun et al., VLDB 2011) used them for similarity search, and metapath2vec (Dong, Chawla and Swami, KDD 2017) used metapath-guided random walks to learn embeddings.

AΦ=AAP APA  (Φ=APA),AΦ=AAPAPVAVPAPA  (Φ=APVPA),NΦ(i)={j:(AΦ)ij>0}A_{\Phi}=A_{AP}\,A_{PA}\ \ (\Phi=\text{APA}),\qquad A_{\Phi}=A_{AP}A_{PV}A_{VP}A_{PA}\ \ (\Phi=\text{APVPA}),\qquad \mathcal N^{\Phi}(i)=\{j:(A_\Phi)_{ij}>0\}
AAPA_{AP}
∣A∣×∣P∣|A|\times|P| author–paper incidence; APA=AAP⊤A_{PA}=A_{AP}^\top
(AΦ)ij(A_\Phi)_{ij}
number of path instances of type Φ\Phi from ii to jj; the diagonal is non-zero, so i∈NΦ(i)i\in\mathcal N^\Phi(i)

HAN: node-level, then semantic-level attention

HAN first projects each node into a shared space with a matrix for its type, hi′=Mτ(i)hih'_i=M_{\tau(i)}h_i. For each metapath Φ\Phi it then runs Graph Attention Network (GAT)-style attention over the metapath neighbours, with its own attention vector aΦa_\Phi and KK heads concatenated.

αijΦ=softmax⁡j∈NΦ(i)(σ(aΦ⊤[hi′ ∥ hj′])),ziΦ=σ(∑j∈NΦ(i)αijΦ hj′)\alpha^{\Phi}_{ij}=\operatorname{softmax}_{j\in\mathcal N^{\Phi}(i)}\Big(\sigma\big(a_\Phi^\top[h'_i\,\Vert\,h'_j]\big)\Big),\qquad z^{\Phi}_i=\sigma\Big(\sum_{j\in\mathcal N^{\Phi}(i)}\alpha^{\Phi}_{ij}\,h'_j\Big)
αijΦ\alpha^{\Phi}_{ij}
how much neighbour j matters to i along metapath Φ; asymmetric, like GAT
σ\sigma
nonlinearity; LeakyReLU inside the score, as in GAT
ziΦz^{\Phi}_i
node i's embedding from metapath Φ alone

The metapath embeddings are then fused by a semantic-level attention. A one-layer MLP scores each metapath, the score is averaged over all target nodes, and a softmax over metapaths gives one weight βΦ\beta_\Phi per metapath, shared by every node. The fused embedding goes to a softmax classifier trained with cross-entropy.

wΦ=1∣V∣∑i∈Vq⊤tanh⁡(WziΦ+b),βΦ=exp⁡(wΦ)∑Φ′exp⁡(wΦ′),Z=∑ΦβΦ ZΦw_\Phi=\frac{1}{|V|}\sum_{i\in V}q^\top\tanh\big(W z^{\Phi}_i+b\big),\qquad \beta_\Phi=\frac{\exp(w_\Phi)}{\sum_{\Phi'}\exp(w_{\Phi'})},\qquad Z=\sum_{\Phi}\beta_\Phi\,Z^{\Phi}
q, W, bq,\ W,\ b
semantic attention vector and MLP, shared across metapaths
βΦ\beta_\Phi
importance of metapath Φ for the task; HAN reports these as an interpretable by-product
Typed nodes and edges: aggregate per relation or metapath, then combine1 · schema, metapathsAauthorPpaperVvenuewritesinAPA: co-authorsAPAAPVPA: same venueAPVPAA_APA = A_AP · A_PA(path counts; > 0 = neighbour)2 · HAN: two levels of attentionnode level, Φ = APAh' = M_type · hα_ij over N^APA(i)node level, Φ = APVPAsame, own a_Φα_ij over N^APVPA(i)z^APAz^APVPAsemantic levelw_Φ = mean qᵀtanh(Wz+b)β_Φ = softmax(w_Φ)Z = Σ_Φ β_Φ Z^Φ→ softmax classifierneighbour sets include i itself;K heads, concatenatedα^Φ_ij = softmax_j(LeakyReLU(a_Φᵀ[h'_i ‖ h'_j]))z^Φ_i = σ(Σ_j α^Φ_ij h'_j); each β_Φ is shared by all nodes3 · HGT: one layer, no hand-picked metapaths; parameters indexed by typessource s: τ(s) = AK-Lin_τ(s), M-Lin_τ(s)per node typeedge e: φ(e) = writesW^ATT_φ, W^MSG_φ, μ⟨τs,φ,τt⟩per relation typetarget t: τ(t) = PQ-Lin_τ(t), softmax N(t)A-Lin_τ(t) + residualscore = K(s)·W^ATT_φ(e)·Q(t)ᵀ · μ/√d per head; L layers reach metapaths of length ≤ L
fig.Left: the schema (author–writes–paper–in–venue) and two metapaths; their adjacency is a product of incidence matrices. Middle: HAN attends over each metapath's neighbour set separately (node level) and fuses the results with one learned weight per metapath (semantic level). Bottom: HGT skips metapaths. On every edge it uses source-type key and message projections, target-type query projections, and relation-specific matrices, so L layers can weight any type sequence of length up to L.

HGT: type-dependent transformer attention

HAN needs a hand-chosen list of metapaths and discards the intermediate nodes (MAGNN, Fu et al., WWW 2020, encodes them). HGT attends over the raw typed edges instead. For an edge e=(s,t)e=(s,t) every projection is indexed by types: keys and messages by the source type, queries by the target type, and two matrices by the relation.

ATTi(s,e,t)=(Ki(s) Wϕ(e)ATT Qi(t)⊤)⋅μ⟨τ(s),ϕ(e),τ(t)⟩d,Ki(s)=K-Linearτ(s)i(H(s)),Qi(t)=Q-Linearτ(t)i(H(t))\text{ATT}^{i}(s,e,t)=\Big(K^i(s)\,W^{\text{ATT}}_{\phi(e)}\,Q^i(t)^{\top}\Big)\cdot\frac{\mu_{\langle\tau(s),\phi(e),\tau(t)\rangle}}{\sqrt d},\qquad K^i(s)=\text{K-Linear}^i_{\tau(s)}\big(H(s)\big),\quad Q^i(t)=\text{Q-Linear}^i_{\tau(t)}\big(H(t)\big)
ii
attention head; scores are normalised by a softmax over all neighbours of tt, as in Multi-head attention
Wϕ(e)ATT, μW^{\text{ATT}}_{\phi(e)},\ \mu
relation-specific bilinear matrix and a learned prior for each (source type, relation, target type) triple
message\text{message}
M-Linearτ(s)i(H(s)) Wϕ(e)MSG\text{M-Linear}^i_{\tau(s)}(H(s))\,W^{\text{MSG}}_{\phi(e)}; the aggregate passes through a target-type A-Linearτ(t)\text{A-Linear}_{\tau(t)} with a residual connection

The projection matrices grow with the number of node and edge types. Only the scalar prior μ\mu is kept per (type, relation, type) triple, so the model does not need a separate weight for every triple. HGT also adds a relative temporal encoding for edges with timestamps and a type-balanced subgraph sampler (HGSampling) for web-scale training.

Datasets and reported results

ACM (HAN)
3,025 papers, 5,835 authors, 56 subjects; target: paper class (3); metapaths PAP and PSP
Other HAN data
DBLP (authors; APA, APCPA, APTPA) and IMDB (movies; MAM, MDM)
HGT
Open Academic Graph with 179 million nodes and 2 billion edges; reported 9–21% gains over GNN baselines on its downstream tasks
ogbn-mag
736,389 papers, 1,134,649 authors, 8,740 institutions, 59,965 fields of study; 4 relations; predict each paper's venue (349 classes); split by year
Reality check
Lv et al. (KDD 2021, HGB benchmark): a well-tuned GAT with edge-type embeddings (Simple-HGN) matched or beat most specialised models
6 venues · 24 papers · 18 authors in 3 areas · metapath A–P–A (co-author)VPAV0V1V2V3V4V50123✗4567891011121314151617✗author-area accuracy vs β00.51own features only 0.556β = 0: APVPA onlyβ = 1: APA onlyauthor 4 (area 0), self includedAPA: 4 neighbours, 2 same areaAPVPA: 9 neighbours, 5 same areahomophily: APA 0.741, APVPA 0.600z = 0.5·mean_APA(x) + 0.5·mean_APVPA(x)author 4: predicted area 0 ✓accuracy16/18 = 0.889HAN learns β by semantic attention
highlight
fig.18 authors in 3 areas, 24 papers, 6 venues; author features are a one-hot of the area plus Gaussian noise σ. Each metapath view averages features over its neighbour set (self included); z = β·APA + (1 − β)·APVPA. APA gives 3.0 neighbours on average with homophily 0.741; APVPA gives 8.33 with 0.600. At σ = 1.0, own features classify 10/18 = 0.556, APVPA alone 0.722, APA alone 0.667, and β = 0.5 reaches 16/18 = 0.889. At σ = 0.8 any β in 0.5–0.9 gets 18/18. At σ = 1.6 the best is APVPA alone (0.722): the useful β depends on the data, which is why HAN learns it.

Worked example: two metapaths, one author

  1. 1Graph. Authors a1..a4a_1..a_4, papers p1p_1 (by a1,a2a_1,a_2, at v1v_1), p2p_2 (by a3a_3, at v1v_1), p3p_3 (by a3,a4a_3,a_4, at v2v_2). Scalar features x=(1.0, 0.6, −0.2, −1.0)x=(1.0,\,0.6,\,-0.2,\,-1.0).
  2. 2APA. AAPAPAA_{AP}A_{PA} has rows a1:(1,1,0,0)a_1:(1,1,0,0) and a3:(0,0,2,1)a_3:(0,0,2,1). So NAPA(a1)={a1,a2}\mathcal N^{APA}(a_1)=\{a_1,a_2\} and NAPA(a3)={a3,a4}\mathcal N^{APA}(a_3)=\{a_3,a_4\}.
  3. 3APVPA. AAPVA_{APV} has rows (1,0),(1,0),(1,1),(0,1)(1,0),(1,0),(1,1),(0,1). Then AAPVAAPV⊤A_{APV}A_{APV}^\top has rows a1:(1,1,1,0)a_1:(1,1,1,0) and a3:(1,1,2,1)a_3:(1,1,2,1). Through venue v1v_1, a1a_1 is now linked to a3a_3, with whom it never wrote a paper.
  4. 4Node level (uniform attention). za1APA=(1.0+0.6)/2=0.8z^{APA}_{a_1}=(1.0+0.6)/2=0.8 and za1APVPA=(1.0+0.6−0.2)/3=0.4667z^{APVPA}_{a_1}=(1.0+0.6-0.2)/3=0.4667. For a3a_3: zAPA=−0.6z^{APA}=-0.6 and zAPVPA=(1.0+0.6−0.2−1.0)/4=0.1z^{APVPA}=(1.0+0.6-0.2-1.0)/4=0.1.
  5. 5Semantic level. With scores wAPA=0.8w_{APA}=0.8, wAPVPA=0.2w_{APVPA}=0.2, the softmax gives β=(0.6457, 0.3543)\beta=(0.6457,\,0.3543). Then Za1=0.6457⋅0.8+0.3543⋅0.4667=0.6819Z_{a_1}=0.6457\cdot0.8+0.3543\cdot0.4667=0.6819 and Za3=0.6457⋅(−0.6)+0.3543⋅0.1=−0.352Z_{a_3}=0.6457\cdot(-0.6)+0.3543\cdot0.1=-0.352.
python
1import torch, torch.nn.functional as F
2import torch_geometric.transforms as T
3from torch_geometric.datasets import DBLP
4from torch_geometric.nn import HGTConv, Linear
5
6# node types: author, paper, term, conference; conference has no features
7data = DBLP("data/DBLP", transform=T.Constant(node_types="conference"))[0]
8
9class HGT(torch.nn.Module):
10 def __init__(self, metadata, hidden=64, out=4, heads=2, layers=2):
11 super().__init__()
12 self.inp = torch.nn.ModuleDict({t: Linear(-1, hidden) for t in metadata[0]}) # per-type input
13 self.convs = torch.nn.ModuleList(HGTConv(hidden, hidden, metadata, heads) for _ in range(layers))
14 self.out = Linear(hidden, out)
15 def forward(self, x_dict, edge_index_dict):
16 x = {t: self.inp[t](x).relu() for t, x in x_dict.items()}
17 for conv in self.convs:
18 x = conv(x, edge_index_dict)
19 return self.out(x["author"])
20
21model = HGT(data.metadata())
22with torch.no_grad(): # initialise the lazy Linear(-1, ...) layers
23 model(data.x_dict, data.edge_index_dict)
24opt = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=1e-3)
25y, tr = data["author"].y, data["author"].train_mask
26for epoch in range(100):
27 opt.zero_grad()
28 loss = F.cross_entropy(model(data.x_dict, data.edge_index_dict)[tr], y[tr])
29 loss.backward(); opt.step()
30pred = model(data.x_dict, data.edge_index_dict).argmax(-1)
31te = data["author"].test_mask
32print("test acc", (pred[te] == y[te]).float().mean().item())
A 2-layer HGT for author classification on DBLP (PyTorch Geometric; not run here, since this environment has no torch)

Use when

  • +Handles different feature spaces and featureless types through per-type projections.
  • +HAN's β weights show which metapaths the task uses.
  • +HGT needs no hand-picked metapaths and scales with sampling to graphs with hundreds of millions of nodes.

Avoid when

  • −HAN depends on choosing metapaths by hand and drops the intermediate nodes.
  • −Metapath adjacency through hub types is dense and expensive.
  • −Many typed parameters can overfit small graphs; strong homogeneous baselines are often close.

Check yourself

In HAN, what does the semantic-level attention produce?

Check yourself

Why does HGT not need a list of metapaths?

Remember

  • ✓Heterogeneous graph: node types τ(v)\tau(v) and edge types ϕ(e)\phi(e); the schema lists which types can link.
  • ✓Metapaths (APA, APVPA) define homogeneous neighbour graphs by products of incidence matrices.
  • ✓HAN: type projection, GAT-style attention within each metapath, then one softmax weight βΦ\beta_\Phi per metapath.
  • ✓HGT: type-specific K/Q/V, relation-specific attention and message matrices, and a type-triple prior μ; no hand-picked metapaths.
  • ✓Dense metapaths through hubs are costly and noisy; sample or use relation-level models.
  • ✓Benchmark against a tuned GAT or R-GCN with typed inputs; the HGB study found many gains vanished.
← back to Deep Learning roadmap