Definition
Heterogeneous graph neural network is a GNN for a graph with a node-type map and an edge-type map , , whose parameters depend on those types. Two reference models: HAN, "Heterogeneous Graph Attention Network" (Wang et al., WWW 2019), and HGT, "Heterogeneous Graph Transformer" (Hu, Dong, Wang and Sun, WWW 2020).
Typical inputs are academic graphs (authors, papers, venues, fields), e-commerce graphs (users, items, shops) and knowledge graphs. The task is usually node classification on one target type, such as a paper's venue or an author's area, or link prediction between types, such as user–item edges. The network schema lists the types and which pairs can be linked: author–writes–paper, paper–published in–venue.
A homogeneous GNN treats all of this as one graph, and that causes two problems. Node types have different feature spaces, of different dimension or with no features at all, so one weight matrix cannot read them all. Relations have different meanings: being cited by a paper is not the same as being written by an author. Summing all neighbours with one weight blurs them together. The earliest fix is Relational GCN, which gives each relation its own weight matrix. Heterogeneous GNNs add type-aware attention on top.
Metapaths
A metapath is a type sequence through the schema, such as (APA, co-authors) or APVPA (authors who published at the same venue). It turns the heterogeneous graph into one homogeneous graph per metapath, with adjacency given by a product of bipartite adjacency matrices. Metapaths predate GNNs: PathSim (Sun et al., VLDB 2011) used them for similarity search, and metapath2vec (Dong, Chawla and Swami, KDD 2017) used metapath-guided random walks to learn embeddings.
- author–paper incidence;
- number of path instances of type from to ; the diagonal is non-zero, so
HAN: node-level, then semantic-level attention
HAN first projects each node into a shared space with a matrix for its type, . For each metapath it then runs Graph Attention Network (GAT)-style attention over the metapath neighbours, with its own attention vector and heads concatenated.
- how much neighbour j matters to i along metapath Φ; asymmetric, like GAT
- nonlinearity; LeakyReLU inside the score, as in GAT
- node i's embedding from metapath Φ alone
The metapath embeddings are then fused by a semantic-level attention. A one-layer MLP scores each metapath, the score is averaged over all target nodes, and a softmax over metapaths gives one weight per metapath, shared by every node. The fused embedding goes to a softmax classifier trained with cross-entropy.
- semantic attention vector and MLP, shared across metapaths
- importance of metapath Φ for the task; HAN reports these as an interpretable by-product
HGT: type-dependent transformer attention
HAN needs a hand-chosen list of metapaths and discards the intermediate nodes (MAGNN, Fu et al., WWW 2020, encodes them). HGT attends over the raw typed edges instead. For an edge every projection is indexed by types: keys and messages by the source type, queries by the target type, and two matrices by the relation.
- attention head; scores are normalised by a softmax over all neighbours of , as in Multi-head attention
- relation-specific bilinear matrix and a learned prior for each (source type, relation, target type) triple
- ; the aggregate passes through a target-type with a residual connection
The projection matrices grow with the number of node and edge types. Only the scalar prior is kept per (type, relation, type) triple, so the model does not need a separate weight for every triple. HGT also adds a relative temporal encoding for edges with timestamps and a type-balanced subgraph sampler (HGSampling) for web-scale training.
Datasets and reported results
- ACM (HAN)
- 3,025 papers, 5,835 authors, 56 subjects; target: paper class (3); metapaths PAP and PSP
- Other HAN data
- DBLP (authors; APA, APCPA, APTPA) and IMDB (movies; MAM, MDM)
- HGT
- Open Academic Graph with 179 million nodes and 2 billion edges; reported 9–21% gains over GNN baselines on its downstream tasks
- ogbn-mag
- 736,389 papers, 1,134,649 authors, 8,740 institutions, 59,965 fields of study; 4 relations; predict each paper's venue (349 classes); split by year
- Reality check
- Lv et al. (KDD 2021, HGB benchmark): a well-tuned GAT with edge-type embeddings (Simple-HGN) matched or beat most specialised models
Worked example: two metapaths, one author
- 1Graph. Authors , papers (by , at ), (by , at ), (by , at ). Scalar features .
- 2APA. has rows and . So and .
- 3APVPA. has rows . Then has rows and . Through venue , is now linked to , with whom it never wrote a paper.
- 4Node level (uniform attention). and . For : and .
- 5Semantic level. With scores , , the softmax gives . Then and .
1import torch, torch.nn.functional as F2import torch_geometric.transforms as T3from torch_geometric.datasets import DBLP4from torch_geometric.nn import HGTConv, Linear56# node types: author, paper, term, conference; conference has no features7data = DBLP("data/DBLP", transform=T.Constant(node_types="conference"))[0]89class HGT(torch.nn.Module):10 def __init__(self, metadata, hidden=64, out=4, heads=2, layers=2):11 super().__init__()12 self.inp = torch.nn.ModuleDict({t: Linear(-1, hidden) for t in metadata[0]}) # per-type input13 self.convs = torch.nn.ModuleList(HGTConv(hidden, hidden, metadata, heads) for _ in range(layers))14 self.out = Linear(hidden, out)15 def forward(self, x_dict, edge_index_dict):16 x = {t: self.inp[t](x).relu() for t, x in x_dict.items()}17 for conv in self.convs:18 x = conv(x, edge_index_dict)19 return self.out(x["author"])2021model = HGT(data.metadata())22with torch.no_grad(): # initialise the lazy Linear(-1, ...) layers23 model(data.x_dict, data.edge_index_dict)24opt = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=1e-3)25y, tr = data["author"].y, data["author"].train_mask26for epoch in range(100):27 opt.zero_grad()28 loss = F.cross_entropy(model(data.x_dict, data.edge_index_dict)[tr], y[tr])29 loss.backward(); opt.step()30pred = model(data.x_dict, data.edge_index_dict).argmax(-1)31te = data["author"].test_mask32print("test acc", (pred[te] == y[te]).float().mean().item())Use when
- +Handles different feature spaces and featureless types through per-type projections.
- +HAN's β weights show which metapaths the task uses.
- +HGT needs no hand-picked metapaths and scales with sampling to graphs with hundreds of millions of nodes.
Avoid when
- −HAN depends on choosing metapaths by hand and drops the intermediate nodes.
- −Metapath adjacency through hub types is dense and expensive.
- −Many typed parameters can overfit small graphs; strong homogeneous baselines are often close.
Check yourself
In HAN, what does the semantic-level attention produce?
Check yourself
Why does HGT not need a list of metapaths?
Remember
- ✓Heterogeneous graph: node types and edge types ; the schema lists which types can link.
- ✓Metapaths (APA, APVPA) define homogeneous neighbour graphs by products of incidence matrices.
- ✓HAN: type projection, GAT-style attention within each metapath, then one softmax weight per metapath.
- ✓HGT: type-specific K/Q/V, relation-specific attention and message matrices, and a type-triple prior μ; no hand-picked metapaths.
- ✓Dense metapaths through hubs are costly and noisy; sample or use relation-level models.
- ✓Benchmark against a tuned GAT or R-GCN with typed inputs; the HGB study found many gains vanished.