# Heterogeneous graph neural networks

> Heterogeneous GNNs handle typed nodes and edges: HAN (2019) attends within metapath neighbourhoods, then across metapaths with learned weights; HGT (2020) uses type-specific Q/K/V and relation matrices, so no metapaths are hand-picked.

Source: https://www.mlfoundry.in/learn/heterogeneous-graph-neural-networks  
Section: Deep Learning › Graph Neural Networks (GNNs)  
Level: advanced · 8 min read · updated 2026-10-21

**Heterogeneous graph neural network:** is a GNN for a graph $G=(V,E)$ with a node-type map $\tau:V\to\mathcal A$ and an edge-type map $\phi:E\to\mathcal R$, $|\mathcal A|+|\mathcal R|>2$, whose parameters depend on those types. Two reference models: HAN, "Heterogeneous Graph Attention Network" (Wang et al., WWW 2019), and HGT, "Heterogeneous Graph Transformer" (Hu, Dong, Wang and Sun, WWW 2020).

Typical inputs are academic graphs (authors, papers, venues, fields), e-commerce graphs (users, items, shops) and knowledge graphs. The task is usually node classification on one target type, such as a paper's venue or an author's area, or link prediction between types, such as user–item edges. The **network schema** lists the types and which pairs can be linked: author–writes–paper, paper–published in–venue.

A homogeneous GNN treats all of this as one graph, and that causes two problems. Node types have different feature spaces, of different dimension or with no features at all, so one weight matrix cannot read them all. Relations have different meanings: being cited by a paper is not the same as being written by an author. Summing all neighbours with one weight blurs them together. The earliest fix is **Relational GCN**, which gives each relation its own weight matrix. Heterogeneous GNNs add type-aware attention on top.

## Metapaths

A **metapath** is a type sequence through the schema, such as $A\xrightarrow{\text{writes}}P\xrightarrow{\text{written by}}A$ (APA, co-authors) or APVPA (authors who published at the same venue). It turns the heterogeneous graph into one homogeneous graph per metapath, with adjacency given by a product of bipartite adjacency matrices. Metapaths predate GNNs: PathSim (Sun et al., VLDB 2011) used them for similarity search, and metapath2vec (Dong, Chawla and Swami, KDD 2017) used metapath-guided random walks to learn embeddings.

$$
A_{\Phi}=A_{AP}\,A_{PA}\ \ (\Phi=\text{APA}),\qquad A_{\Phi}=A_{AP}A_{PV}A_{VP}A_{PA}\ \ (\Phi=\text{APVPA}),\qquad \mathcal N^{\Phi}(i)=\{j:(A_\Phi)_{ij}>0\}
$$

- $A_{AP}$: $|A|\times|P|$ author–paper incidence; $A_{PA}=A_{AP}^\top$
- $(A_\Phi)_{ij}$: number of path instances of type $\Phi$ from $i$ to $j$; the diagonal is non-zero, so $i\in\mathcal N^\Phi(i)$

## HAN: node-level, then semantic-level attention

HAN first projects each node into a shared space with a matrix for its type, $h'_i=M_{\tau(i)}h_i$. For each metapath $\Phi$ it then runs **Graph Attention Network (GAT)**-style attention over the metapath neighbours, with its own attention vector $a_\Phi$ and $K$ heads concatenated.

$$
\alpha^{\Phi}_{ij}=\operatorname{softmax}_{j\in\mathcal N^{\Phi}(i)}\Big(\sigma\big(a_\Phi^\top[h'_i\,\Vert\,h'_j]\big)\Big),\qquad z^{\Phi}_i=\sigma\Big(\sum_{j\in\mathcal N^{\Phi}(i)}\alpha^{\Phi}_{ij}\,h'_j\Big)
$$

- $\alpha^{\Phi}_{ij}$: how much neighbour j matters to i along metapath Φ; asymmetric, like GAT
- $\sigma$: nonlinearity; LeakyReLU inside the score, as in GAT
- $z^{\Phi}_i$: node i's embedding from metapath Φ alone

The metapath embeddings are then fused by a **semantic-level attention**. A one-layer MLP scores each metapath, the score is averaged over all target nodes, and a softmax over metapaths gives one weight $\beta_\Phi$ per metapath, shared by every node. The fused embedding goes to a softmax classifier trained with cross-entropy.

$$
w_\Phi=\frac{1}{|V|}\sum_{i\in V}q^\top\tanh\big(W z^{\Phi}_i+b\big),\qquad \beta_\Phi=\frac{\exp(w_\Phi)}{\sum_{\Phi'}\exp(w_{\Phi'})},\qquad Z=\sum_{\Phi}\beta_\Phi\,Z^{\Phi}
$$

- $q,\ W,\ b$: semantic attention vector and MLP, shared across metapaths
- $\beta_\Phi$: importance of metapath Φ for the task; HAN reports these as an interpretable by-product

> Figure: Left: the schema (author–writes–paper–in–venue) and two metapaths; their adjacency is a product of incidence matrices. Middle: HAN attends over each metapath's neighbour set separately (node level) and fuses the results with one learned weight per metapath (semantic level). Bottom: HGT skips metapaths. On every edge it uses source-type key and message projections, target-type query projections, and relation-specific matrices, so L layers can weight any type sequence of length up to L.

## HGT: type-dependent transformer attention

HAN needs a hand-chosen list of metapaths and discards the intermediate nodes (MAGNN, Fu et al., WWW 2020, encodes them). HGT attends over the raw typed edges instead. For an edge $e=(s,t)$ every projection is indexed by types: keys and messages by the source type, queries by the target type, and two matrices by the relation.

$$
\text{ATT}^{i}(s,e,t)=\Big(K^i(s)\,W^{\text{ATT}}_{\phi(e)}\,Q^i(t)^{\top}\Big)\cdot\frac{\mu_{\langle\tau(s),\phi(e),\tau(t)\rangle}}{\sqrt d},\qquad K^i(s)=\text{K-Linear}^i_{\tau(s)}\big(H(s)\big),\quad Q^i(t)=\text{Q-Linear}^i_{\tau(t)}\big(H(t)\big)
$$

- $i$: attention head; scores are normalised by a softmax over all neighbours of $t$, as in **Multi-head attention**
- $W^{\text{ATT}}_{\phi(e)},\ \mu$: relation-specific bilinear matrix and a learned prior for each (source type, relation, target type) triple
- $\text{message}$: $\text{M-Linear}^i_{\tau(s)}(H(s))\,W^{\text{MSG}}_{\phi(e)}$; the aggregate passes through a target-type $\text{A-Linear}_{\tau(t)}$ with a residual connection

The projection matrices grow with the number of node and edge types. Only the scalar prior $\mu$ is kept per (type, relation, type) triple, so the model does not need a separate weight for every triple. HGT also adds a relative temporal encoding for edges with timestamps and a type-balanced subgraph sampler (HGSampling) for web-scale training.

**Datasets and reported results**

- **ACM (HAN):** 3,025 papers, 5,835 authors, 56 subjects; target: paper class (3); metapaths PAP and PSP
- **Other HAN data:** DBLP (authors; APA, APCPA, APTPA) and IMDB (movies; MAM, MDM)
- **HGT:** Open Academic Graph with 179 million nodes and 2 billion edges; reported 9–21% gains over GNN baselines on its downstream tasks
- **ogbn-mag:** 736,389 papers, 1,134,649 authors, 8,740 institutions, 59,965 fields of study; 4 relations; predict each paper's venue (349 classes); split by year
- **Reality check:** Lv et al. (KDD 2021, HGB benchmark): a well-tuned GAT with edge-type embeddings (Simple-HGN) matched or beat most specialised models

> Figure: 18 authors in 3 areas, 24 papers, 6 venues; author features are a one-hot of the area plus Gaussian noise σ. Each metapath view averages features over its neighbour set (self included); z = β·APA + (1 − β)·APVPA. APA gives 3.0 neighbours on average with homophily 0.741; APVPA gives 8.33 with 0.600. At σ = 1.0, own features classify 10/18 = 0.556, APVPA alone 0.722, APA alone 0.667, and β = 0.5 reaches 16/18 = 0.889. At σ = 0.8 any β in 0.5–0.9 gets 18/18. At σ = 1.6 the best is APVPA alone (0.722): the useful β depends on the data, which is why HAN learns it.

## Worked example: two metapaths, one author

1. **Graph.** Authors $a_1..a_4$, papers $p_1$ (by $a_1,a_2$, at $v_1$), $p_2$ (by $a_3$, at $v_1$), $p_3$ (by $a_3,a_4$, at $v_2$). Scalar features $x=(1.0,\,0.6,\,-0.2,\,-1.0)$.
2. **APA.** $A_{AP}A_{PA}$ has rows $a_1:(1,1,0,0)$ and $a_3:(0,0,2,1)$. So $\mathcal N^{APA}(a_1)=\{a_1,a_2\}$ and $\mathcal N^{APA}(a_3)=\{a_3,a_4\}$.
3. **APVPA.** $A_{APV}$ has rows $(1,0),(1,0),(1,1),(0,1)$. Then $A_{APV}A_{APV}^\top$ has rows $a_1:(1,1,1,0)$ and $a_3:(1,1,2,1)$. Through venue $v_1$, $a_1$ is now linked to $a_3$, with whom it never wrote a paper.
4. **Node level (uniform attention).** $z^{APA}_{a_1}=(1.0+0.6)/2=0.8$ and $z^{APVPA}_{a_1}=(1.0+0.6-0.2)/3=0.4667$. For $a_3$: $z^{APA}=-0.6$ and $z^{APVPA}=(1.0+0.6-0.2-1.0)/4=0.1$.
5. **Semantic level.** With scores $w_{APA}=0.8$, $w_{APVPA}=0.2$, the softmax gives $\beta=(0.6457,\,0.3543)$. Then $Z_{a_1}=0.6457\cdot0.8+0.3543\cdot0.4667=0.6819$ and $Z_{a_3}=0.6457\cdot(-0.6)+0.3543\cdot0.1=-0.352$.

```python
import torch, torch.nn.functional as F
import torch_geometric.transforms as T
from torch_geometric.datasets import DBLP
from torch_geometric.nn import HGTConv, Linear

# node types: author, paper, term, conference; conference has no features
data = DBLP("data/DBLP", transform=T.Constant(node_types="conference"))[0]

class HGT(torch.nn.Module):
    def __init__(self, metadata, hidden=64, out=4, heads=2, layers=2):
        super().__init__()
        self.inp = torch.nn.ModuleDict({t: Linear(-1, hidden) for t in metadata[0]})  # per-type input
        self.convs = torch.nn.ModuleList(HGTConv(hidden, hidden, metadata, heads) for _ in range(layers))
        self.out = Linear(hidden, out)
    def forward(self, x_dict, edge_index_dict):
        x = {t: self.inp[t](x).relu() for t, x in x_dict.items()}
        for conv in self.convs:
            x = conv(x, edge_index_dict)
        return self.out(x["author"])

model = HGT(data.metadata())
with torch.no_grad():                     # initialise the lazy Linear(-1, ...) layers
    model(data.x_dict, data.edge_index_dict)
opt = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=1e-3)
y, tr = data["author"].y, data["author"].train_mask
for epoch in range(100):
    opt.zero_grad()
    loss = F.cross_entropy(model(data.x_dict, data.edge_index_dict)[tr], y[tr])
    loss.backward(); opt.step()
pred = model(data.x_dict, data.edge_index_dict).argmax(-1)
te = data["author"].test_mask
print("test acc", (pred[te] == y[te]).float().mean().item())
```

A 2-layer HGT for author classification on DBLP (PyTorch Geometric; not run here, since this environment has no torch)

> **Metapath graphs can be dense:** A metapath through a hub type explodes. If one venue has $n$ papers, APVPA links every pair of their authors, so one venue adds on the order of $n^2$ edges. In the figure APVPA already has 8.33 neighbours per author against APA's 3.0, with lower homophily. Precompute metapath adjacency sparsely, cap or sample the neighbours, and prefer relation-level models (R-GCN, HGT) when the schema has hubs.

> **Types are a prior, not magic:** Typed parameters help most when the types really differ: different feature spaces, or relations with opposite meanings. The HGB study found that many published gains disappeared under careful tuning of homogeneous baselines given edge-type embeddings. Start with a typed linear input layer plus GAT or R-GCN, and add metapaths or HGT only if they beat that.

**Strengths**

- Handles different feature spaces and featureless types through per-type projections.
- HAN's β weights show which metapaths the task uses.
- HGT needs no hand-picked metapaths and scales with sampling to graphs with hundreds of millions of nodes.

**Limitations**

- HAN depends on choosing metapaths by hand and drops the intermediate nodes.
- Metapath adjacency through hub types is dense and expensive.
- Many typed parameters can overfit small graphs; strong homogeneous baselines are often close.

**Question:** In HAN, what does the semantic-level attention produce?

- A. A separate weight for each neighbour of each node
- B. One weight per metapath, shared by all target nodes, used to sum the per-metapath embeddings
- C. A weight per node type for the input projection
- D. The metapath adjacency matrices

**Answer:** B. One weight per metapath, shared by all target nodes, used to sum the per-metapath embeddings — $w_\Phi$ averages $q^\top\tanh(Wz^\Phi_i+b)$ over all nodes, so after the softmax there is one $\beta_\Phi$ per metapath. Per-neighbour weights are the node-level $\alpha^\Phi_{ij}$.

**Question:** Why does HGT not need a list of metapaths?

- A. It uses only the most frequent metapath
- B. Its attention on each typed edge is parameterised by source type, relation and target type, so stacked layers weight type sequences implicitly
- C. It converts the graph to a homogeneous one first
- D. It samples metapaths at random during training

**Answer:** B. Its attention on each typed edge is parameterised by source type, relation and target type, so stacked layers weight type sequences implicitly — Each layer aggregates over one hop of typed edges with type-specific $K$, $Q$, message and $W^{\text{ATT}}_{\phi(e)}$, $W^{\text{MSG}}_{\phi(e)}$. After $L$ layers a node has attended along every typed path of length $\le L$, with learned weights. That is a soft, learned version of metapath selection.

**Key points**

- Heterogeneous graph: node types $\tau(v)$ and edge types $\phi(e)$; the schema lists which types can link.
- Metapaths (APA, APVPA) define homogeneous neighbour graphs by products of incidence matrices.
- HAN: type projection, GAT-style attention within each metapath, then one softmax weight $\beta_\Phi$ per metapath.
- HGT: type-specific K/Q/V, relation-specific attention and message matrices, and a type-triple prior μ; no hand-picked metapaths.
- Dense metapaths through hubs are costly and noisy; sample or use relation-level models.
- Benchmark against a tuned GAT or R-GCN with typed inputs; the HGB study found many gains vanished.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
