Sentiment analysis

Predict the polarity of a text, sentence or aspect (binary, 5-way or per target); neural models beat lexicons by composing meaning, so “not very good” and “great cast but dull plot” come out right.

8 min read714 wordscore0 viewsupdated 2026-10-15

Definition

Sentiment analysis predicts the evaluative polarity expressed in text: binary (positive/negative), fine-grained (five classes from very negative to very positive, as in SST-5), or per aspect (one polarity for each target, such as the food and the service of one restaurant review). The landmark neural result is the Recursive Neural Tensor Network trained on the Stanford Sentiment Treebank (Socher et al., 2013).

As a machine-learning problem, sentence- or document-level sentiment is Text classification with a polarity label set, and the same encoder-plus-softmax head applies. This article covers what is specific to sentiment: compositionality, negation, and aspects.

Early systems summed word scores from a sentiment lexicon such as the MPQA subjectivity lexicon (Wilson, Wiebe and Hoffmann, 2005) or SentiWordNet (Esuli and Sebastiani, 2006). Pang, Lee and Vaithyanathan (2002) trained naive Bayes, maximum-entropy and SVM classifiers on bag-of-words features of movie reviews and found sentiment harder than topic classification. The reason is that polarity depends on how words combine.

A bag of words cannot tell "not good" from "good", and it scores "a great cast but a dull, boring plot" as neutral because the +2 and the two −1s cancel. Lexicon systems patch this with hand rules. SO-CAL (Taboada et al., 2011) uses shift negation, which moves a score a fixed amount toward the opposite pole instead of flipping its sign, so "not excellent" stays mildly positive.

Compositionality: the Sentiment Treebank

Socher et al. (2013) parsed 11,855 movie-review sentences and had crowd workers label every phrase in every parse tree, 215,154 unique phrases in all, on a five-point scale. This turned compositionality into supervision: a model sees not only that "not very good" is negative but also that "very good" is very positive.

compositional: Stanford Sentiment Treebank5 labels at every node (−− − 0 + ++)0not0very+good++−“very good”: ++“not very good”: −rootparent p from children a, b (RNTN, d-dim):p = tanh([a;b]ᵀ V[1:d] [a;b] + W[a;b])label = softmax(W_s p) over 5 classesone V, W, W_s shared by every nodeaspect-based: one polarity per aspectpair 1 · aspect = foodpair 2 · aspect = service“the pasta was great butthe service was slow”[CLS] sentence [SEP]food[SEP]BERT → [CLS] → softmaxpositive[CLS] sentence [SEP]service[SEP]BERT → [CLS] → softmaxnegativelabels: positive, negative, neutral, conflictgranularitydocumentIMDb: 25k + 25k reviewssentenceSST-2 (binary), SST-5 (5-way)aspectSemEval-2014 Task 4one label per review → per sentence (and phrase) → per (sentence, aspect) pair
fig.Left: in the Treebank every node of the parse carries one of five labels, and a recursive network computes each parent from its two children with the same weights everywhere, so "very good" (++) and "not very good" (−) are both supervised. Right: aspect-based sentiment pairs the sentence with each aspect and classifies the pair, giving positive for food and negative for service. Bottom: the three granularities and their standard datasets.

The Recursive Neural Tensor Network (RNTN) composes vectors bottom-up. A bilinear tensor term lets each child change how the other is interpreted, which a plain recursive network p=tanh⁡(W[a;b])p=\tanh(W[a;b]) can express only weakly:

p=tanh⁡ ⁣([a;b]⊤V[1:d][a;b]+W[a;b]),y^p=softmax(Ws p)∈Δ5p=\tanh\!\Big([a;b]^\top V^{[1:d]}[a;b]+W[a;b]\Big),\qquad \hat y_p=\mathrm{softmax}(W_s\,p)\in\Delta^{5}
a,b∈Rda,b\in\mathbb{R}^d
vectors of the left and right child (word vectors at the leaves)
V[1:d]∈R2d×2d×dV^{[1:d]}\in\mathbb{R}^{2d\times2d\times d}
tensor; slice ii gives coordinate ii of the bilinear term [a;b]⊤V[i][a;b][a;b]^\top V^{[i]}[a;b]
W∈Rd×2d,  Ws∈R5×dW\in\mathbb{R}^{d\times2d},\;W_s\in\mathbb{R}^{5\times d}
composition and classifier weights, shared by every node of every tree

Training applies cross-entropy at every node, not just the root. The RNTN reached 85.4% on binary and 45.7% on five-class sentence-level accuracy, and it handled negated phrases and "X but Y" contrasts better than bag-of-words baselines. Tree-LSTMs (Tai, Socher and Manning, 2015) later replaced the tensor with gated composition.

Fine-tuned Transformers then made parse trees unnecessary as input: self-attention learns which word scopes over which. BERT-large scores 94.9% on the GLUE SST-2 test set. The Treebank's phrase labels are still used as extra training data.

Datasets

SST-5
Five-class sentence sentiment; 8,544 train, 1,101 dev, 2,210 test sentences, plus phrase-level labels for training.
SST-2
The same sentences with neutral removed, binary; part of GLUE.
IMDb
50,000 polar movie reviews, 25,000 train and 25,000 test (Maas et al., 2011).
SemEval-2014 Task 4
Aspect-based sentiment on laptop and restaurant reviews (Pontiki et al., 2014): aspect term extraction and polarity, aspect category detection and polarity; labels positive, negative, neutral and conflict.

Negation, scope and contrast

The figure below makes composition explicit with hand-written rules on a parse tree, the kind of behaviour an RNTN or a Transformer must learn from data. With a negator applied to a subtree scored vv, the two classic rules are:

switch: not(v)=−v,shift: not(v)=v−s⋅sign⁡(v)\text{switch: } \mathrm{not}(v)=-v,\qquad \text{shift: } \mathrm{not}(v)=v-s\cdot\operatorname{sign}(v)
vv
score of the negated phrase, here on a [−2,2][-2,2] scale
ss
shift amount; SO-CAL uses 4 on its ±5 scale
not/never: v ↦ v − 1.50·sign(v) · very: v ↦ 1.5v · L but R: 0.25L + 0.75R · else sumnode scores clipped to [−2, 2] · negative · positive0shift0not+1.5very0very+1goodroot = 0 → SST-5 class: neutralphraselexiconswitchshift 1.50not good+1 +−1 −−0.5 −not terrible−2 −−+2 ++−0.5 −not very good+1 +−1.5 −−0 0never boring and very funny0 0+2 +++2 ++a great cast but a dull , boring plot0 0−1 −−1 −a brilliant , moving film+2 +++2 +++2 ++SST-5 bins: ≤ −1.2 · ≤ −0.4 · < 0.4 · < 1.2 · above · rules hand-set, not trained
model
fig.Sentiment computed bottom-up over the parse tree with hand-set rules (shift negation with s = 1.5 by default; the root is binned into the five SST classes). "not very good": the lexicon sum says +1 (positive); switch negation turns very good's +1.5 into −1.5 (very negative); shift gives 0 (neutral). "not terrible": lexicon −2, switch +2 (very positive, too strong), shift −0.5 (mildly negative). The contrast "a great cast but a dull, boring plot" is 0 for the lexicon but −1 (negative) once the clause after "but" gets weight 0.75. Raise s to 2 and "not terrible" becomes exactly neutral.

Worked example: a great cast but a dull, boring plot (tree rules)

  1. 1Leaves: great =+2=+2, dull =−1=-1, boring =−1=-1; "a", "cast", ",", "plot" =0=0.
  2. 2Left clause: "great cast" =2+0=2=2+0=2, "a great cast" =2=2.
  3. 3Right clause: "boring plot" =−1=-1, ", boring plot" =−1=-1, "dull , boring plot" =−1+(−1)=−2=-1+(-1)=-2, "a dull , boring plot" =−2=-2.
  4. 4Contrast: 0.25⋅2+0.75⋅(−2)=0.5−1.5=−10.25\cdot2+0.75\cdot(-2)=0.5-1.5=-1, which bins to negative. The lexicon sum 2−1−1=02-1-1=0 bins to neutral.

Aspect-based sentiment

"The pasta was great but the service was slow" has no single correct sentence label. Aspect-based sentiment analysis (ABSA) instead predicts a polarity for each target. The targets are either aspect terms in the text ("pasta") or predefined categories (food, service, price, ambience in the SemEval restaurant data).

A strong, simple approach with BERT is sentence-pair classification. Sun, Huang and Qiu (2019) build an auxiliary sentence from the aspect, feed "[CLS] review [SEP] aspect sentence [SEP]", and classify the [CLS] vector. One review becomes one example per aspect, and self-attention across the two segments links the aspect to the opinion words that describe it.

python
1from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline
2
3# sentence-level polarity: DistilBERT fine-tuned on SST-2
4clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
5print(clf(["not very good", "not terrible", "a great cast but a dull, boring plot"]))
6
7# aspect-based: one (sentence, aspect) pair per target, 4-way head
8name = "bert-base-uncased"
9tok = AutoTokenizer.from_pretrained(name)
10model = AutoModelForSequenceClassification.from_pretrained(
11 name, num_labels=4) # positive / negative / neutral / conflict; fine-tune before use
12review = "The pasta was great but the service was slow."
13batch = tok([review, review], ["food", "service"], # -> [CLS] review [SEP] aspect [SEP]
14 padding=True, return_tensors="pt")
15logits = model(**batch).logits # shape (2, 4): one row per aspect
A fine-tuned SST-2 classifier, and the sentence-pair input used for aspect-based sentiment (not run here)

Use when

  • +Lexicon and rule systems need no training data, are transparent, and work as a quick baseline.
  • +Phrase-level supervision (SST) teaches composition directly, and fine-tuned Transformers learn negation scope from data.
  • +Aspect-level output is far more useful for product analytics than one score per review.

Avoid when

  • −Lexicons fail on negation, contrast, sarcasm and domain-specific words.
  • −Fine-grained labels are noisy: annotators often disagree between adjacent classes, which caps SST-5 accuracy.
  • −ABSA needs aspect-level annotation, which is expensive, and multiplies inference cost by the number of aspects.

Check yourself

On a ±2 scale, "excellent" scores +2. With shift negation at s = 1.5, what does "not excellent" score, and why prefer that over switching?

Check yourself

Why did the Stanford Sentiment Treebank label every phrase rather than only whole sentences?

Remember

  • ✓Granularity: document (IMDb), sentence (SST-2 / SST-5), aspect (SemEval-2014 Task 4).
  • ✓Bag-of-words and lexicon sums fail on negation and contrast; SO-CAL-style shift negation is the classic rule-based patch.
  • ✓RNTN: p=tanh⁡([a;b]⊤V[a;b]+W[a;b])p=\tanh([a;b]^\top V[a;b]+W[a;b]) at every parse node, trained with a 5-class loss on all 215,154 SST phrases; 85.4% binary, 45.7% fine-grained.
  • ✓Fine-tuned Transformers learn scope from data; BERT-large reaches 94.9% on SST-2.
  • ✓ABSA as sentence-pair classification: [CLS] review [SEP] aspect [SEP] → polarity, one pass per aspect.
← back to Deep Learning roadmap