# Sentiment analysis

> Predict the polarity of a text, sentence or aspect (binary, 5-way or per target); neural models beat lexicons by composing meaning, so “not very good” and “great cast but dull plot” come out right.

Source: https://www.mlfoundry.in/learn/sentiment-analysis  
Section: Deep Learning › Deep Learning for NLP  
Level: core · 8 min read · updated 2026-10-15

**Sentiment analysis:** predicts the evaluative polarity expressed in text: binary (positive/negative), fine-grained (five classes from very negative to very positive, as in SST-5), or per **aspect** (one polarity for each target, such as the food and the service of one restaurant review). The landmark neural result is the Recursive Neural Tensor Network trained on the Stanford Sentiment Treebank (Socher et al., 2013).

As a machine-learning problem, sentence- or document-level sentiment is **Text classification** with a polarity label set, and the same encoder-plus-softmax head applies. This article covers what is specific to sentiment: compositionality, negation, and aspects.

Early systems summed word scores from a **sentiment lexicon** such as the MPQA subjectivity lexicon (Wilson, Wiebe and Hoffmann, 2005) or SentiWordNet (Esuli and Sebastiani, 2006). Pang, Lee and Vaithyanathan (2002) trained naive Bayes, maximum-entropy and SVM classifiers on bag-of-words features of movie reviews and found sentiment harder than topic classification. The reason is that polarity depends on how words combine.

A bag of words cannot tell "not good" from "good", and it scores "a great cast but a dull, boring plot" as neutral because the +2 and the two −1s cancel. Lexicon systems patch this with hand rules. SO-CAL (Taboada et al., 2011) uses **shift negation**, which moves a score a fixed amount toward the opposite pole instead of flipping its sign, so "not excellent" stays mildly positive.

## Compositionality: the Sentiment Treebank

Socher et al. (2013) parsed 11,855 movie-review sentences and had crowd workers label every phrase in every parse tree, 215,154 unique phrases in all, on a five-point scale. This turned compositionality into supervision: a model sees not only that "not very good" is negative but also that "very good" is very positive.

> Figure: Left: in the Treebank every node of the parse carries one of five labels, and a recursive network computes each parent from its two children with the same weights everywhere, so "very good" (++) and "not very good" (−) are both supervised. Right: aspect-based sentiment pairs the sentence with each aspect and classifies the pair, giving positive for food and negative for service. Bottom: the three granularities and their standard datasets.

The Recursive Neural Tensor Network (RNTN) composes vectors bottom-up. A bilinear tensor term lets each child change how the other is interpreted, which a plain recursive network $p=\tanh(W[a;b])$ can express only weakly:

$$
p=\tanh\!\Big([a;b]^\top V^{[1:d]}[a;b]+W[a;b]\Big),\qquad \hat y_p=\mathrm{softmax}(W_s\,p)\in\Delta^{5}
$$

- $a,b\in\mathbb{R}^d$: vectors of the left and right child (word vectors at the leaves)
- $V^{[1:d]}\in\mathbb{R}^{2d\times2d\times d}$: tensor; slice $i$ gives coordinate $i$ of the bilinear term $[a;b]^\top V^{[i]}[a;b]$
- $W\in\mathbb{R}^{d\times2d},\;W_s\in\mathbb{R}^{5\times d}$: composition and classifier weights, shared by every node of every tree

Training applies cross-entropy at **every** node, not just the root. The RNTN reached 85.4% on binary and 45.7% on five-class sentence-level accuracy, and it handled negated phrases and "X but Y" contrasts better than bag-of-words baselines. Tree-LSTMs (Tai, Socher and Manning, 2015) later replaced the tensor with gated composition.

Fine-tuned Transformers then made parse trees unnecessary as input: self-attention learns which word scopes over which. BERT-large scores 94.9% on the GLUE SST-2 test set. The Treebank's phrase labels are still used as extra training data.

**Datasets**

- **SST-5:** Five-class sentence sentiment; 8,544 train, 1,101 dev, 2,210 test sentences, plus phrase-level labels for training.
- **SST-2:** The same sentences with neutral removed, binary; part of GLUE.
- **IMDb:** 50,000 polar movie reviews, 25,000 train and 25,000 test (Maas et al., 2011).
- **SemEval-2014 Task 4:** Aspect-based sentiment on laptop and restaurant reviews (Pontiki et al., 2014): aspect term extraction and polarity, aspect category detection and polarity; labels positive, negative, neutral and conflict.

## Negation, scope and contrast

The figure below makes composition explicit with hand-written rules on a parse tree, the kind of behaviour an RNTN or a Transformer must learn from data. With a negator applied to a subtree scored $v$, the two classic rules are:

$$
\text{switch: } \mathrm{not}(v)=-v,\qquad \text{shift: } \mathrm{not}(v)=v-s\cdot\operatorname{sign}(v)
$$

- $v$: score of the negated phrase, here on a $[-2,2]$ scale
- $s$: shift amount; SO-CAL uses 4 on its ±5 scale

> Figure: Sentiment computed bottom-up over the parse tree with hand-set rules (shift negation with s = 1.5 by default; the root is binned into the five SST classes). "not very good": the lexicon sum says +1 (positive); switch negation turns very good's +1.5 into −1.5 (very negative); shift gives 0 (neutral). "not terrible": lexicon −2, switch +2 (very positive, too strong), shift −0.5 (mildly negative). The contrast "a great cast but a dull, boring plot" is 0 for the lexicon but −1 (negative) once the clause after "but" gets weight 0.75. Raise s to 2 and "not terrible" becomes exactly neutral.

**Worked example: a great cast but a dull, boring plot (tree rules)**

1. Leaves: great $=+2$, dull $=-1$, boring $=-1$; "a", "cast", ",", "plot" $=0$.
2. Left clause: "great cast" $=2+0=2$, "a great cast" $=2$.
3. Right clause: "boring plot" $=-1$, ", boring plot" $=-1$, "dull , boring plot" $=-1+(-1)=-2$, "a dull , boring plot" $=-2$.
4. Contrast: $0.25\cdot2+0.75\cdot(-2)=0.5-1.5=-1$, which bins to **negative**. The lexicon sum $2-1-1=0$ bins to neutral.

## Aspect-based sentiment

"The pasta was great but the service was slow" has no single correct sentence label. Aspect-based sentiment analysis (ABSA) instead predicts a polarity for each target. The targets are either aspect terms in the text ("pasta") or predefined categories (food, service, price, ambience in the SemEval restaurant data).

A strong, simple approach with BERT is sentence-pair classification. Sun, Huang and Qiu (2019) build an auxiliary sentence from the aspect, feed "[CLS] review [SEP] aspect sentence [SEP]", and classify the [CLS] vector. One review becomes one example per aspect, and self-attention across the two segments links the aspect to the opinion words that describe it.

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

# sentence-level polarity: DistilBERT fine-tuned on SST-2
clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
print(clf(["not very good", "not terrible", "a great cast but a dull, boring plot"]))

# aspect-based: one (sentence, aspect) pair per target, 4-way head
name = "bert-base-uncased"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(
    name, num_labels=4)        # positive / negative / neutral / conflict; fine-tune before use
review = "The pasta was great but the service was slow."
batch = tok([review, review], ["food", "service"],      # -> [CLS] review [SEP] aspect [SEP]
            padding=True, return_tensors="pt")
logits = model(**batch).logits                          # shape (2, 4): one row per aspect
```

A fine-tuned SST-2 classifier, and the sentence-pair input used for aspect-based sentiment (not run here)

> **Polarity is domain-dependent:** "Unpredictable" praises a film plot and condemns a car's steering; "long" is good for battery life and bad for a queue. Lexicons and models trained on one domain lose accuracy on another, so evaluate on in-domain data. Sarcasm ("great, another delay") reverses surface polarity and remains a common error.

**Strengths**

- Lexicon and rule systems need no training data, are transparent, and work as a quick baseline.
- Phrase-level supervision (SST) teaches composition directly, and fine-tuned Transformers learn negation scope from data.
- Aspect-level output is far more useful for product analytics than one score per review.

**Limitations**

- Lexicons fail on negation, contrast, sarcasm and domain-specific words.
- Fine-grained labels are noisy: annotators often disagree between adjacent classes, which caps SST-5 accuracy.
- ABSA needs aspect-level annotation, which is expensive, and multiplies inference cost by the number of aspects.

**Question:** On a ±2 scale, "excellent" scores +2. With shift negation at s = 1.5, what does "not excellent" score, and why prefer that over switching?

- A. −2: negation always flips the sign
- B. +0.5: it moves toward negative but stays mildly positive, matching how people read it
- C. 0: negation always gives neutral
- D. +2: negators carry no sentiment

**Answer:** B. +0.5: it moves toward negative but stays mildly positive, matching how people read it — Shift gives $2-1.5\cdot\operatorname{sign}(2)=0.5$. "Not excellent" usually means "fine but not great", not "terrible", which is what switching ($-2$) would claim.

**Question:** Why did the Stanford Sentiment Treebank label every phrase rather than only whole sentences?

- A. To make the dataset larger without collecting more reviews, and to supervise how sentiment composes at every node
- B. Because sentence labels were too expensive
- C. Because recursive networks cannot be trained on sentence labels
- D. To support aspect-based sentiment

**Answer:** A. To make the dataset larger without collecting more reviews, and to supervise how sentiment composes at every node — The 11,855 sentences yield 215,154 labelled phrases, and a loss at every node tells the model how each composition ("very" + "good", "not" + "very good") should change polarity, instead of inferring it from one root label.

**Key points**

- Granularity: document (IMDb), sentence (SST-2 / SST-5), aspect (SemEval-2014 Task 4).
- Bag-of-words and lexicon sums fail on negation and contrast; SO-CAL-style shift negation is the classic rule-based patch.
- RNTN: $p=\tanh([a;b]^\top V[a;b]+W[a;b])$ at every parse node, trained with a 5-class loss on all 215,154 SST phrases; 85.4% binary, 45.7% fine-grained.
- Fine-tuned Transformers learn scope from data; BERT-large reaches 94.9% on SST-2.
- ABSA as sentence-pair classification: [CLS] review [SEP] aspect [SEP] → polarity, one pass per aspect.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
