Definition
Sentiment analysis predicts the evaluative polarity expressed in text: binary (positive/negative), fine-grained (five classes from very negative to very positive, as in SST-5), or per aspect (one polarity for each target, such as the food and the service of one restaurant review). The landmark neural result is the Recursive Neural Tensor Network trained on the Stanford Sentiment Treebank (Socher et al., 2013).
As a machine-learning problem, sentence- or document-level sentiment is Text classification with a polarity label set, and the same encoder-plus-softmax head applies. This article covers what is specific to sentiment: compositionality, negation, and aspects.
Early systems summed word scores from a sentiment lexicon such as the MPQA subjectivity lexicon (Wilson, Wiebe and Hoffmann, 2005) or SentiWordNet (Esuli and Sebastiani, 2006). Pang, Lee and Vaithyanathan (2002) trained naive Bayes, maximum-entropy and SVM classifiers on bag-of-words features of movie reviews and found sentiment harder than topic classification. The reason is that polarity depends on how words combine.
A bag of words cannot tell "not good" from "good", and it scores "a great cast but a dull, boring plot" as neutral because the +2 and the two −1s cancel. Lexicon systems patch this with hand rules. SO-CAL (Taboada et al., 2011) uses shift negation, which moves a score a fixed amount toward the opposite pole instead of flipping its sign, so "not excellent" stays mildly positive.
Compositionality: the Sentiment Treebank
Socher et al. (2013) parsed 11,855 movie-review sentences and had crowd workers label every phrase in every parse tree, 215,154 unique phrases in all, on a five-point scale. This turned compositionality into supervision: a model sees not only that "not very good" is negative but also that "very good" is very positive.
The Recursive Neural Tensor Network (RNTN) composes vectors bottom-up. A bilinear tensor term lets each child change how the other is interpreted, which a plain recursive network can express only weakly:
- vectors of the left and right child (word vectors at the leaves)
- tensor; slice gives coordinate of the bilinear term
- composition and classifier weights, shared by every node of every tree
Training applies cross-entropy at every node, not just the root. The RNTN reached 85.4% on binary and 45.7% on five-class sentence-level accuracy, and it handled negated phrases and "X but Y" contrasts better than bag-of-words baselines. Tree-LSTMs (Tai, Socher and Manning, 2015) later replaced the tensor with gated composition.
Fine-tuned Transformers then made parse trees unnecessary as input: self-attention learns which word scopes over which. BERT-large scores 94.9% on the GLUE SST-2 test set. The Treebank's phrase labels are still used as extra training data.
Datasets
- SST-5
- Five-class sentence sentiment; 8,544 train, 1,101 dev, 2,210 test sentences, plus phrase-level labels for training.
- SST-2
- The same sentences with neutral removed, binary; part of GLUE.
- IMDb
- 50,000 polar movie reviews, 25,000 train and 25,000 test (Maas et al., 2011).
- SemEval-2014 Task 4
- Aspect-based sentiment on laptop and restaurant reviews (Pontiki et al., 2014): aspect term extraction and polarity, aspect category detection and polarity; labels positive, negative, neutral and conflict.
Negation, scope and contrast
The figure below makes composition explicit with hand-written rules on a parse tree, the kind of behaviour an RNTN or a Transformer must learn from data. With a negator applied to a subtree scored , the two classic rules are:
- score of the negated phrase, here on a scale
- shift amount; SO-CAL uses 4 on its ±5 scale
Worked example: a great cast but a dull, boring plot (tree rules)
- 1Leaves: great , dull , boring ; "a", "cast", ",", "plot" .
- 2Left clause: "great cast" , "a great cast" .
- 3Right clause: "boring plot" , ", boring plot" , "dull , boring plot" , "a dull , boring plot" .
- 4Contrast: , which bins to negative. The lexicon sum bins to neutral.
Aspect-based sentiment
"The pasta was great but the service was slow" has no single correct sentence label. Aspect-based sentiment analysis (ABSA) instead predicts a polarity for each target. The targets are either aspect terms in the text ("pasta") or predefined categories (food, service, price, ambience in the SemEval restaurant data).
A strong, simple approach with BERT is sentence-pair classification. Sun, Huang and Qiu (2019) build an auxiliary sentence from the aspect, feed "[CLS] review [SEP] aspect sentence [SEP]", and classify the [CLS] vector. One review becomes one example per aspect, and self-attention across the two segments links the aspect to the opinion words that describe it.
1from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline23# sentence-level polarity: DistilBERT fine-tuned on SST-24clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")5print(clf(["not very good", "not terrible", "a great cast but a dull, boring plot"]))67# aspect-based: one (sentence, aspect) pair per target, 4-way head8name = "bert-base-uncased"9tok = AutoTokenizer.from_pretrained(name)10model = AutoModelForSequenceClassification.from_pretrained(11 name, num_labels=4) # positive / negative / neutral / conflict; fine-tune before use12review = "The pasta was great but the service was slow."13batch = tok([review, review], ["food", "service"], # -> [CLS] review [SEP] aspect [SEP]14 padding=True, return_tensors="pt")15logits = model(**batch).logits # shape (2, 4): one row per aspectUse when
- +Lexicon and rule systems need no training data, are transparent, and work as a quick baseline.
- +Phrase-level supervision (SST) teaches composition directly, and fine-tuned Transformers learn negation scope from data.
- +Aspect-level output is far more useful for product analytics than one score per review.
Avoid when
- −Lexicons fail on negation, contrast, sarcasm and domain-specific words.
- −Fine-grained labels are noisy: annotators often disagree between adjacent classes, which caps SST-5 accuracy.
- −ABSA needs aspect-level annotation, which is expensive, and multiplies inference cost by the number of aspects.
Check yourself
On a ±2 scale, "excellent" scores +2. With shift negation at s = 1.5, what does "not excellent" score, and why prefer that over switching?
Check yourself
Why did the Stanford Sentiment Treebank label every phrase rather than only whole sentences?
Remember
- ✓Granularity: document (IMDb), sentence (SST-2 / SST-5), aspect (SemEval-2014 Task 4).
- ✓Bag-of-words and lexicon sums fail on negation and contrast; SO-CAL-style shift negation is the classic rule-based patch.
- ✓RNTN: at every parse node, trained with a 5-class loss on all 215,154 SST phrases; 85.4% binary, 45.7% fine-grained.
- ✓Fine-tuned Transformers learn scope from data; BERT-large reaches 94.9% on SST-2.
- ✓ABSA as sentence-pair classification: [CLS] review [SEP] aspect [SEP] → polarity, one pass per aspect.