The question
Shoppers lean on star ratings, but the text of a review often tells a different story. We measured how often the sentiment of the text and the star rating disagree, and tested two explanations: delivery problems, and how much people write.
Data and methods
A balanced sample of 100,000 verified Amazon Electronics reviews, 20,000 per star level. NLTK preprocessing, VADER sentiment, a TF-IDF and logistic regression classifier, LDA topics and a keyword-based logistics proxy checked with keyword-in-context reading. Every claim is backed by a significance test and an effect size, with robustness checks across thresholds, review length and time.
Findings
| Question | Result |
|---|---|
| How big is the gap? | 44.5% of reviews mismatch, Cramér's V = 0.32, stable across thresholds and periods |
| Do logistics complaints matter? | They cost 0.60 stars on average, Cohen's d = 0.43 |
| Does length matter? | Long reviews mismatch more: 50.2% vs 40.2% |
| Can text alone predict the rating? | Partly: macro F1 = 0.655 |
Negative reviews are often written in a factual tone with few evaluative words, which a lexicon model reads as neutral. Part of the gap is therefore a limit of VADER, and a transformer model fine-tuned on reviews is the next step.
