Methodology
This methodology is continuously refined based on empirical data. All changes are versioned and documented.
How the Rating Works
Every analyzed article from a media outlet receives an individual score. Article scores are aggregated into an overall outlet rating. The entire process is fully automated — with no editorial influence or outlet-specific weighting.
1. Article Score: Four-Factor Model
Each text is analyzed for over 30 persuasion techniques across 9 categories. The score (0-100) consists of four weighted components:
Density
max. 40 pointsMeasures the frequency of detected techniques per 100 words. Uses logarithmic scaling (Zipf distribution), so the first techniques carry more weight than additional accumulations.
Intensity
max. 35 pointsCombines the severity of each technique (low/medium/high) with category relevance and text position. Techniques at the beginning and end of a text receive slightly higher weight (primacy-recency effect).
Coverage
max. 15 pointsMeasures the diversity of techniques used via Shannon entropy. A text using many different categories scores higher than one repeating a single technique.
Compound Effect
max. 10 pointsCaptures reinforcement effects when multiple techniques co-occur in the same text passage. Reinforcement grows exponentially with group size.
Normalization
The raw score (0-100) is transformed using a sigmoid function: Score = 100 × (1 − e^(−Raw/50)). This produces diminishing returns at the extremes — a moderate increase in techniques has more impact than a further increase at already high density.
2. Outlet Rating: Aggregation
Individual article scores are aggregated into an overall outlet rating:
Source Weighting
Each analysis receives a credibility weight based on its source: Registered users (1.0), Automated (0.8), Anonymous (0.7), Seed data (0.5).
Trimmed Mean
With 10 or more analyses, the top and bottom 10% of scores are excluded for outlier resistance. Below 10 analyses, all data points are used.
Inversion
The composite score is inverted: Composite = 100 − Average. Fewer detected techniques = higher score = better rating.
Preliminary Flag
Ratings based on fewer than 10 analyses are marked as "preliminary" to indicate the limited data basis.
3. Grade Scale
The composite score is mapped to a letter grade. Thresholds are calibrated so that the B range — where most differentiation occurs — receives the widest spread.
| Grade | Composite | Avg. Techniques | Meaning |
|---|---|---|---|
| A+ | ≥ 97 | ≤ 3% | No persuasion techniques detected |
| A | ≥ 92 | ≤ 8% | Nearly free of persuasion techniques |
| A- | ≥ 85 | ≤ 15% | Very few persuasion techniques |
| B+ | ≥ 78 | ≤ 22% | Light persuasion techniques |
| B | ≥ 72 | ≤ 28% | Moderate persuasion techniques |
| B- | ≥ 65 | ≤ 35% | Notable persuasion techniques |
| C+ | ≥ 55 | ≤ 45% | Significant persuasion techniques |
| C | ≥ 46 | ≤ 54% | Heavy persuasion techniques |
| C- | ≥ 38 | ≤ 62% | Very heavy persuasion techniques |
| D | ≥ 18 | ≤ 82% | Predominantly persuasive |
| F | < 18 | > 82% | Almost entirely persuasive |
4. Trend Calculation
For each outlet, 7-day and 30-day trends are calculated. These compare the source-weighted average of recent analyses against the current composite score. A positive trend indicates improvement (fewer techniques detected), a negative trend indicates decline.
5. Neutrality Principles
- ✓Fully automated — no editorial influence on scores or ratings
- ✓No outlet-specific weighting — every analysis is processed identically
- ✓No assumptions — the methodology neither favors nor penalizes any outlet regardless of its orientation or reputation
- ✓Traceable — every rating can be traced back to its underlying individual analyses
- ✓Versioned — methodology changes are documented and published
6. Scientific Foundations
The scoring algorithm is based on established research approaches:
- Propaganda Analysis (Institute for Propaganda Analysis, 1937-1942) — Taxonomy of persuasion techniques
- Shannon Information Theory (1948) — Entropy-based diversity measurement
- Psycholinguistics (Weber-Fechner, Zipf) — Logarithmic perception scaling
- Memory Research (Ebbinghaus, 1885) — Primacy-recency effects
- Media Analysis (Herman & Chomsky, 1988) — Propaganda model and framing
7. Recalibration
The current grade scale (v1.0) is based on domain-informed estimates. Once sufficient empirical data is available (target: 500+ analyses, 50+ outlets), the scale will be recalibrated based on the actual score distribution. Each recalibration will be published as a new methodology version with a complete changelog.