Skip to content

feat/improved bias calculation#29

Merged
jomkv merged 1 commit into
mainfrom
feat/bias-validation
Apr 22, 2026
Merged

feat/improved bias calculation#29
jomkv merged 1 commit into
mainfrom
feat/bias-validation

Conversation

@Mark-0909

@Mark-0909 Mark-0909 commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator

Major changes:

  • updated weights based on the discussed websites
  • force database session to expire old searches to eliminate state leakage.
  • stream can now track and dedup processed ids.

BIAS IMPLEMENTATION

A. Bias Divergence (The "Echo Chamber" Detector)
Goal: Measures how diverse your source set is.

The Logic: It looks at the spread of news sources across the political spectrum (from Left to Right).
The Math: It calculates the Standard Deviation of all source bias values. If all sources are in the same location (e.g., all are "Left-Center"), the deviation is 0. If they are spread from far-left to far-right, the deviation is high.
The Scale:
1.0 (Maximum Divergence): You have a perfectly represented sample of the whole political spectrum.
-1.0 (Minimum Divergence): You are in an echo chamber; all sources share the same ideological lean.

B. Bias Consistency (The "Polarization" Detector)
Goal: Detects if sources are "picking sides" based on their politics.

The Logic: It calculates the correlation between a source's lean and the verdict they gave.
The Math: It uses the Absolute Pearson Correlation Coefficient (r).
If Left sources always say "Support" and Right sources always say "Refute," then r is 1.0.
If their verdicts have nothing to do with their bias (e.g., everyone agrees or the disagreement is random), r is 0.0.
The Scale:
1.0 (High Consistency/Polarization): The claim is highly partisan. Ideology is a perfect predictor of the verdict.
-1.0 (No Consistency): There is no partisan pattern. The media is either in consensus or the disagreement is not driven by bias.

@github-actions

Copy link
Copy Markdown
## Accuracy Check Results

- Normal dataset (jomTestCases): **71.72%** (71/99)
- Negated dataset (negatedClaims): **81.82%** (81/99)

- Evaluation step outcome: success/success
- Run: https://github.com/jomkv/truescope-api/actions/runs/24721485779

@jomkv jomkv merged commit 726e3e5 into main Apr 22, 2026
1 check passed
@jomkv jomkv deleted the feat/bias-validation branch April 24, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants