Leaderboard Score Direct Manipulation Vulnerability (#83)

[Ayaanshaikh12243]

Fix: Leaderboard Score Direct Manipulation Vulnerability (#83)

close #83

Overview

This PR implements server-side validation, session-based score verification, and an admin review system to prevent teams from cheating on the leaderboard through direct score manipulation.

Problem

Leaderboard points were calculated client-side before submission, with no server-side validation:

• Teams could submit fake time_spent values (0 seconds, negative values)
• Hints and attempts counts could be falsified
• No verification that submitted data matches actual session timing
• Race conditions could cause double-point awards
• Impossible completion times (solved in <1 second) accepted without question
• No audit trail or admin ability to review suspicious scores
• Legitimate teams disadvantaged against cheaters

Solution Implemented

1) Server-Side Score Validation & Calculation

Modified file:

• supabase/functions/validate-flag/index.ts

Changes:

• Session-based validation: Compare submitted time_spent against elapsed time from session_start_time
• Value sanitization: Clamp all scores to reasonable ranges (time: 0-86400s, attempts: 1-1000, hints: 0-100)
• Impossible value rejection: Reject negative time, zero attempts, unreasonable hints
• Server-calculated points: Never trust client-provided points
  • Base: 100 points
  • Time bonus: +50 (if <5 min), +25 (if <10 min)
  • Hints penalty: -5 per hint used
  • Attempts bonus: +25 (1st try), +10 (2-3 attempts)
  • Minimum: 50 points (guaranteed participation award)
• Validation function call: Invoke register_leaderboard_submission() RPC to check for score anomalies
• Anomaly detection: Flag submissions with timing mismatches, impossible patterns, hint/attempt discrepancies

2) Database-Layer Integrity Tracking

New migration:

• Databases/supabase/migrations/20260304_leaderboard_score_integrity.sql

Adds:

• challenge_sessions table: Track per-challenge per-team interactions

  • hint_reveal_count: Server-authoritative hints used count
  • wrong_attempt_count: Server-authoritative wrong attempts count
  • submitted_*: Client-submitted values for comparison
  • flagged_for_review: Auto-set if validation fails

• leaderboard_integrity_flags table: Admin review queue

  • flag_type: 'time_anomaly', 'impossible_time', 'hints_mismatch', 'attempts_mismatch', etc.
  • severity: 'low', 'medium', 'high', 'critical'
  • evidence: JSON with submitted vs stored values for audit
  • reviewed_by, action, admin_notes: Admin response tracking

• Validation columns on leaderboard:

  • validation_level: 'unvalidated', 'validated', 'flagged', 'rejected'
  • validation_warnings: Array of anomalies detected
  • session_start_time, server_received_time: Timing audit trail
  • points, hints_used: From server calculation, not client

• Validation function: validate_leaderboard_submission()

  • Checks 1: Negative/invalid values → CRITICAL
  • Checks 2-6: Time anomalies, hints/attempts mismatches, suspicious patterns → HIGH/MEDIUM/LOW
  • Returns: validation status, warnings, severity level, flag reason

• Auto-flagging triggers:

  • enforce_leaderboard_integrity(): Validates on INSERT, sets validation_level
  • flag_suspicious_leaderboard_entries(): Creates flag record for review if anomalies detected

• Helper functions:

  • record_hint_reveal(): Increment server-side hint count when hint revealed
  • record_wrong_attempt(): Increment server-side wrong attempt count on failed submission

• Admin views:

  • admin_review_queue: Unflagged submissions awaiting review (sorted by severity)
  • leaderboard_with_validation: All submissions with validation status

3) Frontend Session Tracking (Anti-Cheat Instrumentation)

Modified file:

• src/components/ChallengePage.tsx

Changes:

• Challenge session registration: When challenge loads, upsert to challenge_sessions with session_start_time
• Hint reveal tracking: When hint revealed, call record_hint_reveal() RPC to increment server counter
• Ensures server has authoritative count of hints/attempts for validation at submission time

4) Admin Review Queue & Actions

New Edge Functions:

get-admin-review-queue/index.ts:

• Retrieve all flagged submissions awaiting review
• Filter by severity, sort by severity/date/team
• Pagination support
• Return statistics: total flagged, count by severity
• For admins to review suspicious scores

admin-flag-action/index.ts:

• Admin approves, rejects, or penalizes flagged submissions
• Actions:
  • approved: Trust the submission, mark reviewed
  • rejected: Discard the submission from leaderboard
  • penalize: Remove 25% of earned points (minimum 50) from team as punishment
• Audit trail: Records which admin took action and when
• Deducts penalty automatically on penalize action

Security Improvements

• Server-side authority: All score calculations happen server-side, client values only used for context
• Timing validation: Compare submitted time against true session elapsed time
• Attempt verification: Server counters prevent false claim of "1st attempt"
• Hint integrity: Database tracks actual hint reveals, not claimed hints
• Anomaly detection: Automatic flagging of impossible patterns (0-second solutions, negative time, etc.)
• Audit trail: All scores have timing data, validation status, and admin review trail
• Penalty system: Cheaters lose points, deterring future attempts
• Defense in depth: Client tracking + server validation + database constraints + admin review

Files Changed

• Databases/supabase/migrations/20260304_leaderboard_score_integrity.sql – new database layer with validation & flagging
• supabase/functions/validate-flag/index.ts – server-side score validation and calculation
• supabase/functions/get-admin-review-queue/index.ts – new Edge function for admin review
• supabase/functions/admin-flag-action/index.ts – new Edge function for admin actions
• src/components/ChallengePage.tsx – session tracking and hint recording

How It Works (End-to-End)

1. Challenge starts: ChallengePage registers session in challenge_sessions table
2. Team submits flag: Sends submitted_time_spent, submitted_attempts, submitted_hints_used
3. Server validates:
   • Calls register_leaderboard_submission() RPC
   • Compares submitted values against server counters
   • Checks for timing anomalies (submitted > elapsed time)
   • Detects impossible patterns (< 1 second, negative values, etc.)
4. Points calculated server-side:
   • Time bonus based on elapsed time, not submitted time
   • Hints penalty based on server counter, not claimed hints
   • Attempts bonus verified against wrong attempt count
5. Leaderboard record created:
   • If validation passes: validation_level = 'validated'
   • If warnings detected: validation_level = 'flagged', flag record created
   • Admin review queue populated with flagged entries
6. Admin review (asynchronous):
   • Admin reviews flagged submission evidence
   • Approves (trust submission), rejects (discard), or penalizes (deduct points)
   • Admin notes recorded for audit

Testing Checklist

• Submit valid flag with reasonable time → validation_level = 'validated', correct points awarded
• Submit with time_spent = 0 → flagged as impossible_time, high severity
• Submit with time_spent > actual_elapsed_time + 5s → flagged as time_anomaly, medium severity
• Submit with hints_used = 0 but revealed 3 hints → flagged as hints_mismatch
• Submit with attempts = 1 but 5 wrong attempts recorded → flagged as attempts_mismatch
• View admin review queue → see all flagged submissions with severity breakdown
• Admin approves flag → submission kept, marked reviewed, removed from queue
• Admin penalizes flag → 25% points deducted from team, marked reviewed
• Verify challenge_sessions table populated after submission
• Verify leaderboard_integrity_flags table has audit evidence (JSON)

Deployment Notes

1. Run database migration: supabase db push
2. Deploy Edge Functions:
   • supabase functions deploy validate-flag
   • supabase functions deploy get-admin-review-queue
   • supabase functions deploy admin-flag-action
3. Test end-to-end: submit a valid flag, verify leaderboard entry, check validation status
4. Monitor admin review queue for any legitimate false positives
5. Admins review flagged submissions and take action

Follow-up

• Create admin dashboard to visualize leaderboard integrity metrics
• Add webhook notifications for critical/high severity flags (real-time admin alerting)
• Implement team reputation score (automatic penalties across multiple flags)
• Add rate limiting on leaderboard submission Edge Function to prevent spam
• Periodic administrative review of approved submissions to catch systematic cheaters

[vercel]

@Ayaanshaikh12243 is attempting to deploy a commit to the csxark's projects Team on Vercel.

A member of the Team first needs to authorize it.