Evidence-Grounded Chargeback Decision and Recovery Intelligence
Razorpay AI Buildathon 2026 · Track 02 — AI Risk Manager
VerifyFlow AI is an evidence-grounded chargeback decision system.
It helps merchants decide whether a dispute should be:
- FIGHT — contest the dispute because expected recovery exceeds expected cost.
- CONCEDE — accept the dispute because contesting is not economically justified.
- HUMAN_REVIEW — route the case to an analyst because evidence or confidence is insufficient.
VerifyFlow does not treat a model prediction as the final authority. The model estimates contestability. A deterministic policy evaluates evidence sufficiency, confidence, dispute value, operational cost, and expected recovery before producing a decision.
The system also generates an evidence packet in which factual claims must reference supplied evidence IDs.
Current implementation uses synthetic data for demonstration and evaluation. It is not presented as production-ready chargeback automation.
A chargeback should not be contested only because a model assigns it a high probability.
A contest is worthwhile only when:
expected recovery > total contest cost
The economic calculation is:
expected_recovery = p_win × dispute_amount
total_cost = chargeback_fee + evidence_cost + admin_cost
net_ev = expected_recovery - total_cost
The decision engine applies safety gates before using this calculation:
if evidence_completeness < minimum:
HUMAN_REVIEW
if confidence < minimum:
HUMAN_REVIEW
if net_ev > 0:
FIGHT
otherwise:
CONCEDE
This prevents the system from confusing:
- suspicious behaviour with provable evidence,
- model confidence with legal or operational certainty,
- high dispute value with automatic approval,
- fluent generated text with verified facts.
A merchant may believe a dispute is fraudulent but still lack enough evidence to contest it successfully.
VerifyFlow therefore calculates an evidence completeness score from observable evidence:
| Evidence component | Weight |
|---|---|
| Delivery proof | 25% |
| Authentication proof | 20% |
| Transaction match | 15% |
| Customer history | 15% |
| Customer communication | 10% |
| Dispute timing | 10% |
| Refund status | 5% |
Each component is normalized to a value between 0 and 1.
evidence_completeness =
0.25 × delivery_proof
+ 0.20 × authentication_proof
+ 0.15 × transaction_match
+ 0.15 × customer_history
+ 0.10 × communication_record
+ 0.10 × dispute_timing
+ 0.05 × refund_status
If evidence completeness falls below the configured threshold, VerifyFlow routes the case to human review instead of generating an unsupported contest recommendation.
Evidence can include:
- transaction authentication,
- delivery proof,
- delivery timestamp,
- customer communication,
- refund status,
- device consistency,
- IP consistency,
- order history,
- dispute reason,
- product or service fulfilment.
Every evidence item records its availability, reliability, source, and timestamp where available.
Dispute received
|
v
Evidence ingestion
|
v
Evidence normalization
|
v
Evidence completeness and reliability checks
|
v
Feature construction
|
v
Contestability probability
|
v
Deterministic policy gates
|
+--------------------+
| |
v v
Economic evaluation Human-review gate
| |
v v
FIGHT / CONCEDE HUMAN_REVIEW
|
v
Evidence packet generation
|
v
Citation and unsupported-claim validation
|
v
Audit record
VerifyFlow separates probabilistic components from decision authority.
The model estimates contestability probability.
It does not create the final decision.
The language model may extract facts or draft a narrative from supplied evidence.
It must not:
- invent missing facts,
- create unsupported claims,
- override the policy decision,
- fabricate evidence IDs,
- decide whether a dispute should be fought.
The deterministic policy engine is the decision authority.
It evaluates:
- evidence completeness,
- confidence,
- expected recovery,
- total cost,
- net expected value,
- reason codes,
- review requirements.
The system records decision inputs, policy versions, evidence references, and packet validation results.
VerifyFlow builds structured evidence before generating prose.
Example evidence record:
{
"evidence_id": "E1",
"claim": "Order marked delivered",
"value": true,
"source": "order_record",
"timestamp": "2026-08-30T14:22:00"
}Generated claims must reference evidence IDs:
{
"claim_id": "C1",
"claim": "The order was marked delivered.",
"evidence_ids": ["E1"],
"supported": true
}The packet validator checks:
- unsupported claim count,
- missing citation count,
- contradictory evidence,
- invalid evidence IDs,
- claims that do not match supplied evidence.
Target:
unsupported_claim_count = 0
missing_citation_count = 0
If packet validation fails, the system rejects the generated draft and uses a safer fallback.
Combines structured evidence, model probability, and deterministic economics.
Routes incomplete, contradictory, or low-confidence cases to an analyst.
Uses dispute amount and contest costs instead of relying only on a fixed probability threshold.
Explains why the system selected FIGHT, CONCEDE, or HUMAN_REVIEW.
Creates a structured packet with evidence references and a readable narrative.
Rejects unsupported generated claims.
The system can continue with deterministic behaviour when optional dependencies are unavailable.
Displays decision distribution, evidence completeness, expected value, review rate, and evaluation results.
Provides a simple white-and-blue interface for reviewing cases, inspecting evidence, and testing decisions.
- Open the Streamlit application.
- Select a synthetic dispute case.
- Review transaction and evidence details.
- Inspect evidence completeness.
- View the contestability probability.
- Review cost and expected recovery.
- Inspect the final policy decision.
- Open the generated evidence packet.
- Review citations and validation status.
- Inspect evaluation and production-readiness notes.
The demo is designed to show the complete decision path rather than only displaying a prediction score.
Dispute amount: ₹8,900
Estimated win probability: 0.62
Chargeback fee: ₹350
Evidence cost: ₹75
Administrative cost: ₹100
Expected recovery:
0.62 × ₹8,900 = ₹5,518
Total cost:
₹350 + ₹75 + ₹100 = ₹525
Net expected value:
₹5,518 - ₹525 = ₹4,993
Decision:
FIGHT
A case with incomplete delivery evidence may still be routed to:
HUMAN_REVIEW
even when its behavioural risk score is high.
That is intentional. Suspicion is not equivalent to evidence.
The project evaluates more than classification accuracy.
Relevant metrics include:
- precision,
- recall,
- F1 score,
- ROC-AUC,
- PR-AUC,
- calibration quality,
- evidence completeness,
- human-review rate,
- unsupported claim count,
- missing citation count,
- expected recovery,
- total contest cost,
- net expected value,
- baseline comparison,
- decision latency.
The evaluation should compare VerifyFlow against simple baselines such as:
- contest everything,
- concede everything,
- fixed probability threshold,
- fixed dispute-amount threshold,
- evidence-only heuristic.
A financial policy should be judged using money-denominated outcomes, not classification metrics alone.
The current demonstration uses synthetic dispute records.
Synthetic data is useful for:
- testing decision logic,
- testing missing evidence,
- testing contradictory evidence,
- testing policy boundaries,
- testing packet validation,
- testing fallback behaviour,
- reproducing evaluation runs.
Synthetic results do not establish production performance.
Real deployment would require:
- real labelled dispute outcomes,
- merchant-specific cost parameters,
- probability recalibration,
- reason-code-specific modelling,
- legal and compliance review,
- human approval workflow,
- monitoring for distribution shift,
- access controls and audit retention.
Synthetic data demonstration
| Requirement | Purpose |
|---|---|
| Real labelled dispute outcomes | Measure actual contest success |
| Merchant-specific cost configuration | Reflect real acquirer and analyst costs |
| Probability recalibration | Align predictions with observed outcomes |
| Reason-code-specific models | Handle different dispute mechanisms |
| Legal and compliance review | Validate operational and regulatory use |
| Human approval workflow | Prevent unsupported automatic submissions |
| Monitoring and drift detection | Detect changing dispute behaviour |
| Access control and audit retention | Protect sensitive merchant data |
These are not hidden weaknesses. They are explicit deployment boundaries.
The system should not be marketed as production-accurate until these requirements are satisfied.
VerifyFlow follows a defense-only design.
Important safeguards:
- no fabricated evidence,
- no unsupported packet claims,
- no automatic override by the language model,
- bounded optional-agent behaviour,
- explicit human-review path,
- secret values loaded from environment variables,
- no credentials committed to Git,
- structured validation at API boundaries,
- audit records for important decisions,
- graceful degradation when optional services fail.
Do not commit:
.env
API keys
private certificates
database files
model secrets
customer data
VerifyFlow-AI/
├── app.py
├── README.md
├── LICENSE
├── requirements.txt
├── make.ps1
├── Makefile
├── .gitignore
├── backend/
│ ├── src/
│ │ ├── api/
│ │ │ ├── main.py
│ │ │ ├── middleware.py
│ │ │ └── routes/
│ │ ├── config.py
│ │ ├── schemas/
│ │ ├── ingest/
│ │ ├── extraction/
│ │ ├── features/
│ │ ├── models/
│ │ ├── policy/
│ │ ├── packet/
│ │ ├── llm/
│ │ ├── audit/
│ │ └── storage/
│ ├── data_gen/
│ ├── eval/
│ └── tests/
├── data/
│ ├── synthetic/
│ └── examples/
├── docs/
└── scripts/
python -m pip install -r requirements.txtpython -m streamlit run app.py$env:PYTHONPATH="$PWD\backend\src"
pytest backend/tests -qpython -m compileall backend app.py$env:PYTHONPATH="$PWD\backend\src"
python -m uvicorn api.main:app --reloadAPI documentation:
http://127.0.0.1:8000/docs
PowerShell helper:
.\make.ps1 help
.\make.ps1 install
.\make.ps1 data
.\make.ps1 train
.\make.ps1 eval
.\make.ps1 test
.\make.ps1 serve
.\make.ps1 appIf the project uses only the Streamlit demonstration:
python -m streamlit run app.pyThe current user-facing application is Streamlit.
Recommended deployment flow:
- Push repository to GitHub.
- Open Streamlit Community Cloud.
- Select repository:
Soumadeep46/VerifyFlow-AI - Select branch:
main - Set main file:
app.py - Add required secrets only through the deployment platform.
- Deploy.
- Test every main navigation page.
- Verify synthetic-data disclaimer is visible.
- Share the generated application URL.
The repository no longer depends on a React, Vite, Netlify, or Vercel frontend.
Use environment variables for deployment-specific values.
Example:
VERIFYFLOW_ENV=demo
VERIFYFLOW_LOG_LEVEL=INFO
VERIFYFLOW_CHARGEBACK_FEE_INR=350
VERIFYFLOW_EVIDENCE_COST_INR=75
VERIFYFLOW_ADMIN_COST_INR=100
VERIFYFLOW_MIN_EVIDENCE_COMPLETENESS=0.50
VERIFYFLOW_MIN_CONFIDENCE=0.60
Do not place secrets directly in source code.
- Data is synthetic.
- The model has not been validated on merchant production data.
- Cost parameters are illustrative.
- Probability estimates require recalibration for each merchant.
- Dispute reason-code behaviour requires deeper scheme-specific validation.
- Evidence availability does not guarantee representment success.
- Human review remains necessary for ambiguous or contradictory cases.
- The Streamlit interface is a demonstration surface, not a complete merchant operations platform.
- Production deployment requires legal, compliance, privacy, and security review.
VerifyFlow is not merely a fraud classifier.
Its central contribution is the combination of:
evidence sufficiency
+
contestability estimation
+
cost-aware decisioning
+
human-review routing
+
citation-validated evidence packets
The system answers a more useful operational question:
“Can we prove this dispute well enough, and is contesting it economically justified?”
That question is different from:
“Does this transaction look suspicious?”
For a reviewer with limited time:
app.py— end-to-end Streamlit demonstration.backend/src/policy/engine.py— deterministic decision authority.backend/src/policy/economics.py— expected-value calculation.backend/src/evidence/completeness.py— evidence sufficiency logic.backend/src/packet/citation_validator.py— packet integrity checks.backend/eval/metrics.py— evaluation metrics.backend/tests/— behavioural and safety tests.docs/deployment-readiness.md— production boundary.
MIT License.