Competition and Experimental Research Engineering Skill

English | 简体中文

A reusable human-AI protocol for multi-round competitions and experiment-driven papers. It anchors decisions to verifiable evidence, persistent notes, metric contracts, claim-evidence links, explicit route states, and immutable baselines.

The method was distilled from an LLM inference optimization competition: 23 finalists from 200+ teams, sixth place in the national final, and an online-stage score improvement from 71.9 to 89.11 with no rule violations. The repository generalizes that experience, but each competition and research type still needs evaluation-specific adaptation.

Scope

Use it for projects that:

• Run through multiple experiment rounds over days or weeks.
• Have a score, leaderboard, benchmark, performance target, or replication criterion.
• Need human and AI collaborators to resume across sessions.
• Carry meaningful submission, compute, compliance, or reproducibility risk.
• Turn competition engineering into a paper, technical report, or reproducible artifact.
• Need to adopt a mature workspace with substantial experiment history.

It should not activate automatically for prose-only writing, ordinary literature reviews, one-off benchmarks, or non-experimental code changes.

Core mechanisms

Mechanism	Purpose
Evidence hierarchy	Orders official results, raw artifacts, Git state, notes, and chat claims
Single dynamic entry	00_先看这里.md holds the metric contract, current recommendation, and next action
Conditional blockers	Stop on conflicting unknown changes, missing authorization, or invalid evidence
Experiment states	Separate pass, fail, inconclusive, and invalid outcomes
Separate ledgers	_evaluations.md for local evaluation and _submissions.md for real external submissions
Immutable baseline	Identifies baselines by commit SHA or content hash
Claim-evidence	Traces each paper claim to experiments, code, data, statistics, and figures
Workspace audit	Detects stale paths, authority drift, duplicate directories, and oversized AGENTS

Installation

Codex

User-level installation:

New-Item -ItemType Directory -Force "$HOME\.agents\skills" | Out-Null
git clone https://github.com/yfgug/competition-engineering-skill.git "$HOME\.agents\skills\competition-engineering"

Repository-scoped installation:

New-Item -ItemType Directory -Force ".agents\skills" | Out-Null
git clone https://github.com/yfgug/competition-engineering-skill.git ".agents\skills\competition-engineering"

These are the current official Codex skill discovery paths. The repository includes agents/openai.yaml. Invoke it explicitly with $competition-engineering in Codex or @competition-engineering in ChatGPT, or allow normal automatic selection.

Claude Code

mkdir -p .claude/skills
cp -r competition-engineering .claude/skills/

AGENTS.md or rule-based agents

Read and follow <path-to>/competition-engineering/SKILL.md for multi-round competition and experimental-research work in this repo.

The skill instructions are currently Chinese-first. The host must let the agent read SKILL.md and access references/, assets/, and scripts/ as needed.

Scaffold a project

Competition project:

node .\scripts\scaffold.cjs "D:\path\to\competition" --dry-run
node .\scripts\scaffold.cjs "D:\path\to\competition"

Experimental research project:

node .\scripts\scaffold.cjs "D:\path\to\research" --profile research --dry-run
node .\scripts\scaffold.cjs "D:\path\to\research" --profile research

The research profile adds paper/CLAIMS.md, paper/ARTIFACTS.md, and data/README.md. Existing files are skipped unless --force is explicitly supplied.

<project>/
|-- 00_先看这里.md
|-- README.md
|-- AGENTS.md
|-- notes/
|-- scripts/
|-- results/
|-- backups/
|-- source/
|-- deliver/
|-- archive/
|-- tmp/
|-- paper/                 # research profile
`-- data/                  # research profile

Adopt an existing workspace

Run the read-only audit before applying templates:

node .\scripts\audit_workspace.cjs "D:\path\to\existing-project"
node .\scripts\audit_workspace.cjs "D:\path\to\existing-project" --json
node .\scripts\audit_workspace.cjs "D:\path\to\existing-project" --ready --strict
node .\scripts\audit_workspace.cjs "D:\path\to\existing-project" --ready --profile competition --strict

It reports missing entrypoints, stale absolute paths, potentially duplicate directories, oversized AGENTS files, legacy notes, and notes newer than the current entry. --ready also detects unresolved placeholders in the active competition or research handoff files. For a legacy workspace whose entry does not declare its project type, pass --profile competition or --profile research explicitly; a conflicting entry declaration remains a warning.

For a large first-party scan, exclude copied repositories, generated documentation, or vendor trees explicitly and raise the deterministic scan limit only when needed:

node .\scripts\audit_workspace.cjs "D:\path\to\existing-project" --all `
  --exclude "**/vendor/**" --exclude "analysis_results/**/repo" --max-files 20000

--exclude accepts repeatable relative-path globs. Warnings are investigation leads; the audit never modifies the target.

Quick start

1. Define the project type, primary metric, direction, constraints, noise method, and promotion rule in 00_先看这里.md.
2. Translate official rules, research protocols, and data licenses into stable constraints in AGENTS.md.
3. Establish a Git baseline or content-hash snapshot and reproduce the baseline.
4. Record local evaluations in _evaluations.md and real external submissions in _submissions.md.
5. Preserve hypotheses, boundaries, raw evidence, and outcomes in experiment notes.
6. For papers, trace claims and figures through paper/CLAIMS.md and paper/ARTIFACTS.md.

Repository layout

|-- SKILL.md
|-- CITATION.cff
|-- CHANGELOG.md
|-- agents/openai.yaml
|-- references/
|   |-- methodology.md
|   |-- evaluation.md
|   |-- research.md
|   |-- adoption.md
|   |-- templates.md
|   `-- adaptations.md
|-- assets/scaffold/
|-- assets/research/
|-- scripts/scaffold.cjs
|-- scripts/audit_workspace.cjs
|-- examples/qwen-inference-optimization.md
`-- tests/

Case study

The sanitized LLM inference optimization case study shows how to adopt a mature workspace without bulk-renaming historical notes or immediately moving overlapping result directories. It also demonstrates the boundary between competition evidence and paper claims.

Citation and releases

GitHub can generate citation formats from CITATION.cff. Pin public or paper-facing use to a tagged release instead of a moving branch, and see CHANGELOG.md for versioned changes.

Validation

The scripts require Node.js 22 or newer. CI covers Node.js 22 and 24 on Windows and Linux.

node --check .\scripts\scaffold.cjs
node --check .\scripts\audit_workspace.cjs
node --test .\tests\scaffold.test.cjs .\tests\audit-workspace.test.cjs
node .\scripts\validate_skill.cjs

GitHub Actions runs the same checks on Windows and Linux. Validate the skill frontmatter with Codex's skill-creator/scripts/quick_validate.py when available.

License

MIT