From 071dc42b031d989c4a11676b81a6ec25bd6842dd Mon Sep 17 00:00:00 2001 From: Rafael Rivera Soto Date: Mon, 29 Jun 2026 12:43:45 -0400 Subject: [PATCH 1/2] removed SUBMISSION.md --- SUBMISSION.md | 131 -------------------------------------------------- 1 file changed, 131 deletions(-) delete mode 100644 SUBMISSION.md diff --git a/SUBMISSION.md b/SUBMISSION.md deleted file mode 100644 index e5f08e0..0000000 --- a/SUBMISSION.md +++ /dev/null @@ -1,131 +0,0 @@ -# Submitting a model to STEB - -If you've evaluated a new model with STEB and want it on the [public leaderboard](https://rrivera1849.github.io/STEB/leaderboard/), open a PR with three small changes. The whole submission is usually a four-line metadata block plus the JSON files STEB already wrote when you ran the benchmark. - -Inspired by — and intentionally lighter than — [MTEB's contribution flow](https://github.com/embeddings-benchmark/mteb): no per-model Python wrapper, no HuggingFace dataset, no leaderboard re-fit on the maintainer side. - -## The contributor workflow - -Five steps, ~10 minutes once your model is evaluated. - -### 1. Get the canonical baseline results - -The leaderboard's headline numbers are computed against ~40 baseline models the maintainers have already evaluated. Download them once: - -```bash -./scripts/download_results.sh -``` - -This pulls a single tarball (~10 MB compressed) from the maintainer's Google Drive and extracts it into `./results/`. The first time it runs you'll need `gdown` (already in `requirements.txt`). Pass `--purge` to force a fresh download. - -### 2. Run STEB on your model, into `submitted_results/` - -Point `STEB_RESULTS_DIR` at the community tree so your model's metrics land directly where the contribution belongs, instead of mixing into the canonical baselines: - -```bash -export STEB_RESULTS_DIR=./submitted_results -steb "your-org/your-model" -``` - -This writes `submitted_results/////metrics.json` for every dataset/task combination. - -`` is the part after the last `/` in your HF id (e.g. `your-model` for `your-org/your-model`). It's the directory name STEB picks automatically. - -### 3. Regenerate the Excel workbook - -```bash -unset STEB_RESULTS_DIR # so benchmark_clustering reads ./results/ by default -python -m scripts.benchmark_clustering -``` - -`benchmark_clustering` auto-merges `./submitted_results/` with `./results/` (maintainer baselines win on the rare collision), so the resulting `scores.xlsx` shows your model alongside every other. Sanity-check the `STEB_operational` and `STEB_definitional` sheets — your model should appear with reasonable numbers. - -### 4. Declare the submission - -Three small edits in the repo: - -- **`SUBMISSIONS.yaml`** — append a 4-key entry: - - ```yaml - - short_name: - hf_id: - run_command: steb "" - contributor: - ``` - -- **`scripts/models_all.txt`** — add `/` (one new line). - -- (Already done in step 2:) the `submitted_results///...` subtree is in place. - -### 5. Validate and open the PR - -```bash -python scripts/validate_submission.py -``` - -If it prints `OK — all checks passed`, open the PR with the model_submission template. CI runs the same validator on every PR that touches `SUBMISSIONS.yaml`, `submitted_results/`, or `scripts/models_all.txt`. - -## `SUBMISSIONS.yaml` schema - -Four required keys per entry. Order in the file doesn't matter; entries are append-only. - -```yaml -- short_name: LUAR-MUD # Must match the directory name under submitted_results/ - hf_id: rrivera1849/LUAR-MUD # HuggingFace identifier (or a stable URL if not on HF) - run_command: steb "rrivera1849/LUAR-MUD" # Exact CLI used to produce the metrics files - contributor: rrivera1849 # Your GitHub handle -``` - -That's the entire schema. Anything beyond these four keys is a future addition; don't pre-emptively invent fields. - -## Why so lightweight - -We considered a richer per-model YAML (paper, license, training data, embedding dim, parameter count, hardware, …) but explicitly decided not to require it in the first pass: - -- **Friction kills contributions.** Every required field is a chance for a would-be contributor to give up. -- **The benchmark numbers are the point.** Metadata is supporting evidence; the scores tell the story. -- **Additive migration is free.** If we later want to filter the leaderboard by license or training-data type, we add an optional field and backfill the entries that care. No schema migration, no broken PRs. - -If you want to volunteer extra context, put it in the PR description rather than the YAML. - -## What the validator checks (`scripts/validate_submission.py`) - -1. `SUBMISSIONS.yaml` parses, and every entry has the four required keys with non-empty string values. -2. Every entry's `short_name` corresponds to at least one results subtree under `submitted_results///...`. -3. Every metrics file under `submitted_results/` parses as JSON. -4. Every entry's `hf_id` (or the bare `short_name`) is present in `scripts/models_all.txt`. - -The validator returns non-zero on any failure and prints a copy-pasteable summary suitable for a PR comment. - -## After your PR merges — how the leaderboard refreshes - -The [public leaderboard](https://rrivera1849.github.io/STEB/leaderboard/) is a static Markdown page (`docs/leaderboard.md`) generated from `scores.xlsx`. The maintainer refresh recipe: - -```bash -./scripts/download_results.sh # ensures ./results/ is up-to-date (skips if present) -python -m scripts.benchmark_clustering # regenerates scores.xlsx, auto-merging submitted_results/ -python scripts/build_leaderboard.py # rewrites docs/leaderboard.md -git commit -am "Refresh leaderboard" -git push # triggers the docs deploy → GH Pages -``` - -A maintainer typically does this after merging a batch of submissions, not after every single one. A future enhancement will run this in CI on push-to-main so the refresh becomes automatic; the path is open now that the canonical results are reachable by a download. - -## Decisions log - -| Question | Decision | -|---|---| -| Single results tree, or split? | Split: `results/` is maintainer-owned (downloaded via `download_results.sh`), `submitted_results/` is community. | -| Auto-ingest `submitted_results/`? | Yes — `benchmark_clustering` reads it whenever it exists. | -| Required vs optional metadata fields? | All four `SUBMISSIONS.yaml` keys are required; no optional fields in Level 1. | -| CI validation? | Yes — GitHub Actions runs `validate_submission.py` on PRs touching `SUBMISSIONS.yaml`, `submitted_results/`, or `scripts/models_all.txt`. | -| Where do the canonical results live? | Google Drive tarball, downloaded via `scripts/download_results.sh`. Not committed to git (62 MB / ~15k JSON files). | -| Leaderboard surface? | Static Markdown rendered via mkdocs to GitHub Pages, refreshed by a maintainer running `build_leaderboard.py` (auto-refresh in CI is the next iteration). | -| Restrictive model licenses for metadata redistribution? | Open: assumed acceptable for Level 1. | - -## What's deliberately not in scope yet - -- **Auto-refresh of the leaderboard on every submission merge.** Manual today; the gdown-based download script unblocks a future CI workflow that does `download_results.sh` → `benchmark_clustering` → `build_leaderboard.py` → commit. -- **Versioning of the benchmark suite.** If we add or remove datasets later, older submissions may be missing coverage. No re-run policy or version pinning yet. -- **Maintainer re-runs of contributor numbers.** The validator's checks plus the auto-included `submitted_results/` ingestion are the only guards against bad numbers. Spot-checks are case-by-case. -- **License taxonomy.** Apache-2.0 redistribution of bare benchmark scores is treated as acceptable for any model license; if you disagree, open an issue. From 8d0be2c24e87e2fa713f9694b17883cb8df0ab83 Mon Sep 17 00:00:00 2001 From: Rafael Rivera Soto Date: Mon, 29 Jun 2026 12:49:45 -0400 Subject: [PATCH 2/2] Test the model submission flow with a fake submission MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit End-to-end exercise of the submission path documented in SUBMISSION.md: - New SUBMISSIONS.yaml entry for `validation-test-model` (hf_id: steb-tests/validation-test-model). - 30 datasets / 90 metrics.json files added under submitted_results//validation-test-model/ — copied verbatim from a real bert-large-uncased run so the leaderboard pipeline gets a complete row to render. - scripts/models_all.txt gets the new model so the default --models-file filter picks it up. `python scripts/validate_submission.py` passes. This PR should NOT be merged into main; it exists to exercise CI on the validate_submission workflow and to confirm the end-to-end PR template, validator, and auto-merge ingestion paths work as designed. Close once verified. --- SUBMISSIONS.yaml | 5 +- scripts/models_all.txt | 50 +++++++++++++++++++ .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + .../metrics.json | 1 + 92 files changed, 144 insertions(+), 1 deletion(-) create mode 100644 scripts/models_all.txt create mode 100644 submitted_results/20_Newsgroups_Fixed/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/20_Newsgroups_Fixed/validation-test-model/2_115/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/20_Newsgroups_Fixed/validation-test-model/3_76/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ag_news/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ag_news/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ag_news/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/bible_versions/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/bible_versions/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/bible_versions/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_age/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_age/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_age/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_gender/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_gender/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/blog_gender/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ceeces/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ceeces/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/ceeces/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/core/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/core/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/core/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/corpus-of-diverse-styles/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/corpus-of-diverse-styles/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/corpus-of-diverse-styles/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/eWAVE/validation-test-model/1_29/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/eWAVE/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/eWAVE/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/emotion/validation-test-model/1_66/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/emotion/validation-test-model/2_33/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/emotion/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/endive/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/endive/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/endive/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/enron_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/enron_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/enron_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/fce_l1/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/fce_l1/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/fce_l1/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/financial_phrasebank/validation-test-model/1_138/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/financial_phrasebank/validation-test-model/2_69/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/financial_phrasebank/validation-test-model/3_46/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech/validation-test-model/2_119/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech/validation-test-model/3_79/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech_and_offensive_language/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech_and_offensive_language/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/hate_speech_and_offensive_language/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/jigsaw_toxicity_pred/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/jigsaw_toxicity_pred/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/jigsaw_toxicity_pred/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/masc_text_genre/validation-test-model/1_150/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/masc_text_genre/validation-test-model/2_75/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/masc_text_genre/validation-test-model/3_50/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_age/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_age/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_age/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_education/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_education/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_education/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_ethnic/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_ethnic/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_ethnic/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_gender/validation-test-model/1_55/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_gender/validation-test-model/2_27/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_gender/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_politics/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_politics/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_politics/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_tod/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_tod/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/pastel_tod/validation-test-model/3_166/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/reuters21578/validation-test-model/1_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/reuters21578/validation-test-model/2_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/reuters21578/validation-test-model/3_25/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/sms_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/sms_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/sms_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/stackexchange_politeness/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/stackexchange_politeness/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/stackexchange_politeness/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/telegram-spam-ham/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/telegram-spam-ham/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/telegram-spam-ham/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/twitter-airline-sentiment/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/twitter-airline-sentiment/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/twitter-airline-sentiment/validation-test-model/3_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/yelp_polarity/validation-test-model/1_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/yelp_polarity/validation-test-model/2_200/all_to_all_pair_classification/metrics.json create mode 100644 submitted_results/yelp_polarity/validation-test-model/3_200/all_to_all_pair_classification/metrics.json diff --git a/SUBMISSIONS.yaml b/SUBMISSIONS.yaml index ad1768b..b3adb1c 100644 --- a/SUBMISSIONS.yaml +++ b/SUBMISSIONS.yaml @@ -11,4 +11,7 @@ # run_command: steb "rrivera1849/LUAR-MUD" # contributor: rrivera1849 -[] +- short_name: validation-test-model + hf_id: steb-tests/validation-test-model + run_command: steb "steb-tests/validation-test-model" + contributor: rrivera1849 diff --git a/scripts/models_all.txt b/scripts/models_all.txt new file mode 100644 index 0000000..1fdc3d2 --- /dev/null +++ b/scripts/models_all.txt @@ -0,0 +1,50 @@ +# Style-Specific Models +rrivera1849/LUAR-MUD +rivera1849/LUAR-CRUD +StyleDistance/styledistance +StyleDistance/mstyledistance +StyleDistance/styledistance_synthetic_only +Blablablab/multilingual-style-representation +AnnaWegmann/Style-Embedding +AIDA-UPM/star +lisa_checkpoint + +# General Sentence Embedders (Semantic) +sentence-transformers/all-mpnet-base-v2 +Alibaba-NLP/gte-base-en-v1.5 +Alibaba-NLP/gte-large-en-v1.5 +intfloat/e5-base-v2 +intfloat/e5-large-v2 +BAAI/bge-base-en-v1.5 +BAAI/bge-large-en-v1.5 +jinaai/jina-embeddings-v3 +Qwen/Qwen3-Embedding-8B + +# MLM Models +google-bert/bert-large-cased +google-bert/bert-large-uncased +FacebookAI/roberta-base +FacebookAI/roberta-large +microsoft/deberta-v3-base +microsoft/deberta-v3-large +answerdotai/ModernBERT-base +answerdotai/ModernBERT-large + +# Causal LMs +openai-community/gpt2-xl +facebook/opt-1.3b +Qwen/Qwen2-0.5B +Qwen/Qwen3-0.6B-Base +Qwen/Qwen3.5-0.8B-Base +Qwen/Qwen3.5-2B-Base +Qwen/Qwen3.5-4B-Base + +# Pre-defined Features +neurobiber +surface_pos.yaml +tfidfngrams_fineweb_sample10bt_1-2grams.pkl +tfidfngrams_fineweb_sample10bt_1-3grams.pkl +tfidfngrams_mud_subset_1-2grams.pkl +tfidfngrams_mud_subset_1-3grams.pkl +functionwordfreq +steb-tests/validation-test-model diff --git a/submitted_results/20_Newsgroups_Fixed/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/20_Newsgroups_Fixed/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..a424dae --- /dev/null +++ b/submitted_results/20_Newsgroups_Fixed/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4013303959965187, "auc": 0.6339963160397382, "auc@0.01": 0.5121829925629868, "auc@0.05": 0.5356410322937223, "auc@0.10": 0.5539260802658549, "auc@0.20": 0.578427304595437, "auc@0.30": 0.5953093802007976, "auc@0.50": 0.617079626396456} \ No newline at end of file diff --git a/submitted_results/20_Newsgroups_Fixed/validation-test-model/2_115/all_to_all_pair_classification/metrics.json b/submitted_results/20_Newsgroups_Fixed/validation-test-model/2_115/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..e674055 --- /dev/null +++ b/submitted_results/20_Newsgroups_Fixed/validation-test-model/2_115/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.32202135774218144, "auc": 0.7342779789905957, "auc@0.01": 0.5235624232035904, "auc@0.05": 0.5669291515070555, "auc@0.10": 0.5988760230768002, "auc@0.20": 0.6404687020511014, "auc@0.30": 0.6686039825299165, "auc@0.50": 0.7043434900659067} \ No newline at end of file diff --git a/submitted_results/20_Newsgroups_Fixed/validation-test-model/3_76/all_to_all_pair_classification/metrics.json b/submitted_results/20_Newsgroups_Fixed/validation-test-model/3_76/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ba4894b --- /dev/null +++ b/submitted_results/20_Newsgroups_Fixed/validation-test-model/3_76/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.261737292540106, "auc": 0.8079924321403176, "auc@0.01": 0.5368475118653352, "auc@0.05": 0.5993672319932957, "auc@0.10": 0.6444285799120579, "auc@0.20": 0.6996071197292185, "auc@0.30": 0.7340189257666577, "auc@0.50": 0.775372674076575} \ No newline at end of file diff --git a/submitted_results/ag_news/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/ag_news/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..bae085f --- /dev/null +++ b/submitted_results/ag_news/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.38202261306528346, "auc": 0.6657447470948493, "auc@0.01": 0.5184481728681094, "auc@0.05": 0.547790002899111, "auc@0.10": 0.5697931747553557, "auc@0.20": 0.5980706352218268, "auc@0.30": 0.6167575158780011, "auc@0.50": 0.6411760402707984} \ No newline at end of file diff --git a/submitted_results/ag_news/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/ag_news/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..c0eb398 --- /dev/null +++ b/submitted_results/ag_news/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.2810083333342752, "auc": 0.7884009052554438, "auc@0.01": 0.5388744497192832, "auc@0.05": 0.5975211474036851, "auc@0.10": 0.6372998273990567, "auc@0.20": 0.6859078014493766, "auc@0.30": 0.7169113376831051, "auc@0.50": 0.7552122582007259} \ No newline at end of file diff --git a/submitted_results/ag_news/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/ag_news/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..3570072 --- /dev/null +++ b/submitted_results/ag_news/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.21376884422146133, "auc": 0.8615562985500419, "auc@0.01": 0.5756790942779223, "auc@0.05": 0.6646975003221234, "auc@0.10": 0.7158904211297716, "auc@0.20": 0.7706690814460031, "auc@0.30": 0.8013188942055046, "auc@0.50": 0.8351687778475712} \ No newline at end of file diff --git a/submitted_results/bible_versions/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/bible_versions/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..520a304 --- /dev/null +++ b/submitted_results/bible_versions/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4820481550969356, "auc": 0.5251746892189744, "auc@0.01": 0.4991139807357408, "auc@0.05": 0.5023628859153828, "auc@0.10": 0.5053712381367599, "auc@0.20": 0.5099623336720184, "auc@0.30": 0.5136698286271166, "auc@0.50": 0.5189917912404284} \ No newline at end of file diff --git a/submitted_results/bible_versions/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/bible_versions/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..80081ef --- /dev/null +++ b/submitted_results/bible_versions/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4585824527252503, "auc": 0.5577510311041816, "auc@0.01": 0.49952630291285033, "auc@0.05": 0.5058124903938189, "auc@0.10": 0.5118665927292714, "auc@0.20": 0.5214284898341534, "auc@0.30": 0.5292076602814493, "auc@0.50": 0.5416969954235463} \ No newline at end of file diff --git a/submitted_results/bible_versions/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/bible_versions/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..4704360 --- /dev/null +++ b/submitted_results/bible_versions/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.43701802704056086, "auc": 0.5855254031569679, "auc@0.01": 0.49998976847626503, "auc@0.05": 0.5094418252549376, "auc@0.10": 0.518576378898723, "auc@0.20": 0.5330605384260505, "auc@0.30": 0.5447600584494162, "auc@0.50": 0.563527344527698} \ No newline at end of file diff --git a/submitted_results/blog_age/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_age/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..3bb0315 --- /dev/null +++ b/submitted_results/blog_age/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4672696817415931, "auc": 0.548190696747627, "auc@0.01": 0.5080876991041191, "auc@0.05": 0.5163508468267263, "auc@0.10": 0.5210696266419819, "auc@0.20": 0.5265309966111421, "auc@0.30": 0.5300031474091591, "auc@0.50": 0.5373985682114275} \ No newline at end of file diff --git a/submitted_results/blog_age/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_age/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..87ba8ee --- /dev/null +++ b/submitted_results/blog_age/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.42424166666651886, "auc": 0.6190203652289223, "auc@0.01": 0.5247755387770792, "auc@0.05": 0.5512608118942289, "auc@0.10": 0.5654861280084633, "auc@0.20": 0.5806382215863267, "auc@0.30": 0.5897735966050295, "auc@0.50": 0.6018492172901544} \ No newline at end of file diff --git a/submitted_results/blog_age/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_age/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..c7b63dc --- /dev/null +++ b/submitted_results/blog_age/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.37981574539363405, "auc": 0.6776137963428253, "auc@0.01": 0.5587851316885937, "auc@0.05": 0.5966699079442798, "auc@0.10": 0.614514701798466, "auc@0.20": 0.6341842887275886, "auc@0.30": 0.6463206722610876, "auc@0.50": 0.6616873992183138} \ No newline at end of file diff --git a/submitted_results/blog_gender/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_gender/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7b06110 --- /dev/null +++ b/submitted_results/blog_gender/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4737437185929649, "auc": 0.5452255568467337, "auc@0.01": 0.5073302915330421, "auc@0.05": 0.5172485375595929, "auc@0.10": 0.5251779985453584, "auc@0.20": 0.5346425094221106, "auc@0.30": 0.5407678718346635, "auc@0.50": 0.5450772487437187} \ No newline at end of file diff --git a/submitted_results/blog_gender/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_gender/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..0c6d57a --- /dev/null +++ b/submitted_results/blog_gender/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4380499999997744, "auc": 0.5819443435929648, "auc@0.01": 0.5087858418979319, "auc@0.05": 0.5218517716789074, "auc@0.10": 0.530245441020894, "auc@0.20": 0.5415374162479062, "auc@0.30": 0.5507362960636516, "auc@0.50": 0.5668487298994974} \ No newline at end of file diff --git a/submitted_results/blog_gender/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/blog_gender/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..a40ba3e --- /dev/null +++ b/submitted_results/blog_gender/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.44152500000115263, "auc": 0.5850884899497488, "auc@0.01": 0.5097726698820737, "auc@0.05": 0.5219873276639608, "auc@0.10": 0.5319367230891299, "auc@0.20": 0.5468228974734786, "auc@0.30": 0.5570109364223076, "auc@0.50": 0.570526682998325} \ No newline at end of file diff --git a/submitted_results/ceeces/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/ceeces/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..845fe2f --- /dev/null +++ b/submitted_results/ceeces/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.49128140703517587, "auc": 0.5167092730876606, "auc@0.01": 0.503444836900864, "auc@0.05": 0.5068402103823964, "auc@0.10": 0.5091435279467513, "auc@0.20": 0.5111131863988151, "auc@0.30": 0.5121314023959668, "auc@0.50": 0.5133072827935976} \ No newline at end of file diff --git a/submitted_results/ceeces/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/ceeces/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ca289d0 --- /dev/null +++ b/submitted_results/ceeces/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.47383584589660915, "auc": 0.5377749538595756, "auc@0.01": 0.5035894646319257, "auc@0.05": 0.5094163001617775, "auc@0.10": 0.5138171584530842, "auc@0.20": 0.5196461932734661, "auc@0.30": 0.5239581783372199, "auc@0.50": 0.5306851718499906} \ No newline at end of file diff --git a/submitted_results/ceeces/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/ceeces/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..d8d96ef --- /dev/null +++ b/submitted_results/ceeces/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.46257956448911214, "auc": 0.5582268277498603, "auc@0.01": 0.5054692214562483, "auc@0.05": 0.513999420464144, "auc@0.10": 0.5205446309397866, "auc@0.20": 0.5293633395178051, "auc@0.30": 0.5356504535203249, "auc@0.50": 0.544494650046529} \ No newline at end of file diff --git a/submitted_results/core/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/core/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..89591b0 --- /dev/null +++ b/submitted_results/core/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.38053598862086707, "auc": 0.6615111864440619, "auc@0.01": 0.5130051821333715, "auc@0.05": 0.5366800769659495, "auc@0.10": 0.5565143338994337, "auc@0.20": 0.5851490898100317, "auc@0.30": 0.6065808327473636, "auc@0.50": 0.6349636653606515} \ No newline at end of file diff --git a/submitted_results/core/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/core/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..2cfe274 --- /dev/null +++ b/submitted_results/core/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.29249999999989934, "auc": 0.7767099373309818, "auc@0.01": 0.5280846501762191, "auc@0.05": 0.5770433488596451, "auc@0.10": 0.6165158006126428, "auc@0.20": 0.6656252769612646, "auc@0.30": 0.6978181065347949, "auc@0.50": 0.7390134675289045} \ No newline at end of file diff --git a/submitted_results/core/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/core/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..c48e7e5 --- /dev/null +++ b/submitted_results/core/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.23639305678140632, "auc": 0.8449525081114173, "auc@0.01": 0.5462203733697567, "auc@0.05": 0.6168105860469497, "auc@0.10": 0.6632777898603257, "auc@0.20": 0.7212576841648812, "auc@0.30": 0.7593743039425749, "auc@0.50": 0.8062018234873993} \ No newline at end of file diff --git a/submitted_results/corpus-of-diverse-styles/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/corpus-of-diverse-styles/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..9f686ef --- /dev/null +++ b/submitted_results/corpus-of-diverse-styles/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.37071974864607876, "auc": 0.675118822557, "auc@0.01": 0.5237299660706456, "auc@0.05": 0.5553085263678, "auc@0.10": 0.5776549984972711, "auc@0.20": 0.6067226995658965, "auc@0.30": 0.6267278099845361, "auc@0.50": 0.6531402413292358} \ No newline at end of file diff --git a/submitted_results/corpus-of-diverse-styles/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/corpus-of-diverse-styles/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..e7351b9 --- /dev/null +++ b/submitted_results/corpus-of-diverse-styles/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.29146681818260733, "auc": 0.7770571928672287, "auc@0.01": 0.540940447336145, "auc@0.05": 0.5905884303803399, "auc@0.10": 0.6260456568838101, "auc@0.20": 0.6715857028542622, "auc@0.30": 0.7022907751771335, "auc@0.50": 0.7421817415950275} \ No newline at end of file diff --git a/submitted_results/corpus-of-diverse-styles/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/corpus-of-diverse-styles/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..8db547a --- /dev/null +++ b/submitted_results/corpus-of-diverse-styles/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.2431395454553403, "auc": 0.8370089328117862, "auc@0.01": 0.5606827609537367, "auc@0.05": 0.6273849799751033, "auc@0.10": 0.6719593922745524, "auc@0.20": 0.7254914016449326, "auc@0.30": 0.7595046836699296, "auc@0.50": 0.8017357592591055} \ No newline at end of file diff --git a/submitted_results/eWAVE/validation-test-model/1_29/all_to_all_pair_classification/metrics.json b/submitted_results/eWAVE/validation-test-model/1_29/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..b1d11a6 --- /dev/null +++ b/submitted_results/eWAVE/validation-test-model/1_29/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3652390343743223, "auc": 0.6774448124519938, "auc@0.01": 0.5217125742066812, "auc@0.05": 0.5450242524670565, "auc@0.10": 0.562919235967708, "auc@0.20": 0.5900651345041358, "auc@0.30": 0.6121872099142383, "auc@0.50": 0.6464300124917483} \ No newline at end of file diff --git a/submitted_results/eWAVE/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/eWAVE/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..4a9bd34 --- /dev/null +++ b/submitted_results/eWAVE/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3128472222223015, "auc": 0.754933680112293, "auc@0.01": 0.527094520078723, "auc@0.05": 0.5593002197369219, "auc@0.10": 0.5857139938720916, "auc@0.20": 0.6259284083322388, "auc@0.30": 0.6587895464430846, "auc@0.50": 0.7073610137903862} \ No newline at end of file diff --git a/submitted_results/eWAVE/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/eWAVE/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..e185806 --- /dev/null +++ b/submitted_results/eWAVE/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.29541666666675065, "auc": 0.7780844198872786, "auc@0.01": 0.5314576910316477, "auc@0.05": 0.5723278128741897, "auc@0.10": 0.6034141939994915, "auc@0.20": 0.6484213935856147, "auc@0.30": 0.6822448557039563, "auc@0.50": 0.7313280885668277} \ No newline at end of file diff --git a/submitted_results/emotion/validation-test-model/1_66/all_to_all_pair_classification/metrics.json b/submitted_results/emotion/validation-test-model/1_66/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..d79d90b --- /dev/null +++ b/submitted_results/emotion/validation-test-model/1_66/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.48088578088549006, "auc": 0.5270985930030926, "auc@0.01": 0.5019240585224776, "auc@0.05": 0.5047788264571481, "auc@0.10": 0.5075519773939376, "auc@0.20": 0.512358117353781, "auc@0.30": 0.5164986462002973, "auc@0.50": 0.5226235806615359} \ No newline at end of file diff --git a/submitted_results/emotion/validation-test-model/2_33/all_to_all_pair_classification/metrics.json b/submitted_results/emotion/validation-test-model/2_33/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..156fb4d --- /dev/null +++ b/submitted_results/emotion/validation-test-model/2_33/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4674873737377847, "auc": 0.5484388864927203, "auc@0.01": 0.5018683880953216, "auc@0.05": 0.5084274974082137, "auc@0.10": 0.5145855169385931, "auc@0.20": 0.5214700732712287, "auc@0.30": 0.5264457558352389, "auc@0.50": 0.5361789639070019} \ No newline at end of file diff --git a/submitted_results/emotion/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/emotion/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..4bc33ba --- /dev/null +++ b/submitted_results/emotion/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4395200000007822, "auc": 0.5753061866666667, "auc@0.01": 0.5034827470686767, "auc@0.05": 0.5115399658119658, "auc@0.10": 0.5189150877192983, "auc@0.20": 0.5281038518518518, "auc@0.30": 0.5367915294117647, "auc@0.50": 0.5555387022222222} \ No newline at end of file diff --git a/submitted_results/endive/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/endive/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..e689b21 --- /dev/null +++ b/submitted_results/endive/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.24744556113902821, "auc": 0.8261551332774986, "auc@0.01": 0.5175331115375874, "auc@0.05": 0.5780896363584304, "auc@0.10": 0.6281480130183079, "auc@0.20": 0.6929651338094795, "auc@0.30": 0.7347331029466504, "auc@0.50": 0.7848239472640983} \ No newline at end of file diff --git a/submitted_results/endive/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/endive/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..776b9a2 --- /dev/null +++ b/submitted_results/endive/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.27951833333372655, "auc": 0.7914592344988832, "auc@0.01": 0.5278942296631117, "auc@0.05": 0.5965690140159486, "auc@0.10": 0.6409565501484029, "auc@0.20": 0.6920913244850797, "auc@0.30": 0.7224804721729563, "auc@0.50": 0.7587023458123954} \ No newline at end of file diff --git a/submitted_results/endive/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/endive/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..2ae2d9a --- /dev/null +++ b/submitted_results/endive/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3850167504187605, "auc": 0.6779950762423228, "auc@0.01": 0.5227896191594489, "auc@0.05": 0.5722480938882446, "auc@0.10": 0.603272739299568, "auc@0.20": 0.6342288246983374, "auc@0.30": 0.6492436489911431, "auc@0.50": 0.6612765234971152} \ No newline at end of file diff --git a/submitted_results/enron_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/enron_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..e4f6853 --- /dev/null +++ b/submitted_results/enron_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.43937185929648165, "auc": 0.5861581306532663, "auc@0.01": 0.5108619574000657, "auc@0.05": 0.5240399722973843, "auc@0.10": 0.533784463435599, "auc@0.20": 0.5464150003489671, "auc@0.30": 0.5552839319144743, "auc@0.50": 0.5697091306532663} \ No newline at end of file diff --git a/submitted_results/enron_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/enron_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..61d1343 --- /dev/null +++ b/submitted_results/enron_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3700249999991313, "auc": 0.678892335741206, "auc@0.01": 0.5218257746016515, "auc@0.05": 0.5531739047803118, "auc@0.10": 0.5735771257603809, "auc@0.20": 0.6003595058626465, "auc@0.30": 0.61994627364765, "auc@0.50": 0.6494961867671691} \ No newline at end of file diff --git a/submitted_results/enron_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/enron_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..a07ae54 --- /dev/null +++ b/submitted_results/enron_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.31767500000031906, "auc": 0.7437799343592965, "auc@0.01": 0.5414443132749173, "auc@0.05": 0.5831480575956707, "auc@0.10": 0.6151002595212907, "auc@0.20": 0.6554391026312116, "auc@0.30": 0.6821110521972609, "auc@0.50": 0.7147680385259632} \ No newline at end of file diff --git a/submitted_results/fce_l1/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/fce_l1/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..17a492f --- /dev/null +++ b/submitted_results/fce_l1/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4887415329768271, "auc": 0.5224029815873016, "auc@0.01": 0.501387019754859, "auc@0.05": 0.5037640048840049, "auc@0.10": 0.5063230788081314, "auc@0.20": 0.5112420223398001, "auc@0.30": 0.5140885519244734, "auc@0.50": 0.5161433577425044} \ No newline at end of file diff --git a/submitted_results/fce_l1/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/fce_l1/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..99e7858 --- /dev/null +++ b/submitted_results/fce_l1/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4683333333333334, "auc": 0.5418499549973836, "auc@0.01": 0.5017822235194812, "auc@0.05": 0.5059124045673496, "auc@0.10": 0.5095473629127765, "auc@0.20": 0.5166103261817547, "auc@0.30": 0.521798224930998, "auc@0.50": 0.5307006007326007} \ No newline at end of file diff --git a/submitted_results/fce_l1/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/fce_l1/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..83111fe --- /dev/null +++ b/submitted_results/fce_l1/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4626190476190483, "auc": 0.5527374631083202, "auc@0.01": 0.5021471565046582, "auc@0.05": 0.5073699096995801, "auc@0.10": 0.5132352089013743, "auc@0.20": 0.5215097273097273, "auc@0.30": 0.526832300099527, "auc@0.50": 0.5375288916797488} \ No newline at end of file diff --git a/submitted_results/financial_phrasebank/validation-test-model/1_138/all_to_all_pair_classification/metrics.json b/submitted_results/financial_phrasebank/validation-test-model/1_138/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..181f589 --- /dev/null +++ b/submitted_results/financial_phrasebank/validation-test-model/1_138/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4398603617899084, "auc": 0.5774740381408742, "auc@0.01": 0.5037913192255237, "auc@0.05": 0.5146171331238687, "auc@0.10": 0.5268550203858025, "auc@0.20": 0.5459010389360346, "auc@0.30": 0.5585332489158915, "auc@0.50": 0.5729955843131758} \ No newline at end of file diff --git a/submitted_results/financial_phrasebank/validation-test-model/2_69/all_to_all_pair_classification/metrics.json b/submitted_results/financial_phrasebank/validation-test-model/2_69/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..d8abb5f --- /dev/null +++ b/submitted_results/financial_phrasebank/validation-test-model/2_69/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.38803637396987123, "auc": 0.6351181085019965, "auc@0.01": 0.5050186104551375, "auc@0.05": 0.5183883219666284, "auc@0.10": 0.5353749972998003, "auc@0.20": 0.5632111518415616, "auc@0.30": 0.5851231458832167, "auc@0.50": 0.6160351180942434} \ No newline at end of file diff --git a/submitted_results/financial_phrasebank/validation-test-model/3_46/all_to_all_pair_classification/metrics.json b/submitted_results/financial_phrasebank/validation-test-model/3_46/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..1c82779 --- /dev/null +++ b/submitted_results/financial_phrasebank/validation-test-model/3_46/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.36294896030146434, "auc": 0.6702146415065239, "auc@0.01": 0.5052658432019278, "auc@0.05": 0.5246023388813997, "auc@0.10": 0.5447966311158235, "auc@0.20": 0.578009038706082, "auc@0.30": 0.6043662277077316, "auc@0.50": 0.6414163183758538} \ No newline at end of file diff --git a/submitted_results/hate_speech/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..dfdbebc --- /dev/null +++ b/submitted_results/hate_speech/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.47137500000019505, "auc": 0.5436135612437186, "auc@0.01": 0.5037009955556678, "auc@0.05": 0.5115580788558175, "auc@0.10": 0.5191756760777572, "auc@0.20": 0.5297681297110552, "auc@0.30": 0.5362616279929057, "auc@0.50": 0.5421144137353434} \ No newline at end of file diff --git a/submitted_results/hate_speech/validation-test-model/2_119/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech/validation-test-model/2_119/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..b49ef83 --- /dev/null +++ b/submitted_results/hate_speech/validation-test-model/2_119/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.44813219405310967, "auc": 0.5739644760775529, "auc@0.01": 0.506493006107479, "auc@0.05": 0.5229710950904126, "auc@0.10": 0.5342096564931356, "auc@0.20": 0.5475992886046286, "auc@0.30": 0.5552705695664997, "auc@0.50": 0.566319110400094} \ No newline at end of file diff --git a/submitted_results/hate_speech/validation-test-model/3_79/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech/validation-test-model/3_79/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..800ae98 --- /dev/null +++ b/submitted_results/hate_speech/validation-test-model/3_79/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.43342413074726427, "auc": 0.597829742079487, "auc@0.01": 0.510395320090051, "auc@0.05": 0.5336045146365971, "auc@0.10": 0.548944891207236, "auc@0.20": 0.5653188100692254, "auc@0.30": 0.5755920919477364, "auc@0.50": 0.5877507340961551} \ No newline at end of file diff --git a/submitted_results/hate_speech_and_offensive_language/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech_and_offensive_language/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..94e91c6 --- /dev/null +++ b/submitted_results/hate_speech_and_offensive_language/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4672026800669948, "auc": 0.5488203267029592, "auc@0.01": 0.5012921110297439, "auc@0.05": 0.505540046385775, "auc@0.10": 0.5115055140468424, "auc@0.20": 0.5219384296094671, "auc@0.30": 0.5289452799126351, "auc@0.50": 0.5380125803089522} \ No newline at end of file diff --git a/submitted_results/hate_speech_and_offensive_language/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech_and_offensive_language/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..3d1554b --- /dev/null +++ b/submitted_results/hate_speech_and_offensive_language/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4371524288109808, "auc": 0.5946423435231714, "auc@0.01": 0.5057674750076457, "auc@0.05": 0.5213539206001517, "auc@0.10": 0.5348564680125775, "auc@0.20": 0.551674267750791, "auc@0.30": 0.5626710295759845, "auc@0.50": 0.5769150451330728} \ No newline at end of file diff --git a/submitted_results/hate_speech_and_offensive_language/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/hate_speech_and_offensive_language/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..5fe5493 --- /dev/null +++ b/submitted_results/hate_speech_and_offensive_language/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.43693333333300477, "auc": 0.5993087635399219, "auc@0.01": 0.5090622957052151, "auc@0.05": 0.5278049055820413, "auc@0.10": 0.540777308325252, "auc@0.20": 0.5560981849835598, "auc@0.30": 0.5657002304551078, "auc@0.50": 0.578753389633352} \ No newline at end of file diff --git a/submitted_results/jigsaw_toxicity_pred/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/jigsaw_toxicity_pred/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..79c2485 --- /dev/null +++ b/submitted_results/jigsaw_toxicity_pred/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4538693467336668, "auc": 0.5663973771984925, "auc@0.01": 0.5061684521855508, "auc@0.05": 0.5175563329467852, "auc@0.10": 0.5255705087939698, "auc@0.20": 0.5367290105039084, "auc@0.30": 0.5447258609222583, "auc@0.50": 0.5557445766331658} \ No newline at end of file diff --git a/submitted_results/jigsaw_toxicity_pred/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/jigsaw_toxicity_pred/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..678b990 --- /dev/null +++ b/submitted_results/jigsaw_toxicity_pred/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.41231155778894474, "auc": 0.6206968071608041, "auc@0.01": 0.5090069695209717, "auc@0.05": 0.5248156294291972, "auc@0.10": 0.5373030613594287, "auc@0.20": 0.5571836779383026, "auc@0.30": 0.5736446404818208, "auc@0.50": 0.5967584845058627} \ No newline at end of file diff --git a/submitted_results/jigsaw_toxicity_pred/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/jigsaw_toxicity_pred/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..3e71268 --- /dev/null +++ b/submitted_results/jigsaw_toxicity_pred/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3564824120603815, "auc": 0.6864400778894473, "auc@0.01": 0.5172036848816949, "auc@0.05": 0.5419394182450715, "auc@0.10": 0.5653700376884422, "auc@0.20": 0.599154259142937, "auc@0.30": 0.6241370578382107, "auc@0.50": 0.6584023341708543} \ No newline at end of file diff --git a/submitted_results/masc_text_genre/validation-test-model/1_150/all_to_all_pair_classification/metrics.json b/submitted_results/masc_text_genre/validation-test-model/1_150/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..d12fd21 --- /dev/null +++ b/submitted_results/masc_text_genre/validation-test-model/1_150/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.27288826092042245, "auc": 0.8124144034191825, "auc@0.01": 0.6121159875366639, "auc@0.05": 0.6707366322677992, "auc@0.10": 0.7012089858538171, "auc@0.20": 0.7342924232651074, "auc@0.30": 0.7551437966506916, "auc@0.50": 0.7832798612557665} \ No newline at end of file diff --git a/submitted_results/masc_text_genre/validation-test-model/2_75/all_to_all_pair_classification/metrics.json b/submitted_results/masc_text_genre/validation-test-model/2_75/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..717bada --- /dev/null +++ b/submitted_results/masc_text_genre/validation-test-model/2_75/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.24493124703651015, "auc": 0.8454697968488648, "auc@0.01": 0.6490892253321512, "auc@0.05": 0.7061433852727174, "auc@0.10": 0.7350914494802914, "auc@0.20": 0.7678621545248268, "auc@0.30": 0.7889186056647297, "auc@0.50": 0.8171671000780198} \ No newline at end of file diff --git a/submitted_results/masc_text_genre/validation-test-model/3_50/all_to_all_pair_classification/metrics.json b/submitted_results/masc_text_genre/validation-test-model/3_50/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ffd7b86 --- /dev/null +++ b/submitted_results/masc_text_genre/validation-test-model/3_50/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.22808187134346788, "auc": 0.8592702487547189, "auc@0.01": 0.6633574187480641, "auc@0.05": 0.7223324324563739, "auc@0.10": 0.7529621843819416, "auc@0.20": 0.7867447303862553, "auc@0.30": 0.8074073208969645, "auc@0.50": 0.8337192817968062} \ No newline at end of file diff --git a/submitted_results/pastel_age/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_age/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..8a36357 --- /dev/null +++ b/submitted_results/pastel_age/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.46055555555555566, "auc": 0.5550459851851851, "auc@0.01": 0.5018559463986599, "auc@0.05": 0.5068377587844255, "auc@0.10": 0.5114727485380117, "auc@0.20": 0.5169102057613169, "auc@0.30": 0.5237407116920842, "auc@0.50": 0.5381218765432099} \ No newline at end of file diff --git a/submitted_results/pastel_age/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_age/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..2e49ade --- /dev/null +++ b/submitted_results/pastel_age/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.47104000000011303, "auc": 0.5334462933333334, "auc@0.01": 0.4994251256281407, "auc@0.05": 0.4996425299145299, "auc@0.10": 0.501159298245614, "auc@0.20": 0.5043385185185185, "auc@0.30": 0.5065333333333333, "auc@0.50": 0.5159912533333333} \ No newline at end of file diff --git a/submitted_results/pastel_age/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_age/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..193d7dd --- /dev/null +++ b/submitted_results/pastel_age/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4573333333333334, "auc": 0.5595728, "auc@0.01": 0.5003738693467337, "auc@0.05": 0.5021609572649572, "auc@0.10": 0.5070545964912281, "auc@0.20": 0.5176008888888889, "auc@0.30": 0.5265014379084967, "auc@0.50": 0.5406160355555555} \ No newline at end of file diff --git a/submitted_results/pastel_education/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_education/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..f44fab1 --- /dev/null +++ b/submitted_results/pastel_education/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.47434285714314367, "auc": 0.5352239285714286, "auc@0.01": 0.5024958123953098, "auc@0.05": 0.5076159951159951, "auc@0.10": 0.5095901002506266, "auc@0.20": 0.5159091931216931, "auc@0.30": 0.5208268674136322, "auc@0.50": 0.5287321111111111} \ No newline at end of file diff --git a/submitted_results/pastel_education/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_education/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..d007873 --- /dev/null +++ b/submitted_results/pastel_education/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4529166666666666, "auc": 0.560389130952381, "auc@0.01": 0.5029528595357741, "auc@0.05": 0.5067946275946276, "auc@0.10": 0.5113820175438597, "auc@0.20": 0.5204670965608466, "auc@0.30": 0.5283700980392156, "auc@0.50": 0.5425978888888888} \ No newline at end of file diff --git a/submitted_results/pastel_education/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_education/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..b39bc26 --- /dev/null +++ b/submitted_results/pastel_education/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.43920000000106685, "auc": 0.5863000595238095, "auc@0.01": 0.5023247188322565, "auc@0.05": 0.5099892551892552, "auc@0.10": 0.5159101503759398, "auc@0.20": 0.5252915674603175, "auc@0.30": 0.5336229925303455, "auc@0.50": 0.553172619047619} \ No newline at end of file diff --git a/submitted_results/pastel_ethnic/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_ethnic/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..5629161 --- /dev/null +++ b/submitted_results/pastel_ethnic/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.432962962963061, "auc": 0.600597160493827, "auc@0.01": 0.5040469838906466, "auc@0.05": 0.5125610214202807, "auc@0.10": 0.5181552523283517, "auc@0.20": 0.530635093735711, "auc@0.30": 0.5425745501492778, "auc@0.50": 0.563293037037037} \ No newline at end of file diff --git a/submitted_results/pastel_ethnic/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_ethnic/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..1d366be --- /dev/null +++ b/submitted_results/pastel_ethnic/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3876571428575581, "auc": 0.6718188480725624, "auc@0.01": 0.5041155892842899, "auc@0.05": 0.5143542764114193, "auc@0.10": 0.5243452440625374, "auc@0.20": 0.5447599899218947, "auc@0.30": 0.566643866435463, "auc@0.50": 0.6058610551776266} \ No newline at end of file diff --git a/submitted_results/pastel_ethnic/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_ethnic/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ab783bf --- /dev/null +++ b/submitted_results/pastel_ethnic/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.37809523809522655, "auc": 0.6853738231292517, "auc@0.01": 0.5038380109162593, "auc@0.05": 0.5173514739229025, "auc@0.10": 0.529415156939969, "auc@0.20": 0.553230183925422, "auc@0.30": 0.5755698546085101, "auc@0.50": 0.6174329614512472} \ No newline at end of file diff --git a/submitted_results/pastel_gender/validation-test-model/1_55/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_gender/validation-test-model/1_55/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..da8d784 --- /dev/null +++ b/submitted_results/pastel_gender/validation-test-model/1_55/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4792286501391064, "auc": 0.5329718241490509, "auc@0.01": 0.5023641394153274, "auc@0.05": 0.5030676269733508, "auc@0.10": 0.5045093696294701, "auc@0.20": 0.5082529199762136, "auc@0.30": 0.5118691551900648, "auc@0.50": 0.5186829296948673} \ No newline at end of file diff --git a/submitted_results/pastel_gender/validation-test-model/2_27/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_gender/validation-test-model/2_27/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..eb4174f --- /dev/null +++ b/submitted_results/pastel_gender/validation-test-model/2_27/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4545038866035858, "auc": 0.5675744742198027, "auc@0.01": 0.5072245126050198, "auc@0.05": 0.5088566222578783, "auc@0.10": 0.511027793221618, "auc@0.20": 0.51595007844902, "auc@0.30": 0.5226599861534158, "auc@0.50": 0.5359748306961638} \ No newline at end of file diff --git a/submitted_results/pastel_gender/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_gender/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ff28466 --- /dev/null +++ b/submitted_results/pastel_gender/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4575999999999731, "auc": 0.5628666666666666, "auc@0.01": 0.504355108877722, "auc@0.05": 0.5147076923076923, "auc@0.10": 0.5144140350877193, "auc@0.20": 0.5164074074074074, "auc@0.30": 0.5203764705882353, "auc@0.50": 0.5313991111111112} \ No newline at end of file diff --git a/submitted_results/pastel_politics/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_politics/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..1324028 --- /dev/null +++ b/submitted_results/pastel_politics/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4929648241206047, "auc": 0.5107897365298715, "auc@0.01": 0.5009925990926155, "auc@0.05": 0.5018011682343341, "auc@0.10": 0.5023493601046166, "auc@0.20": 0.5037390404956883, "auc@0.30": 0.5046087427880142, "auc@0.50": 0.5068527252931323} \ No newline at end of file diff --git a/submitted_results/pastel_politics/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_politics/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..68b68cc --- /dev/null +++ b/submitted_results/pastel_politics/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.486733333333047, "auc": 0.519584663176996, "auc@0.01": 0.5020826845001108, "auc@0.05": 0.5034420700367936, "auc@0.10": 0.5048190657233536, "auc@0.20": 0.5075143396380049, "auc@0.30": 0.5097556700988602, "auc@0.50": 0.5131499682672622} \ No newline at end of file diff --git a/submitted_results/pastel_politics/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_politics/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..5d3393f --- /dev/null +++ b/submitted_results/pastel_politics/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4774749999989543, "auc": 0.5338305937325516, "auc@0.01": 0.5036165332525273, "auc@0.05": 0.5062690904665779, "auc@0.10": 0.5084301779364659, "auc@0.20": 0.512522093104411, "auc@0.30": 0.5163063345047678, "auc@0.50": 0.5230902914572865} \ No newline at end of file diff --git a/submitted_results/pastel_tod/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_tod/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..79c571a --- /dev/null +++ b/submitted_results/pastel_tod/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.48780904522612184, "auc": 0.517631284258794, "auc@0.01": 0.5009588577308653, "auc@0.05": 0.5022000938023451, "auc@0.10": 0.5032230708807194, "auc@0.20": 0.5050491367950866, "auc@0.30": 0.5067832972706671, "auc@0.50": 0.5103489225963149} \ No newline at end of file diff --git a/submitted_results/pastel_tod/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_tod/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..4e0cd95 --- /dev/null +++ b/submitted_results/pastel_tod/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4771811811811812, "auc": 0.5320761270854272, "auc@0.01": 0.5020560402515089, "auc@0.05": 0.5039771638964051, "auc@0.10": 0.5059718521555144, "auc@0.20": 0.5099632022962032, "auc@0.30": 0.513961966055769, "auc@0.50": 0.52118662360134} \ No newline at end of file diff --git a/submitted_results/pastel_tod/validation-test-model/3_166/all_to_all_pair_classification/metrics.json b/submitted_results/pastel_tod/validation-test-model/3_166/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7003a35 --- /dev/null +++ b/submitted_results/pastel_tod/validation-test-model/3_166/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.46478080998644966, "auc": 0.5519854318234948, "auc@0.01": 0.5035032267482735, "auc@0.05": 0.507336123416587, "auc@0.10": 0.5114044790365371, "auc@0.20": 0.5187708286176991, "auc@0.30": 0.5253408369044961, "auc@0.50": 0.5358970166064346} \ No newline at end of file diff --git a/submitted_results/reuters21578/validation-test-model/1_25/all_to_all_pair_classification/metrics.json b/submitted_results/reuters21578/validation-test-model/1_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..0d7977e --- /dev/null +++ b/submitted_results/reuters21578/validation-test-model/1_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.38305555555536264, "auc": 0.6457851784511784, "auc@0.01": 0.5081506860903846, "auc@0.05": 0.5515338340671674, "auc@0.10": 0.5732774233563708, "auc@0.20": 0.598133305274972, "auc@0.30": 0.614820954644484, "auc@0.50": 0.6390395735129069} \ No newline at end of file diff --git a/submitted_results/reuters21578/validation-test-model/2_25/all_to_all_pair_classification/metrics.json b/submitted_results/reuters21578/validation-test-model/2_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7e29f8b --- /dev/null +++ b/submitted_results/reuters21578/validation-test-model/2_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3540571428581234, "auc": 0.6784827142857143, "auc@0.01": 0.5097894233070113, "auc@0.05": 0.5599941391941392, "auc@0.10": 0.6046642857142858, "auc@0.20": 0.6486690476190476, "auc@0.30": 0.669989005602241, "auc@0.50": 0.6859343333333333} \ No newline at end of file diff --git a/submitted_results/reuters21578/validation-test-model/3_25/all_to_all_pair_classification/metrics.json b/submitted_results/reuters21578/validation-test-model/3_25/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..69ceca4 --- /dev/null +++ b/submitted_results/reuters21578/validation-test-model/3_25/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.41761904761904634, "auc": 0.6694851156462585, "auc@0.01": 0.5180824758714206, "auc@0.05": 0.58331270422699, "auc@0.10": 0.6187380833034968, "auc@0.20": 0.6469699168556311, "auc@0.30": 0.656298252634387, "auc@0.50": 0.653800223733938} \ No newline at end of file diff --git a/submitted_results/sms_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/sms_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..bb4fe6e --- /dev/null +++ b/submitted_results/sms_spam/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.35129999999981254, "auc": 0.7159472603643215, "auc@0.01": 0.6231850521451479, "auc@0.05": 0.6704638126530087, "auc@0.10": 0.6886504529225073, "auc@0.20": 0.7040450341987716, "auc@0.30": 0.7111974467927874, "auc@0.50": 0.7169139836683417} \ No newline at end of file diff --git a/submitted_results/sms_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/sms_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..57f4509 --- /dev/null +++ b/submitted_results/sms_spam/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.24622499999967465, "auc": 0.8351227292713568, "auc@0.01": 0.7157766533673392, "auc@0.05": 0.7504554986470815, "auc@0.10": 0.7687686161068501, "auc@0.20": 0.7902724630094919, "auc@0.30": 0.803946436841068, "auc@0.50": 0.8208304966499163} \ No newline at end of file diff --git a/submitted_results/sms_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/sms_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..70320dc --- /dev/null +++ b/submitted_results/sms_spam/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.16577889447232053, "auc": 0.907376720477387, "auc@0.01": 0.7474263907477084, "auc@0.05": 0.8032872116995232, "auc@0.10": 0.8322549441285374, "auc@0.20": 0.8620977430555556, "auc@0.30": 0.8783149251157749, "auc@0.50": 0.8954946407035176} \ No newline at end of file diff --git a/submitted_results/stackexchange_politeness/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/stackexchange_politeness/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..2a073fc --- /dev/null +++ b/submitted_results/stackexchange_politeness/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.48254606365206376, "auc": 0.5257845604410943, "auc@0.01": 0.5008527422147028, "auc@0.05": 0.5024560337299031, "auc@0.10": 0.504293065106233, "auc@0.20": 0.5075547744509585, "auc@0.30": 0.5109715323622469, "auc@0.50": 0.5165429870649544} \ No newline at end of file diff --git a/submitted_results/stackexchange_politeness/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/stackexchange_politeness/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7175f86 --- /dev/null +++ b/submitted_results/stackexchange_politeness/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.47043551088777213, "auc": 0.5430444477945282, "auc@0.01": 0.5010108856678703, "auc@0.05": 0.5042599013586451, "auc@0.10": 0.5077135660025273, "auc@0.20": 0.5133662564365035, "auc@0.30": 0.5185100587359455, "auc@0.50": 0.5278095961287921} \ No newline at end of file diff --git a/submitted_results/stackexchange_politeness/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/stackexchange_politeness/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..28e46da --- /dev/null +++ b/submitted_results/stackexchange_politeness/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4495333333331425, "auc": 0.5701392920156337, "auc@0.01": 0.5015428447373663, "auc@0.05": 0.507489689902504, "auc@0.10": 0.5133402123923712, "auc@0.20": 0.522887108963025, "auc@0.30": 0.531461511807403, "auc@0.50": 0.5470975351758793} \ No newline at end of file diff --git a/submitted_results/telegram-spam-ham/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/telegram-spam-ham/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7262921 --- /dev/null +++ b/submitted_results/telegram-spam-ham/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.41514999999983987, "auc": 0.6236442647613065, "auc@0.01": 0.5513078426049848, "auc@0.05": 0.5769582173688957, "auc@0.10": 0.5910208509653531, "auc@0.20": 0.6054174605667225, "auc@0.30": 0.6137830445117746, "auc@0.50": 0.623204236599665} \ No newline at end of file diff --git a/submitted_results/telegram-spam-ham/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/telegram-spam-ham/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..0b7f9fe --- /dev/null +++ b/submitted_results/telegram-spam-ham/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.33474874371864316, "auc": 0.728025506281407, "auc@0.01": 0.5731291981263099, "auc@0.05": 0.6177016396082979, "auc@0.10": 0.6440368090452262, "auc@0.20": 0.6733021339335566, "auc@0.30": 0.6909938879446251, "auc@0.50": 0.711509709798995} \ No newline at end of file diff --git a/submitted_results/telegram-spam-ham/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/telegram-spam-ham/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..fffb244 --- /dev/null +++ b/submitted_results/telegram-spam-ham/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.2699246231155752, "auc": 0.8027773030778894, "auc@0.01": 0.5992014090553268, "auc@0.05": 0.6592862292230383, "auc@0.10": 0.6924086931367364, "auc@0.20": 0.7301027987158013, "auc@0.30": 0.7532163008670805, "auc@0.50": 0.7806571817420436} \ No newline at end of file diff --git a/submitted_results/twitter-airline-sentiment/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/twitter-airline-sentiment/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..bdf071f --- /dev/null +++ b/submitted_results/twitter-airline-sentiment/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.45331658291441124, "auc": 0.5645545917783361, "auc@0.01": 0.5042327845256432, "auc@0.05": 0.5133963292244699, "auc@0.10": 0.5217366066443327, "auc@0.20": 0.5338620946166015, "auc@0.30": 0.5426995099407714, "auc@0.50": 0.5543948466406105} \ No newline at end of file diff --git a/submitted_results/twitter-airline-sentiment/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/twitter-airline-sentiment/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..7aaa669 --- /dev/null +++ b/submitted_results/twitter-airline-sentiment/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.42718592964776975, "auc": 0.6026435443188163, "auc@0.01": 0.5077911717998143, "auc@0.05": 0.5226417643774428, "auc@0.10": 0.5341030859707896, "auc@0.20": 0.5503606144146659, "auc@0.30": 0.5628765729519055, "auc@0.50": 0.5817225356411688} \ No newline at end of file diff --git a/submitted_results/twitter-airline-sentiment/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/twitter-airline-sentiment/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..8812db1 --- /dev/null +++ b/submitted_results/twitter-airline-sentiment/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3888916666664879, "auc": 0.6529844066862088, "auc@0.01": 0.5125115527385672, "auc@0.05": 0.5332034803647869, "auc@0.10": 0.5511212748684945, "auc@0.20": 0.577304229480737, "auc@0.30": 0.5964002611094689, "auc@0.50": 0.6239872021217198} \ No newline at end of file diff --git a/submitted_results/yelp_polarity/validation-test-model/1_200/all_to_all_pair_classification/metrics.json b/submitted_results/yelp_polarity/validation-test-model/1_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..3320d86 --- /dev/null +++ b/submitted_results/yelp_polarity/validation-test-model/1_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4538500000006369, "auc": 0.5626615665829147, "auc@0.01": 0.5022957437943486, "auc@0.05": 0.5083047223295967, "auc@0.10": 0.5144696244379794, "auc@0.20": 0.5241475825307091, "auc@0.30": 0.5321739734210267, "auc@0.50": 0.5459566850921272} \ No newline at end of file diff --git a/submitted_results/yelp_polarity/validation-test-model/2_200/all_to_all_pair_classification/metrics.json b/submitted_results/yelp_polarity/validation-test-model/2_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..ff10b9a --- /dev/null +++ b/submitted_results/yelp_polarity/validation-test-model/2_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.4185499999997272, "auc": 0.6109901661432161, "auc@0.01": 0.5084747702078231, "auc@0.05": 0.5231983861615771, "auc@0.10": 0.5363252826633166, "auc@0.20": 0.5559666282802903, "auc@0.30": 0.5704224541826781, "auc@0.50": 0.5909442826633166} \ No newline at end of file diff --git a/submitted_results/yelp_polarity/validation-test-model/3_200/all_to_all_pair_classification/metrics.json b/submitted_results/yelp_polarity/validation-test-model/3_200/all_to_all_pair_classification/metrics.json new file mode 100644 index 0000000..14a6fca --- /dev/null +++ b/submitted_results/yelp_polarity/validation-test-model/3_200/all_to_all_pair_classification/metrics.json @@ -0,0 +1 @@ +{"eer": 0.3801749999996554, "auc": 0.6633528467336683, "auc@0.01": 0.516214947223555, "auc@0.05": 0.5431798415152687, "auc@0.10": 0.5640829459798995, "auc@0.20": 0.5932742785106087, "auc@0.30": 0.6128532189131934, "auc@0.50": 0.6392424736180905} \ No newline at end of file