EvoLens uses real spectra and ground truth to discover candidate spectral channels, compare predictive models, challenge the result, apply the selected eye to compatible new data, and export reproducible evidence.
Open hosted reference · Quick start · Scientific integrity · Documentation
EvoLens is a virtual design and verification laboratory for scientists and engineering teams asking whether a hyperspectral instrument can be approximated by a smaller, task-specific sensing design. It preserves the scientific controls needed by experts while making data readiness, channel discovery, model evidence, fragility, and next actions understandable to a first-time spectral-ML user.
It does not manufacture or certify optical hardware. Every output is a candidate virtual sensor that still requires optical engineering, fabrication, calibration, and physical validation.
Hyperspectral instruments may measure hundreds or thousands of narrow bands. That density is valuable for discovery, but a field instrument, embedded system, or production line may have hard limits on channels, cost, power, latency, and calibration. Manually choosing a compact sensor for one prediction task is difficult because channel response, model choice, data leakage, domain shift, and hardware tolerances interact.
EvoLens turns that question into a traceable experiment:
Which physically plausible spectral response functions retain useful task information on groups that were not used to select the design—and where does the candidate fail?
EvoLens is not a text wrapper around an optimizer.
- The scientific engine calculates Gaussian channel responses and validation objectives.
- Models train on the actual selected response transform, not on a label attached to a full-spectrum model.
- Selected-eye and full-spectrum models remain separate versioned artifacts.
- Acquisition scenes, sites, specimens, or subjects remain isolated across splits.
- Held-out test data stays sealed until the candidate and model are frozen.
- Full-spectrum, evenly spaced, RGB-like where valid, and repeated-random baselines reuse the same split.
- Verification can mark an attractive validation result Fragile because of a generalization gap, unstable seed, drift, ablation, or calibration evidence.
- GPT-5.6 proposes structured experiments and explains real evidence; it cannot invent channels, metrics, or physical validation.
flowchart LR
Q["1. Define question"] --> D["2. Prepare real data"]
D --> S["3. Discover virtual sensor"]
S --> M["4. Compare models"]
M --> V["5. Verify evidence"]
V -->|"fragile: revise constraints"| S
V --> E["6. Inspect visual evidence"]
E --> I["7. Apply to compatible data"]
I --> X["8. Export reproducible package"]
The workflow is connected, not rigid. A verification failure can return to Design Lab; new data can invalidate an earlier split; a model comparison can motivate a different channel budget. Every stage retains the exact dataset, preparation, split, candidate, model, and evidence identifiers that produced it.
| Project evidence | Sensor and model evidence |
|---|---|
 |
 |
 |
 |
 |
 |
 |
flowchart TB
UI["Next.js product UI"] --> API["FastAPI application boundary"]
API --> DB["SQLModel + SQLite\nprojects and evidence lineage"]
API --> ENG["evolens_engine\nsplits · transforms · models · optimization"]
ENG --> RESP["Gaussian spectral responses\ncenter · FWHM · exposure"]
RESP --> EYE["Selected-eye transformation"]
EYE --> REG["Classical + PyTorch model registry"]
REG --> POLICY["Result status policy"]
POLICY --> VERIFY["Verification Lab\nleakage · stability · robustness · calibration"]
API --> ART["Immutable and content-addressed artifacts"]
API --> GPT["GPT-5.6 boundary\ncompact evidence · typed proposal · budget"]
GPT --> ALLOW["Schema + allowlist + user approval"]
ALLOW --> ENG
The Next.js UI presents backend records; it does not manufacture metrics. FastAPI owns
the product boundary and persistent project graph. The independent Python engine owns
group-aware splits, train-only preprocessing, transforms, fitting, metrics, evolution,
perturbations, inference compatibility, and exports. RESULT_STATUS_POLICY keeps a
measured candidate from being promoted when verification says otherwise.
EvoLens includes two compact, reproducible real-data derivatives. Raw upstream files
are not committed. Rights and exact SHA-256 values are in
DATA_LICENSES.md.
| Reference | Source evidence | Candidate | Grouped validation | Held-out test | Full-spectrum held-out | Status |
|---|---|---|---|---|---|---|
| HYPSO-1 sea/land/cloud | 112 verified bands; five documented scenes | 4 channels | balanced accuracy 0.9344 | balanced accuracy 0.7639 | 0.8056 | Fragile |
| OSSL ICRAF–ISRIC soil carbon | 1,076 VisNIR bands; site-grouped layers | 4 channels | R² 0.5957 | R² 0.2820 | 0.3769 | Fragile |
The HYPSO candidate compresses 112 source bands to four response functions, but its validation-to-test gap is 0.1706 and its held-out result trails both full-spectrum and the four-channel evenly spaced baseline. The OSSL candidate compresses 1,076 bands to four, but its held-out R² falls by 0.3137 from validation and random channel dropout is severe. These are scientifically useful negative findings—not deployment-ready wins.
- HYPSO-1 Sea–Land–Cloud, NTNU / DataverseNO V1, doi:10.18710/ZR0MVS, is CC BY-NC-SA 4.0. The included material is noncommercial and ShareAlike.
- OSSL v1.2 ICRAF–ISRIC VisNIR uses only the rights-filtered
ICRAF.ISRICcomponent, CC BY 4.0. OSSL is heterogeneous; this license is not assigned to other components. - Indian Pines is guided import only because authoritative redistribution terms and exact wavelength calibration are not registered.
GPT-5.6 acts as a controlled scientific collaborator, not the measurement engine.
It receives a compact evidence manifest: current project and page, task, dataset identity, aggregate split information, candidate response parameters, measured metrics, baselines, verification status, limitations, and artifact hashes. Raw spectra, labels, image arrays, private files, filesystem paths, environment values, and API credentials are excluded.
The model returns a typed proposal with a hypothesis, rationale, uncertainty, experiment type, parameters, control, falsification criterion, and stop condition. EvoLens validates that proposal against an allowlist and explicit budgets. In local-full mode a user must approve execution. Deterministic code performs the experiment and feeds the measured result back for interpretation. The audit record stores the returned model and response ID. Model text never becomes a measured artifact.
EvoLens runs fully without an OpenAI key. The interface says that GPT is offline and keeps
all deterministic workflows available. To enable live GPT locally, put only these values
in the root .env file:
OPENAI_API_KEY=
OPENAI_MODEL=gpt-5.6-solFill the key after = on your machine or in the Railway API service's secret manager.
Never commit .env, paste the key into source, or expose it through NEXT_PUBLIC_*.
Codex worked in the primary EvoLens repository as an engineering collaborator, with each change verified against the scientific invariants. Concretely, it:
- shaped the monorepo into a Next.js product, FastAPI boundary, SQLModel state graph, and independent scientific engine;
- implemented and tested optimization, candidate genomes, Gaussian sensor transforms, baselines, robustness, and group-isolated evaluation;
- migrated persistent state while preserving backward-compatible artifact linkage;
- traced the selected sensor into the selected-eye model instead of reusing the full-spectrum feature matrix;
- connected project overview, Design, Model, Verification, Inference, Visual Evidence, Scientist, and Export to exact versioned artifacts;
- added leakage, determinism, provenance, failure-state, API, and browser tests;
- performed route-by-route visual QA and repaired state, copy, responsiveness, and error behavior discovered in the running application;
- prepared public history, licensing, non-root images, hosted security controls, Railway configuration, CI, and one-command release verification.
Codex did not supply private research data or fabricate reference evidence. The public metrics come from named artifacts produced by EvoLens code.
- Python 3.12.13; the package supports the Python 3.12 line only.
- Node.js 22.23.1 and npm 10.9.8.
- macOS and Linux are supported through a Unix-like shell. CPU-only use is supported; CUDA is not required.
- Native Windows has not been validated. Use WSL2 with Ubuntu and Docker Desktop, or contribute a verified Windows run before describing it as supported.
.python-version and .nvmrc pin the reference runtimes. Python production and
development locks are requirements.lock and requirements-dev.lock; npm uses
apps/web/package-lock.json.
git clone <GITHUB_REPOSITORY_URL> evolens
cd evolens
cp .env.example .env
make install
make startOpen:
- Web: http://127.0.0.1:3000
- API: http://127.0.0.1:8000
- Local API docs: http://127.0.0.1:8000/docs
The two reviewed reference artifacts are already initialized. make dataset-list verifies
their readiness without downloading anything. The application stays useful in offline GPT
mode and does not fetch public data on install or startup.
make status # process IDs and health
make stop # stop only recorded EvoLens services
make start # build/start again and wait for healthFor foreground hot reload, use make dev. Local state, logs, uploads, and exports live
under ignored paths and remain on the machine.
| Capability | local_full |
hosted_reference |
|---|---|---|
| Public reference evidence | Yes | Yes |
| Public safe exports | Yes | Yes |
| Bounded reference Scientist | Yes | Yes |
| Private uploads and custom projects | Yes | Blocked |
| Search, training, verification jobs | Yes | Blocked for anonymous users |
| Compatible-data inference | Yes | Reference evidence only |
| Project rename/archive/delete | Yes | Blocked |
| Complete expert controls | Yes | Read-only results |
| API docs | Enabled on loopback | Disabled by default |
The hosted strategy is deliberately read-only until isolated visitor projects, durable quotas, and tenant authorization can be implemented safely. See Deployment for Railway's two-service architecture and exact settings.
Private work belongs in local_full. Supported inputs are:
- CSV / TSV tables;
- NumPy NPZ / NPY matrices or cubes;
- ENVI header plus paired image binary;
- HDF5;
- NetCDF;
- MATLAB arrays, including v7.3 through HDF5;
- TIFF / GeoTIFF;
- zipped Zarr stores.
Map the spectral matrix/cube, target or label map, optional sample IDs, and a scientifically meaningful grouping unit such as acquisition, site, specimen, or subject. Provide a finite, strictly increasing wavelength vector only when physically supported; otherwise choose band indices. Classification targets need class labels; regression targets need units. Spatial cubes without groups use deterministic blocks so adjacent pixels do not trivially cross splits. See Data import guide.
Review the data terms before downloading. Fetch is explicit, checksummed, and never runs as part of install, tests, image build, startup, or a GPT request.
make dataset-list
make dataset-fetch DATASET=hypso1-sea-land-cloud
make dataset-prepare DATASET=hypso1-sea-land-cloud
make reference-build DATASET=hypso1-sea-land-cloud
make dataset-fetch DATASET=ossl-icraf-isric-visnir
make dataset-prepare DATASET=ossl-icraf-isric-visnir
make reference-build DATASET=ossl-icraf-isric-visnirEach derivative records source/version, upstream and local hashes, exclusions, calibration, group semantics, split seed, task/target, output hash, and the exact rebuild command.
make test # backend, product API, and scientific-engine tests
make check # Ruff, formatting, strict mypy, ESLint, TypeScript, Next build
npm --prefix apps/web run test:e2e
make security-check # Python/npm dependency audit + tracked source/history scan
make env-check # local or production environment safety
docker compose build # deployment images
make release-check # complete clean-tree release gate, including Docker builds
make release-package # ignored report with commit, results, hashes, and warningsCI runs without a live key and without downloading public datasets. Browser tests mock the API contract with labeled measured fixtures; numerical/scientific tests use isolated arrays only inside tests and never expose them as product evidence.
apps/
api/ FastAPI product and hosted security boundary
web/ Next.js product
packages/evolens_engine/ Independent scientific engine
config/ Product and scientific configuration
datasets/manifests/ Source, rights, wavelength, and split metadata
data/showcases/ Reviewed compact public derivatives and measured results
scripts/ Setup, data preparation, migration, release, and startup
tests/ Scientific, API, state, hosted-mode, and regression tests
docs/ Product, methodology, evidence, rights, and deployment guides
docs/assets/ Approved interface captures and social preview
deploy/railway/ Service-specific Railway configuration
docker/ Non-root API and web production images
submission/ Separate project description and recording support
.github/workflows/ CI, audits, browser tests, and container builds
- No test-set selection. Candidates, hyperparameters, channel count, and stopping use training and validation only. Held-out evidence opens after freeze.
- Grouped isolation. Scene/site/specimen/subject is atomic across top-level splits.
- Train-only preprocessing. Fitted transforms never learn from validation or test.
- No invented wavelengths. An unverified axis remains a band index.
- Versioned evidence. Datasets, preparations, splits, candidates, models, verification, inference, and exports carry IDs and hashes.
- Fragility is a result. Generalization gaps, unstable seeds, drift, ablation, and calibration failures stay visible and can send the workflow back to design.
- Physical boundary. Virtual performance is not fabrication, calibration, optical tolerance, device latency, or target-environment proof.
- Privacy boundary. Private data stays in ignored local storage and raw content is not sent to GPT.
Read the full Scientific integrity policy, Methodology, and Result status policy.
Private hyperspectral research motivated the product direction and the need for compact task-specific sensing. No private thesis data, unpublished metric, proprietary image, or confidential result is included in this repository or used as public evidence.
This repository contains the EvoLens scientific platform: optimization and model infrastructure, persistent project evidence chain, selected-eye linkage, GPT-5.6 Scientist, scientific verification, inference, visual product, exports, public reference adapters, tests, and deployment tooling. See Prior work and Codex collaboration.
- Candidate virtual sensors are not manufactured hardware.
- Both current reference candidates are fragile; neither is a deployment claim.
- Public datasets may not match a user's geography, instrument, lab protocol, class prevalence, or future operating domain.
- Hosted-reference mode blocks unsafe anonymous workflows instead of pretending to be a complete multi-tenant research environment.
- Device latency, memory, power, calibration drift, optical cross-talk, and environmental performance require target-device and physical benchmarking.
- Indian Pines and any source with unclear rights remain guided import only.
- SQLite is appropriate for the current single-service deployment, not an unreviewed horizontally scaled write workload.
- Product guide
- Scientific workflow
- Architecture
- Methodology
- Simple and Expert modes
- Sensor-to-model linkage
- Model Lab
- Verification Lab
- Visual Evidence
- Inference Lab
- GPT-5.6 Scientist
- Independent verification
- Data rights and attribution
- Public release audit
- GitHub publication and release
- Local runbook
- Railway deployment
- Security policy
- Contributing
- v0.1.0 release notes
Original EvoLens software and documentation are licensed under
Apache-2.0. Dataset content, prepared derivatives, measured reference results,
and screenshots containing scientific evidence retain their separate terms. See
DATA_LICENSES.md, THIRD_PARTY_NOTICES.md,
and Data rights and attribution.
EvoLens acknowledges NTNU / DataverseNO for HYPSO-1; Soil Spectroscopy for Global Good, ICRAF, and ISRIC for the OSSL reference component; and the maintainers of its open-source Python and JavaScript dependencies.