Skip to content

chore: clean up Nextflow run-time warnings — unmatched config selectors and redundant .first() calls #468

Description

@m-huertasp

Summary

A normal pipeline run prints two recurring families of warnings:

  1. WARN: There's no process matching config selector: … — a withName: selector in a config file matched no process in that run.
  2. WARN: The operator 'first' is useless when applied to a value channel ….first() was called on a channel that is already a single-value channel.

Neither breaks the run, but together they produce dozens of lines of noise that bury genuinely useful warnings. This issue tracks cleaning both up.


A. Dead config selectors — referenced process exists nowhere → delete

(verified: zero matches anywhere under .nf)

  • SUBSETPILEUPconf/modules.config:135
  • READSPOSBEDconf/modules.config:572
  • BBGTOOLS:DEEPCSA:OMEGANONPROT.*:SUBSETPANEL*conf/base.config:92, conf/exome.config:96. No SUBSETPANEL process; the real alias is SUBSETOMEGA.
  • BBGTOOLS:DEEPCSA:MUTRATE.*:MUTRATE*conf/base.config:102, conf/exome.config:102. No MUTRATE subworkflow/process exists.
  • BBGTOOLS:DEEPCSA:OMEGA.*:ESTIMATOR_.*conf/base.config:112. The only aliases are ESTIMATOR/ESTIMATORGLOBALLOC (no ESTIMATOR_<x>), so this catch-all never matches.
  • CREATESAMPLEPANELS{ALL,PROTAFFECT,NONPROTAFFECT,EXONS,INTRONS,SYNONYMOUS}conf/tools/panels.config:60-110. The module modules/local/createpanels/sample/main.nf defines CREATESAMPLEPANELS but it is never included or invoked (real processes are CREATECONSENSUSPANELS*). Decide: wire the module in, or delete both the orphan module and these 6 selectors.

B. Param-gated selectors — process exists, runs only when a flag is on → warning is expected when off

Confirm the gate, then decide whether to keep + document, or relocate the selector into the feature-specific config so it isn't loaded when the feature is disabled:

  • CREATEPANELS:CUSTOMPROCESSING.*params.customize_annotation (conf/base.config:47)
  • SIGNATURESNONPROT:SIGPROFILERASSIGNMENT* → non-protein signatures flags (conf/base.config:116)
  • MUTATEDCELLSVAF:MUTATEDGENOMESFROMVAFAMparams.mutated_cells_vaf (+omega+omega_globalloc) (conf/base.config:130, conf/tmp_quick_fixes.config:18)
  • MUT_PREPROCESSING:FILTERNANOSEQSNPparams.nanoseq_snp (conf/base.config:148)
  • EXPECTEDMUTATEDCELLS:.*params.expected_mutated_cells (conf/tmp_quick_fixes.config:23)
  • OMEGANONPROTMULTI:.*params.omega_multi (+non-prot) (conf/tools/omega.config:76)
  • OMEGANONPROT:.*GLOBALLOC / OMEGANONPROTMULTI:.*GLOBALLOCparams.omega_globalloc (both are OMEGA_ANALYSIS aliases) (conf/tools/omega.config:89-93)
  • .*:REGRESSIONS.*:{EDITCONFIG,CREATE_INPUT,MODELS,PLOT} → regression flags (conf/modules.config:653-685)
  • .*INTRONS:.* → introns analysis flags (conf/modules.config:151)
  • .*NONPROT:SUBSETMUTPROFILE → non-protein mutprofile flags (conf/modules.config:418)

C. Runtime-only selectors not found in tracked configs — investigate local overrides

The run also reported OMEGA.*:ESTIMATOR_DNDSCV*, OMEGA.*:ESTIMATOR_DRIVERMUTRATE*, and OMEGA.*:ESTIMATOR_(?!DNDSCV|DRIVERMUTRATE).*, which are not in any committed config. Confirm whether they come from an uncommitted local config or another branch, and fold them into the audit.

D. Redundant .first() on value channels

What the warning is telling us. A Nextflow channel comes in two flavours. A queue channel is like a conveyor belt: items pass by once and whoever takes one takes it for good. A value (single-value) channel is like a sticky note pinned to the wall: it holds one thing that everyone can read as many times as they like. .first() grabs the first item off the belt and pins it to the wall — it turns a queue channel into a value channel, which is how you make a single file reusable by many processes. The warning simply means we called .first() on something that was already pinned to the wall, so the call does nothing.

Why we have it. Three patterns in the code:

  1. .first() twice across a subworkflow boundary. A subworkflow's emit: block already pins the file with .first(), and then the main workflow pins it again:
    • exons_consensus_expanded_panel — pinned at enrichpanels/main.nf:77, re-pinned at deepcsa.nf:465, 495, 517, 531, 641.
    • domains_in_panel — pinned at createpanels/main.nf:176, re-pinned at deepcsa.nf:645.
    • dna2protein_mapping_depth_exons — pinned at enrichpanels/main.nf:81, re-pinned at deepcsa.nf:646.
  2. .first() on a take: input the caller already pinned — e.g. mutationdensities.first() at omega/main.nf:112. mutationdensities, bedfile, and panel_captured_rich are take: inputs (omega/main.nf:30-37) that arrive already pinned. These show up as check channel mutationdensities/bedfile/panel.
  3. .first() on a process output / param channel that's already single-valued — the warnings with no channel name, e.g. .join(profile).first() at omega/main.nf:58.

The fix. Pin each singleton once, at its source — the subworkflow emit: block, or the channel.fromPath(params.x, checkIfExists: true).first() where it's first read off the belt — and drop the .first() everywhere downstream that consumes an already-pinned .out.X or take: input. Don't strip blindly: .first() on a genuine queue channel feeding several processes is still needed. The -- check channel <name> hints point straight at the redundant ones.

Acceptance criteria

  • A: every dead selector removed (or CREATESAMPLEPANELS module wired in instead).
  • B: each selector confirmed param-gated and either documented as expected-when-off or relocated alongside its feature.
  • C: source of the ESTIMATOR_* selectors identified.
  • D: redundant .first() removed at the consumer sites above, keeping one .first() at each singleton's source.
  • A clean default run produces no no process matching config selector warnings for any section-A selector and no operator 'first' is useless warnings.
  • D: a full run on Seqera Platform completes successfully with the cleaned-up channels (no regressions from the removed .first() calls).

Related to issue #461

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions