Skip to content

LINT-KEYWORD substring-matches inside strings and comments (// Note: warns "Keyword No"), and reports only the first hit per keyword #707

Description

@logbie

Summary

LINT-KEYWORD does a raw substring search over the whole source file, so it fires on text inside string literals, inside comments, and inside ordinary words. store s as "MNOP" is reported as Keyword 'NO' should be lowercase; so is a comment reading // Note: or // TODO:. Because source.find returns only the first hit per keyword, the rule is also unable to report genuine violations after the first.

Reproduction

store s as "MNOP"
display s

Command:

wfl --lint l2_lint_keyword_in_string.wfl

Expected

Rule LINT-KEYWORD found 0 issues, exit 0. There is no keyword in this program —
MNOP is the content of a string literal. The rule's own description is
"Keywords should be lowercase" (src/linter/mod.rs:236), and a sequence of
characters inside a string literal is not a keyword.

Actual

Rule LINT-KEYWORD found 1 issues
Linting complete. Found 1 issues
Lint warnings:
warning[LINT-KEYWORD]: Keyword 'NO' should be lowercase
 = Change to 'no'

Exit code: 1. Control: store s as "MXYP" → 0 issues, confirming the match is
the substring NO inside MNOP.

Scope is wider than string literals — it also hits comments and prose

Each row is a two-line program with the shown line added; all are valid, correct
WFL:

line warnings
// Note: this explains the next step 1 — Keyword 'No' should be lowercase
// TODO: revisit 1 — Keyword 'TO' should be lowercase
store greeting as "Nothing to see" 2 — Keyword 'No', Keyword 'Nothing'
store label as "Ineligible" 1 — Keyword 'In' should be lowercase

So an ordinary English comment beginning "Note" or "TODO", and any capitalized
word containing a keyword substring, produce a lint warning and a nonzero exit.

Root cause

KeywordCasingRule::apply (src/linter/mod.rs:229-326) ignores the parsed AST
(_program: &Program, :242) and searches the raw source text:

// src/linter/mod.rs:292
if let Some(pos) = source.find(&uppercase_keyword) { ... }
// src/linter/mod.rs:308
if let Some(pos) = source.find(&mixed_case_keyword) { ... }

for each of 26 keywords, in both UPPERCASE and Mixedcase forms. There is no
tokenization, no word-boundary check, and no exclusion of string literals or
comments. The Mixedcase variant is what makes No, To, In, Nothing,
For, Each, End, Check, If, Count, From match ordinary English —
those are common English words and prefixes, so any file with prose comments is
likely to trip several.

It is also a false-negative

source.find returns only the first occurrence, so at most one diagnostic is
produced per keyword per casing — 52 possible warnings in a file of any size. A
file containing fifty genuine STORE keywords reports one. The rule can neither
avoid reporting non-keywords nor finish reporting real ones.

Suggested fix

Use the token stream (or the AST the rule already receives) instead of
source.find, and report a keyword only where the lexer actually produced a
keyword token. That fixes the false positives and the truncation together. If a
text-level implementation must be kept as an interim step, at minimum skip string
literals and comment text and require word boundaries — but the mixed-case English
collisions (No, To, In) argue for doing it properly on tokens.

Related

Same root-cause family as the companion LINT-INDENT issue in this batch: four of
the six rules receive the parsed Program and ignore it (_program at
src/linter/mod.rs:174, :242, :341, :386). For TrailingWhitespaceRule and
LineLengthRule that is legitimate. For IndentationRule and this rule it is the
direct cause of the defect. Independently fixable; noted so they are not triaged
as unrelated.

Environment

  • wfl --version: WebFirst Language (WFL) version 26.8.4
  • binary: system install C:\Program Files\wfl\bin\wfl.exe
  • commit: c277d8f
  • OS: Windows 11 Pro 10.0.26200
  • build: release

Also reproduces identically on the repo build G:\repos\wfl\target\release\wfl.exe
(version 26.8.2, 1 issue) — not a regression.
No .wflcfg in scope for the repro.

Context

Found while porting G:/repos/JShrink/src/JShrink/Minifier.php (a 738-line PHP
JavaScript minifier) to WFL. The port needs an uppercase-alphabet constant to test
PHP's \w character class:

store word_chars as "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_"

which contains MNOP and therefore reports Keyword 'NO' should be lowercase
verified, 1 issue. The string cannot be changed; it is the definition of the
character class.

This did not block the port (the program runs and analyzes clean; nothing was
stubbed), but combined with the LINT-INDENT issue it means --lint cannot be
used as a gate on this codebase, and there is no in-language way to silence a
single warning.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions