Add maxsize directive to skip oversized files when scanning

[techjoec]

Motivation

qgrep has no notion of text vs binary — buildAppendFile() opens every file "rb" and indexes the raw bytes. That is a reasonable design, but it means the only lever for keeping huge files out of a database is the include/exclude path regexes, and those cannot express "this tree, but not the multi-gigabyte artifacts in it".

I hit this on a corpus that mixes source with firmware images, disk dumps and compressed archives in the same directories. Extension patterns get you most of the way, but any tree where a large blob shares an extension with source (or has none) pulls that blob into the index byte-for-byte, and the only workaround is to generate an explicit file list outside qgrep and feed it in as a project — which then has to be regenerated whenever the tree changes.

What this adds

A maxsize directive for project files:

path /some/tree
include \.(c|h)$
maxsize 16M

• Accepts a plain byte count or a K/M/G suffix, with an optional trailing B (16M, 512K, 1024, 1MB).
• Applies to the group it appears in and is inherited by nested groups, which can override it — same shape as include/exclude, so a root limit can be relaxed or tightened per subtree.
• 0 or absent means unlimited, so behaviour is unchanged for every existing project.
• Malformed values throw during parse and are reported with the usual file(line): prefix rather than being silently ignored.

Implementation notes

traverseDirectory() already hands its callback the file size, so the check in getProjectGroupFilesRec() costs nothing extra. Explicit file entries are filtered against the same limit after getFileAttributes().

Also adds #include <stdlib.h> to project.cpp for strtoull — it was previously reaching it transitively; main.cpp includes it explicitly for its own strtoul, so this follows the existing convention and avoids depending on a transitive include on MSVC.

Testing

Built with make (gcc, -Wall -Werror, clean). Verified manually:

• no directive → unchanged behaviour
• maxsize 1M / 100K on a tree of small/medium/large files → only files at or under the limit indexed
• root maxsize 1M with a nested group overriding to 100K → each group honours its own limit
• suffix forms 1048576, 1024K, 1M, 1m, 1MB all equivalent
• empty / abc / 10X → parse errors with file and line

Happy to adjust the name, the suffix handling, or the inheritance semantics if you'd prefer something different.

🤖 Generated with Claude Code