bug(marketplace): marketplaces.json is written non-atomically and a corrupt registry is silently persisted as empty

[BenHao-WTG]

Summary

marketplaces.json can be silently emptied and stays empty, permanently unregistering every marketplace in a workspace. The symptom is the workspace suddenly reporting no marketplaces, and every plugin becoming unresolvable.

This is the same symptom as #449 (closed by #451), but that fix addressed one trigger (installing an invalid plugin). The two underlying defects that let any trigger cause permanent data loss are still on main (8cf4c78).

Evidence

Observed on a real workspace, cargowiseII-allagents:

.allagents/marketplaces.json        0 bytes
~/.allagents/marketplaces.json      {"version":1,"marketplaces":{}}

workspace.yaml still referenced five plugins, all @wtg-ai-prompts:

plugins:
  - cargowise@wtg-ai-prompts
  - cargowise-customs@wtg-ai-prompts
  - crikey@wtg-ai-prompts
  - cargowise-customs-jp@wtg-ai-prompts
  - skill-usage-metrics@wtg-ai-prompts

So the project registry was truncated to zero bytes, and the user registry held exactly the empty-fallback object — i.e. the damaged state had already been read back as "empty" and written to disk as legitimate.

Defect 1 — the registry is written non-atomically

https://github.com/EntityProcess/allagents/blob/main/src/core/marketplace.ts#L322-L333

export async function saveRegistryToPath(
  registry: MarketplaceRegistry,
  registryPath: string,
): Promise<void> {
  const dir = dirname(registryPath);
  if (!existsSync(dir)) {
    await mkdir(dir, { recursive: true });
  }
  await writeFile(registryPath, `${JSON.stringify(registry, null, 2)}\n`);
}

writeFile truncates in place. There is no temp-file-and-rename and no lock, so a process killed mid-write — or two processes writing concurrently — leaves a zero-byte or partial file. That matches the 0-byte file above.

Concurrency is reachable in normal use because plugin list and skill list are not read-only. Both resolve plugin specs, and resolvePluginSpec re-clones a stale marketplace:

https://github.com/EntityProcess/allagents/blob/main/src/core/marketplace.ts#L1671-L1705

const results = await updateMarketplace(registration.key, options.workspacePath);
...
const refreshResult = await refreshMarketplace(registration);

Both paths end in saveRegistryToPath (L1428, L1588). So two concurrent allagents invocations against one workspace — a terminal command while a scheduled allagents update runs, or any tool that fans out several commands — can race on the same file.

Defect 2 — a corrupt registry is silently reported as empty, then persisted

https://github.com/EntityProcess/allagents/blob/main/src/core/marketplace.ts#L303-L317

export async function loadRegistryFromPath(
  registryPath: string,
): Promise<MarketplaceRegistry> {
  if (!existsSync(registryPath)) {
    return { version: 1, marketplaces: {} };
  }

  try {
    const content = await readFile(registryPath, 'utf-8');
    return JSON.parse(content) as MarketplaceRegistry;
  } catch {
    return { version: 1, marketplaces: {} };
  }
}

The bare catch makes three very different states indistinguishable:

• the file does not exist (legitimately empty)
• the file is corrupt or truncated (data loss in progress)
• the file could not be read — EACCES, EBUSY, antivirus lock, etc. (transient)

All three return an empty registry. The damage then becomes permanent: the next mutation loads empty, adds one entry, and saves — discarding every other registration. This is why the user-level file contains a pristine {"version":1,"marketplaces":{}} rather than anything corrupt.

This is what makes the bug recur rather than self-heal. Defect 1 causes one corruption event; defect 2 converts it into permanent silent data loss and erases the evidence.

Steps to reproduce

Defect 2 alone, deterministically:

cd <workspace>
allagents plugin marketplace add https://github.com/some/repo --name m1 --scope project
printf '' > .allagents/marketplaces.json          # simulate an interrupted write
allagents plugin marketplace list                 # reports no marketplaces, no error
allagents plugin marketplace add https://github.com/other/repo --name m2 --scope project
cat .allagents/marketplaces.json                  # m1 is gone for good

Expected: step 3 reports the registry is unreadable; step 4 refuses to overwrite it.
Actual: the corruption is reported as "empty" and then silently overwritten.

Suggested fixes

1. Write atomically.

const tmp = `${registryPath}.${process.pid}.tmp`;
await writeFile(tmp, `${JSON.stringify(registry, null, 2)}\n`);
await rename(tmp, registryPath);   // atomic within a filesystem

A reader then sees either the old file or the new one, never a truncated one.

2. Distinguish "absent" from "unreadable". Return an empty registry only for ENOENT. For a parse failure or any other read error, throw (or return a discriminated result) so callers can refuse to overwrite:

try {
  const content = await readFile(registryPath, 'utf-8');
  return JSON.parse(content) as MarketplaceRegistry;
} catch (err) {
  if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
    return { version: 1, marketplaces: {} };
  }
  throw new Error(
    `Marketplace registry at ${registryPath} is unreadable: ${(err as Error).message}. ` +
    `Refusing to overwrite it — fix or delete the file to continue.`,
  );
}

3. Optional — back up before overwrite. Copying the previous file to marketplaces.json.bak on each save makes recovery trivial and costs almost nothing.

4. Optional — a cross-process lock on the registry path, so concurrent invocations serialise rather than interleave.

Fixes 1 and 2 are independently valuable: 1 prevents the corruption, 2 stops it becoming permanent and makes it diagnosable.

Impact

Silent, permanent loss of all marketplace registrations. There is no error message, and the resulting file is valid JSON, so nothing downstream flags it. Recovery requires knowing to re-run plugin marketplace add with the exact --name the plugin specs expect — anything else leaves every plugin orphaned.

Environment

• allagents 1.13.4 observed; defects confirmed still present on main at 8cf4c78 (1.13.6-next.1)
• Windows 11, PowerShell
• Related: marketplaces.json will be cleaned when we install an invalid plugin #449 (same symptom, different trigger), fixed by fix(marketplace): handle invalid registrations safely #451