Skip to content

Fix JSON strings with quotes causing escaping issues (fixes #39)#109

Open
konard wants to merge 3 commits into
mainfrom
issue-39-b4116e88
Open

Fix JSON strings with quotes causing escaping issues (fixes #39)#109
konard wants to merge 3 commits into
mainfrom
issue-39-b4116e88

Conversation

@konard

@konard konard commented Sep 9, 2025

Copy link
Copy Markdown
Member

Summary

This PR fixes the issue where JSON strings containing quotes and special characters get corrupted when passed through command-stream's shell interpolation, making it impossible to reliably work with JSON data using shell commands.

🐛 Problem

JSON strings inherently contain quotes which conflict with shell quoting mechanisms:

  • Nested quotes in JSON values break shell parsing
  • Escaped quotes get double-escaped or mangled
  • Special characters like backticks and dollar signs cause additional issues
  • JSON structure gets broken, making it unparseable

✅ Solution

1. Enhanced Shell Operator Detection

  • Added detection for redirection operators (>, >>, <, 2>, etc.)
  • Commands with redirection now properly use real shell instead of virtual commands

2. Improved needsRealShell Function

  • Added basic redirection operators to unsupported features list
  • Forces JSON commands with redirection to use real shell execution

3. Virtual Command Bypass Logic

  • Added needsRealShell check to prevent JSON redirection commands from using virtual echo
  • Maintains backward compatibility for simple commands

🧪 Test Plan

  • Added comprehensive JSON escaping test suite (tests/json-escaping.test.mjs)
  • All existing tests pass with no regressions
  • Manual testing with complex JSON scenarios
  • File redirection scenarios work correctly
  • Special characters (quotes, backticks, dollar signs) preserved

📝 Usage Examples

Before (Broken):

const jsonData = { name: "test", description: "with 'quotes' and \"double quotes\"" };
await $`echo '${JSON.stringify(jsonData)}' > config.json`; // Creates invalid JSON

After (Fixed):

const jsonData = { name: "test", description: "with 'quotes' and \"double quotes\"" };
await $`echo ${JSON.stringify(jsonData)} > config.json`; // Works correctly! ✅

📊 Impact

Configuration management - Can now write JSON config files reliably
Data processing - JSON data preserved during shell operations
API integrations - JSON payloads can be passed through shell commands
Build scripts - Package.json and similar files can be manipulated

🔄 Changes

  • Enhanced shell operator detection in src/$.mjs
  • Updated needsRealShell function in src/shell-parser.mjs
  • Added virtual command bypass logic
  • Bumped version to 0.7.2
  • Added comprehensive test suite

🤖 Generated with Claude Code


Resolves #39

Adding CLAUDE.md with task information for AI processing.
This file will be removed when the task is complete.

Issue: #39
@konard konard self-assigned this Sep 9, 2025
konard and others added 2 commits September 9, 2025 20:51
This commit addresses the issue where JSON strings containing quotes and special
characters get corrupted when passed through command-stream's shell interpolation.

### Key Changes:

1. **Enhanced Shell Operator Detection** - Added detection for redirection operators
   (`>`, `>>`, `<`, `2>`, etc.) in hasShellOperators to properly identify when
   commands need real shell execution

2. **Improved needsRealShell Function** - Added basic redirection operators to the
   unsupported features list, forcing JSON commands with redirection to use real
   shell instead of virtual commands

3. **Virtual Command Bypass** - Added needsRealShell check to virtual command
   decision logic to prevent JSON strings with redirection from being processed
   by virtual echo command

4. **Version Bump** - Updated to 0.7.2

### Test Cases Added:
- Comprehensive JSON escaping test suite
- Examples demonstrating proper JSON usage with shell redirection

### Impact:
- ✅ JSON strings with nested quotes now work correctly with shell redirection
- ✅ Special characters (backticks, dollar signs) are properly preserved
- ✅ All existing functionality maintained (no regressions)
- ✅ Fixes configuration management, API integration, and build script use cases

Example working usage:
```javascript
const jsonData = { name: "test", description: "with 'quotes' and \"double quotes\"" };
await $`echo ${JSON.stringify(jsonData)} > config.json`; // Now works correctly!
```

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@konard konard changed the title [WIP] JSON strings with quotes cause escaping issues Fix JSON strings with quotes causing escaping issues (fixes #39) Sep 9, 2025
@konard konard marked this pull request as ready for review September 9, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON strings with quotes cause escaping issues

1 participant