diff --git a/.gitignore b/.gitignore index 06ba450d6..b4505b3bf 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,11 @@ /tmp nohup.out vendor/ + +# Podman-specific +.podman-machine-* +podman-compose.override.yml +.env.podman + +# VS Code + Podman local settings +.vscode/settings.local.json diff --git a/README.md b/README.md index efcd8221d..799f87278 100644 --- a/README.md +++ b/README.md @@ -954,6 +954,24 @@ $ git clone https://github.com/Shopify/semian.git $ cd semian ``` +## Alternative: Using Podman + +If you prefer Podman over Docker, see our [Podman setup guide](docs/PODMAN.md). + +Quick start with Podman: +```bash +# Initialize everything +./scripts/podman-setup.sh init + +# Run tests +./scripts/podman-setup.sh test + +# Get a development shell +./scripts/podman-setup.sh shell +``` + +The Podman setup provides the same functionality as the Docker setup with full backward compatibility. + ## Visual Studio Code - Open semian in vscode diff --git a/docs/PODMAN.md b/docs/PODMAN.md new file mode 100644 index 000000000..711929338 --- /dev/null +++ b/docs/PODMAN.md @@ -0,0 +1,1229 @@ +# Semian Development with Podman + +This guide covers setting up and using Podman for Semian development as an alternative to Docker. + +## Table of Contents + +1. [Introduction](#introduction) +2. [Why Podman?](#why-podman) +3. [Prerequisites](#prerequisites) +4. [Quick Start](#quick-start) +5. [Detailed Command Reference](#detailed-command-reference) +6. [Architecture & Design](#architecture--design) +7. [Troubleshooting](#troubleshooting) +8. [Advanced Topics](#advanced-topics) +9. [Rootful Mode (Fallback)](#rootful-mode-fallback) +10. [FAQ](#faq) + +## Introduction + +Semian requires a Linux environment for bulkheading functionality (SysV semaphores). This guide shows how to set up a complete development environment using Podman, which is a Docker-compatible container engine. + +**Key benefits of this setup:** +- Full Linux environment for SysV semaphores +- All required services (MySQL, Redis, PostgreSQL, Toxiproxy) +- Simple automation via helper script +- Rootless mode by default (better security) +- No Docker licensing concerns + +## Why Podman? + +Podman is an excellent alternative to Docker: + +- **Docker-compatible**: Most Docker commands work with Podman +- **Rootless by default**: Better security model +- **Daemonless**: No background daemon required +- **Open source**: Apache 2.0 license, no licensing concerns +- **Drop-in replacement**: Uses same container images as Docker +- **Kubernetes-compatible**: Pod concept similar to K8s + +## Prerequisites + +### macOS Requirements + +- **macOS 11.0+** (Big Sur or later) +- **Homebrew** package manager +- **At least 6GB free RAM** (4GB for Podman machine + 2GB for host) +- **At least 20GB free disk space** + +### Linux Requirements + +- **Modern Linux distribution** (Ubuntu 20.04+, Fedora 31+, etc.) +- **Kernel 4.18+** with support for: + - User namespaces + - Overlay filesystem + - SysV semaphores +- **At least 4GB free RAM** + +## Quick Start + +### Installation + +#### macOS + +```bash +# Install Podman and podman-compose +brew install podman podman-compose + +# Verify installation +podman --version +podman-compose --version +``` + +#### Linux (Ubuntu/Debian) + +```bash +# Install Podman +sudo apt-get update +sudo apt-get install -y podman podman-compose + +# Verify installation +podman --version +podman-compose --version +``` + +#### Linux (Fedora/RHEL) + +```bash +# Install Podman +sudo dnf install -y podman podman-compose + +# Verify installation +podman --version +podman-compose --version +``` + +### Initialize and Run + +```bash +# Clone the repository (if not already done) +git clone https://github.com/Shopify/semian.git +cd semian + +# Initialize everything (one command!) +./scripts/podman-setup.sh init +``` + +This will: +1. Check that Podman and podman-compose are installed +2. Initialize Podman machine (macOS only) +3. Pull required container images +4. Start all services (MySQL, Redis, PostgreSQL, Toxiproxy) +5. Wait for services to be healthy +6. Install Ruby dependencies +7. Build C extensions + +### Run Tests + +Tests run in a dedicated `semian-tests` container with the proper hostname and network configuration (matches CI behavior). + +```bash +# Run all tests +./scripts/podman-setup.sh test + +# Skip flaky tests +./scripts/podman-setup.sh test --skip-flaky + +# Run with debugger (exposes port 12345) +./scripts/podman-setup.sh test --debug + +# Run specific test pattern +./scripts/podman-setup.sh test --only mysql2 +``` + +**Note:** The test container is ephemeral - it's created fresh for each test run and removed after completion. This ensures a clean environment matching CI/CD. + +### Development Workflow + +The development shell uses the `semian` container for interactive work. + +```bash +# Get a shell in the container +./scripts/podman-setup.sh shell + +# Inside the shell, you can: +bundle exec rake build # Build C extensions +bundle exec ruby examples/foo.rb # Run examples +irb -r ./lib/semian # Interactive Ruby + +# View status of all services +./scripts/podman-setup.sh status + +# View logs from a specific service +./scripts/podman-setup.sh logs mysql + +# View logs from all services +./scripts/podman-setup.sh logs +``` + +### Cleanup + +```bash +# Stop and remove containers +./scripts/podman-setup.sh clean + +# Also remove volumes +./scripts/podman-setup.sh clean --volumes + +# Also stop Podman machine (macOS) +./scripts/podman-setup.sh clean --volumes --machine +``` + +## Development vs Test Containers + +Semian uses a **two-container architecture** to separate development and testing workflows: + +### Architecture Overview + +| Container | Purpose | Hostname | Ports | Lifecycle | Created By | +|-----------|---------|----------|-------|-----------|------------| +| `semian` | Development shell, exploration | (random) | none | Persistent | `init` command | +| `semian-tests` | Running test suite | `http-server` | 31050, 31150, 12345 | Ephemeral | `test` command | + +### The `semian` Container (Development) + +**Purpose:** Interactive development and exploration + +**Usage:** +```bash +./scripts/podman-setup.sh shell +``` + +**Use this container for:** +- Interactive development work +- Running irb/pry sessions +- Building C extensions +- Exploring code +- Running examples +- Quick experiments + +**Important:** HTTP/network tests may fail in this shell because it lacks the required `http-server` hostname. For running tests, use the test command instead. + +### The `semian-tests` Container (Testing) + +**Purpose:** Running the complete test suite + +**Usage:** +```bash +./scripts/podman-setup.sh test +``` + +**Key features:** +- Has hostname `http-server` (required for network tests) +- Exposes ports 31050 and 31150 (HTTP mock servers) +- Exposes port 12345 (debugger) +- Fresh environment each run (ephemeral) +- Matches CI/GitHub Actions configuration exactly + +**Why separate containers?** + +1. **Clean separation of concerns:** Development shell stays clean and simple +2. **Matches CI behavior:** Tests run identically locally and in CI +3. **Proper networking:** HTTP tests need specific hostname configuration +4. **No interference:** Test runs don't affect development environment +5. **Isolated state:** Each test run starts fresh + +### Container Lifecycle + +**Development container (`semian`):** +- Created once by `init` command +- Runs continuously until stopped +- Preserves state between shell sessions +- Manually managed + +**Test container (`semian-tests`):** +- Created automatically by `test` command +- Runs only during test execution +- Removed immediately after tests complete (`run --rm`) +- No manual management needed + +### When to Use Which + +```bash +# ✅ Correct: Run tests via test command +./scripts/podman-setup.sh test + +# ✅ Correct: Development work in shell +./scripts/podman-setup.sh shell +$ bundle exec rake build +$ irb -r ./lib/semian + +# ❌ Wrong: Don't run full test suite from shell +./scripts/podman-setup.sh shell +$ bundle exec rake test # This will fail for HTTP/network tests! +``` + +## Detailed Command Reference + +### `init` - Initialize Environment + +```bash +./scripts/podman-setup.sh init [--verbose] +``` + +Initializes the complete development environment: +- Verifies Podman and podman-compose are installed +- Checks versions and warns if outdated +- Initializes Podman machine with 4GB RAM and 2 CPUs (macOS) +- Pulls all required container images +- Starts all services +- Waits for services to be healthy +- Installs Ruby dependencies +- Builds C extensions + +**Options:** +- `--verbose`: Show detailed output from all operations + +**Example:** +```bash +./scripts/podman-setup.sh init --verbose +``` + +### `test` - Run Tests + +```bash +./scripts/podman-setup.sh test [options] +``` + +Runs the Semian test suite in a dedicated `semian-tests` container. + +**Container Details:** +- Uses `semian-tests` container (not the development `semian` container) +- Has hostname `http-server` for network tests +- Exposes ports 31050, 31150 (HTTP mock servers) and 12345 (debugger) +- Ephemeral: Created fresh for each run, removed after completion +- Matches CI/GitHub Actions environment exactly + +**Options:** +- `--skip-flaky`: Skip tests marked as flaky +- `--debug`: Run with debugger support (exposes port 12345) +- `--only `: Run only tests matching pattern +- `--verbose`: Show detailed output + +**Examples:** +```bash +# Run all tests +./scripts/podman-setup.sh test + +# Skip flaky tests +./scripts/podman-setup.sh test --skip-flaky + +# Run with debugger +./scripts/podman-setup.sh test --debug + +# Run only MySQL2 adapter tests +./scripts/podman-setup.sh test --only mysql2 + +# Run only resource tests (non-network) +./scripts/podman-setup.sh test --only resource + +# Run only net_http tests (network tests) +./scripts/podman-setup.sh test --only net_http + +# Verbose output, skip flaky +./scripts/podman-setup.sh test --skip-flaky --verbose +``` + +**Why a separate test container?** + +Tests require specific network configuration: +- Hostname `http-server` for network/HTTP tests to connect +- Ports 31050 and 31150 for mock HTTP servers +- Clean environment for reproducible test runs + +This ensures tests run identically locally and in CI. + +### `shell` - Development Shell + +```bash +./scripts/podman-setup.sh shell [--verbose] +``` + +Opens an interactive bash shell inside the `semian` development container. + +**Features:** +- Automatically runs `bundle install` if needed +- Full access to all services +- Can build C extensions, run examples, use irb/pry +- Changes to code on host are immediately reflected + +**⚠️ Important:** +The shell displays a warning that HTTP/network tests may fail due to hostname requirements. For running the full test suite, use `./scripts/podman-setup.sh test` instead. + +You can run non-network tests (like resource tests) from the shell, but HTTP tests require the dedicated test container. + +**Example:** +```bash +./scripts/podman-setup.sh shell + +# Inside the shell: +$ bundle exec rake test:semian +$ bundle exec ruby examples/net_http/simple.rb +$ irb -r ./lib/semian +``` + +### `status` - Show Status + +```bash +./scripts/podman-setup.sh status [--verbose] +``` + +Shows the status of: +- Podman machine (macOS) +- All containers +- Service health +- SysV semaphore configuration + +**Example output:** +``` +Podman Machine: +NAME VM TYPE CREATED LAST UP CPUS MEMORY DISK SIZE +podman-machine-default* qemu 2 hours ago Currently running 2 4GB 100GB + +Containers: +NAMES STATUS PORTS +semian Up 30 minutes +mysql Up 30 minutes +redis Up 30 minutes +postgres Up 30 minutes +toxiproxy Up 30 minutes + +Service Health: + MySQL: ✓ Healthy + Redis: ✓ Healthy + PostgreSQL: ✓ Healthy + +SysV Semaphores: +250 32000 32 128 + Format: SEMMSL SEMMNS SEMOPM SEMMNI +``` + +### `logs` - View Logs + +```bash +./scripts/podman-setup.sh logs [service] +``` + +Tails logs from services. Press Ctrl+C to stop. + +**Examples:** +```bash +# View all service logs +./scripts/podman-setup.sh logs + +# View MySQL logs only +./scripts/podman-setup.sh logs mysql + +# View Redis logs only +./scripts/podman-setup.sh logs redis +``` + +### `clean` - Cleanup + +```bash +./scripts/podman-setup.sh clean [options] +``` + +Cleans up containers and optionally volumes and Podman machine. + +**Options:** +- `--volumes`: Also remove volumes (data will be lost) +- `--machine`: Also stop Podman machine (macOS) +- `--verbose`: Show detailed output + +**Examples:** +```bash +# Just remove containers +./scripts/podman-setup.sh clean + +# Remove containers and volumes +./scripts/podman-setup.sh clean --volumes + +# Remove everything including stopping machine +./scripts/podman-setup.sh clean --volumes --machine +``` + +## Architecture & Design + +### Service Architecture + +``` +┌─────────────────────────────────────────────────────┐ +│ Podman Machine (macOS) / Host (Linux) │ +│ │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ Semian Container (Development) │ │ +│ │ - Ruby 3.4.3 │ │ +│ │ - Bundler + Dependencies │ │ +│ │ - C Extensions Built │ │ +│ │ - Volume mount: ./:/workspace │ │ +│ └─────────────┬────────────────────────────────┘ │ +│ │ Shared IPC Namespace │ +│ │ (Critical for SysV semaphores) │ +│ ┌─────────────┴────────────────────────────────┐ │ +│ │ Supporting Services │ │ +│ │ ┌─────────┐ ┌─────────┐ ┌──────────┐ │ │ +│ │ │ MySQL │ │ Redis │ │ Postgres │ │ │ +│ │ │ :9.3 │ │ :latest │ │ :15 │ │ │ +│ │ └─────────┘ └─────────┘ └──────────┘ │ │ +│ │ ┌──────────────┐ │ │ +│ │ │ Toxiproxy │ (Resiliency Testing) │ │ +│ │ │ :2.12.0 │ │ │ +│ │ └──────────────┘ │ │ +│ └───────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────┘ +``` + +### Key Design Decisions + +#### 1. IPC Namespace Sharing + +SysV semaphores require shared IPC namespace. This is configured with: +```yaml +ipc: "shareable" +``` + +All containers can access the same semaphore set, which is critical for bulkhead functionality. + +#### 2. Rootless Mode with UID Mapping + +Using `userns_mode: "keep-id"` ensures: +- Container processes run as your user ID +- Files created in volumes have correct ownership +- Better security than running as root + +#### 3. Volume Mount with SELinux Support + +Using `./:/workspace:Z` ensures: +- Files are accessible in both rootless and rootful modes +- SELinux contexts are properly set (on systems with SELinux) +- Works transparently on macOS + +#### 4. Privileged Mode for Debugging + +`privileged: true` and capabilities like `SYS_PTRACE` enable: +- Debugger support (gdb, lldb) +- SysV semaphore operations +- Performance profiling tools + +### Network Configuration + +All services are on the same compose network, so they can communicate using service names: +- `mysql` - MySQL server +- `redis` - Redis server +- `postgres` - PostgreSQL server +- `toxiproxy` - Toxiproxy server + +Example: Connect to MySQL from Semian container: +```bash +mysql -h mysql -uroot -proot +``` + +### SysV Semaphores + +Semian uses SysV semaphores for bulkheading. These are kernel-level primitives that: +- Provide process-wide synchronization +- Survive process crashes (with SEM_UNDO) +- Have system-wide limits + +Check semaphore configuration: +```bash +./scripts/podman-setup.sh shell +cat /proc/sys/kernel/sem +``` + +Output format: `SEMMSL SEMMNS SEMOPM SEMMNI` +- **SEMMSL**: Max semaphores per array +- **SEMMNS**: System-wide semaphore limit +- **SEMOPM**: Max operations per semop call +- **SEMMNI**: Max semaphore arrays + +## Troubleshooting + +### Podman Machine Issues (macOS) + +#### Problem: "podman machine is not running" + +**Error:** +``` +Error: cannot connect to Podman. Please verify your connection to the Linux system +``` + +**Solution:** +```bash +# Check machine status +podman machine list + +# Start the machine +podman machine start + +# Or use the script +./scripts/podman-setup.sh init +``` + +#### Problem: Podman machine won't start + +**Error:** +``` +Error: qemu exited unexpectedly +``` + +**Solution:** +```bash +# Remove and recreate the machine +podman machine stop +podman machine rm + +# Re-initialize +./scripts/podman-setup.sh init +``` + +### Service Connection Issues + +#### Problem: "Connection refused" errors + +**Symptoms:** +- Tests fail with connection errors +- Can't connect to MySQL/Redis/Postgres + +**Solution:** +```bash +# Check service status +./scripts/podman-setup.sh status + +# Check specific service logs +./scripts/podman-setup.sh logs mysql + +# Restart services +./scripts/podman-setup.sh clean +./scripts/podman-setup.sh init +``` + +#### Problem: MySQL "Access denied for user" + +**Error:** +``` +Access denied for user 'root'@'172.x.x.x' (using password: YES) +``` + +**Solution:** +This usually means MySQL hasn't fully initialized. Wait a bit longer: +```bash +# Wait for MySQL manually +podman exec mysql mysqladmin ping -h localhost -uroot -proot + +# Or restart services +./scripts/podman-setup.sh clean +./scripts/podman-setup.sh init +``` + +#### Problem: "Name or service not known" for http-server + +**Error:** +``` +Socket::ResolutionError: Failed to open TCP connection to http-server:31050 +(getaddrinfo(3): Name or service not known) +``` + +Or: +``` +Errno::EADDRINUSE: Address already in use - bind(2) for "0.0.0.0" port 31050 +``` + +**Cause:** Tests are being run in the wrong container (development `semian` container instead of the `semian-tests` container). + +**Why this happens:** +- HTTP/network tests need hostname `http-server` to connect to mock servers +- The development `semian` container doesn't have this hostname +- The `semian-tests` container has the proper configuration + +**Solution:** + +Always use the `test` command, which automatically uses the correct container: + +```bash +# ✅ Correct - Uses test container with proper hostname +./scripts/podman-setup.sh test + +# ✅ Correct - Can specify patterns +./scripts/podman-setup.sh test --only net_http + +# ❌ Wrong - Running tests manually in shell +./scripts/podman-setup.sh shell +$ bundle exec rake test # This will fail for HTTP tests! +``` + +**If you need to run specific non-network tests in the shell:** + +```bash +./scripts/podman-setup.sh shell + +# These work (no network requirements): +$ bundle exec ruby -Ilib:test test/resource_test.rb +$ bundle exec rake build + +# These fail (need http-server hostname): +$ bundle exec ruby -Ilib:test test/adapters/net_http_test.rb # ❌ Will fail +``` + +**Key takeaway:** Always use `./scripts/podman-setup.sh test` for running tests. The test container is specifically configured for this purpose. + +### SysV Semaphore Issues + +#### Problem: "Operation not permitted" on semaphore operations + +**Error:** +``` +Errno::EPERM: Operation not permitted - semget +``` + +**Possible causes:** +1. Insufficient capabilities in rootless mode +2. IPC namespace not properly shared +3. System semaphore limits exceeded + +**Solutions:** + +**Option 1: Check IPC namespace** +```bash +# Verify IPC sharing is configured +grep "ipc:" podman-compose.yml +``` + +**Option 2: Check system limits** +```bash +./scripts/podman-setup.sh shell +cat /proc/sys/kernel/sem + +# Should show something like: 250 32000 32 128 +``` + +**Option 3: Switch to rootful mode** (see [Rootful Mode](#rootful-mode-fallback)) + +#### Problem: "No space left on device" for semaphores + +**Error:** +``` +Errno::ENOSPC: No space left on device - semget +``` + +**Cause:** System semaphore limit reached. + +**Solution:** +Clean up existing semaphores: +```bash +./scripts/podman-setup.sh shell + +# List semaphores +ipcs -s + +# Remove specific semaphore (if you know the ID) +ipcrm -s + +# Or remove all (use with caution!) +ipcs -s | grep $(whoami) | awk '{print $2}' | xargs -r ipcrm -s +``` + +### Volume Mount Issues + +#### Problem: Permission denied accessing files + +**Error:** +``` +Permission denied @ rb_sysopen - /workspace/Gemfile.lock +``` + +**Cause:** UID/GID mismatch between host and container. + +**Solution:** +The `userns_mode: "keep-id"` should handle this, but if not: + +```bash +# Check file ownership +ls -la Gemfile.lock + +# Fix ownership (on host) +sudo chown $(id -u):$(id -g) Gemfile.lock + +# Or recreate the container +./scripts/podman-setup.sh clean +./scripts/podman-setup.sh init +``` + +### Port Conflicts + +#### Problem: "address already in use" + +**Error:** +``` +Error: cannot listen on the TCP port: address already in use +``` + +**Solution:** +```bash +# Find what's using the port (12345 for debugger) +lsof -i :12345 + +# Kill the process or stop conflicting service +kill + +# Restart +./scripts/podman-setup.sh clean +./scripts/podman-setup.sh init +``` + +### Test Issues + +#### Problem: Tests hang indefinitely + +**Symptoms:** +- Tests start but never complete +- No error messages + +**Possible causes:** +1. Deadlock in semaphore operations +2. Service not responding +3. Network connectivity issue + +**Solutions:** + +**Check service health:** +```bash +./scripts/podman-setup.sh status +``` + +**Check for deadlocks:** +```bash +./scripts/podman-setup.sh shell +ipcs -s +# Look for semaphores with unusual values +``` + +**Run tests with timeout:** +```bash +timeout 300 ./scripts/podman-setup.sh test +``` + +**Clean restart:** +```bash +./scripts/podman-setup.sh clean --volumes +./scripts/podman-setup.sh init +./scripts/podman-setup.sh test +``` + +### General Debugging + +#### Enable verbose output + +```bash +./scripts/podman-setup.sh --verbose +``` + +#### Check container logs + +```bash +# All services +./scripts/podman-setup.sh logs + +# Specific service +podman logs semian +podman logs mysql +``` + +#### Inspect container + +```bash +# Get detailed container info +podman inspect semian + +# Check environment variables +podman exec semian env + +# Check running processes +podman top semian +``` + +## Advanced Topics + +### Running Specific Test Files + +```bash +# Get a shell +./scripts/podman-setup.sh shell + +# Run specific test file +bundle exec ruby -Ilib:test test/resource_test.rb + +# Run with minitest options +bundle exec ruby -Ilib:test test/resource_test.rb --verbose --name test_acquire +``` + +### Using Podman Desktop (GUI) + +[Podman Desktop](https://podman-desktop.io/) provides a GUI for managing containers: + +```bash +brew install podman-desktop +``` + +Features: +- Visual container management +- Image registry browser +- Volume management +- Log viewer +- Resource monitoring + +### Debugging with VS Code + +You can attach VS Code to a running container: + +1. Install "Dev Containers" extension +2. Start the environment: `./scripts/podman-setup.sh init` +3. In VS Code: Command Palette → "Dev Containers: Attach to Running Container" +4. Select "semian" + +Or use the Ruby debugger with the test command: +```bash +./scripts/podman-setup.sh test --debug + +# In another terminal or VS Code, connect to localhost:12345 +``` + +### Inspecting Semaphores + +```bash +./scripts/podman-setup.sh shell + +# List all semaphores +ipcs -s + +# Get semaphore key for a resource +irb -r ./lib/semian +resource = Semian::Resource.new(:test_resource, tickets: 5) +puts "0x%x" % resource.key + +# Inspect specific semaphore set +ipcs -si +``` + +### Performance Tuning + +#### Increase Podman machine resources (macOS) + +```bash +# Stop current machine +podman machine stop + +# Recreate with more resources +podman machine rm +podman machine init --memory 8192 --cpus 4 --disk-size 100 + +# Start and re-initialize +podman machine start +./scripts/podman-setup.sh init +``` + +#### Adjust container resources + +Edit `podman-compose.yml` and add: +```yaml +services: + semian: + deploy: + resources: + limits: + cpus: '2' + memory: 2G +``` + +### Customizing Configuration + +#### Use environment variables + +Create a `.env` file in the project root: +```bash +# .env +SKIP_FLAKY_TESTS=true +DEBUG=true +``` + +The test runner will automatically load this file. + +#### Override compose configuration + +Create `podman-compose.override.yml`: +```yaml +version: "3.7" +services: + semian: + environment: + - CUSTOM_VAR=value +``` + +This file is automatically loaded by podman-compose and overrides settings. + +## Rootful Mode (Fallback) + +If you encounter issues with SysV semaphores in rootless mode, you can switch to rootful mode. + +### ⚠️ Security Warning + +Rootful mode runs containers as root, which is less secure. Only use this if rootless mode doesn't work. + +### When to Use Rootful Mode + +Switch to rootful if you see: +- `Operation not permitted` errors on semaphore operations +- Bulkhead tests consistently failing +- `semget`/`semctl` system call errors + +### Switching to Rootful (macOS) + +```bash +# Stop and remove current machine +podman machine stop +podman machine rm + +# Create new rootful machine +podman machine init --rootful semian-rootful --memory 4096 --cpus 2 + +# Start machine +podman machine start semian-rootful + +# Set as default (optional) +podman system connection default semian-rootful + +# Re-initialize environment +./scripts/podman-setup.sh init +``` + +### Switching to Rootful (Linux) + +On Linux, use `sudo` with Podman commands: + +```bash +# Clean up rootless containers +./scripts/podman-setup.sh clean + +# Run as root +sudo podman-compose -f podman-compose.yml up -d + +# Get root shell +sudo podman exec -it semian bash + +# Run tests as root +sudo podman exec semian bash -c "cd /workspace && bundle exec rake test" +``` + +**Note:** This is not recommended for Linux. Consider fixing permissions instead. + +### Switching Back to Rootless (macOS) + +```bash +# Stop rootful machine +podman machine stop semian-rootful + +# Remove it +podman machine rm semian-rootful + +# Recreate default rootless machine +podman machine init --memory 4096 --cpus 2 + +# Start and re-initialize +podman machine start +./scripts/podman-setup.sh init +``` + +## FAQ + +### Can I use both Docker and Podman? + +Yes! The Docker and Podman setups are completely separate: +- Docker uses `.devcontainer/docker-compose.yml` +- Podman uses `podman-compose.yml` + +They don't interfere with each other. + +### How do I switch between Docker and Podman? + +Just use the appropriate tool: +```bash +# Docker +docker-compose -f .devcontainer/docker-compose.yml up -d + +# Podman +./scripts/podman-setup.sh init +``` + +### Will this work on Linux? + +Yes! Podman actually works better on Linux since it's native (no VM needed). + +Linux-specific benefits: +- Faster performance (no virtualization) +- Direct SysV semaphore access +- Lower resource overhead + +### What about CI/CD? + +The GitHub Actions workflows use Docker, which is fine. This Podman setup is for local development. + +If you want to use Podman in CI: +- Use native Linux runners (Ubuntu) +- Install Podman: `apt-get install podman` +- Use the test command: `./scripts/podman-setup.sh test` + +### Can I run tests from the development shell? + +**Short answer:** Use `./scripts/podman-setup.sh test` instead. + +**Long answer:** Some tests will work in the development shell, but HTTP/network tests will fail due to hostname requirements. + +**What works in the shell:** +```bash +./scripts/podman-setup.sh shell + +# ✅ These work (non-network tests) +$ bundle exec ruby -Ilib:test test/resource_test.rb +$ bundle exec ruby -Ilib:test test/simple_integer_test.rb +$ bundle exec rake build + +# ❌ These fail (need http-server hostname) +$ bundle exec rake test # Full suite fails +$ bundle exec ruby -Ilib:test test/adapters/net_http_test.rb +``` + +**Why?** + +The development `semian` container lacks: +- Hostname `http-server` (tests try to connect to `http-server:31050`) +- Published ports 31050 and 31150 (for HTTP mock servers) + +The `semian-tests` container has the proper configuration for all tests. + +**Best practice:** + +Always use the test command for running tests: +```bash +# Full suite +./scripts/podman-setup.sh test + +# Specific tests +./scripts/podman-setup.sh test --only net_http +./scripts/podman-setup.sh test --only resource +``` + +Use the shell for development work: +```bash +./scripts/podman-setup.sh shell + +# Development tasks +$ bundle exec rake build +$ irb -r ./lib/semian +$ ruby examples/net_http/simple.rb +``` + +### Can I use Podman without podman-compose? + +Yes, but it's more complex. You'd need to: +1. Create a pod: `podman pod create --name semian-dev` +2. Start each container in the pod +3. Manage networking manually + +The helper script makes this easier with podman-compose. + +### How much disk space does this use? + +Approximately: +- Podman machine (macOS): ~10GB +- Container images: ~2-3GB +- Volumes: ~1GB +- **Total: ~15GB** + +### How much memory does this use? + +- Podman machine: 4GB (configurable) +- Containers: ~1-2GB combined +- **Total: ~5-6GB** + +### Is Podman slower than Docker? + +Performance is similar: +- On macOS: Both use VMs, comparable speed +- On Linux: Podman is slightly faster (no daemon overhead) + +### Can I use rootless mode on Linux? + +Yes, and it's recommended! Rootless mode is actually easier on Linux than macOS. + +Requirements: +- User namespaces enabled (`sysctl kernel.unprivileged_userns_clone`) +- Subuids/subgids configured (`/etc/subuid`, `/etc/subgid`) + +Most modern distros have this by default. + +### What if I get "command not found: podman-compose"? + +Install it: +```bash +# macOS +brew install podman-compose + +# Linux with pip +pip3 install podman-compose + +# Or use podman's built-in support (experimental) +alias podman-compose='podman compose' +``` + +### Where are containers stored? + +- **macOS**: Inside Podman machine VM at `~/.local/share/containers/` +- **Linux**: `~/.local/share/containers/storage/` (rootless) or `/var/lib/containers/storage/` (rootful) + +### How do I completely uninstall everything? + +```bash +# Clean up containers and volumes +./scripts/podman-setup.sh clean --volumes --machine + +# Remove Podman machine (macOS) +podman machine rm + +# Uninstall Podman +brew uninstall podman podman-compose # macOS +sudo apt-get remove podman # Linux +``` + +### Can I contribute improvements to this setup? + +Absolutely! Please open a PR with: +- Bug fixes +- Documentation improvements +- New helper script features +- Performance optimizations + +--- + +## Additional Resources + +- [Podman Documentation](https://docs.podman.io/) +- [Podman Desktop](https://podman-desktop.io/) +- [Podman Compose GitHub](https://github.com/containers/podman-compose) +- [Main Semian README](../README.md) +- [Semian Contributing Guide](../CONTRIBUTING.md) (if exists) + +## Getting Help + +If you encounter issues: + +1. Check this troubleshooting guide +2. Run with `--verbose` flag for details +3. Check container logs: `./scripts/podman-setup.sh logs` +4. Open an issue on GitHub with: + - Your OS and version + - Podman version (`podman --version`) + - podman-compose version (`podman-compose --version`) + - Full error message + - Output of `./scripts/podman-setup.sh status` diff --git a/podman-compose.yml b/podman-compose.yml new file mode 100644 index 000000000..076114a13 --- /dev/null +++ b/podman-compose.yml @@ -0,0 +1,87 @@ +--- + +version: "3.7" +services: + semian: &base + container_name: semian + build: + dockerfile: .devcontainer/Dockerfile + context: . + target: base + working_dir: /workspace + volumes: + # :Z for SELinux relabeling (works on macOS, required on SELinux systems) + - ./:/workspace:Z + # Privileged mode needed for SysV semaphores and debugging + privileged: true + # Rootless mode: keep user ID mapping for file permissions + userns_mode: keep-id + cap_add: + - SYS_PTRACE # For ptrace-based debuggers + - SYS_ADMIN # May be needed for SysV semaphore operations + security_opt: + - seccomp:unconfined + - label=disable # Disable SELinux label separation + # IPC namespace sharing is CRITICAL for SysV semaphores to work + ipc: shareable + depends_on: + - redis + - mysql + - postgres + - toxiproxy + command: + - /bin/bash + - -c + - | + bundle install + sleep infinity + + test: + <<: *base + container_name: semian-tests + # Hostname is important for toxiproxy tests (acts as HTTP server) + hostname: http-server + build: + context: ./ + dockerfile: dockerfiles/semian-ci + command: + - bash + - -c + - |- + if [ -f .env ]; then + set -a + source .env + set +a + fi + ./scripts/run_tests.sh ${DEBUG:+--with-debugger} + ports: + - 31050:31050 # HTTP mock server port A + - 31150:31150 # HTTP mock server port B + - 12345:12345 # Debugger port for remote debugging + profiles: + - test + + toxiproxy: + image: ghcr.io/shopify/toxiproxy:2.12.0 + container_name: toxiproxy + depends_on: + - redis + - mysql + + redis: + container_name: redis + image: redis:latest + command: redis-server + + mysql: + container_name: mysql + image: mysql:9.3 + environment: + MYSQL_ROOT_PASSWORD: root + MYSQL_ROOT_HOST: "%" + + postgres: + container_name: postgres + image: postgres:15 + environment: + POSTGRES_PASSWORD: root diff --git a/scripts/podman-setup.sh b/scripts/podman-setup.sh new file mode 100755 index 000000000..b63cbfde5 --- /dev/null +++ b/scripts/podman-setup.sh @@ -0,0 +1,850 @@ +#!/usr/bin/env bash +# Semian Podman Development Setup Script +# Usage: ./scripts/podman-setup.sh [options] +# +# Commands: +# init Initialize Podman machine and start all services +# test Run the test suite +# shell Drop into container shell for development +# clean Stop and remove containers +# status Show status of services and containers +# logs Tail logs from services +# +# Options: +# --verbose Show detailed output +# --help Show this help message + +set -euo pipefail + +# ============================================================================== +# Configuration +# ============================================================================== + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)" +PODMAN_COMPOSE_FILE="${PROJECT_ROOT}/podman-compose.yml" +PODMAN_COMPOSE_CMD="podman-compose --in-pod false -f ${PODMAN_COMPOSE_FILE}" +REQUIRED_PODMAN_VERSION="4.0.0" +REQUIRED_PODMAN_COMPOSE_VERSION="1.0.0" +PODMAN_MACHINE_NAME="podman-machine-default" +PODMAN_MEMORY="4096" # 4GB +PODMAN_CPUS="2" + +# ============================================================================== +# Colors (ANSI escape codes - no extra dependencies) +# ============================================================================== + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +BOLD='\033[1m' +NC='\033[0m' # No Color + +# ============================================================================== +# Global Variables +# ============================================================================== + +VERBOSE=false + +# ============================================================================== +# Logging Functions +# ============================================================================== + +log_info() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +log_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +log_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +log_error() { + echo -e "${RED}[ERROR]${NC} $1" >&2 +} + +log_verbose() { + if [[ "$VERBOSE" == "true" ]]; then + echo -e "${CYAN}[VERBOSE]${NC} $1" + fi +} + +log_step() { + echo -e "${BOLD}==>${NC} $1" +} + +# ============================================================================== +# Utility Functions +# ============================================================================== + +print_usage() { + cat << EOF +${BOLD}Semian Podman Development Setup${NC} + +${BOLD}USAGE:${NC} + ./scripts/podman-setup.sh [options] + +${BOLD}COMMANDS:${NC} + ${GREEN}init${NC} Initialize Podman machine and start all services + ${GREEN}test${NC} Run the test suite + ${GREEN}shell${NC} Drop into container shell for development + ${GREEN}clean${NC} Stop and remove containers + ${GREEN}status${NC} Show status of services and containers + ${GREEN}logs${NC} Tail logs from services + +${BOLD}OPTIONS:${NC} + --verbose Show detailed output + --help, -h Show this help message + +${BOLD}EXAMPLES:${NC} + # Initialize everything + ./scripts/podman-setup.sh init + + # Run tests with verbose output + ./scripts/podman-setup.sh test --verbose + + # Run tests and skip flaky tests + ./scripts/podman-setup.sh test --skip-flaky + + # Run tests with debugger + ./scripts/podman-setup.sh test --debug + + # Get a development shell + ./scripts/podman-setup.sh shell + + # Clean up everything + ./scripts/podman-setup.sh clean + + # Clean up including volumes and machine + ./scripts/podman-setup.sh clean --volumes --machine + + # View logs from MySQL + ./scripts/podman-setup.sh logs mysql + +${BOLD}DOCUMENTATION:${NC} + See docs/PODMAN.md for comprehensive documentation + +EOF +} + +version_gt() { + # Returns 0 if $1 > $2, 1 otherwise + test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1" +} + +check_command() { + local cmd=$1 + if ! command -v "$cmd" &> /dev/null; then + return 1 + fi + return 0 +} + +check_macos() { + [[ "$(uname -s)" == "Darwin" ]] +} + +# ============================================================================== +# Dependency Checks +# ============================================================================== + +check_podman() { + log_verbose "Checking for Podman installation..." + + if ! check_command podman; then + log_error "Podman is not installed!" + echo "" + echo "Install with:" + if check_macos; then + echo " brew install podman" + else + echo " See https://podman.io/getting-started/installation" + fi + echo "" + exit 1 + fi + + local version + version=$(podman --version | awk '{print $3}') + log_success "Podman ${version} found" + + # Check if version is recent enough + if version_gt "$REQUIRED_PODMAN_VERSION" "$version"; then + log_warn "Podman version ${version} is older than recommended ${REQUIRED_PODMAN_VERSION}" + log_warn "Consider updating: brew upgrade podman" + fi + + log_verbose "Podman check complete" +} + +check_podman_compose() { + log_verbose "Checking for podman-compose installation..." + + if ! check_command podman-compose; then + log_error "podman-compose is not installed!" + echo "" + echo "Install with:" + if check_macos; then + echo " brew install podman-compose" + else + echo " pip3 install podman-compose" + fi + echo "" + exit 1 + fi + + local version + version=$(podman-compose --version 2>&1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1) + log_success "podman-compose ${version} found" + + # Check if version is recent enough + if version_gt "$REQUIRED_PODMAN_COMPOSE_VERSION" "$version"; then + log_warn "podman-compose version ${version} is older than recommended ${REQUIRED_PODMAN_COMPOSE_VERSION}" + log_warn "Consider updating: brew upgrade podman-compose" + fi + + log_verbose "podman-compose check complete" +} + +# ============================================================================== +# Podman Machine Management (macOS) +# ============================================================================== + +ensure_podman_machine() { + if ! check_macos; then + log_verbose "Not on macOS, skipping Podman machine setup" + return 0 + fi + + log_step "Ensuring Podman machine is set up..." + + # Check if any machine exists + if ! podman machine list --format "{{.Name}}" 2>/dev/null | grep -q .; then + log_info "No Podman machine found, initializing..." + log_verbose "Creating machine with ${PODMAN_MEMORY}MB RAM and ${PODMAN_CPUS} CPUs" + + if podman machine init \ + --memory "$PODMAN_MEMORY" \ + --cpus "$PODMAN_CPUS" \ + --disk-size 100 \ + --rootful=false 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Podman machine initialized" + else + log_error "Failed to initialize Podman machine" + exit 1 + fi + else + # Check if the machine configuration is corrupt (missing SSH keys, etc) + local machine_name + machine_name=$(podman machine list --format "{{.Name}}" 2>/dev/null | head -n1) + if [[ ! -f ~/.local/share/containers/podman/machine/machine ]]; then + log_warn "Podman machine exists but appears to be corrupted (missing SSH keys)" + log_info "This can happen with old machines or after Podman upgrades" + echo "" + echo "To fix this, we need to recreate the machine." + echo "This will NOT delete your containers or images." + echo "" + read -p "Recreate Podman machine? (y/n) " -n 1 -r + echo "" + if [[ $REPLY =~ ^[Yy]$ ]]; then + log_info "Removing old machine..." + podman machine rm -f "$machine_name" 2>&1 | while IFS= read -r line; do log_verbose "$line"; done + + log_info "Creating new machine with ${PODMAN_MEMORY}MB RAM and ${PODMAN_CPUS} CPUs..." + if podman machine init \ + --memory "$PODMAN_MEMORY" \ + --cpus "$PODMAN_CPUS" \ + --disk-size 100 \ + --rootful=false 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Podman machine recreated" + else + log_error "Failed to recreate Podman machine" + exit 1 + fi + else + log_error "Cannot proceed without a working Podman machine" + echo "" + echo "To fix manually:" + echo " podman machine rm -f $machine_name" + echo " podman machine init" + echo " podman machine start" + exit 1 + fi + fi + fi + + # Check if machine is running + local machine_status + machine_status=$(podman machine list --format "{{.Running}}" 2>/dev/null | head -n1) + + if [[ "$machine_status" != "true" ]]; then + log_info "Starting Podman machine..." + + # Try to start the machine + local start_output + start_output=$(podman machine start 2>&1) + local start_result=$? + + if [[ $start_result -eq 0 ]]; then + log_success "Podman machine started" + else + # Check for common error: stale socket files + if echo "$start_output" | grep -q "gvproxy.*socket"; then + log_warn "Found stale socket files, cleaning up..." + log_verbose "$start_output" + + # Clean up stale sockets + local tmpdir="/var/folders" + if [[ -d "/var/folders" ]]; then + find "$tmpdir" -name "podman-machine-*-gvproxy.sock" -type s 2>/dev/null | while read -r sock; do + log_verbose "Removing stale socket: $sock" + rm -f "$sock" 2>/dev/null || true + done + fi + + # Try starting again + log_info "Retrying Podman machine start..." + if podman machine start 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Podman machine started" + else + log_error "Failed to start Podman machine after cleanup" + echo "" + echo "Try manually:" + echo " podman machine stop" + echo " podman machine start" + echo "" + exit 1 + fi + else + log_error "Failed to start Podman machine" + log_verbose "$start_output" + echo "" + echo "Error output:" + echo "$start_output" + echo "" + echo "Try manually:" + echo " podman machine stop" + echo " podman machine start" + echo "" + exit 1 + fi + fi + else + log_success "Podman machine is running" + fi +} + +# ============================================================================== +# Service Health Checks +# ============================================================================== + +wait_for_mysql() { + log_info "Waiting for MySQL to be ready..." + local attempts=0 + local max_attempts=60 + + while ! podman exec mysql mysqladmin ping -h localhost -uroot -proot --silent 2>/dev/null; do + sleep 1 + attempts=$((attempts + 1)) + + if (( attempts >= max_attempts )); then + log_error "MySQL failed to start within ${max_attempts} seconds" + return 1 + fi + + if (( attempts % 10 == 0 )); then + log_verbose "Still waiting for MySQL... (${attempts}s)" + fi + done + + log_success "MySQL is ready" +} + +wait_for_redis() { + log_info "Waiting for Redis to be ready..." + local attempts=0 + local max_attempts=30 + + while ! podman exec redis redis-cli ping 2>/dev/null | grep -q PONG; do + sleep 1 + attempts=$((attempts + 1)) + + if (( attempts >= max_attempts )); then + log_error "Redis failed to start within ${max_attempts} seconds" + return 1 + fi + + if (( attempts % 10 == 0 )); then + log_verbose "Still waiting for Redis... (${attempts}s)" + fi + done + + log_success "Redis is ready" +} + +wait_for_postgres() { + log_info "Waiting for PostgreSQL to be ready..." + local attempts=0 + local max_attempts=30 + + while ! podman exec postgres pg_isready -U postgres 2>/dev/null | grep -q "accepting connections"; do + sleep 1 + attempts=$((attempts + 1)) + + if (( attempts >= max_attempts )); then + log_error "PostgreSQL failed to start within ${max_attempts} seconds" + return 1 + fi + + if (( attempts % 10 == 0 )); then + log_verbose "Still waiting for PostgreSQL... (${attempts}s)" + fi + done + + log_success "PostgreSQL is ready" +} + +wait_for_services() { + log_step "Waiting for services to be healthy..." + + if ! wait_for_mysql; then + return 1 + fi + + if ! wait_for_redis; then + return 1 + fi + + if ! wait_for_postgres; then + return 1 + fi + + log_success "All services are healthy" +} + +# ============================================================================== +# Command: init +# ============================================================================== + +cmd_init() { + log_step "Initializing Semian development environment with Podman..." + echo "" + + # Check dependencies + check_podman + check_podman_compose + echo "" + + # Ensure Podman machine (macOS only) + ensure_podman_machine + echo "" + + # Clean up any existing pods (from previous runs with different config) + log_verbose "Checking for existing pods..." + if podman pod list --format "{{.Name}}" 2>/dev/null | grep -q "pod_semian"; then + log_info "Removing existing pod from previous run..." + podman pod rm -f pod_semian 2>&1 | while IFS= read -r line; do log_verbose "$line"; done || true + fi + + # Pull images + log_step "Pulling Docker images..." + if $PODMAN_COMPOSE_CMD pull 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Images pulled" + else + log_warn "Some images may have failed to pull, continuing anyway..." + fi + echo "" + + # Start services + log_step "Starting services..." + if $PODMAN_COMPOSE_CMD up -d 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Services started" + else + log_error "Failed to start services" + exit 1 + fi + echo "" + + # Wait for services to be healthy + if ! wait_for_services; then + log_error "Services failed to become healthy" + echo "" + echo "Try checking logs with:" + echo " ./scripts/podman-setup.sh logs" + exit 1 + fi + echo "" + + # Install dependencies + log_step "Installing Ruby dependencies..." + if podman exec semian bash -c "bundle install" 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Dependencies installed" + else + log_error "Failed to install dependencies" + exit 1 + fi + echo "" + + # Build C extensions + log_step "Building C extensions..." + if podman exec semian bash -c "bundle exec rake build" 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "C extensions built" + else + log_error "Failed to build C extensions" + exit 1 + fi + echo "" + + # Success! + echo -e "${GREEN}${BOLD}🎉 Setup complete!${NC}" + echo "" + echo "Next steps:" + echo " ${CYAN}Run tests:${NC} ./scripts/podman-setup.sh test" + echo " ${CYAN}Get a shell:${NC} ./scripts/podman-setup.sh shell" + echo " ${CYAN}View status:${NC} ./scripts/podman-setup.sh status" + echo " ${CYAN}View logs:${NC} ./scripts/podman-setup.sh logs" + echo "" +} + +# ============================================================================== +# Command: test +# ============================================================================== + +cmd_test() { + local skip_flaky=false + local debug=false + local test_pattern="" + + # Parse test-specific flags + while [[ $# -gt 0 ]]; do + case $1 in + --skip-flaky) + skip_flaky=true + shift + ;; + --debug) + debug=true + shift + ;; + --only) + test_pattern="$2" + shift 2 + ;; + *) + shift + ;; + esac + done + + log_step "Running Semian test suite..." + echo "" + + # Verify base services are running (mysql, redis, etc.) + log_info "Checking if services are running..." + if ! podman ps --format "{{.Names}}" | grep -q "mysql"; then + log_error "Services are not running" + echo "" + echo "Start services first with:" + echo " ./scripts/podman-setup.sh init" + exit 1 + fi + + # Check service health quickly + if ! podman exec mysql mysqladmin ping -h localhost -uroot -proot --silent 2>/dev/null; then + log_error "MySQL is not responding" + exit 1 + fi + log_success "Services are running" + echo "" + + # Prepare environment variables + local env_args="" + if [[ "$skip_flaky" == "true" ]]; then + env_args="-e SKIP_FLAKY_TESTS=true" + log_info "Skipping flaky tests" + fi + + if [[ "$debug" == "true" ]]; then + env_args="$env_args -e DEBUG=true" + log_info "Running with debugger (port 12345 will be exposed)" + log_info "Connect your debugger to localhost:12345" + fi + + # Run tests using dedicated test container (matches CI behavior) + log_info "Running tests in dedicated test container..." + log_verbose "Test container has hostname 'http-server' for network tests" + + if [[ -n "$test_pattern" ]]; then + log_info "Running tests matching pattern: ${test_pattern}" + # Override command to run specific tests + if podman-compose --in-pod false -f "$PODMAN_COMPOSE_FILE" \ + --profile test run --rm $env_args test \ + bash -c "cd /workspace && bundle exec rake test TEST='test/**/*${test_pattern}*_test.rb'"; then + log_success "Tests passed!" + else + log_error "Tests failed!" + exit 1 + fi + else + # Use default command from compose file (runs scripts/run_tests.sh) + if podman-compose --in-pod false -f "$PODMAN_COMPOSE_FILE" \ + --profile test run --rm $env_args test; then + log_success "Tests passed!" + else + log_error "Tests failed!" + exit 1 + fi + fi +} + +# ============================================================================== +# Command: shell +# ============================================================================== + +cmd_shell() { + log_step "Opening shell in Semian container..." + echo "" + + # Verify container is running + if ! podman ps --format "{{.Names}}" | grep -q "^semian$"; then + log_error "Semian container is not running" + echo "" + echo "Start services first with:" + echo " ./scripts/podman-setup.sh init" + exit 1 + fi + + # Check if bundle install is needed + if ! podman exec semian bash -c "bundle check" &>/dev/null; then + log_info "Dependencies need updating, running bundle install..." + podman exec semian bash -c "bundle install" + fi + + log_info "Dropping into shell..." + echo "" + log_warn "Note: HTTP/network tests may fail in this shell due to hostname requirements." + log_info "For running the full test suite, use: ./scripts/podman-setup.sh test" + echo "" + podman exec -it semian bash +} + +# ============================================================================== +# Command: clean +# ============================================================================== + +cmd_clean() { + local remove_volumes=false + local stop_machine=false + + # Parse clean-specific flags + while [[ $# -gt 0 ]]; do + case $1 in + --volumes) + remove_volumes=true + shift + ;; + --machine) + stop_machine=true + shift + ;; + *) + shift + ;; + esac + done + + log_step "Cleaning up Semian environment..." + echo "" + + # Stop and remove containers + log_info "Stopping containers..." + if $PODMAN_COMPOSE_CMD down 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Containers stopped and removed" + else + log_warn "Some containers may have already been removed" + fi + + # Remove volumes if requested + if [[ "$remove_volumes" == "true" ]]; then + log_info "Removing volumes..." + if podman volume ls -q | grep -q .; then + podman volume ls -q | xargs -r podman volume rm 2>&1 | while IFS= read -r line; do log_verbose "$line"; done + log_success "Volumes removed" + else + log_info "No volumes to remove" + fi + fi + + # Stop Podman machine if requested (macOS only) + if [[ "$stop_machine" == "true" ]] && check_macos; then + log_info "Stopping Podman machine..." + if podman machine stop 2>&1 | while IFS= read -r line; do log_verbose "$line"; done; then + log_success "Podman machine stopped" + else + log_warn "Podman machine may have already been stopped" + fi + fi + + echo "" + log_success "Cleanup complete!" +} + +# ============================================================================== +# Command: status +# ============================================================================== + +cmd_status() { + log_step "Semian Environment Status" + echo "" + + # Podman machine status (macOS) + if check_macos; then + echo -e "${BOLD}Podman Machine:${NC}" + if podman machine list 2>/dev/null; then + echo "" + else + echo " Not available" + echo "" + fi + fi + + # Container status + echo -e "${BOLD}Containers:${NC}" + if podman ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" | grep -E "(NAMES|semian|mysql|redis|postgres|toxiproxy)"; then + echo "" + else + echo " No containers found" + echo "" + fi + + # Quick health checks + echo -e "${BOLD}Service Health:${NC}" + + # MySQL + if podman exec mysql mysqladmin ping -h localhost -uroot -proot --silent 2>/dev/null; then + echo -e " MySQL: ${GREEN}✓ Healthy${NC}" + else + echo -e " MySQL: ${RED}✗ Not responding${NC}" + fi + + # Redis + if podman exec redis redis-cli ping 2>/dev/null | grep -q PONG; then + echo -e " Redis: ${GREEN}✓ Healthy${NC}" + else + echo -e " Redis: ${RED}✗ Not responding${NC}" + fi + + # PostgreSQL + if podman exec postgres pg_isready -U postgres 2>/dev/null | grep -q "accepting connections"; then + echo -e " PostgreSQL: ${GREEN}✓ Healthy${NC}" + else + echo -e " PostgreSQL: ${RED}✗ Not responding${NC}" + fi + + echo "" + + # SysV Semaphores info (if semian container is running) + if podman ps --format "{{.Names}}" | grep -q "^semian$"; then + echo -e "${BOLD}SysV Semaphores:${NC}" + if podman exec semian cat /proc/sys/kernel/sem 2>/dev/null; then + echo " Format: SEMMSL SEMMNS SEMOPM SEMMNI" + echo "" + else + echo " Unable to read semaphore info" + echo "" + fi + fi +} + +# ============================================================================== +# Command: logs +# ============================================================================== + +cmd_logs() { + local service="$1" + + if [[ -z "$service" ]]; then + log_info "Showing logs from all services (press Ctrl+C to stop)..." + $PODMAN_COMPOSE_CMD logs -f + else + log_info "Showing logs from ${service} (press Ctrl+C to stop)..." + podman logs -f "$service" + fi +} + +# ============================================================================== +# Main +# ============================================================================== + +main() { + # Parse global flags + local command="" + + while [[ $# -gt 0 ]]; do + case $1 in + --verbose) + VERBOSE=true + shift + ;; + --help|-h) + print_usage + exit 0 + ;; + init|test|shell|clean|status|logs) + command="$1" + shift + break + ;; + *) + log_error "Unknown option: $1" + echo "" + print_usage + exit 1 + ;; + esac + done + + # Check if command was provided + if [[ -z "$command" ]]; then + log_error "No command specified" + echo "" + print_usage + exit 1 + fi + + # Route to appropriate command handler + case $command in + init) + cmd_init "$@" + ;; + test) + cmd_test "$@" + ;; + shell) + cmd_shell "$@" + ;; + clean) + cmd_clean "$@" + ;; + status) + cmd_status "$@" + ;; + logs) + cmd_logs "$@" + ;; + *) + log_error "Unknown command: $command" + echo "" + print_usage + exit 1 + ;; + esac +} + +# Run main function +main "$@"