MLX Server Manager is a local macOS control surface for explicit, profile-selected OpenAI-compatible local servers, including mlx_lm.server and Rapid-MLX. Its purpose is to make local MLX servers easier to operate without becoming part of the inference request path.
The current Direct Mode path remains:
OpenAI-compatible client -> mlx_lm.server or adopted external server -> MLX model
The app manages setup, lifecycle, diagnostics, profile metadata, logs, memory visibility, and connection settings. It does not proxy inference requests in the current architecture.
- Preserve
mlx-lmruntime performance as the top priority. - Make
mlx-lmusable for users who are not comfortable with CLI workflows. - Adopt useful features from other local LLM tools when they do not conflict with
mlx-lmperformance, safety, or Direct Mode boundaries.
The product should stay focused on making local mlx_lm.server operation clear, explicit, and fast. It should reduce routine CLI friction while keeping users in control of server ownership, launch arguments, profile metadata, and OpenAI-compatible connection details.
New features should be evaluated by whether they help users run or connect to mlx_lm.server more confidently without adding hidden runtime behavior.
mlx-lm generation performance is the top priority.
The runtime path should stay thin and direct. In the current architecture:
- There is no inference proxy.
- There is no request rewriting.
- There is no hidden routing layer between the client and server.
- Background features must not interfere with active generation.
- The app should avoid CPU, memory, and disk I/O contention during managed server operation.
- New features must not degrade managed
mlx_lm.serverruntime behavior.
Features that are useful but expensive should be opt-in, visible, and easy to avoid during active generation.
Users should not need to memorize common mlx_lm.server commands to operate a local server.
The GUI should expose common setup and operating workflows:
- executable path setup,
- model profile selection,
- Start / Stop / Restart for app-managed servers,
- readiness checks via
/v1/models, - port conflict visibility,
- logs and memory display for managed servers,
- OpenAI-compatible connection settings copy actions.
Advanced users should still be able to inspect and copy command or configuration details. The app should explain Direct Mode and OpenAI-compatible connection settings clearly instead of hiding the underlying server model.
The current boundary is:
OpenAI-compatible client -> mlx_lm.server or adopted external server -> MLX model
Current boundaries:
- No inference proxy.
- No general-purpose or persistent conversation workspace; Test Chat is an explicit diagnostic request surface.
- No hidden or automatic multi-backend routing; each profile selects its backend explicitly.
- No hidden request rewriting.
- No automatic external process ownership changes.
- Stop and Restart apply only to app-managed processes.
- Adopted External Server is connection context only, not process ownership.
These boundaries keep the app understandable and preserve the performance characteristics of the selected server.
Useful features from other local LLM tools can be considered when they pass performance, safety, and scope checks.
A candidate feature should:
- preserve
mlx-lmruntime performance, - preserve Direct Mode unless a future release explicitly changes architecture,
- avoid silent server Start / Stop / Restart behavior,
- keep process ownership visible,
- avoid importing or exporting secrets unexpectedly,
- avoid hidden network calls,
- keep user-visible control over files, downloads, endpoints, and launch arguments.
Convenience is welcome when it makes local mlx-lm safer or easier without making runtime behavior harder to reason about.
Current releases do not include:
- inference proxy,
- general-purpose or persistent chat workspace,
- hidden or automatic multi-backend router,
- automatic model deletion,
- automatic server start after import or download workflows,
- hidden background downloads,
- external process takeover,
- direct model-directory deletion or unconfirmed cache cleanup,
- replacement for dedicated package managers or Hugging Face tooling.
Explicit Hugging Face search and download are accepted convenience features only when user-triggered, visible, and outside the inference path.
Future candidates can be considered if they preserve performance, safety, and Direct Mode boundaries:
- runtime health, capability detection, and failure-recovery polish,
- deeper local cache awareness,
- full model-card browsing boundaries,
- profile templates,
- logs and diagnostics polish,
- connection settings polish,
- optional speed test polish.
Each candidate should define what it changes, what it refuses to change, and how it avoids interfering with active managed generation.
Import conflict handling should follow the same rule. v3.3.0 implements Rename for profile-name conflicts as an explicit metadata operation. v3.4.0 implements Replace only for one unambiguous existing profile target, with explicit confirmation. v3.5.0 adds deterministic fixtures and service-level tests for the current import/export schema and conflict behavior. v4.0.0 treats Import / Export as stable within this metadata-only boundary. Replace remains metadata-only and must not start servers, call readiness endpoints, alter external process ownership, or interfere with active generation.
Historical release-planning documents remain useful records of how the current workspace evolved, but they are not authoritative for present scope. Current product and engineering decisions must follow Stable Scope, this document, and AGENTS.md.
Model download is implemented only as an explicit, user-triggered convenience workflow. It remains separate from the inference path and from server lifecycle automation.
Current download boundaries:
- download must not be coupled to automatic server start,
- download must not happen silently,
- download must not delete models,
- target location, progress, and failure state should be clear,
- Hugging Face token handling must remain outside app persistence unless a future release defines explicit safe storage,
- downloaded model metadata must not be confused with profile import/export metadata.
Direct model-directory deletion remains outside the current scope. Matching Hugging Face cache cleanup is available only as an explicit, confirmed option during profile deletion, with path and source safety checks; it is never triggered by import, download, or server lifecycle automation.
The app should not silently collect, transmit, import, or export private runtime data.
Safety boundaries:
- no model weights in profile import/export,
- no Hugging Face cache mutation in profile import/export,
- no API keys, tokens, or secrets in profile import/export,
- no committed runtime settings or model files,
- no hidden LAN/WAN exposure,
- no external process stop, restart, kill, or ownership takeover,
- no telemetry, analytics, crash reporting, or external log sending.
Runtime logs can contain local paths because they reflect local execution. Public docs, screenshots, examples, and exported profile metadata should avoid personal paths and secrets.
Releases should be described by their effect on runtime behavior:
- docs-only releases update project guidance, public docs, screenshots, or plans,
- app releases may change UI, state, services, or launch behavior,
- binary asset releases should be explicit when users need a new
.app.
When a feature is only a future candidate, docs should say so clearly. When a feature is implemented, docs should describe the implemented boundary rather than imply broader automation.