Skip to content

Known limitation: Apple Foundation Model refuses MCP tool-use for third-party service domains #22

Description

@sfegette

Summary

MCP server support is correctly wired at the infrastructure layer (apfel loads tool definitions, server starts with --mcp flags), but the Apple Foundation Model refuses to invoke tools for requests involving named third-party services (Figma, etc.).

Observed behavior

With a local Figma MCP server configured and apfel restarted with --mcp <path>, asking "list my recent Figma files" produces:

I'm sorry, but as an LLM created by Apple, I cannot assist you with that request. Attempting to view or access personal or proprietary files without proper authorization is illegal.

The model pattern-matches third-party service names → unauthorized access → hard refusal, and hallucinates legal justifications for declining.

Root cause

This is not a grapfel or apfel bug. It is baked into the Apple Foundation Model's instruction tuning. The --permissive flag removes content safety filtering (violence, explicit content) but does not override refusals rooted in the model's RLHF training around perceived third-party data access.

Scope

  • MCP infrastructure is working correctly (server starts, tool definitions load)
  • Neutral/local MCP tools (filesystem, calculator, shell utilities) may work — untested
  • Any domain the model associates with proprietary third-party services is likely to refuse
  • The model has no mechanism to understand "I have legitimate tool grants for this service"

Resolution path

Blocked on Apple updating the Foundation Model's instruction tuning to support agentic tool-use contexts. No grapfel-side fix is possible. Track Apple Intelligence / FoundationModels framework release notes for changes to tool-use behavior.

Workaround

None currently. Advise users that MCP is available and correctly wired, but the underlying model's agentic capabilities are limited to domains it does not associate with third-party service access.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions