975B Total Parameters | 41B Active Parameters | 1M Token Context Window
🤗 Hugging Face Page •
📝 Official Blog Post •
📖 Tinker Cookbook •
📄 Documentation •
Inkling is an open-weights, general-purpose multimodal Mixture-of-Experts (MoE) model designed for high-performance and efficient reasoning across multiple modalities. Trained by Thinking Machines, Inkling reasoning natively spans text, images, and audio/video, balancing cost and performance through controllable thinking effort.
Inkling is designed to be a balanced foundation model—extremely strong across agentic workflows, complex coding tasks, instruction following, and calibrated forecasting, rather than narrowly optimizing for a single benchmark. Alongside Inkling, a preview of Inkling-Small is shared—a lighter-weight version with 12B active parameters (276B total) that achieves comparable reasoning strength at lower cost and latency.
- Native Multimodality: Encoder-free audio and vision processing, accepting WAV files (16kHz) and any pixel-based image format.
- Controllable Thinking Effort: Dynamic test-time compute scaling (from
0.2to0.99) allowing developers to optimize for token cost vs. response quality. - Calibrated Forecasting (Epistemics): Trained via Reinforcement Learning (RL) with abstention-aware rewards to gauge uncertainty and avoid hallucinations.
- Agentic Tool Use: Natively trained to operate within agent harnesses and execute shell or browser tools autonomously.
Inkling supports local deployment and inference across several major open-source frameworks. Choose the framework that fits your hardware configuration below.
vLLM provides high-throughput inference with PagedAttention support.
-
Installation:
pip install vllm>=0.4.0 -
Run Server:
vllm serve thinkingmachines/Inkling --port 8000 --trust-remote-code
-
Test API Call:
curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "thinkingmachines/Inkling", "messages": [{"role": "user", "content": "Explain Mixture-of-Experts in one sentence."}] }'
SGLang supports fast execution and structured outputs.
-
Installation:
pip install sglang[all]
-
Run Server:
python -m sglang.launch_server --model-path thinkingmachines/Inkling --port 30000 --trust-remote-code
-
Test Client Script:
import openai client = openai.Client(base_url="http://127.0.0.1:30000/v1", api_key="EMPTY") response = client.chat.completions.create( model="thinkingmachines/Inkling", messages=[{"role": "user", "content": "Solve: 25 * 45"}] ) print(response.choices[0].message.content)
Ideal for quick prototyping or integration into existing PyTorch pipelines.
-
Installation:
pip install transformers torch accelerate
-
Run Inference:
from transformers import AutoModelForCausalLM, AutoTokenizer import torch model_id = "thinkingmachines/Inkling" tokenizer = AutoTokenizer.from_pretrained(model_id) # Load in BF16 (requires GPU with ~80GB VRAM for full model or quantization) model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto" ) inputs = tokenizer("How does sliding window attention work?", return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=100) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
TokenSpeed provides accelerated inference on various hardware using low-level optimizations.
-
Installation:
pip install tokenspeed
-
Run Model:
tokenspeed run --model thinkingmachines/Inkling
For running quantized weights (NVFP4) on consumer-grade hardware or NVIDIA Blackwell systems.
-
Installation:
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build -GGUIDE cmake --build build --config Release -
Run Quantized Inference:
./build/bin/llama-cli -m thinkingmachines/Inkling-NVFP4 --prompt "What is the capital of France?"
If you prefer not to host the model locally, Inkling is accessible online through playgrounds and cloud APIs.
Try Inkling in a web interface for free (limited time):
- Navigate to the Tinker Console Playground.
- Chat, experiment with tools, and configure thinking effort dynamically.
Inkling is hosted globally by key cloud providers:
| Provider | Endpoint API / Python Example |
|---|---|
| Together AI | api.together.xyz (Use together Python library or OpenAI-compatible client) |
| Fireworks AI | api.fireworks.ai (Use standard OpenAI SDK configured with Fireworks base URL) |
| Modal | Dynamically host serverless endpoints via modal.com |
| Databricks | Serverless Real-Time Inference endpoints |
| Baseten | Dedicated autoscaling deployment endpoints |
import openai
client = openai.OpenAI(
base_url="https://api.fireworks.ai/inference/v1",
api_key="YOUR_FIREWORKS_API_KEY"
)
response = client.chat.completions.create(
model="accounts/fireworks/models/inkling",
messages=[{"role": "user", "content": "Summarize the history of mechanical escapements."}],
extra_body={"thinking_effort": 0.9} # Adjust thinking effort (0.0 to 1.0)
)
print(response.choices[0].message.content)Inkling was subjected to rigorous testing across three distinct benchmarks representing different operational challenges. Below are the prompts, constraints, and documented capabilities evaluated:
- Objective: Verify agentic web development, layout design, and strict technical constraint following.
- Target Capability: Single-shot generation of complex frontend state management and interaction.
- The Prompt:
"Build a single-page Kanban board web application using only plain HTML, CSS, and Vanilla JavaScript. The application must include three columns: 'To Do', 'In Progress', and 'Done'. Users must be able to add new task cards, drag and drop cards between columns, and delete cards. Store the board state in the browser's localStorage so the data persists after a page refresh. Keep all code in a single file and use a clean, modern, minimalist color palette."
- Result / Behavior: Inkling built the entire app in a single turn without external dependencies, adhering strictly to the constraints. It handled CSS drag-and-drop API bindings, event listeners, and localStorage JSON serialization perfectly. On the Design Arena Agentic Web Dev leaderboard, Inkling consistently ranks among the strongest open-weights models (comparable to Claude Opus 4.6).
- Objective: Evaluate forecasting capabilities, uncertainty calibration, and structured factor breakdown without hallucinating certainty.
- Target Capability: Abstention-aware forecasting and reasoning under high ambiguity.
- The Prompt:
"What is the probability that a human will successfully land on Mars and return to Earth safely by the end of the year 2035? Break down the specific technical, economic, and political variables you are considering. Provide a calibrated probability estimate (as a percentage), and clearly state the conditions, missing information, or future events that would cause you to significantly revise this estimate up or down."
- Result / Behavior: Rather than presenting a generic answer or asserting absolute confidence, Inkling utilized its calibration training. It systematically structured the response into technical (propulsion, life support), economic (NASA/private funding), and political (global space treaties, election-cycle priorities) variables, provided a precise, calibrated probability percentage, and listed quantitative boundary conditions (e.g., success of SLS/Starship milestones) that would adjust its confidence interval. On ForecastBench, Inkling matches frontier models (Brier Index 61.1 without search, 63.7 with search).
- Objective: Test long-context coherence, complex instruction following, and high-end formatting aesthetics.
- Target Capability: Long-form narrative structure, layout diversification, and factual accuracy.
- The Prompt:
"Create a premium, editorial-style technical zine titled 'The Mechanics of Time: A Brief History of Escapements'. Explore the evolution of clockwork from the verge escapement to the coaxial escapement. The publication should feel like a refined, high-end engineering journal. Use elegant markdown typography, generous whitespace, and varied editorial layouts. Include a cover page, an introduction, three distinct chronological features, and a reference page. Keep the writing atmospheric but strictly grounded in mechanical facts."
- Result / Behavior: Evaluated by rubric and claims graders, the model generated a beautifully structured multi-page markdown document. It correctly mapped the chronological mechanics of the Verge, Anchor, Deadbeat, and Coaxial escapements with high factual accuracy, while organizing the text into a visually distinct editorial layout (using whitespace, blockquotes, and tables) that felt premium.
graph TD
A[Multimodal Inputs] -->|Audio dMel Spectrogram| B(Embedding Layer)
A -->|Image 40x40 Patches| B
A -->|Text UTF-8| B
B --> C[Decoder-Only Transformer]
C --> D[Attention: Interleaved Sliding Window & Global 5:1]
D --> E[MoE Layer: 256 Routed Experts + 2 Shared Experts]
E -->|Sigmoid Routing: 6 Active Experts| F[Output Text Tokens]
- Architecture: Sparse Mixture-of-Experts (MoE) Decoder-only Transformer.
- Routing System: Sigmoid-based router with auxiliary-loss-free load balancing. Each token activates 6 out of 256 routed experts, plus 2 shared experts active on every token.
- Attention Mechanism: Interleaved sliding-window and global attention layers at a 5:1 ratio, using 8 KV heads.
- Positional Embedding: Relative positional embedding (Shaw & Huang) for superior sequence extrapolation (up to 1M tokens) compared to standard RoPE.
- Feature Convolutions: Short convolutions applied after KV projections in attention layers, and on the MLP residual branch outputs.
- Optimization & Training: Pretrained on 45 Trillion tokens using Muon for large matrices and Adam for other parameters. Fine-tuned using large-scale asynchronous Reinforcement Learning (RL) spanning over 30M rollouts.
| File / Folder | Type | Description |
|---|---|---|
| README.md | File | Comprehensive documentation outlining the model details, local setups, playground/API use, evaluated tasks, tech stack, and roadmap. |
- Autonomous Web Developers: Building and iterating on complete, standalone frontend interfaces and mini-tools in a single shot.
- Calibrated Intelligence & Forecasting: Synthesizing complex global variables into probabilistic reports for finance, geopolitics, and technology planning.
- Multimodal Customer Assistance: Audio-native voice assistants and document-reading support agents that directly reason over visual files and recordings.
- Desktop Publishing & Editorial Generators: Synthesizing factual engineering or research documents into beautiful, custom-formatted newsletters, zines, and PDFs.
- Self-Finetuning Pipelines: Utilizing Tinker APIs to write objective functions, generate synthetic datasets, and fine-tune specialized downstream weights.
- Native Multimodal Outputs: Direct generation of synthesizable audio and image assets along with response text.
- Multi-Agent Coding Harnesses: Direct integration with continuous refinement loops (e.g., self-correcting unit tests during generation).
- Advanced Local Quantization (GGUF/EXL2): Optimizing NVFP4 formats for execution on standard 16GB/24GB consumer GPUs.
- Expanded Audio Window Lengths: Extending 16kHz audio input processing support from 20 minutes to several hours of continuous recordings.
- Localized RL for Calibration: Allowing users to feed their own resolved historical event datasets to tune the model's forecasting confidence intervals locally.
- Safety Mitigations: Inkling has been trained to reject weapons, violence, and malicious cyber requests (scoring 98.6% on StrongREJECT and 78.0% on FORTRESS Adversarial).
- Bias & Limitations: Like all LLMs, Inkling may hallucinate, show demographic biases from its training corpus, or show degraded performance on highly recursive multi-turn conversations.
- Usage Recommendations: Do not deploy Inkling in safety-critical, medical, or legal domains without additional domain-specific verification and human-in-the-loop oversight.
Inkling AI • Mixture of Experts • Multimodal LLM • Thinking Machines • vLLM • SGLang • Unsloth • Together AI • Fireworks AI • Local Model Deployment • Open Weights LLM • Agentic Coding • Epistemics • Calibrated Forecasting • Technical Escapements Zine
Developed by Thinking Machines. Distributed under the Apache 2.0 License.
