support glm model - #163
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4ebf1d4024
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| int maxContextLength = ResolveConfiguredContextLength(); | ||
|
|
||
| if (NativeRequested(backend)) | ||
| { | ||
| // The whole model lives in the native executor: no managed weight | ||
| // tensors, no managed caches. | ||
| InitNativeExecutor(ggufPath, backend, tpDegree, maxContextLength); |
There was a problem hiding this comment.
When MAX_CONTEXT is unset, this passes the GGUF's advertised 1,048,576-token context directly to the native executor, whose slot_alloc eagerly allocates F16 MLA and indexer caches for every layer at that full size. For GLM-5.2 this is roughly 93 GiB per slot, so even the documented 3×97-GiB setup cannot fit the 218-GiB weights plus the default cache and the example launch fails unless users discover and set MAX_CONTEXT or offload weights. Use a practical implicit cap, reserving the advertised/full context for an explicit override or dynamically grow native caches.
Useful? React with 👍 / 👎.
| sb.Append("<think></think>"); | ||
| content = content.Trim(); | ||
| if (content.Length > 0) | ||
| sb.Append(content); |
There was a problem hiding this comment.
When a follow-up request contains an assistant ChatMessage with structured ToolCalls, this branch renders only Content and ignores m.ToolCalls. Tool-calling responses commonly have empty content, so the subsequent tool message becomes <|assistant|><think></think><|observation|>... with no preceding <tool_call>...</tool_call> for the model to associate with the observation, breaking multi-turn GLM tool use. Serialize the stored calls in GLM's XML format before rendering the tool responses.
Useful? React with 👍 / 👎.
Engine comparison — TensorSharp vs llama.cpp (PR smoke)No report artifact was produced — the benchmark failed before generating results (see the workflow logs). |
1 participant
No description provided.