GLM 4.6V

Long story short, I only want to run local models. I hear many good things of 4.6, but is far too large to run locally. 4.6V-flash would fit on my GPU. How do the models compare in roleplaying?

https://preview.redd.it/dyfhb6nhwy5g1.jpg?width=10101&format=pjpg&auto=webp&s=d03177e251a72b04491b10634e66bdde1a9544c5

GLM-4.6V series model includes two versions: GLM-4.6V (106B), a foundation model designed for cloud and high-performance cluster scenarios, and GLM-4.6V-Flash (9B), a lightweight model optimized for local deployment and low-latency applications. GLM-4.6V scales its context window to 128k tokens in training, and achieves SoTA performance in visual understanding among models of similar parameter scales. Crucially, we integrate native Function Calling capabilities for the first time. This effectively bridges the gap between "visual perception" and "executable action" providing a unified technical foundation for multimodal agents in real-world business scenarios.

Beyond achieves SoTA performance across major multimodal benchmarks at comparable model scales. GLM-4.6V introduces several key features:

* **Native Multimodal Function Calling** Enables native vision-driven tool use. Images, screenshots, and document pages can be passed directly as tool inputs without text conversion, while visual outputs (charts, search images, rendered pages) are interpreted and integrated into the reasoning chain. This closes the loop from perception to understanding to execution.
* **Interleaved Image-Text Content Generation** Supports high-quality mixed media creation from complex multimodal inputs. GLM-4.6V takes a multimodal context—spanning documents, user inputs, and tool-retrieved images—and synthesizes coherent, interleaved image-text content tailored to the task. During generation it can actively call search and retrieval tools to gather and curate additional text and visuals, producing rich, visually grounded content.
* **Multimodal Document Understanding** GLM-4.6V can process up to 128K tokens of multi-document or long-document input, directly interpreting richly formatted pages as images. It understands text, layout, charts, tables, and figures jointly, enabling accurate comprehension of complex, image-heavy documents without requiring prior conversion to plain text.
* **Frontend Replication & Visual Editing** Reconstructs pixel-accurate HTML/CSS from UI screenshots and supports natural-language-driven edits. It detects layout, components, and styles visually, generates clean code, and applies iterative visual modifications through simple user instructions.

[https://huggingface.co/zai-org/GLM-4.6V](https://huggingface.co/zai-org/GLM-4.6V)

please notice that llama.cpp support for GLM 4.5V is still draft

[https://github.com/ggml-org/llama.cpp/pull/16600](https://github.com/ggml-org/llama.cpp/pull/16600)

Looks incredible for your own machine.

GLM-4.6V-Flash (9B), a lightweight model optimized for local deployment and low-latency applications. GLM-4.6V scales its context window to 128k tokens in training, and achieves SoTA performance in visual understanding among models of similar parameter scales. Crucially, we integrate native Function Calling capabilities for the first time. This effectively bridges the gap between "visual perception" and "executable action" providing a unified technical foundation for multimodal agents in real-world business scenarios.

https://huggingface.co/zai-org/GLM-4.6V-Flash

you need this

[https://www.reddit.com/r/LocalLLaMA/comments/1pnz1je/support\_for\_glm4v\_vision\_encoder\_has\_been\_merged/](https://www.reddit.com/r/LocalLLaMA/comments/1pnz1je/support_for_glm4v_vision_encoder_has_been_merged/)

Hey guys, we did a refresh of quants (quality of life updates) for GLM 4.5, 4.6, 4.6V-Flash and 4.7

llama.cpp and other inference engines like LM Studio now support more features including but not limited to:

1. Non ascii decoding for tools (affects non English languages) For eg before the default (ensure\_ascii=True) would cause "café" → "caf\\u00e9", whilst now ensure\_ascii=False would tokenize "café" → "café". I would re-download our quants if you use languages other than English.
2. Converts reasoning content parsing to original \[0\], \[-1\] from our changes of |first and |last. We used to change \[0\] to |first and \[-1\] to |last so we be compatible with LM Studio and llama-cli. With the upgrade of llama-cli to use llama-server, we can revert this. llama-server also didn't like |first, so we fixed it as well.
3. Many of you reported Chinese thinking with the GLM-4.6V-Flash GGUFs. After investigating, we confirmed the same behavior appears in all uploads regardless of uploader (e.g., LM Studio and bartowski). LM Studio’s Q8\_0, bartowski’s BF16, and our BF16 all produce Chinese “thinking,” so this is just the way Z . ai intended for the model and is not unique to our uploads. [See our investigation here.](https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF/discussions/4#694bdbac91a48021d8b210a1)

Also other changes:

1. **Added lot of tool calls in our calibration dataset - makes tool calling better, especially for smaller quants.**
2. **A bit more calibration data for GLM models., adding a teeny tiny bit more accuracy overall.**

**This does mean you need to re-download them to use the latest changes**

GGUFs which received Quality of Life updates:

* [https://huggingface.co/unsloth/GLM-4.6V-GGUF](https://huggingface.co/unsloth/GLM-4.6V-GGUF)
* [https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF](https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF)
* [https://huggingface.co/unsloth/GLM-4.7-GGUF](https://huggingface.co/unsloth/GLM-4.7-GGUF)
* [https://huggingface.co/unsloth/GLM-4.6-GGUF](https://huggingface.co/unsloth/GLM-4.6-GGUF)
* [https://huggingface.co/unsloth/GLM-4.5-GGUF](https://huggingface.co/unsloth/GLM-4.5-GGUF)
* [https://huggingface.co/unsloth/GLM-4.5-Air-GGUF](https://huggingface.co/unsloth/GLM-4.5-Air-GGUF)

Our guides are all in our docs or model cards: [https://unsloth.ai/docs/models/glm-4.7](https://unsloth.ai/docs/models/glm-4.7)

Thanks so much guys! :)