GPT-5 mini vs DeepSeek V4 Flash

WTF, I could just use the expensive ai models of opencode go for planning, writing specs and then use opencode zen deepseek v4 flash max for implementation. I am loving this opencode, loving the freebies

I asked copilot to check correct answers in a quiz for me

https://preview.redd.it/5b49y505cz5g1.jpg?width=482&format=pjpg&auto=webp&s=7bfd451451d464fe2dd5ee4d31dc9a6137f39bde

Just wanted to share that I used u/LegacyRemaster slightly modified (Q4\_K\_M conversion support) DeepSeek V4 [CUDA repo](https://github.com/Fringe210/llama.cpp-deepseek-v4-flash-cuda) (based on u/antirez [work](https://github.com/antirez/llama.cpp-deepseek-v4-flash)) to convert and run Q4\_K\_M [DeepSeek V4 Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) on my Epyc workstation (Genoa 9374F, 12 x 96GB RAM, single RTX PRO 6000 Max-Q) and it worked right from the start:

(base) phm@epyc:~/projects/llama.cpp-deepseek-v4-flash-cuda/build-cuda$ ./bin/llama-cli -m ../models/DeepSeek-V4-Pro-Q4_K_M.gguf --no-repack -ub 128 --chat-template-file ../models/templates/deepseek-ai-DeepSeek-V3.2.jinja
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 97247 MiB):
Device 0: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition, compute capability 12.0, VMM: yes, VRAM: 97247 MiB

Loading model...

▄▄ ▄▄
██ ██
██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
██ ██
▀▀ ▀▀

build : b8936-44c7b01de
model : DeepSeek-V4-Pro-Q4_K_M.gguf
modalities : text

available commands:
/exit or Ctrl+C stop or exit
/regen regenerate the last response
/clear clear the chat history
/read <file> add a text file
/glob <pattern> add text files using globbing pattern

> who are you?

[Start thinking]
Okay, the user is asking "who are you?" This is a simple, introductory question. I need to introduce myself clearly and warmly. I should state my name, creator, and key features that are most relevant to a new user. I can mention that I'm free, my context window, knowledge cutoff, file support, and availability on web and app. I'll end with an open invitation for further questions to keep the conversation going.
[End thinking]

Hi there! I'm DeepSeek, an AI assistant created by the Chinese company DeepSeek (深度求索). I'm here to help you with questions, creative tasks, problem-solving, and pretty much anything you're curious about!

Here's a bit about me:
- **Free to use** - no charges for chatting with me
- **1M context window** - I can handle huge amounts of text at once (like entire book trilogies!)
- **Knowledge cutoff: May 2025** - I'm reasonably up-to-date
- **File upload support** - I can read text from images, PDFs, Word docs, Excel files, and more
- **Web search capability** - though you need to manually enable it via the search button
- **Available on web and mobile app** - with voice input support on the app

I'm a pure text-based model, so I can't "see" images directly, but I can read any text in uploaded files. I aim to be warm, helpful, and detailed in my responses.

What can I help you with today? 😊

[ Prompt: 12.2 t/s | Generation: 8.6 t/s ]

> /exit

Exiting...
common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
common_memory_breakdown_print: | - CUDA0 (RTX PRO 6000 Blackwell Max-Q Workstation Edition) | 97247 = 4022 + ( 92472 = 87766 + 84 + 4621) + 753 |
common_memory_breakdown_print: | - Host | 793994 = 793954 + 0 + 39 |
~llama_context: CUDA_Host compute buffer size of 39.1719 MiB, does not match expectation of 15.3535 MiB

The model file is 859GB.

Update: ran some lineage-bench prompts to see if the model has healthy brain and no problems so far.

Hi everyone,

We ran task benchmarks on the GPT-5 series models, and as per general consensus, they are likely not a break through in intelligence. But they are a good replacement of o3, o1 and gpt-4.1. And lower latency and the cost improvements are impressive! Likely really good models for chatgpt, even though users have to get used to them.

**For builders, perhaps one way to look at it:**

o3 and gpt-4.1 -> gpt-5

o1 -> gpt-5-mini

o1-mini -> gpt-5-nano

**But let's look at a tricky failure case to be aware of.**

Part of our context oriented task evals, we task the model to read a travel journal and count the number of visited cities:

Question: "How many cities does the author mention"

Expected: 19

GPT-5: 12

Models that consistently gets this right is gemini-2.5-flash, gemini-2.5-pro, claude-sonnet-4, claude-opus-4, claude-sonnet-3.7, claude-3.5-sonnet, gpt-oss-120b, grok-4.

To be a good model for building with, context attention is one of the primary criterias. What makes Anthropic models stand out is how well they have been utilising the context window even since sonnet-3.5. Gemini series and Grok seems to be putting attention to this as well.

You can read more about our task categories and eval methods here: [https://opper.ai/models](https://opper.ai/models)

For those building with it, anyone else seeing similar strengths/weaknesses?

Hi LocalLlama.

Here are the results from the March run of the GACL. A few observations from my side:

* **GPT-5.4** clearly leads among the major models at the moment.
* **Qwen3.5-27B** performed better than every other Qwen model except **397B**, trailing it by only **0.04 points**. In my opinion, it’s an outstanding model.
* **Kimi2.5** is currently the top **open-weight** model, ranking **#6 globally**, while **GLM-5** comes next at **#7 globally**.
* Significant difference between Opus and Sonnet, more than I expected.
* **GPT models dominate the Battleship game.** However, **Tic-Tac-Toe** didn’t work well as a benchmark since nearly all models performed similarly. I’m planning to replace it with another game next month. Suggestions are welcome.

For context, **GACL** is a league where models generate **agent code** to play **seven different games**. Each model produces **two agents**, and each agent competes against every other agent except its paired “friendly” agent from the same model. In other words, the models themselves don’t play the games but they generate the agents that do. Only the top-performing agent from each model is considered when creating the leaderboards.

All **game logs, scoreboards, and generated agent codes** are available on the league page.

[Github Link](https://github.com/summersonnn/Game-Agent-Coding-Benchmark)

[League Link](https://gameagentcodingleague.com/leaderboard.html)