@vick21 The fan is actually pretty quiet. The model has 125B total parameters with 6B active, 51B n-gram embedding, and 4B MTP, soI have to offload 50% of the experts. The GPU isn’t running at full speed because SSD streaming is the bottleneck. This would be an amazing model to run on macs with 128GB. :)
@chikim That's a really good news! I had to lower my Qwen3.8-27b to oQ4e eventually and that really helped while not sacrificing any quality. Will give the flash a try over the weekend.
@vick21 Hope it works for you. I’m curious to hear what you think of it. The benchmarks on the model card show that it’s roughly comparable to Opus-4.6-Max, but more importantly, the general vibe on the internet seems to check out as well. The complain is it just takes a lot longer and more tokens. Even just the fact that it's the first local model I could use with VOCR for computer use is very exciting! It's great mystery to me why Gemini-3.8-flash still fails clicking the right coordinates!
@chikim And the file is really large, wow!
@vick21 Yes, it's 125B parameters model. That's why I can only offload 55% resident experts in memory, and stream the rest from SSD.