But on my old M1 Max, with "Reasoning strength: low" in the system prompt, it's just kind of nice to use. It's not really slow (despite being a dense model it is faster to actual solutions than the Qwen 3.6 35B MoE), it writes well, etc.
So I use it as a developer-friend and researcher at the moment. Though it is actually the only model I've deployed code from! In general I am still writing my own (because I'm stuck in a bespoke framework on my main project) but it wrote me a plugin I needed for another app without any difficulty.
It's the first local LLM I've considered building hardware for.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T18:52:30.000Z","created_at_i":1792781550,"id":50213688,"options":[],"parent_id":50213408,"points":null,"story_id":50209907,"text":"> I still broadly just prefer working with Muse Glimmer
Welcome to our very exclusive club of now two people! Ever since it came out I\u2019ve been trying to find a better model, but just can\u2019t.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T18:30:47.000Z","created_at_i":1792780247,"id":50213408,"options":[],"parent_id":50213222,"points":null,"story_id":50209907,"text":"Yeah. I didn't find these any good either.
However, Gemma 4 26B is a reasonably good general chat inquiry model (it's also not too bad at commonplace code questions, if not as good for agentic code).
So this could be a good, fast model for that, in some situations; I may test it.
I still broadly just prefer working with Muse Glimmer, to be honest; we get along OK. I am trying out the Nemotron Bolt 74B/A9B MoE and I really do think it could be quite good, but it's still slow on Metal.","title":null,"type":"comment","url":null},{"author":"Foobar8568","children":[{"author":"simonw","children":[{"author":"dofm","children":[{"author":"electroglyph","children":[{"author":"dofm","children":[],"created_at":"2026-10-23T21:01:28.000Z","created_at_i":1792789288,"id":50215270,"options":[],"parent_id":50214939,"points":null,"story_id":50209907,"text":"I have found it to be OK on sort of general purpose overview code stuff. At the moment it's too slow in llama.cpp on Mac for me to bother testing further.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T20:33:38.000Z","created_at_i":1792787618,"id":50214939,"options":[],"parent_id":50213946,"points":null,"story_id":50209907,"text":"that nemotron model is such a dog, like all the other nemotrons","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T19:12:15.000Z","created_at_i":1792782735,"id":50213946,"options":[],"parent_id":50213733,"points":null,"story_id":50209907,"text":"It does. It's quite good with the low setting. I prefer Muse Glimmer for, I dunno, hard to explain comfort reasons.
I had to hack the template of Qwen 3.8-27B to get it to go into low reasoning mode. Not sure if that is fixed.
Have you tried Nemotron 74B-9B? (Slow on Mac with llama.cpp due to a missing tensor type)","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T18:55:43.000Z","created_at_i":1792781743,"id":50213733,"options":[],"parent_id":50213678,"points":null,"story_id":50209907,"text":"Qwen 3.8 27B defaults to "xhigh" (its maximum level) for thinking, which makes it diabolically slow. Setting it to "medium" or "low" speeds it up a lot.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T18:51:41.000Z","created_at_i":1792781501,"id":50213678,"options":[],"parent_id":50213222,"points":null,"story_id":50209907,"text":"But Qwen 3.8 27b is unbearable slow, you get 10minutes of "let me think" over any trivial stuff.","title":null,"type":"comment","url":null},{"author":"verdverm","children":[],"created_at":"2026-10-23T23:57:21.000Z","created_at_i":1792799841,"id":50216984,"options":[],"parent_id":50213222,"points":null,"story_id":50209907,"text":"This is my experience with Fireworks Ember-1 (reduced thinking Kimi K3)
It hallucinated and claimed task completion almost every session. Zero trust for this concept of fine tuning out excessive thinking, seems to do more harm than good. Eventually, as token vendors continue to commoditize, we will not care.","title":null,"type":"comment","url":null},{"author":"sheepscreek","children":[{"author":"joker99","children":[],"created_at":"2026-10-24T08:59:37.000Z","created_at_i":1792832377,"id":50219403,"options":[],"parent_id":50217634,"points":null,"story_id":50209907,"text":"Please share it. Fellow M1 (Ultra) user here","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T01:43:06.000Z","created_at_i":1792806186,"id":50217634,"options":[],"parent_id":50213222,"points":null,"story_id":50209907,"text":"For most folks in this thread running old M1 Max like me, look into MTPLX models (and the namesake inference engine).
I\u2019ve built something custom that\u2019s pretty darn fast but not ready for prime time yet. Might share it eventually.","title":null,"type":"comment","url":null},{"author":"BottleCapAI","children":[],"created_at":"2026-10-24T09:46:06.000Z","created_at_i":1792835166,"id":50219630,"options":[],"parent_id":50213222,"points":null,"story_id":50209907,"text":"There is a trade-off, but on Gemma we reduced thinking tokens by 39% on average for just a 0.15 percentage-point drop in average accuracy across 12 benchmarks. We'd encourage you to give this release a try.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T18:16:46.000Z","created_at_i":1792779406,"id":50213222,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"I have tried the Thinking cap and the Swift model for QWEN-3.8 27B and my experience was its not as smart and capable, the results on the same task were less impressive. For the amount of tokens saved it might very well be worth it but I wanted to run the xhigh precisely because I wanted the peak result. I haven't tested this but I suspect it ends up much the same trade off, in theory it does well in the benchmarks but in practice on your own tasks its not as capable.","title":null,"type":"comment","url":null},{"author":"syntaxing","children":[{"author":"julianlam","children":[],"created_at":"2026-10-25T03:21:10.000Z","created_at_i":1792898470,"id":50227965,"options":[],"parent_id":50213988,"points":null,"story_id":50209907,"text":"Base qwen 3.8 27B was so bad at over thinking that I couldn't work with it day to day. Swift solves that.
Perhaps I'll give bottlecap AI's fine tune a go.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T19:15:43.000Z","created_at_i":1792782943,"id":50213988,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"I tried using thinkingcap Qwen 27B for about two weeks. It definitely does dumber things than vanilla 27B but isn\u2019t too far off. Swift 1.5 is noticeably dumber that I stopped using it after one day. Oddly, I switched back to regular 27B because the thinkingcap one is noticeably more censored. Anything explicit gets instantly shot down by thinkingcap while vanilla 27B will chug along.","title":null,"type":"comment","url":null},{"author":"arjie","children":[{"author":"dofm","children":[],"created_at":"2026-10-23T22:54:43.000Z","created_at_i":1792796083,"id":50216509,"options":[],"parent_id":50214535,"points":null,"story_id":50209907,"text":"Indeed \u2014 the output token speed is showy but not that useful as a metric.
Muse Glimmer (dense 30B), in practice, can get to better answers more quickly than Qwen 3.6 35B-A3B.
It just appears to be slower word by word.","title":null,"type":"comment","url":null},{"author":"jacquesm","children":[],"created_at":"2026-10-24T00:10:50.000Z","created_at_i":1792800650,"id":50217060,"options":[],"parent_id":50214535,"points":null,"story_id":50209907,"text":"Indeed, which is why I always compared wallclock time at equal power consumption. I think that's the fairest way to compare models.","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T20:01:54.000Z","created_at_i":1792785714,"id":50214535,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"It feels clear to me that smaller models are moving parameter-level thinking into inference time. i.e. they use more tokens to perform computation because they don't perform it in the model itself. There's some conservation of difficulty for tasks and you either do the computation in the model or you do it in the looped iterations of the model.
This is why the massive tok/s you see on many small MoE never manifest as worthwhile outcomes. Sonnet 5.5 can be faster on tasks than DeepSeek 4 Flash Vision simply because it gets there faster.","title":null,"type":"comment","url":null},{"author":"serf","children":[],"created_at":"2026-10-23T20:09:23.000Z","created_at_i":1792786163,"id":50214616,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":""ThinkingCap PRO: want even better efficiency? The PRO version of this model uses up to 87.6% fewer thinking tokens, 62% on average. Faster answers and lower inference costs at scale. Contact us at enterprise@bottlecapai.com."
lol what's old is new again, software selling performance options like it's the beginning of the multi-core days again.","title":null,"type":"comment","url":null},{"author":"mehdi_zare","children":[],"created_at":"2026-10-23T21:28:48.000Z","created_at_i":1792790928,"id":50215624,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"In the case of PR reviews, cost is mostly thinking tokens. For me, what matters is a shorter reasoning at the same accuracy rather than a slightly higher score.","title":null,"type":"comment","url":null},{"author":"zedlasso","children":[],"created_at":"2026-10-23T21:36:10.000Z","created_at_i":1792791370,"id":50215708,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"Byteshape's Quant for Qwen has been surprisingly fast at good at output for my humanities stuff. It's small model punching way above its weight and it being so early in all this nutiness is quite amazing to see.
And hilarious cause what a time to be alive.","title":null,"type":"comment","url":null},{"author":"yieldcrv","children":[],"created_at":"2026-10-23T23:09:11.000Z","created_at_i":1792796951,"id":50216629,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"other improvements in the inference engine and harness net 1.5x+ improvements in tokens/sec, so if you're doing that on your own hardware you don't really need less tokens run, you just need any output done quickly","title":null,"type":"comment","url":null},{"author":"jacquesm","children":[{"author":"lp92","children":[{"author":"jacquesm","children":[{"author":"intoXbox","children":[],"created_at":"2026-10-24T07:50:31.000Z","created_at_i":1792828231,"id":50219096,"options":[],"parent_id":50217481,"points":null,"story_id":50209907,"text":"Better than is not part of the issue. If open weights models are released with a copy-left license as GPL, companies won\u2019t include them in their products and favour commercially permissive models","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T01:13:08.000Z","created_at_i":1792804388,"id":50217481,"options":[],"parent_id":50217463,"points":null,"story_id":50209907,"text":"No, it won't. Using open weight models is using them 'as is'. If you start modifying them and then gatekeep the results you are no better than parties that distribute modified open source binaries without the source.","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T01:09:31.000Z","created_at_i":1792804171,"id":50217463,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"That will just make people less likely to use open models for any commercial work.","title":null,"type":"comment","url":null},{"author":"kadoban","children":[{"author":"jacquesm","children":[{"author":"simcop2387","children":[],"created_at":"2026-10-24T03:00:44.000Z","created_at_i":1792810844,"id":50218014,"options":[],"parent_id":50217642,"points":null,"story_id":50209907,"text":"https://huggingface.co/bottlecapai/ThinkingCap-gemma-4-26B-A...
That claims to have been there for 12 hours when i view it","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T01:44:02.000Z","created_at_i":1792806242,"id":50217642,"options":[],"parent_id":50217504,"points":null,"story_id":50209907,"text":"That's the Google link, here is Bottlecapai on HF:
https://huggingface.co/bottlecapai
I don't see Gemma there.","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T01:17:23.000Z","created_at_i":1792804643,"id":50217504,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"Hm? I see a model download link to hf.
These people building on or tweaking open models is _great_, it should be encouraged. There's a lot of room to spend some effort improving models in various ways that the original creators didn't have time for.","title":null,"type":"comment","url":null},{"author":"deadbabe","children":[],"created_at":"2026-10-24T01:23:45.000Z","created_at_i":1792805025,"id":50217536,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"Get used to it. In the AI era, gatekeeping is how you build value.","title":null,"type":"comment","url":null},{"author":"intoXbox","children":[],"created_at":"2026-10-24T07:52:48.000Z","created_at_i":1792828368,"id":50219106,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"Yeah agreed, I hope the b2b marketplace will automatically make these products redundant if they are mainly targeted at engineers. Unless they offer value somehow else.
I don\u2019t think a license change would have a net positive effect, they will just add their tiny wrapper around another permissive model","title":null,"type":"comment","url":null},{"author":"BottleCapAI","children":[],"created_at":"2026-10-24T09:38:31.000Z","created_at_i":1792834711,"id":50219585,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"Our ThinkingCap weights are publicly available on Hugging Face: BF16 and GGUF. Free to download and run locally.\nhttps://huggingface.co/bottlecapai/ThinkingCap-gemma-4-26B-A...\nhttps://huggingface.co/bottlecapai/ThinkingCap-gemma-4-26B-A...","title":null,"type":"comment","url":null},{"author":"suprjami","children":[],"created_at":"2026-10-24T09:53:13.000Z","created_at_i":1792835593,"id":50219665,"options":[],"parent_id":50217052,"points":null,"story_id":50209907,"text":"The work is released under the Apache 2.0 licence which is permissive.
If this is your concern, complain about the licence, not about people acting within rights granted by the licence.
Copyleft is the only good sort of licence. (oh boy am I gonna get downvoted for that)","title":null,"type":"comment","url":null}],"created_at":"2026-10-24T00:09:19.000Z","created_at_i":1792800559,"id":50217052,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"I really don't like these companies taking open weight models and then gatekeeping the results of their minor contributions.
I hope that the producers of the actual open models will modify their licenses to ensure that this practice becomes a thing of the past.","title":null,"type":"comment","url":null},{"author":"peter_d_sherman","children":[],"created_at":"2026-10-25T02:58:44.000Z","created_at_i":1792897124,"id":50227853,"options":[],"parent_id":50209907,"points":null,"story_id":50209907,"text":"In the training phase, penalizing models not just for wrong answers, but
also for excessive length in the (CoT) thinking trace
seems to be the recipe, or at least part of the training recipe, for training a model to "think" less (and thus output less CoT tokens and thus operate faster) at approximately the same accuracy as the same model, previously trained without penalizing excessive length wrong answers in the thinking trace...
Anyway, well done BottleCap AI!","title":null,"type":"comment","url":null}],"created_at":"2026-10-23T14:43:02.000Z","created_at_i":1792766582,"id":50209907,"options":[],"parent_id":null,"points":125,"story_id":50209907,"text":null,"title":"Gemma-4-26B-A4B with 39-69% less thinking at same accuracy","type":"story","url":"https://bottlecapai.com/post/thinkingcap-gemma-4-26b/"}