Qwen 3.6 27B can do that today, but setup properly and in a good quant, I run an autoround [0] with weights in int8 and attention heads in f16 on a single RTX 6000 Pro Blackwell Max-Q via vllm with mtp=2 and full context, --max-num-seqs 3, KV in f16, mamba f32.
>It would have 99% reliable tool calling
I managed to score 93/100 in tool-eval-bench [1]. For me this is very good already, at least in the pi coding harness I've never had an issue that wasn't auto-fixed in the next turn(s).
>the ability to go "this task is beyond my skills" and refer to a Big Boy Online Model in a gigantic datacenter somewhere
This is heavy on the harness engineering side I think, but also quite contrary to the nature of LLMs today. If you figure this out I'd love to know.
[0] https://huggingface.co/Minachist/Qwen3.6-27B-INT8-AutoRound/...
[1] https://github.com/SeraphimSerapis/tool-eval-bench"},"story_title":{"matchLevel":"none","matchedWords":[],"value":"Local Qwen isn't a worse Opus, it's a different tool"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://blog.alexellis.io/local-ai-is-not-opus/"}},"_tags":["comment","author_redrove","story_48580209"],"author":"redrove","children":[48594632,48594821],"comment_text":"> My dream would be a local model that can do, say, 80% of the day to day tasks I need; "how does X Handler connect to Y storage?", "commit that feature, but leave out the bits that relate to billing" etc.
Qwen 3.6 27B can do that today, but setup properly and in a good quant, I run an autoround [0] with weights in int8 and attention heads in f16 on a single RTX 6000 Pro Blackwell Max-Q via vllm with mtp=2 and full context, --max-num-seqs 3, KV in f16, mamba f32.
>It would have 99% reliable tool calling
I managed to score 93/100 in tool-eval-bench [1]. For me this is very good already, at least in the pi coding harness I've never had an issue that wasn't auto-fixed in the next turn(s).
>the ability to go "this task is beyond my skills" and refer to a Big Boy Online Model in a gigantic datacenter somewhere
This is heavy on the harness engineering side I think, but also quite contrary to the nature of LLMs today. If you figure this out I'd love to know.
[0] https://huggingface.co/Minachist/Qwen3.6-27B-INT8-AutoRound/...
[1] https://github.com/SeraphimSerapis/tool-eval-bench","created_at":"2026-06-18T07:50:03Z","created_at_i":1781769003,"objectID":"48582176","parent_id":48581623,"story_id":48580209,"story_title":"Local Qwen isn't a worse Opus, it's a different tool","story_url":"https://blog.alexellis.io/local-ai-is-not-opus/","updated_at":"2026-06-19T04:39:28Z"}],"hitsPerPage":20,"nbHits":2,"nbPages":1,"page":0,"params":"query=%22tool-eval-bench%22&advancedSyntax=true&analyticsTags=backend","processingTimeMS":5,"processingTimingsMS":{"_request":{"queue":477,"roundTrip":17},"fetch":{"total":1},"getIdx":{"load":{"gens":2,"total":3},"total":3},"total":5},"query":"\"tool-eval-bench\"","serverTimeMS":492}