Why my model sucks at tools and terminal

Tool-Eval-Bench 

Here's what I've got on hand:

🔧 Tool-Call Benchmark
  Server: http://192.168.1.30:1234
  Querying http://192.168.1.30:1234/v1/models … found 5 models
Available models:
  1) qwen/qwen3.6-35b-a3b
  2) mellum2-12b-a2.5b-thinking
  3) qwen/qwen3.5-9b
  4) openai/gpt-oss-20b

Running against my qwen3-9B nsc-ace-saber model went... poorly.

Running against qwen3.6-35b-a3b was significantly nicer.

It scores each tool-calling scenario as pass=2pts / partial=1pt / fail=0pts, so a 138-point total implies a ~69-scenario suite. 138/138 means every scenario passed cleanly; 117/138 (≈85%) means roughly 20+ points were lost to partial credit or outright failures — usually concentrated in "tool selection" and parameter-formatting categories, which is the specific weak spot called out in public runs of this tool.

Two compounding factors explain the gap between those two specific builds:

Quantization depth.
FP8 is a light, near-lossless quantization of the original weights. Q4_K_S is a much more aggressive 4-bit GGUF quant. Tool calling is unusually sensitive to precision loss because it depends on exact JSON syntax, correct argument types, and crisp "which tool do I call" decisions — small probability shifts from heavier quantization show up disproportionately here, even when general chat quality looks fine.

Architecture.
The 27B model is dense (all parameters active every token). The 35B-A3B is a sparse MoE with only ~3B parameters active per token. Even though the MoE has more total parameters, routing decisions inside MoE layers are themselves precision-sensitive, so a 4-bit quant tends to degrade an MoE's tool-call reliability more than the same quant degrades a dense model — the sparsity and the quantization error compound rather than add.

So the 21-point gap is best read as: aggressive 4-bit quantization stacked on top of a sparser architecture, hitting the most precision-dependent part of the eval (structured tool calls) hardest, while leaving general fluency largely intact.
What happens when a model is subjected to: 

• unsloth
• uncensored
• abliterated
• heritic
• GGUF

These five terms operate on completely different axes of the model, and conflating them is easy to do since they often show up together in the same Hugging Face repo name. Here's what each one actually touches, and what that means specifically for tool-call accuracy.

* Unsloth — a fine-tuning/training optimization library, not a safety or quantization technique. It's how people do faster, cheaper LoRA/QLoRA fine-tunes (and it has its own GGUF export pipeline). On its own, Unsloth has no inherent tool-use impact — it's just the tool used to produce a fine-tune. The actual effect on tool-calling accuracy depends entirely on what someone fine-tuned with it. If the training data included good function-calling examples, tool use can improve; if it didn't (e.g., a narrow uncensoring dataset with no tool-call examples), the model can drift away from its original tool-formatting behavior through plain catastrophic forgetting. The "Unsloth" tag tells you almost nothing about tool-use quality by itself — you have to know what was trained.

* Uncensored — the broad umbrella term for "safety alignment removed," usually via continued fine-tuning/DPO on refusal-free data rather than surgical weight editing. This is the riskiest category for tool use precisely because it's a full training pass: if the uncensoring dataset is narrow (mostly chat-style "edgy" conversations) and doesn't include structured-output examples, the fine-tune can quietly erode the model's JSON-formatting and schema-adherence habits as a side effect, even though nobody touched tool-calling directly. The damage is dataset-dependent and uneven — some uncensored models retain tool use fine, others degrade noticeably.

* Heretic — not a model property but the specific automated abliteration tool (by p-e-w) that's become the most common way people abliterate models today. It distinguishes itself by explicitly optimizing to minimize KL-divergence from the original model while suppressing refusals, which in side-by-side comparisons gave it noticeably better capability preservation than older manual abliteration methods (FailSpy) on most metrics — but it still showed the largest average GSM8K degradation of the tools compared (worse than ErisForge/DECCP), confirming that precision-heavy structured tasks remain abliteration's weak point regardless of how good the tool is. So: a Heretic-abliterated model is likely to preserve tool-calling better than a hand-tuned or full-fine-tune "uncensored" version, but it isn't immune to the same structured-output fragility — just less of it.

* GGUF — purely a file/quantization format (llama.cpp ecosystem), completely orthogonal to all of the above. As we covered earlier, this is where the biggest, most predictable tool-use degradation actually comes from: going from FP8/Q8 down through Q5/Q4/Q3 progressively erodes the bit-precision needed for exact JSON syntax, correct argument typing, and tool-selection logic. A Q8_0 or Q6_K GGUF preserves tool-call accuracy close to the original; a Q4_K_S or lower starts showing the kind of partial/failed tool-call scores we discussed with the Qwen3.6 comparison.

The compounding problem: these axes stack. A model that's "unsloth-trained, uncensored, abliterated via Heretic, exported as Q4_K_S GGUF" has touched tool-call reliability four separate times — fine-tuning drift, possible alignment-removal side effects, ablation's structured-output fragility, and quantization precision loss — and each one independently nudges tool-calling accuracy down. None of them alone is usually catastrophic at moderate settings, but stacked together they explain why some heavily-modified community models that read fine in chat will still fumble tool calls noticeably more than the vanilla release.

* Abliterated — a more surgical technique: find the "refusal direction" in the residual stream (difference between activations on harmful vs. harmless prompts) and orthogonalize it out of specific weight matrices (attention output and MLP down-projections), without retraining. This causes less collateral damage than a full uncensoring fine-tune on average, but it's not free — independent comparisons across abliteration tools found that structured/precise tasks degrade disproportionately. GSM8K (math reasoning, which like tool-calling requires precise stepwise output) showed the largest drops of any benchmark tested, up to ~19 points in the worst tool/model combinations. Tool calling sits in that same fragile category — it's a structured-output task, not free-form chat — so abliteration carries a real but generally modest risk of malformed calls, wrong argument values, or schema slips, with severity depending heavily on how aggressively the ablation was tuned (how many layers, how strong the projection).