Sometimes, you don’t need an AI model to generate text.
You need it to route a support ticket, choose the right tool, or make a structured decision.
In my previous JEV vs. SemIf benchmark, I tested whether a small local model could compete with JEV.
This time, I tested Cloudflare’s purpose-built Clef models through llama.cpp.
Three local variants. Three tests. 1,370 decisions per model.
And the biggest surprise?
Clef-Flash beat both 27B Clef variants on two of the three tests.
Do you prefer watching instead? Check out the full breakdown and live benchmark video here:
How Clef Works
Clef takes a state plus typed questions and returns structured decisions with probabilities.
Cloudflare built the 27B model from Qwen3.8-27B with a custom decision head, while Clef-Flash uses a smaller Qwen3.5-9B backbone.
In llama.cpp, both run through:
POST /v1/systemoneThe API supports:
Choice -pick from named options
Score -score against ordered levels
Noul -estimate whether something is true
Benchmark Setup
GPU: RTX 5060 Ti 16GB
CPU: Intel i9-9900K
Runner: llama.cpp + CUDA
Models: Clef Q8_0, Clef Q4_K_M, Clef-Flash Q8_0
Hosted comparison: JEV 1.13
Total: 770 source records / 1,370 decisions per model
These were text-only tests.
Test 1 -Email Triage
300 support tickets, 900 total decisions.
Each ticket required:
Queue
Priority
Human-attention decision
Model | Accuracy | Avg. time / decision |
|---|---|---|
JEV 1.13 | 53.6% | 15 ms |
Clef Q8_0 | 50.8% | 394 ms |
Clef Q4_K_M | 50.7% | 384 ms |
Clef-Flash Q8_0 | 47.2% | 151 ms |
Winner: JEV
The two 27B Clef variants were almost identical.
Flash gave up some accuracy, but was much faster.
Test 2 -AI Model Router
150 prompts routed to:
Cheap model
Mid-tier model
Frontier model
Human approval
Model | Accuracy | Avg. time / decision |
|---|---|---|
JEV 1.13 | 97.3% | 32 ms |
Clef Q8_0 | 93.3% | 552 ms |
Clef Q4_K_M | 93.3% | 508 ms |
Clef-Flash Q8_0 | 94.0% | 134 ms |
Winner: JEV
But Clef-Flash was the strongest local model.
Test 3 -BANKING77
320 examples across 16 closely related banking intents.
Model | Accuracy | Avg. time / decision |
|---|---|---|
JEV 1.13 | 89.1% | 31 ms |
Clef Q8_0 | 90.9% | 800 ms |
Clef Q4_K_M | 90.0% | 762 ms |
Clef-Flash Q8_0 | 92.2% | 222 ms |
Winner: Clef-Flash
All three Clef variants beat JEV on this test.
The number of wrong answers made with at least 80% confidence was also interesting:
Model | Confidently wrong |
|---|---|
JEV 1.13 | 25 |
Clef Q8_0 | 2 |
Clef Q4_K_M | 1 |
Clef-Flash Q8_0 | 2 |
Run Clef Locally
These are the three variants I tested:
Start the server:
./build-cuda/bin/llama-server \
-m "/path/to/Clef-Q8_0.gguf" \
-ngl auto \
-c 16384 \
-fa on \
--host 127.0.0.1 \
--port 8091Then send a decision request:
curl http://127.0.0.1:8091/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"state": "The customer says their invoice was charged twice.",
"questions": {
"route": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {
"billing": "Charges, invoices, and payments.",
"technical": "Software errors and troubleshooting.",
"sales": "Purchasing and product enquiries."
}
}
}
}'The Verdict
Across my tests:
Email triage: JEV won
Model routing: JEV won
BANKING77: Clef-Flash won
Fastest local model: Clef-Flash
Best overall local model: Clef-Flash
The 27B Clef models were solid, but Clef-Flash is the one that stood out.
It’s much smaller, much faster on my hardware, and still delivered the strongest local accuracy on two of the three tests.
JEV still has the edge on some workloads, but local decision models are now competitive enough that I wouldn’t automatically assume the hosted option is better.
Want the Prompts and Datasets?
They’re all in my previous benchmark post.
For more deep dives into open-source AI, benchmarks, and local LLM architectures, make sure to check out my YouTube channel HERE.
Until next time,
Keep building.