Typed, calibrated decisions for Java
Turn "which team handles this ticket?" into a Java type. You get a valid value, probabilities you can rely on, and a gate that sends uncertain cases to a human, from a small local language model or a classifier trained on your own data.
<dependency> <groupId>com.dukehoff.decide</groupId> <artifactId>dukehoff-decide-openai</artifactId> <version>0.3.0</version> </dependency>
@Question("Triage an incoming customer support ticket")
record Triage(
@Question("How urgent is this ticket?") Priority priority,
@Question("Which team should handle it?") Team team,
@Question("Does the customer threaten to cancel?")
boolean churnRisk) {}
DecisionTask<Triage> triage = decider.task(Triage.class);
Decision<Triage> d = triage.decide(State.of(ticketText));
d.value(); // always a valid Triage
d.distribution("priority"); // Map<Priority, Double>
switch (triage.requireConfidence(0.9).decide(state)) {
case Outcome.Confident<Triage>(var dec) -> route(dec.value());
case Outcome.Uncertain<Triage>(var dec, var why) -> toHuman(dec, why);
}
Most questions to an LLM have a few right answers
"Which team handles this ticket", "is this spam", "how relevant is this document": the answer is one of a handful of options. Generating free text for that is slow, hard to parse and impossible to trust. dukehoff-decide asks for exactly that choice and tells you how sure it is.
Always a valid value
Enums, booleans and scales become typed fields. No parsing, no hallucinated categories, no retries.
Probabilities you can rely on
Calibrated on a few hundred labelled cases, so 90% confidence means about 90% correct.
A gate for the unsure cases
Decide the confident cases automatically and send the rest to a human, with a threshold measured on your data.
How it works
One annotated type, one pluggable backend, one calibration step.
Type → questions
Every field becomes "pick one of N options": yes/no, one option per enum constant, one per @Scale step.
Scores per option
A ScoringBackend returns log-probabilities per option: a language model's next token, or a trained classifier.
Honest confidence
Calibration.fit fits temperature and per-option bias, reports accuracy, ECE, confusions and the threshold for a target accuracy.
Typed value + gate
Calibrated probabilities become the record, per-field distributions, and a confident or uncertain outcome.
Start with a language model, switch when the labels are there
Calibration, gating and diagnostics are the same for every backend, so the code that uses the decisions never changes.
Small language model
Works from day one with only the question and option texts. Any OpenAI-compatible server with logprobs: llama.cpp, Ollama, vLLM, LM Studio.
Word counts + logistic regression
LinearClassifier trains in seconds on a few thousand labelled cases, scores in well under a
millisecond and needs no server and no dependencies.
Sentence embeddings + logistic regression
EmbeddingClassifier on a frozen embedding model: best on longer texts written by people,
and a little ahead of word counts on German text.
Combine them
Word counts and embeddings are unsure about different cases. A
Cascade lets the cheap backend decide what it is sure about and passes only the rest on,
in one batch per stage. An Ensemble averages its parts and is calibrated as a whole.
- Each stage calibrated on its own, with its own threshold.
- The embedding is only computed for the cases the linear model is unsure about.
- A language model fits as the last stage, for the few cases both classifiers leave.
Cascade<Triage> triage = Cascade.of(
linearTask.requireConfidence(0.9),
embeddingTask.requireConfidence(0.9));
Outcome<Triage> outcome = triage.decide(state);
List<Outcome<Triage>> all = triage.decideAll(inbox);
// or one backend, calibrated as a whole
ScoringBackend both = Ensemble.of(linear, embedding);
Measured on public data
Whole records decided automatically at a confidence threshold of 0.9, with the accuracy of those decisions. Means over five seeds, 300 test cases each, from the showcase.
| dataset | word counts | bge-m3 embeddings | ensemble | cascade |
|---|---|---|---|---|
| CFPB complaints | 44.2% (96.9%) | 40.5% (97.3%) | 49.7% (97.3%) | 54.3% (96.3%) |
| MASSIVE, English | 77.1% (97.2%) | 74.2% (97.9%) | 80.9% (98.1%) | 85.9% (96.7%) |
| MASSIVE, German | 73.5% (97.4%) | 70.3% (97.1%) | 77.6% (97.4%) | 82.1% (96.1%) |
The showcase also says what cannot be automated: on synthetic support tickets, queue and priority carry too little signal, and no backend decides a whole ticket. Measuring that first is part of the workflow.
Modules
The core has no runtime dependencies. Backends live in their own modules.
dukehoff-decide-coreAPI, schema binding, gating, calibration, diagnostics, cascade and ensemble.dukehoff-decide-openaiBackend for OpenAI-compatible /v1/chat/completions with logprobs.dukehoff-decide-classifierLinearClassifier: TF-IDF + logistic regression, no dependencies.dukehoff-decide-embeddingEmbeddingClassifier and OpenAiEmbedder for /v1/embeddings.dukehoff-decide-spring-boot-starterSpring Boot 4 auto-configuration, DecisionTask<T> beans, health and Micrometer metrics.Calibrate once, decide with confidence
A few hundred labelled cases are enough to start.