dukehoff-decide
Java 21 · Apache 2.0 · on Maven Central

Typed, calibrated decisions for Java

Turn "which team handles this ticket?" into a Java type. You get a valid value, probabilities you can rely on, and a gate that sends uncertain cases to a human, from a small local language model or a classifier trained on your own data.

<dependency>
  <groupId>com.dukehoff.decide</groupId>
  <artifactId>dukehoff-decide-openai</artifactId>
  <version>0.3.0</version>
</dependency>
Triage.java
@Question("Triage an incoming customer support ticket")
record Triage(
    @Question("How urgent is this ticket?") Priority priority,
    @Question("Which team should handle it?") Team team,
    @Question("Does the customer threaten to cancel?")
    boolean churnRisk) {}

DecisionTask<Triage> triage = decider.task(Triage.class);

Decision<Triage> d = triage.decide(State.of(ticketText));
d.value();                    // always a valid Triage
d.distribution("priority");   // Map<Priority, Double>

switch (triage.requireConfidence(0.9).decide(state)) {
    case Outcome.Confident<Triage>(var dec) -> route(dec.value());
    case Outcome.Uncertain<Triage>(var dec, var why) -> toHuman(dec, why);
}

Most questions to an LLM have a few right answers

"Which team handles this ticket", "is this spam", "how relevant is this document": the answer is one of a handful of options. Generating free text for that is slow, hard to parse and impossible to trust. dukehoff-decide asks for exactly that choice and tells you how sure it is.

Always a valid value

Enums, booleans and scales become typed fields. No parsing, no hallucinated categories, no retries.

Probabilities you can rely on

Calibrated on a few hundred labelled cases, so 90% confidence means about 90% correct.

A gate for the unsure cases

Decide the confident cases automatically and send the rest to a human, with a threshold measured on your data.

How it works

One annotated type, one pluggable backend, one calibration step.

01 · Schema

Type → questions

Every field becomes "pick one of N options": yes/no, one option per enum constant, one per @Scale step.

02 · Backend

Scores per option

A ScoringBackend returns log-probabilities per option: a language model's next token, or a trained classifier.

03 · Calibration

Honest confidence

Calibration.fit fits temperature and per-option bias, reports accuracy, ECE, confusions and the threshold for a target accuracy.

04 · Binding

Typed value + gate

Calibrated probabilities become the record, per-field distributions, and a confident or uncertain outcome.

Start with a language model, switch when the labels are there

Calibration, gating and diagnostics are the same for every backend, so the code that uses the decisions never changes.

dukehoff-decide-openai

Small language model

Works from day one with only the question and option texts. Any OpenAI-compatible server with logprobs: llama.cpp, Ollama, vLLM, LM Studio.

dukehoff-decide-classifier

Word counts + logistic regression

LinearClassifier trains in seconds on a few thousand labelled cases, scores in well under a millisecond and needs no server and no dependencies.

dukehoff-decide-embedding

Sentence embeddings + logistic regression

EmbeddingClassifier on a frozen embedding model: best on longer texts written by people, and a little ahead of word counts on German text.

Combine them

Word counts and embeddings are unsure about different cases. A Cascade lets the cheap backend decide what it is sure about and passes only the rest on, in one batch per stage. An Ensemble averages its parts and is calibrated as a whole.

  • Each stage calibrated on its own, with its own threshold.
  • The embedding is only computed for the cases the linear model is unsure about.
  • A language model fits as the last stage, for the few cases both classifiers leave.
Cascade
Cascade<Triage> triage = Cascade.of(
        linearTask.requireConfidence(0.9),
        embeddingTask.requireConfidence(0.9));

Outcome<Triage> outcome = triage.decide(state);
List<Outcome<Triage>> all = triage.decideAll(inbox);

// or one backend, calibrated as a whole
ScoringBackend both = Ensemble.of(linear, embedding);

Measured on public data

Whole records decided automatically at a confidence threshold of 0.9, with the accuracy of those decisions. Means over five seeds, 300 test cases each, from the showcase.

85.9%of MASSIVE requests (English) decided automatically, 96.7% of them correct
+8.6–10.1points more decided by the cascade than by word counts alone
0model servers needed for word counts: one JAR, microseconds per decision
Share decided at threshold 0.9 and accuracy of those, per dataset and backend
datasetword countsbge-m3 embeddingsensemblecascade
CFPB complaints44.2% (96.9%)40.5% (97.3%)49.7% (97.3%)54.3% (96.3%)
MASSIVE, English77.1% (97.2%)74.2% (97.9%)80.9% (98.1%)85.9% (96.7%)
MASSIVE, German73.5% (97.4%)70.3% (97.1%)77.6% (97.4%)82.1% (96.1%)

The showcase also says what cannot be automated: on synthetic support tickets, queue and priority carry too little signal, and no backend decides a whole ticket. Measuring that first is part of the workflow.

Modules

The core has no runtime dependencies. Backends live in their own modules.

dukehoff-decide-coreAPI, schema binding, gating, calibration, diagnostics, cascade and ensemble.
dukehoff-decide-openaiBackend for OpenAI-compatible /v1/chat/completions with logprobs.
dukehoff-decide-classifierLinearClassifier: TF-IDF + logistic regression, no dependencies.
dukehoff-decide-embeddingEmbeddingClassifier and OpenAiEmbedder for /v1/embeddings.
dukehoff-decide-spring-boot-starterSpring Boot 4 auto-configuration, DecisionTask<T> beans, health and Micrometer metrics.

Calibrate once, decide with confidence

A few hundred labelled cases are enough to start.