BranchOne: A System One Decision Model

In September, I wrote about testing and fine-tuning Laya. I liked the idea of giving a model information and a decision, then receiving a structured result. But the output format alone does not tell you whether the decision is useful.

I have now built BranchOne, my own System One decision model for choosing among candidate descriptions supplied with a request.

Give it a situation, a question, and choices, optionally with one image. It returns a probability per choice and the selected candidate’s ID.

Start with the Decision You Need

Imagine a support application receiving this message:

The package arrived, but the item inside is broken.

The application needs to decide what to do next. Here is a BranchOne request:

{
  "state": "The package arrived, but the item inside is broken.",
  "question": "What should happen next?",
  "candidates": [
    {
      "id": "replace",
      "description": "Arrange a replacement for the damaged item."
    },
    {
      "id": "track",
      "description": "Track a parcel that has not arrived."
    },
    {
      "id": "receipt",
      "description": "Send a receipt for a new purchase."
    }
  ]
}

Descriptions tell the model what each option means. IDs are labels for the application.

A different request can supply different questions and choices using the same model weights.

One Shared Model Processes Every Candidate

BranchOne uses Qwen3.5-0.8B-Base as its pretrained backbone. It adds LoRA adapters to the language model and a small scoring head.

Each candidate gets a separate sequence: the same situation and question, followed by its description and the text Decision:.

Qwen processes the sequence. BranchOne passes the final non-padding input representation to its scoring head, without generating an answer sentence.

The representation contains 1,024 values. The scoring head normalizes it, projects it to 256 values, applies a residual MLP, and produces one scalar score.

The backbone, adapters, and head are shared across all candidate paths.

Each path sees its own candidate description. The scores meet when the model computes the probability distribution.

One Output per Candidate

Three candidates produce three representations and three scores through the same head.

scores = []

for candidate in candidates:
    representation = encode(state, question, candidate.description)
    scores.append(shared_head(representation))

probabilities = softmax([score / temperature for score in scores])

This simplified pseudocode produces one score per candidate, without a permanent output neuron for every possible answer.

The implementation accepts 2–150 candidates. You still need to evaluate the number of choices your application actually uses.

More candidates mean more language-model computation. The released inference path processes the common language prefix again for each candidate.

What Training Changes

Training examples include the situation, question, candidate descriptions, and a zero-based gold_index identifying the correct candidate.

For the support example, a training label could be:

{"gold_index": 0}

Here, 0 identifies the replacement option. The label goes into the loss calculation, not the model’s input.

Cross-entropy loss increases when the correct candidate receives a low probability. Backpropagation encourages that candidate to score higher relative to the alternatives.

The original Qwen language and vision weights stay frozen. Training updates the rank-8 LoRA adapters in the language model and the scoring head.

Each question has its own probability distribution, even when candidate counts differ. Training shuffles candidate order and adjusts the correct index.

The current trainer uses one correct candidate per example. Several candidates being simultaneously correct would require different labels and a different training objective.

Images Use the Same Decision Interface

The frozen Qwen vision encoder converts the image into embeddings. BranchOne computes these once and reuses them across candidate paths.

The release allows one image, supplied as a local path or a PIL image, with a budget of up to 256 visual tokens. Each expanded candidate branch has an 8,192-token limit, including the evidence, question, formatting, candidate description, and visual tokens.

Overlong branches raise an error instead of being silently truncated.

Run It in Colab

The trained release on Hugging Face includes the adapters, head, processor, calibration, and inference code. It downloads the pinned Qwen backbone separately.

The supported setup uses Linux x86-64, Python 3.13, and an NVIDIA CUDA GPU. On a fresh Colab GPU runtime, start with this cell:

%%bash
python3 -m pip install uv==0.12.17
python3 -m uv tool run --from huggingface-hub==1.33.0 hf download ndemirdemir/BranchOne --local-dir BranchOne
python3 BranchOne/install.py --venv .venv

The installer creates the Python environment and builds the CUDA kernel. It needs CUDA 12.8 development tools, which it can install on Colab. Setup can take several minutes.

Save the JSON request from the first example as request.json in Colab’s working directory. Then run this in a new cell. Using .venv/bin/python matters: creating the environment does not switch the notebook’s own interpreter.

%%bash
.venv/bin/python - <<'PY'
import json
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "./BranchOne",
    trust_remote_code=True,
    device="cuda",
)

with open("request.json") as stream:
    request = json.load(stream)

result = model.predict(request)
print("Selected:", result["selected_id"])
for candidate, probability in zip(request["candidates"], result["probabilities"]):
    print(f"{probability:.1%} — {candidate['description']}")
PY

My A100 Run

I ran the ten-question timing example in the model card on an NVIDIA A100-SXM4-80GB. It loads the model once, runs a separate warm-up, and then answers ten short text questions with three choices each.

Prediction timing includes input preparation and inference, with CUDA synchronization. Loading, warm-up, and printing are excluded.

Here is what it selected. Percentages are the probability assigned to the selected choice, rounded as printed in the run.

The stock example is worth inspecting. The rule said to reorder below five items, and there were two items left. The correct action was to replenish stock. The model instead selected stopping sales at 35.6%, followed by waiting at 34.4% and replenishing at 30.1%. Those probabilities were close to an even split.

Nine selections matched the intended answers in this run. These are ten illustrative examples, not a representative accuracy benchmark. I would not assume the same results or latency for images, longer inputs, or another GPU.

Train It on Your Own Data

The GitHub repository includes data preparation, training, evaluation, and inference. Supply your own data to train new adapters and a head on pretrained Qwen, without using my BranchOne checkpoint.