When AI only needs to choose: exploring Jev and local Laya

Does choosing a next step need a long answer?

Many moments in an AI workflow do not need a written answer. They need a choice: who should handle this request, what should happen next, or whether to stop and ask the user.

That is what caught my attention about Jev from TypeSafe AI: define the available options, then ask it to choose. LLMs can classify and return structured output too. My question is whether a different decision tool can make the whole workflow faster and cheaper while maintaining acceptable quality.

What part would Jev handle?

TypeSafe describes Jev as a System One model. Its documentation offers three decision primitives: Choice, Score and Noul. This article focuses on Choice: supply a state and a finite set of options, then receive a choice, option probabilities and a confidence value.

A fixed set of options makes the output easier for software to handle; it does not guarantee the right choice. Confidence here is derived from the output distribution, not a direct statement of how likely this decision is to be correct. Using it as an escalation or stopping threshold requires checks on our own data.

The official Jev interface is a cloud API, with data transfer and network latency. To put the selector itself on the device, my friend’s Laya project offers another path to investigate.

A friend’s Laya project makes the local path concrete

My friend yihsinhung’s laya-snake-dgx connects the laya-mlx Snake demo to PyTorch, using Laya weights on DGX Spark GB10. Its README contains the author’s validation results. I have read the documentation and code, but have not reproduced them.

Upstream Laya offers a Jev-compatible decision API with its own open weights. I would describe it as a local, Jev-compatible decision option, rather than a port of TypeSafe’s official Jev weights.

The demo’s policy module loads weights from a local directory or existing cache and calls the local model. Dependencies and weights need preparation first. I want to verify stable offline operation on my target device once setup is complete.

My first candidate: routing between local and cloud models

Imagine a local assistant receiving a task. A simple cleanup might stay on the device, harder reasoning might go to a cloud model, and an ambiguous request might need a question first. I want to compare local Laya and cloud Jev within the same evaluation framework for this selection step.

Local rules must come first: may this data leave the device, which models have the required capabilities, and what budget is allowed? Sending private data to the cloud to ask whether it may be sent there defeats that check. Only permitted, minimal task information should reach the selector.

I would separate the flow into local rule checks, a choice among allowed options, validation of that choice, task execution, and outcome logging. Data that cannot leave stays with local rules or a local model. Timeouts, missing information or low confidence follow a predefined clarification or stop path.

This is still an architectural hypothesis. The selector has a cost too. If a simple rule already routes the task well, another AI call could make it slower. Local Laya also uses hardware, memory, power and maintenance time; self-hosting does not make total cost zero.

Snake is a small enough experiment to start with

I would begin with Snake: represent the body, food and legal directions as structured state, then let the selector choose the next move. Game rules should first remove moves that immediately hit a wall or the snake itself.

My friend’s code offers a concrete example: a planner computes safe routes and food reachability and even marks a preferred direction before Laya scores the choices. When enabled, a guard also corrects unsafe actions. Game outcomes therefore need to separate the contributions of planning, model decisions and guarding.

The baselines should include simple rules and search methods such as BFS or A*, not just a large LLM. Using the same maps and random seeds, I would compare food collected, survival steps, invalid moves and actual time per move.

TypeSafe has shown a Doom example and states that its input was structured state rather than images. That does not establish general gameplay from screenshots. I would test a small game carefully before moving to more complex levels.

Stock selection and trading start as offline research

Another idea is to represent information available at a given market moment and classify it into options such as add to a watchlist, keep observing, or skip. Buy, hold and sell decisions could be a later research topic. There is no backtest or profitability finding here yet.

This is an easy place to confuse valid output with useful judgment. Evaluation needs chronological splits, only information available at decision time, transaction costs and slippage, plus simple rule baselines. I would begin offline, then consider simulated trading; this draft is not a connection to live order execution.

Four questions for the first round

For a minimal routing experiment, prepare a fixed task set and label what can stay local, what requires a cloud model, and what needs clarification. Labels should reflect what candidate models can actually complete, not intuition alone. Keep an untouched test set so thresholds are not tuned only to familiar examples.

First, are the choices useful? Examine errors by category, especially unauthorized data export and tasks assigned to an incapable local model. Second, does the user actually wait less? Measure end-to-end time including network, selection and execution, with both median and slower p95 latency.

Third, does it save money? Include the selector, execution models, retries and output checks, rather than the price of one selection alone. Fourth, when a choice is wrong or a service fails, does the workflow follow the intended clarification, stop or permitted fallback path?

I would compare rules, local Laya, LLM structured output and Jev on the same tasks. Agree on acceptable quality and errors before comparing speed and cost. There are no projected scores to present as results. My friend’s game tests can inform the experiment, but cannot establish routing or stock-selection performance.

What should we work out next?

The next step is to define one task precisely. What information may the router see? What must stay local? How much delay is unacceptable? Those answers determine the data and tests, and whether Jev is a suitable candidate.

The question I want to test is whether this small decision step has a better implementation. Once tested, I will add the conditions, failures and results here. Until then, this stays under To validate.