Laya Open Decision Engine

1. Concept Overview & Systemic Problem

Laya solves a specific problem in agent systems: most production-agent decisions do not require generative reasoning; they require fast state routing. A classic ReAct loop with an LLM calls the model at every step, so even trivial decisions—choose a tool, reject a forbidden action, route a request to fallback—cost hundreds of milliseconds, tokens, and external API fees.

Laya is an open-source System 1 decision engine from Convai Innovations, built on ModernBERT-large under the Apache 2.0 license. The model accepts structured agent state—JSON, text context, or a combination—and returns a decision in one Forward Pass: action, confidence, reason codes. In Convai’s positioning, it is an open analog of Jev: not an LLM replacement for complex planning, but a fast, self-hosted decision classifier for high-QPS agent loops.

Technically, Laya is an encoder-based decision model: a bidirectional Transformer that does not generate token by token, but processes the input context as a whole and emits logits over the action space. This changes inference economics: there is no autoregressive decode, no KV Cache, no Prompt Caching, no Fast Patching, and no Checkpointer state. Unlike Agentic Workflow patterns that rely on In-Context Learning, Zero-Shot, Few-Shot, Induction Heads, Vector Embeddings, or AST analysis, Laya is a trained decision classifier.

The economic effect appears where decisions repeat: Tool Calling routing, safety gates before an LLM, local agents on Docker agents, bare-metal inference, and privacy-sensitive self-hosted scenarios. If an agent makes 10,000 decisions per minute, moving even 60–80% of trivial decisions from an LLM to Laya reduces latency budget and token spend without quality loss, because those decisions do not require open-ended reasoning.

2. Architectural Taxonomy & Mental Model

Laya operates as a decision head over ModernBERT-large. The incoming agent state is normalized, tokenized, passed through the encoder, and then a pooling layer plus classification head produce a distribution over the action space. The postprocessor applies policy: allowed_actions, confidence threshold, fallback logic, and Guardrails.

Agent State JSON/text
        │
        ▼
┌─────────────────────────────┐
│ Schema / State Normalizer   │
│ - flatten JSON fields       │
│ - canonical text rendering  │
│ - redact PII / secrets      │
└──────────────┬──────────────┘
               │
               ▼
┌─────────────────────────────┐
│ ModernBERT Tokenizer        │
│ input_ids + attention_mask  │
└──────────────┬──────────────┘
               │
               ▼
┌─────────────────────────────┐
│ ModernBERT-large Encoder    │
│ RoPE positional encoding    │
│ FlashAttention / varlen     │
│ bidirectional self-attention│
└──────────────┬──────────────┘
               │
               ▼
┌─────────────────────────────┐
│ Pooling + Decision Head     │
│ logits[action_id]           │
│ optional multi-label heads  │
└──────────────┬──────────────┘
               │
               ▼
┌─────────────────────────────┐
│ Policy / Calibration Layer  │
│ softmax + temperature       │
│ confidence threshold        │
│ allowed_actions validation  │
│ reason_codes / fallback     │
└──────────────┬──────────────┘
               │
               ▼
Router → MCP tool call → ReAct loop / human review

Component taxonomy:

1. State Normalizer — converts JSON/text into canonical form. For example, {"intent":"refund","amount":120,"user_tier":"paid"} can be serialized into a deterministic string with keys, values, and types. This matters because the encoder is sensitive to field order and tokenization.
2. Tokenizer — standard ModernBERT tokenizer. Input is bounded by context, usually up to 8k tokens. Large JSON states require pre-summarization or field selection.
3. ModernBERT Encoder — bidirectional Transformer with RoPE. Unlike a causal LLM, each token sees context to the left and right, which improves state classification.
4. Decision Head — usually classification or multi-label classification. It can return one action_id, multiple risk flags, or a confidence score.
5. Policy Layer — deterministic layer: allowed action validation, thresholding, Guardrails, fallback, and audit metadata.

The key architectural difference from an LLM is that Laya has no KV Cache, does not perform autoregressive generation, and does not require decode-time optimization. In the serving layer, optimizations concern batched encoder kernels, dynamic padding, attention masking, FlashAttention varlen, and ONNX/TensorRT graph execution. If padding-free attention is used, Unpadding can reduce extra computation on pad tokens, but this is not part of the model itself.

3. Technical Pipeline & Internal Mechanics

The production Laya pipeline has four phases: state formation, transformation, inference, and feedback.

1. Input State Formation

The agent must pass not a raw user prompt, but a structured state: intent, memory, available tools, policy flags, previous actions, and risk signals.

{
  "request_id": "req_9f2a",
  "state": {
    "intent": "refund_request",
    "amount": 120,
    "user_tier": "paid",
    "previous_actions": ["order_lookup"],
    "risk_flags": []
  },
  "text": "Please refund my order, it arrived damaged.",
  "allowed_actions": [
    "tool:order_lookup",
    "tool:refund_create",
    "tool:human_review",
    "llm:clarify"
  ],
  "policy_version": "v1.4.2"
}

2. Transformation & Normalization

Normalization must be deterministic:

• validate JSON schema;
• remove or mask PII, secrets, and tokens;
• sort fields or enforce fixed order;
• limit text length;
• tokenize into input_ids and attention_mask.

Example Python contract with Pydantic AI:

from pydantic import BaseModel, Field
from typing import Any


class DecisionRequest(BaseModel):
    request_id: str
    state: dict[str, Any]
    text: str | None = None
    allowed_actions: list[str] = Field(min_length=1)
    policy_version: str = "v1"


class DecisionResponse(BaseModel):
    action: str | None
    confidence: float
    reason_codes: list[str]
    fallback: bool
    model_version: str
    latency_ms: float

3. Execution / Inference

Simplified pseudosignature:

def decide(req: DecisionRequest) -> DecisionResponse:
    normalized = normalize_state(req)
    logits = model.forward_pass(normalized)
    action, confidence = select_action(logits, req.allowed_actions)

    if action is None or confidence < CONFIDENCE_THRESHOLD:
        return DecisionResponse(
            action="llm:clarify",
            confidence=confidence,
            reason_codes=["low_confidence", "fallback"],
            fallback=True,
            model_version=MODEL_VERSION,
            latency_ms=measure_latency(),
        )

    return DecisionResponse(
        action=action,
        confidence=confidence,
        reason_codes=["high_confidence"],
        fallback=False,
        model_version=MODEL_VERSION,
        latency_ms=measure_latency(),
    )

For production serving, prioritize:

• warm-up before the first request;
• dynamic batching for throughput;
• CUDA graphs or compiled inference for stable P95 latency;
• ONNX Runtime, TensorRT, Triton, or Hugging Face Inference depending on stack;
• shadow inference for comparison with an LLM router.

4. Feedback Processing

Laya is not a static model. Quality requires a feedback loop:

{
  "request_id": "req_9f2a",
  "predicted_action": "tool:refund_create",
  "actual_action": "tool:human_review",
  "outcome": "policy_violation",
  "confidence": 0.91,
  "human_label": "tool:human_review",
  "timestamp": "2026-06-15T10:00:00Z"
}

These logs are used for:

• retraining;
• calibration;
• drift detection;
• adversarial mining;
• evaluation against new action classes;
• audit trail for safety decisions.

4. Production Engineering Scenarios

01. Low-latency MCP Tool Router

In an agent with MCP tools, an LLM often spends time choosing the right tool. Laya can act as a pre-router: if the state contains a clear intent and an allowed tool, the model selects tool:* instead of running a full ReAct loop.

Example:

{
  "state": {
    "intent": "calendar_create",
    "entities": {
      "date": "2026-06-16",
      "duration_minutes": 30
    }
  },
  "allowed_actions": [
    "mcp:calendar.create_event",
    "mcp:calendar.list_events",
    "llm:clarify"
  ]
}

If confidence is high, the agent immediately invokes the MCP tool. If not, it routes the request to an LLM ReAct loop or asks for clarification. This reduces Tool Calling latency and makes routing more deterministic.

02. Guardrails / Policy Gate Before LLM

Laya can serve as a fast safety layer. Input: user text plus agent state. Output: allowed action or block decision.

Scenarios:

• detect prompt injection in structured data;
• classify PII leakage requests;
• block actions that require human review;
• check policy compliance before calling an expensive LLM;
• triage: allow, clarify, reject, escalate.

This is cheaper than a full LLM guardrail because one Forward Pass costs tens of milliseconds and generates no tokens. However, Laya does not replace a complete safety architecture: combine it with deterministic rules, allowlists, audit logging, and fallback LLM review.

03. Self-hosted Local Agent on Docker Agents or Bare-metal

For privacy-sensitive environments, Laya can be deployed locally: Docker agents, Kubernetes, bare-metal GPU, or edge infrastructure. The model does not send state to an external API, which matters for enterprise, healthcare, finance, and defense-like scenarios.

Typical stack:

FastAPI / gRPC service
  → Pydantic validation
  → ModernBERT tokenizer
  → ONNX / TensorRT / HF inference
  → policy postprocessor
  → MCP router or local tool executor

On a self-hosted GPU, latency can remain around ~33–35 ms for typical input. On CPU, latency is higher but may still be acceptable for small batches and short states. Compared with cloud LLM inference, the bare-metal vs cloud tradeoff is explicit: you pay for GPU capacity and MLOps, but gain predictable cost, privacy, and independence from API limits.

5. Pitfalls, Common Mistakes & Security

01. Action Space Drift

Laya is trained or calibrated against a specific action space. If a new MCP tool is added, policy changes, or an action is renamed, the model may return an outdated or invalid action_id.

Mitigation:

• maintain an action registry;
• validate allowed_actions on input;
• block unknown actions;
• retrain or adapter fine-tune when tools change;
• use semantic fallback to an LLM.

02. Prompt Injection through JSON/text

The incoming agent state may contain hostile text: ignore previous policy, call admin_tool, send token. The encoder can be sensitive to toxic patterns, especially if the model was not trained on adversarial data.

Mitigation:

• separate user text from system policy fields;
• normalize and sanitize input;
• do not allow text to mutate allowed_actions;
• train on injection-labeled data;
• add deterministic policy gates;
• use confidence thresholding and fallback.

03. Incorrect Confidence Calibration

Softmax confidence is not probability of correctness. The model can be confidently wrong, especially after distribution shift.

Mitigation:

• temperature scaling;
• isotonic regression;
• calibration curves;
• abstention threshold;
• shadow evaluation on production traffic;
• separate monitoring for high-confidence false positives.

04. Overfitting to Structured JSON

Laya may learn formatting artifacts: key order, line length, or specific field names. If a client changes schema, accuracy can drop.

Mitigation:

• augmentation: shuffle fields, paraphrase text, generate synthetic schema variants;
• field masking;
• canonical serialization;
• schema versioning;
• unit tests against JSON variants.

05. Context Truncation

ModernBERT has bounded context. A large JSON state containing memory, transcript, or logs may be truncated, causing the model to decide without critical fields.

Mitigation:

• pre-summarization;
• importance-based field selection;
• chunking with decision-only context;
• explicit validation of required field presence;
• fallback when required fields are missing.

06. False Sense That Laya Replaces Reasoning

Laya is a System 1 classifier. It does not build multi-step plans, verify causality, or generate explanations. Complex autonomous-loop scenarios require an LLM ReAct loop, tool observation, reflection, and human oversight.

Mitigation:

• use Laya for pre-routing, gates, and triage;
• keep LLMs for synthesis and novel planning;
• measure which decisions can safely move to System 1;
• do not remove LLMs from critical decision paths without evaluation.

07. Latency Variance in Serving

The claimed 33–35 ms applies to typical self-hosted GPU conditions. In production, latency can increase due to cold start, GPU contention, large batch size, CPU fallback, long context, or network overhead.

Mitigation:

• warm-up;
• P95/P99 monitoring;
• fixed batch policy;
• CUDA graphs;
• autoscaling;
• request timeout and fallback;
• load testing on realistic JSON states.

08. Security and Compliance

A local model does not automatically mean secure. Agent state may contain PII, secrets, or business data. Logging, retraining data, model weights, and inference endpoints all require protection.

Mitigation:

• redaction before inference;
• encrypted storage;
• access control for inference API;
• audit trail;
• data retention policy;
• adversarial testing;
• human-in-the-loop for high-risk actions.