Surface Scout

The Surface Scout is the resident half of the model. It is the dense backbone of the target architecture — embeddings, self-attention, layer norms, routing gates, and shared experts — loaded entirely into Unified Memory, where the heavy pass executes it directly for every token. At startup its 16 GB mapping is prefaulted sequentially in the background, so first-prompt weights stream in at full NVMe bandwidth instead of random-faulting mid-inference.

---

What the Scout Is

The Scout is not a separate model. It is the dense residual stream of the target MoE architecture — the parts that always fire, regardless of routing decisions.

When shatter_moe.py splits the model, the Scout receives:

• model.embed_tokens.weight — the token embedding table
• model.layers.N.self_attn.* — all 4 attention projections (Q/K/V/O) for every layer
• model.layers.N.input_layernorm.weight — pre-attention RMS norms
• model.layers.N.post_attention_layernorm.weight — pre-FFN RMS norms
• model.layers.N.mlp.gate.weight — the MoE routing gate for every MoE layer
• model.layers.N.self_attn.{q,k}_norm.weight — Qwen3's per-head Q/K RMS norms (when present)
• model.layers.N.mlp.shared_expert.* — the shared expert FFN, when the architecture has one (Qwen2-MoE: .shared_expert, DeepSeek: .shared_experts; Qwen3-235B has none)
• model.norm.weight — the final model norm
• lm_head.weight — the unembedding matrix

The routed experts.N.{gate,up,down}_proj weights are the only tensors that go into the vault.

---

What the Scout Does

The Scout's weights are the heavy model's own dense path. Every token — prompt or generated — flows through them: the embedding lookup, all 94 attention blocks, the per-layer router gates, the final norm, and the LM head all read directly from the Scout's resident tensors.

What the Scout is not is a predictor. It runs no forward pass of its own — the Scout class is an artifact: weight accessors, the auto-detected model config, and the compute_top_k routing helper. Routing is computed exactly at every layer, prompt and generation alike — the real gate evaluated on the heavy hidden state, norm_topk_prob semantics per the Qwen3 specification.

Why There Is No Routing Predictor

Because the free oracle sets a bar no predictor clears. The instrumented coverage of the next token's true top-8 experts:

Oracle	Cost per token	Coverage of the true top-8
Retained ring (previous tokens' reads)	free	46.4%
Previous token's top-16 gate ranking	free (already computed)	63.7%
A dense forward pass evaluated as a predictor	~265 ms + prefetch bandwidth on its misses	51.5%

An oracle that charges 265 ms per consultation and is right barely more often than doing nothing cannot justify its seat. Next-token coverage therefore comes from the ring's LRU retention, optionally widened by speculative prefetch of the current token's lower gate ranks (--spec N, through the streamer's low-priority queue). And because execution follows the exact gate, the experts and mixing weights are always the ones Qwen3 was trained to use.

---

The Model Config

At startup, scout.cpp parses the .safetensors JSON header and auto-populates SmoeModelConfig:

struct SmoeModelConfig {
    uint32_t d_model;                   // hidden dimension
    uint32_t vocab_size;                // embedding table rows
    uint32_t ffn_dim;                   // expert intermediate dim (from vault descriptors)
    uint32_t shared_expert_ffn_dim {0}; // 0 = no shared expert (Qwen3-235B)
    uint32_t num_moe_layers;
    uint32_t max_experts_per_layer;     // routing gate rows
    bool     has_dense_layer_0 {true};  // DeepSeek = true, Qwen3 = false
    bool     has_qk_norm {false};       // Qwen3 per-head Q/K RMS norm before RoPE
    uint32_t num_heads {16};            // GQA query heads (64 for Qwen3-235B)
    uint32_t num_kv_heads {16};         // GQA key/value heads (4 for Qwen3-235B)
    uint32_t head_dim {128};
    float    rope_theta {10000.0f};     // 10000 = DeepSeek, 1000000 = Qwen3
    bool     norm_topk_prob {false};    // re-normalise top-k routing weights to sum 1.0
};

Detection is fully automatic. The Scout probes:

• model.embed_tokens.weight → vocab_size, d_model
• model.layers.0.mlp.gate_proj.weight → ffn_dim, sets has_dense_layer_0 = true
• If absent → has_dense_layer_0 = false, probes shared_expert.gate_proj.weight instead
• model.layers.0.mlp.gate.weight or model.layers.1.mlp.gate.weight → max_experts_per_layer

No configuration files. No CLI flags. The model tells S-MoE what it is.

---

KV-Cache

The Scout owns no KV-cache. The engine's single KV-cache is the heavy path's own ring — GQA-sized (num_kv_heads × head_dim = 4 × 128 = 512 for Qwen3-235B, an 8× saving over caching full d_model), pre-allocated at startup to ATTN_CTX = 4096 slots per layer. During decode it is written directly by the GPU (the attn_prep kernel appends K/V in the same command buffer that computes attention — see Metal Compute Kernels); during prefill it is written by the CPU. Both writers share the same zero-copy UMA buffer, registered with Metal once at boot.

---

Zero-Allocation Guarantee

The dense path that reads the Scout's weights allocates zero bytes during execution. All intermediate buffers (heavy_hidden, heavy_normed, the Q/K/V/attention scratch, the gate score buffers) are pre-allocated to maximum size at startup using posix_memalign with 16 KB alignment — which is also what makes every one of them wrappable as a zero-copy MTLBuffer.

Stack-only temporaries (__builtin_alloca) are used for small, bounded scratch arrays that vary by layer count.