Research note | PLI Labs | June 2026

The core result

A four-card Intel Arc Pro B70 workstation, with 32 GB per card and 128 GB aggregate VRAM, served a 122B-class mixture-of-experts GGUF lane through llama.cpp SYCL. The important public claim is not that every B70 stack is solved. The claim is that a specific layer-split profile was measured against real legal prompt tiers and survived.

The promoted profile

The public successful launch profile uses full GPU offload where supported, layer split across the B70 cards, 12K context for the always-on legal readiness lane, large prompt and micro batches, Flash Attention auto mode, FP16 KV tensors, continuous batching, KV offload, and memory mapping disabled.

llama-server \
  --model <local-gguf-path> \
  -ngl 99 \
  -sm layer \
  -c 12288 \
  -b 16384 \
  -ub 16384 \
  -np 1 \
  -cb \
  -ctk f16 \
  -ctv f16 \
  -fa auto \
  --kv-offload \
  --defrag-thold 0.1 \
  --no-mmap

Measured legal throughput

Tier Prompt tokens Output tokens Prompt tok/s Decode tok/s Client tok/s
L0 legal canary9,2741,024609.7935.1723.06
L0 service window9,2741,024613.6435.2423.15
L1 medium legal context17,6462,048584.4434.7922.97
L2 long legal context35,0342,048482.6030.8014.71

What the result means

The B70 lane is a capacity and privacy win. It proves that a four-card non-CUDA system can carry a large local legal AI workload when the runtime and model format are chosen correctly.

It is not a tensor-parallel speed claim. The proven path uses llama.cpp layer split, which avoids collective-communication failure modes but does not make all four cards compute the same layer at the same time.

Guardrails

  • Do not treat model load as proof of useful inference.
  • Do not assume XPU tensor parallelism is production-safe until the exact model and topology pass legal-context gates.
  • Do not treat smaller OpenVINO results as quality-equivalent to a 122B-class legal drafting lane.
  • Do not publish local paths, endpoints, hostnames, failure runbooks, or private service details.

Source links