edgeinfer · browser inference + retrieval

edgeinfer v2

Two-stage Matryoshka retrieval, speculative decoding, and client-side ONNX inference. All numbers below are synthetic/mock local benchmarks, not MTEB or WebGPU measurements.

95/95tests
25.1 mscoarse+re-rank p50 · 100k×768
0.725→1.000Recall@10 · d=64→768
68.4%mock speculative acceptance
MetricValueBoundary
Index build1.76 s100k synthetic chunks
Coarse scanp50 25.08 ms / p95 59.41 msINT8 d=64 + FP32 top-50 re-rank
Speculative decoding0.45 target calls/token; modeled 1.64×Mock latencies; no WebGPU run
ONNX input castingBigInt64Array when metadata says int64Covered by tests

Synthetic and mock benchmarks only. Not MTEB, not WebGPU tokens/sec.

Measured local evidence only. No deployment, customer, or trained-weight claim is made unless explicitly stated.