edgeinfer · browser inference + retrieval
edgeinfer v2
Two-stage Matryoshka retrieval, speculative decoding, and client-side ONNX inference. All numbers below are synthetic/mock local benchmarks, not MTEB or WebGPU measurements.
95/95tests
25.1 mscoarse+re-rank p50 · 100k×768
0.725→1.000Recall@10 · d=64→768
68.4%mock speculative acceptance
| Metric | Value | Boundary |
|---|---|---|
| Index build | 1.76 s | 100k synthetic chunks |
| Coarse scan | p50 25.08 ms / p95 59.41 ms | INT8 d=64 + FP32 top-50 re-rank |
| Speculative decoding | 0.45 target calls/token; modeled 1.64× | Mock latencies; no WebGPU run |
| ONNX input casting | BigInt64Array when metadata says int64 | Covered by tests |
Synthetic and mock benchmarks only. Not MTEB, not WebGPU tokens/sec.
Measured local evidence only. No deployment, customer, or trained-weight claim is made unless explicitly stated.