Skip to content

Wasm gemm gemv forward - #6239

Open
anutosh491 wants to merge 2 commits into
OpenMathLib:developfrom
anutosh491:wasm-gemm-gemv-forward
Open

anutosh491 wants to merge 2 commits into
OpenMathLib:developfrom
anutosh491:wasm-gemm-gemv-forward

Conversation

@anutosh491

Copy link
Copy Markdown

While benchmarking batch-one PyTorch linear layers (I am working on pytorch to wasm now with openblas enabled), I noticed that vector-shaped GEMM calls on Wasm still use the GEMM path. OpenBLAS already has GEMM-to-GEMV forwarding, and the Wasm GEMV kernels from #5990 make this worth enabling.

This enables GEMM_GEMV_FORWARD for Wasm in Make and CMake. It reuses the existing dispatch and kernels without changing their arithmetic.

I've added scalar-reference regression cases for both M=1 and N=1, including real and complex types, transpose combinations and conjugated vector operands. These reuse the existing size and input-generation grids. The existing Wasm CI workflow now also triggers when the relevant build settings change.

For local validation, I built the complete S/D/C/Z BLAS libraries with both Make and CMake, for wasm32 and wasm64, using LLVM 23.1.2. Each configuration passed:

  • 6,844 checks in the upstream numerical suite.
  • A separate 12,480-case scalar-reference GEMM grid covering both matrix orders, transpose/conjugation modes, padded leading dimensions, zero dimensions and alpha/beta combinations.

The existing numerical suite can be run with JOBS=6 WASM_RELAXED_SIMD=0 ./test/wasm/run.sh.

Benchmarks below use Apple M5, Node 26.12.1, Emscripten 6.0.8, LLVM 23.1.2, portable SIMD128 and no threads. Both variants use the same kernels and inputs in one Wasm module; baseline GEMM entry points are compiled with forwarding disabled. Results are medians of nine alternating timing samples after warmup.

All rows are column-major. Dimensions are GEMM's M/N/K; times are microseconds per call.

Wasm Routine M × N × K TransA/TransB Forwarding off Forwarding on Speedup
32 SGEMM 4096 × 1 × 784 T/N 1776.490 263.951 6.73×
64 SGEMM 4096 × 1 × 784 T/N 1777.635 257.652 6.90×
32 SGEMM 1 × 4096 × 784 N/N 2121.802 279.727 7.59×
64 SGEMM 1 × 4096 × 784 N/N 3394.563 426.717 7.96×
32 DGEMM 4096 × 1 × 784 T/N 2183.125 509.565 4.28×
64 DGEMM 4096 × 1 × 784 T/N 2784.979 539.966 5.16×
32 SGEMM 64 × 512 × 1024 N/N 2250.834 2252.896 ≈1.00×
64 SGEMM 64 × 512 × 1024 N/N 2755.636 2754.386 ≈1.00×

I measured 192 cases per width. Gains depend on shape and layout: a tiny row-major ZGEMM case with conjugated operands was about 1.3% slower on wasm64 (0.448 → 0.454 µs). These are measurements from one host, rather than a claim that every call improves.

The CMake portion depends on #6238.

@anutosh491

Copy link
Copy Markdown
Author

Again ccing @jjerphan & @martin-frbg

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant