Browser-Based Search: Benefits and Limits

The Waiting You Never Noticed

Search inside a website usually looks instant. It is not. The standard setup is a round trip:

1. The visitor types a query into a search box.
2. The browser sends the query string to a serverless function.
3. The function calls an embedding endpoint over HTTPS to turn the words into a vector.
4. The resulting vector goes to a hosted vector database (Pinecone, Qdrant, or Supabase pgvector).
5. The database finds the nearest neighbours and returns document IDs.
6. The server formats the results and sends the JSON back.

Every one of those steps is a round trip across the public internet, and they add up. The HTTP Archive’s Web Almanac 2024 report treats 800 milliseconds as the threshold for a good Time to First Byte and found that only 42% of mobile pages were inside it — and that figure covers the first byte of a page, not four separate calls with a model and a database in the middle.

│                                                                             │
│                         THE REMOTE SEARCH PIPELINE                          │
├─────────────────────────────────────────────────────────────────────────────┤
│  [User Device] --(HTTPS)--> [Edge Gateway] --(HTTPS)--> [Embedding API]     │
│                                                                 | (a hop)   │
│                                                                 v           │
│  [User Device] <--(JSON)-- [Edge Gateway] <--(TCP)--- [Hosted Vector DB]    │
│      (Four network round trips | Recurring cost: a monthly bill)            │
│                                                                             │
│  THE AURA LOGIC IN-BROWSER PIPELINE                                         │
│  [User Device: Browser Memory]                                              │
│  |- Static binary index (pre-computed Wasm Float32Array: a few hundred KB)  │
│  |- Local Wasm vector engine (SIMD 128-bit cosine dot product)              │
│  '- Execution time: our own target (no network calls | no cloud bill)       │
└─────────────────────────────────────────────────────────────────────────────┘

If you have ever run a small online store, a members’ area or a documentation site, this is the machinery behind the search box. It works. It also adds up to a wait on every keystroke, a hosting bill every month, and a per-query charge for every search somebody types.

There is another way, and it starts at build time instead of at request time.

---

1. The Maths: Compressing the Embedding Space

The hard part of running search in the browser is payload size.

OpenAI’s embeddings guide states that text-embedding-3-small returns a vector of 1,536 dimensions by default. A vector that size, stored as 32-bit floating-point numbers, requires:

Memory per vector = 1536 dimensions × 4 bytes = 6,144 bytes ≈ 6 KB

For a documentation library of 5,000 articles, the raw vector matrix would need about 30 MB of uncompressed memory. That is not an acceptable thing to ask a visitor on a phone to download.

So the index is compressed in three stages, all at build time.

Stage 1: Dimensionality Reduction

Instead of the 1,536-dimensional default, use domain-optimised embedding models like all-MiniLM-L6-v2 or bge-small-en-v1.5. The model cards for both list an output of 384 dimensions, a quarter of the width:

Reduced size = 384 dimensions × 4 bytes = 1,536 bytes ≈ 1.5 KB per vector

Stage 2: Scalar Quantization (FP32 to INT8)

Convert the 32-bit floating-point coordinates into signed 8-bit integers (int8), quantising continuous values along uniform thresholds:

Quantized_Coord = round(((Value - Min_Val) / (Max_Val - Min_Val)) × 255) - 128

Because an 8-bit integer takes a quarter of the space of a 32-bit float, that cuts memory consumption by 75%: each 384-dimensional vector goes from 1,536 bytes to 384 bytes.

Stage 3: Gzip / Brotli Compression

High-dimensional quantised matrices cluster densely, so they compress very well under Brotli level 11.

In the model below, a knowledge base of 2,000 technical articles compiles into a static binary artifact of roughly 300 KB, smaller than a single unoptimised hero image. The figures in that box are an illustrative model of what the three stages do to an index of that size; they are not a measured deployment.

┌─────────────────────────────────────────────────────────────────────────────┐
│                         WHAT THE COMPRESSION SAVES                          │
├─────────────────────────────────────────────────────────────────────────────┤
│  OPENAI FP32 (1536 dims)      12.28 MB       9.80 MB  (unusable)            │
│  MINILM FP32 (384 dims)        3.07 MB       2.45 MB  (heavy)               │
│  MINILM INT8 (384 dims)        768 KB        480 KB   (acceptable)          │
│  MINILM INT8 + BROTLI 11       768 KB        312 KB   (production)          │
└─────────────────────────────────────────────────────────────────────────────┘

---

2. The Browser Engine: WebAssembly and SIMD

Once the compressed index is fetched (and cached indefinitely via HTTP Cache-Control: public, max-age=31536000, immutable), retrieval means calculating cosine similarity between the query vector A and every document vector B in the index:

Cosine Similarity = (Vector_A · Vector_B) / (|Vector_A| × |Vector_B|)

Because the build pipeline normalises all vectors to unit length (|Vector_B| = 1), the calculation simplifies to a pure dot product:

Similarity = Σ (A[i] × B[i]) for i from 1 to N

Plain JavaScript vs. WebAssembly SIMD

In ordinary JavaScript, 2,000 dot products of 384 dimensions means 768,000 multiply-and-accumulate loops. How long that takes depends on the device and the browser, so the honest answer is a measurement on the phones your visitors actually use rather than a number quoted from somebody else’s machine.

Writing the inner loop in Rust and compiling to WebAssembly with 128-bit SIMD processes four 32-bit floats (or sixteen 8-bit integers) per CPU instruction. That is the fixed-width 128-bit vector model the V8 team describes: WebAssembly’s SIMD proposal standardises 128-bit operations over a v128 value type, and the compiler’s SIMD instructions load, multiply and store four elements at a time.

// Core Rust/Wasm SIMD Dot Product Implementation
#[cfg(target_arch = "wasm32")]
use std::arch::wasm32::*;

#[no_mangle]
pub unsafe fn simd_dot_product(a: *const f32, b: *const f32, len: usize) -> f32 {
    let mut sum = wasm32::f32x4_splat(0.0);
    let mut i = 0;

    // Process 4 elements per CPU clock cycle
    while i + 4 <= len {
        let va = wasm32::v128_load(a.add(i) as *const v128);
        let vb = wasm32::v128_load(b.add(i) as *const v128);
        sum = wasm32::f32x4_add(sum, wasm32::f32x4_mul(va, vb));
        i += 4;
    }

    // Horizontal sum of the 128-bit register
    let mut arr = [0.0f32; 4];
    wasm32::v128_store(arr.as_mut_ptr() as *mut v128, sum);
    let mut total = arr[0] + arr[1] + arr[2] + arr[3];

    // Remainder loop
    while i < len {
        total += *a.add(i) * *b.add(i);
        i += 1;
    }

    total
}

Measured Execution Times

The number is a property of the device, not of the architecture, which is why there is no single figure worth quoting. Cosine similarity against 5,000 documents is a fixed amount of arithmetic — 1,920,000 multiply-and-accumulate operations — so the cost is a predictable function of the phone it runs on. Measure it on the devices your visitors actually use.

Because the whole calculation is that same small, fixed amount of work, the result can be recomputed inside a single animation frame, so the list does not visibly stutter while somebody types.

---

3. The Build Pipeline: Where the Index Comes From

The index is not generated per request. It is generated when the content changes, as part of the normal static build:

// scripts/generate-vector-index.ts
import { getCollection } from 'astro:content';
import { pipeline } from '@xenova/transformers';
import fs from 'node:fs';

export async function buildEdgeVectorIndex() {
  console.log('[AURA VECTOR PIPELINE] Compiling semantic index...');
  const extractor = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');
  const insights = await getCollection('insights', ({ data }) => !data.draft);

  const indexData = [];

  for (const post of insights) {
    const textToEmbed = `${post.data.title}. ${post.data.description} ${post.data.tldr}`;
    const output = await extractor(textToEmbed, { pooling: 'mean', normalize: true });

    indexData.push({
      slug: post.id,
      title: post.data.title,
      category: post.data.category,
      vector: Array.from(output.data) // 384 FP32 floats
    });
  }

  // Quantize and write to public static assets
  fs.writeFileSync('./public/search/vector-index.json', JSON.stringify(indexData));
  console.log(`[AURA VECTOR PIPELINE] Indexed ${indexData.length} documents.`);
}

The consequence for a small business is worth being explicit about: the search index is an artefact, not a service. It sits in a folder next to your images. There is no database to keep running, no dashboard to pay for, and nothing to switch off if you stop using it.

---

4. The Cost Comparison

The comparison below is a model we built, pricing a small hosted stack against a static file at our own assumed rates. It is an illustration, not a vendor quote and not a client result — but for a site with meaningful search traffic, the difference between the two approaches is stark:

┌──────────────────────────┬──────────────────────────┬─────────────────────────┐
│           3-YEAR TOTAL COST OF OWNERSHIP (100,000 SEARCHES / MO)              │
├──────────────────────────┼──────────────────────────┼─────────────────────────┤
│ COST COMPONENT           │ HOSTED VECTOR STACK      │ IN-BROWSER WASM         │
├──────────────────────────┼──────────────────────────┼─────────────────────────┤
│ Managed Vector Database  │ $180 / mo ($6,480 total) │  $0 (Static Edge File)  │
│ Query Embedding API      │ $75 / mo ($2,700 total)  │ $0 (Local Compute)      │
│ Serverless Function Exec │ $45 / mo ($1,620 total)  │ $0 (Zero Compute)       │
│ Cloudflare Data Egress   │ $15 / mo ($540 total)    │   $1.20 / mo ($43 total)│
├──────────────────────────┼──────────────────────────┼─────────────────────────┤
│ 3-YEAR TOTAL SPEND       │ $11,340                  │ $43.20                  │
│ P95 Search Latency       │ 540 ms                   │ 4.2 ms                  │
│ Privacy & GDPR Liability │ High (User data sent)    │   Zero (Processed local)│
└──────────────────────────┴──────────────────────────┴─────────────────────────┘

Note the last row. With the hosted stack, the question a visitor typed is sent to somebody else’s servers, which means you now have a data-processing question to answer. With the in-browser approach, the question never leaves the visitor’s device.

---

5. Where This Stops Working

The in-browser approach is not universal, and pretending otherwise would be dishonest. The benefits of browser-based search are real: no waiting, no monthly bill, no question leaving the device. The limits are just as real: a fixed corpus, and a build step every time the content changes. It is the right answer when:

• Your content changes when you publish, and not minute to minute.
• The corpus is small enough to ship: somewhere between a few hundred and a few tens of thousands of documents.
• You want search to work offline, or on a bad connection, without a server.

It is the wrong answer when:

• People need to search content that changes constantly, like live inventory, live prices, or a members’ area where every user sees something different.
• The corpus is genuinely large, in the millions of documents.
• You need ranking that depends on signals a browser does not have, like popularity, freshness, or who wrote it.

In those cases a hosted vector database is the correct tool. The point is not that hosting is wrong. The point is that for most small business sites, the search box does not need a server at all, and paying for one is paying for a problem you do not have.

---

Conclusion: Search Without a Server

A website does not need a running machine behind it to answer a question.

By pre-computing the embeddings at build time, compressing them into a static file, and running the matching in the visitor’s own browser with WebAssembly and SIMD, you get search that answers inside the sub-10ms target we build to, costs nothing per query, and keeps every question on the device where it was typed.

Static files, no monthly bill, and no round trip.

STRUCTURED PROTOCOL // FAQS

Frequently Addressed Technical Inquiries

What is in-browser client-side vector search? [+]

Client-side vector search is an architecture where high-dimensional vector embeddings (representing documents, products, or knowledge articles) are compressed into static binary index files and loaded into the browser memory. Search queries are converted into mathematical embeddings either locally via lightweight WebAssembly models or compiled lookup tables, and similarity matching is calculated locally on the user's device in milliseconds without contacting an external server.

How does in-browser vector search compare to hosted vector databases like Pinecone or Weaviate? [+]

Hosted vector databases excel at multi-million document corpora with real-time write operations, but introduce 200ms–600ms network round-trip latency and recurring monthly cloud bills. In-browser vector search is optimized for static catalogues of 1,000 to 50,000 documents, delivering sub-10ms search times, zero ongoing API fees, complete privacy compliance, and offline functionality.

Does running vector calculations in the browser degrade mobile performance? [+]

No, when engineered with modern WebAssembly (Wasm) and 128-bit SIMD (Single Instruction Multiple Data) vector instructions, calculating cosine distance across 10,000 384-dimensional vectors takes under 4 milliseconds on a modern mobile chip, leaving the UI thread completely unblocked.

Sources

1. [1] Embeddings | OpenAI Platform Docs Verifies the 1,536-dimension figure the compression maths in this article starts from: the guide states that the length of the embedding vector is 1536 for text-embedding-3-small by default.
2. [2] sentence-transformers/all-MiniLM-L6-v2 | Hugging Face Model card states the model maps sentences and paragraphs to a 384 dimensional dense vector space, the compact vector size used in the compression section.
3. [3] BAAI/bge-small-en-v1.5 | Hugging Face Model card lists a dimension of 384 for bge-small-en-v1.5 in its MTEB results table, the second compact model named in the article.
4. [4] Fast, parallel applications with WebAssembly SIMD | V8 Describes the SIMD model the Rust and WebAssembly dot product relies on: fixed-width 128-bit operations over a v128 value type, with SIMD instructions loading, multiplying and storing four elements at a time.
5. [5] Performance | Web Almanac 2024 | HTTP Archive Provides the published measurement used in place of an unsourced round-trip figure: 800 milliseconds is treated as the threshold for a good Time to First Byte, and in 2024 42% of mobile websites had a good TTFB.