Build Local Semantic Search with Ollama and JavaScript in 5 Steps

Want to search a few development notes by meaning rather than exact keywords? Start with embeddings, not a full RAG framework. This small JavaScript example turns three public sample notes and one question into vectors, then ranks the notes locally.

The scope is deliberately small: no database, generated answer, private files, or claimed benchmark. The API usage follows Ollama’s current official documentation; the ranking code below is an original teaching example, not a report of a personally tested deployment.

1. Prepare a local embedding model

You need Ollama installed and running, plus a supported Node.js installation with npm. Use the official Ollama documentation for platform-specific installation. Download the model:

ollama pull embeddinggemma

Ollama’s embedding guide uses this model for both single inputs and batches. Downloading weights requires network access and disk space. Local inference still consumes RAM and electricity; it is not a zero-resource service.

This tutorial connects only to the local endpoint. Do not replace the model with a cloud model and assume the same data boundary applies. Check the model’s model page and license before distributing a product based on it.

2. Create an isolated JavaScript project

mkdir local-note-search
cd local-note-search
npm init -y
npm i ollama

The installation command is documented in the official JavaScript client. Save the next example as search.mjs; that extension enables JavaScript modules without changing package.json. Keep the generated lockfile if you want subsequent installs to resolve the same dependency versions.

3. Embed notes and a question in one batch

Ollama accepts an array of strings as input. Batch the tiny corpus and question together so their vectors come from the same request and model. For a real application, index documents once and embed only each new query.

import { Ollama } from 'ollama'

const client = new Ollama({ host: 'http://127.0.0.1:11434' })
const notes = [
  'Git branches keep experimental changes separate from the main branch.',
  'Docker volumes preserve database files when containers are recreated.',
  'CSS grid places page elements into rows and columns.',
]
const question = 'How can I keep database data after replacing a container?'

const result = await client.embed({
  model: 'embeddinggemma',
  input: [...notes, question],
  truncate: false,
})
const vectors = result.embeddings
if (vectors.length !== notes.length + 1) {
  throw new Error('Unexpected embedding count')
}
const queryVector = vectors[vectors.length - 1]

The API reference says truncation defaults to true. Setting it to false makes oversized input an error instead of silently embedding a shortened note. That matters when the important sentence is near the end. Shorten or split long documents deliberately rather than assuming a partial vector represents the entire file.

4. Rank by similarity

Append this to the same file:

function dot(a, b) {
  if (a.length !== b.length) throw new Error('Vector size mismatch')
  return a.reduce((sum, value, i) => sum + value * b[i], 0)
}

const ranked = notes.map((text, i) => ({
  text,
  score: dot(vectors[i], queryVector),
})).sort((a, b) => b.score - a.score)

for (const item of ranked) {
  console.log(item.score.toFixed(4), item.text)
}

Ollama documents that /api/embed returns unit-length vectors. For those vectors, the dot product equals cosine similarity. That shortcut is not safe for arbitrary unnormalized vectors from another provider. Higher similarity is a ranking signal, not a probability that a note is correct or an answer is complete.

5. Run, inspect, then expand cautiously

node search.mjs

Inspect the actual ranking on your machine. The container-persistence note is the intended relevant document; no exact score or rank is promised. Try a few hand-written questions with known relevant notes before adding complexity.

Symptom First check
Connection refused Is Ollama running at the explicit local address?
Model unavailable Did the model download finish under the same Ollama service?
Input too long Split the note; keep truncate:false while diagnosing.
Weak ranking Check chunk boundaries and use the same embedding model for indexing and querying.

If memory becomes the blocker, see the existing Ollama memory troubleshooting guide. Embedding workloads differ from chat, so do not treat chat-context tuning as a guaranteed embedding fix.

For more documents, persist vectors with document IDs and the exact embedding configuration. Re-embed when changing models; equal vector dimensions do not make two models interchangeable. Protect stored vectors alongside the original notes, and keep the Ollama endpoint private. A local search prototype does not justify exposing an inference server to the internet.

Documentation checked: November 22, 2026. The cited living documentation does not establish a fixed publication date. No performance or hardware-fit claim is made.


Discover more from Juzhax Technology

Subscribe to get the latest posts sent to your email.

Leave a Reply