Skip to content
Console
Menu

Queues

Workflows

Getting Started

Authentication

KV Store

Search

What a SearchIndex is, how vectors fit, and where documents and queries are served.

A SearchIndex is one customer lexical and vector index of JSON documents. It is not a namespace inside a shared service: every index has its own engine, its own credentials, its own backup, its own region and its own deletion, so nothing you index can be read from anyone else's.

An index is managed for you the way the rest of Data is: you create it, read it and delete it as a Resource, and its spec is small — how vectors fit, its region, and whether a delete is protected.

#Two ways to match, one index

vector_dimensions decides what a query can be:

  • 0 — lexical only. You write JSON documents, a query is text, and the index matches the words.
  • A positive dimension — the lexical index plus cosine queries. You write an embedding with each document, and a query can carry an embedding of its own, or text, or both.

Data never invents embeddings. The vector you write is the vector that is searched, and its length is the index's vector_dimensions.

FieldTypeWhat it is
vector_dimensionsint32The vector dimension; 0 for lexical only.
regionstringThe region; default the project’s home region. Changing it moves the data by signed export and import.
deletion_protectionboolWhile set, Delete fails with DELETION_PROTECTED.

#Where the calls go

The index itself is administered on the one API, like the rest of Data. Its documents and queries are served on the data host, beside the data they read:

Name
orgs/{org}/projects/{project}/envs/{env}/search_indexes/{search_index}
Id
idx_<cell><ulid>
Documents and queries
https://api.data.sylphx.com
Engine port
8108, TLS

data:read covers the reads: reading an index, listing indexes in an environment, reading a document and running a query. Everything that changes an index or its documents needs data:write — and so does connect, which writes nothing but returns the index's own engine credentials. Those credentials are not API keys, and they come back only in that answer.

Index administration has a CLI (sylphx data search-indexes …). Documents and queries do not: they are calls to https://api.data.sylphx.com.