Skip to content

Home

Jump to bottom
min.tian edited this page Jun 21, 2022 · 2 revisions

Feder is a JavaScript library for ANNs (Approximate Nearest Neighbor Searching) visualization.

Feder parses the built index file to obtain its index structure, simulates its search process to obtain detailed vector access records, and finally provides the corresponding visualizations.

Quick Start

The use of feder is divided into the following two necessary parts:

Initialization

With the input of a built index file, feder will parse it to obtain information about its index structure, and provide a search function for a single target vector.

import { Feder } from "@zilliz/feder";

const feder = new Feder({
  filePath,
  source,
});

where filePath is the built index file and source is the name of the index library.

For example,

const filePath =
  "https://assets.zilliz.com/faiss_ivf_flat_voc_17k_ab112eec72.index";
const source = "faiss"; // or hnswlib

The currently supported index types are as follows：

Visualization

After the initialization of the feder, two visualizations will be provided.

  • overview - visualizes the structure of the index.
  • search view - shows the detailed search process of the given target vector.
const overviewDom = feder.overview();
const searchViewDom = feder.search(vector);

feder's visualization function will return a dom node, which you can insert into the dom tree.

document.querySeletor("xxxx").appendChild(overviewDom);
d3.select("xxxx").node().appendChild(overviewDom);

Advanced

Automatic insertion

When initializing the feder, you can optionally set the domSelector parameter, which will cause the dom node generated by each visualization to be automatically inserted into the domSelector dom.

const domSelector = "#custom-node-id";

const feder = new Feder({
  filePath,
  source,
  domSelector,
});

feder.overview(); // overwritten
feder.search(vector); // render

Note that before auto-insertion, feder will clear the contents of the domSelector dom. As in the example above, only the search view will be rendered and the overview will be overwritten.

Set Search Params

Before searching, feder supports setting the query parameters by setSearchParams.

feder
  .setSearchParams({
    k: 10, // ivf_flat, hnsw (top k results)
    nprobe: 8, // ivf_flat
    ef: 12, // hnsw (ef_search)
  })
  .search(target_vector);

feder.search(target_vector); // the same query parameters.

Note that setSearchParams will take effect for all subsequent queries.

Search Internal Vectors

Feder provides additional search functions to support direct querying of the internal vectors.

searchById - Search by id (rowId), where rowId corresponds to the sequential number of the vector inserted during index construction.

feder.searchById(rowId);

searchRandTestVec - Feder also provides a random query that will randomly pick an internal vector as the target.

feder.searchRandTestVec();

View Params

Feder provides viewParams for more setting.

const feder = new Feder({
  ...,
  viewParams,   // default {}
});

General Params

  • width The width of the canvas, default is 1000.
  • height the length of the canvas, default is 600.
  • canvasScale the fineness of the canvasdefault is 3; the larger the finer, but it will consume more resources; since the canvas is a bitmap, it may be jagged, the optimization method is to generate a larger canvas ([width * canvasScale, height * canvasScale]), then reduce it to the corresponding size ([width, height]) by ctx.scale(1/canvasScale, 1/canvasScale).
const viewParams = {
  ...,
  width: 1000,
  height: 600,
  canvasScale: 3,
  ...,
};

Media Support

feder provides the correspondence between the index vector and the original file, and will show its corresponding file directly during the visual presentation.

Only images are currently supported.

const viewParams = {
  ...,
  mediaType: 'img',
  mediaCallback: (rowId) => imgUrl,
  ...,
};

When this feature is used, the target-media url corresponding to the target vector will also need to be entered additionally when searching.

Note that searchById and searchRandTestVec do not require this parameter.

feder.search(targetVector, targetUrl);

Params for IVF_Flat

  • projectMethod currently only supports umap;
  • projectParams Parameters for projection method, default is the default parameters for the specified projection algorithm;
  • coarseSearchWithProjection whether projection is required for voronoi initialization, default is true, off to reduce the computational burden;
  • fineSearchWithProjection whether projection is required for fine-search vectors, default is true, off to reduce the computational burden;
  • projectSeed supports layout reproduction when fineSearchWithProjection is true;

Params for HNSW

  • overviewLevelCount The number of levels to display at overview (from top to bottom), default is 3, the larger the value the more resources will be consumed.
Clone this wiki locally