A lightweight package to help you check whether a prompt to an AI chatbot is relevant to the context. Keep your API costs down by filtering out irrelevant queries!
This package uses the Xenova/mobilebert-uncased-mnli model via the Transformers.js library to perform zero-shot classification.
- Fast and efficient relevance checking (model automatically loads in a few seconds)
- Uses advanced NLP model for accurate classification
- Easy to integrate into existing chatbot systems
- Helps reduce unnecessary API calls to your main LLM
$ npm i llm-gatekeeperimport isRelevant from "llm-gatekeeper";
const prompt = "Should I travel this summer?";
const keywords = ["reading", "books", "essays"];
const relevance = await isRelevant(prompt, keywordArray);
if (relevance === false) {
// do not make API call
console.log(
`Sorry, the chatbot can only answer questions about ${keywords.join(
", or "
)}`
);
} else {
// make API call
}| Parameter | Type | Description |
|---|---|---|
| prompt | string | The input text to check for relevance |
| keywords | string [] | An array of keywords defining the relevant context |
Returns
Promise<boolean>: Resolves to true if the prompt is relevant, false otherwise.
The package uses a pre-trained MobileBERT model for zero-shot classification. The model is lightweight, and optimized for resource-limited devices.
It classifies the prompt into two categories: one containing your keywords, and "something else". If the prompt is more likely to belong to the keyword category, it's considered relevant.
- The model is loaded asynchronously when the package is imported, which may cause a short delay on first use. The model is usually loaded within the first few seconds of page load.
- Once the model is in memory, promises are resolved almost instantaneously.
If you are not getting accurate results, try adding more keywords. Or submit an issue on GitHub!
Thanks to my friend Zein for inspiring the idea for the package. I wouldn't have made it if he wasn't abusing my chatbot on my website and costing me money.