Skip to content

Latest commit

 

History

History
183 lines (133 loc) · 8.56 KB

README.md

File metadata and controls

183 lines (133 loc) · 8.56 KB
Raw
   /     /   ▶ Token Metadata API   
  / --- /      Indexes metadata for all Fungible, Non-Fungible, and Semi-Fungible Tokens in the
 /     /       Stacks blockchain and exposes it via JSON REST API endpoints.

Features

  • Complete SIP-016 metadata ingestion for
  • Automatic metadata refreshes via SIP-019 notifications
  • Metadata localization support
  • Metadata fetching via http:, https:, data: URIs. Also supported via customizable gateways:
    • IPFS
    • Arweave
  • Easy to use REST JSON endpoints with ETag caching
  • Prometheus metrics for job queue status, contract and token counts, API performance, etc.
  • Image cache/CDN support
  • Run modes for auto-scaling

API reference

See the Token Metadata API Reference for more information.

Client library

A TypeScript client library is available for consuming the API. See the official package for more information.

Quick start

System requirements

The Token Metadata Service is a microservice that has hard dependencies on other Stacks blockchain components. Before you start, you'll need to have access to:

  1. A fully synchronized Stacks node
  2. A fully synchronized instance of Chainhook
  3. A local writeable Postgres database for token metadata storage
  4. (Optional) A Google Cloud Storage bucket for storing token images

Running the service

Clone the repo.

Create an .env file and specify the appropriate values to configure access to the Stacks API database, the Token Metadata Service local database, and the Stacks node RPC interface. See env.ts for all available configuration options.

Build the app (NodeJS v18+ is required)

npm install
npm run build

Start the service

npm run start

Run modes

To better support auto-scaling server configurations, this service supports three run modes specified by the RUN_MODE environment variable:

  • default: Runs all background jobs and the API server. Use this when you're running this service only on one instance. This is the default mode.
  • readonly: Runs only the API server. Use this in an auto-scaled cluster when you have multiple readonly instances and just one writeonly instance. This mode needs a writeonly instance to continue populating the token metadata DB.
  • writeonly: Use one of these in an auto-scaled environment so you can continue consuming new token contracts. Use in conjunction with multiple readonly instances as explained above.

Stopping the service

When shutting down, you should always prefer to send the SIGINT signal instead of SIGKILL so the service has time to finish any pending background work and all dependencies are gracefully disconnected.

Using an image cache service

The Token Metadata API allows you to specify a Google Cloud Storage bucket to store the token images scanned by the metadata processor. This is recommended if you're looking to optimize your image speeds when using the API via a wallet, etc. See the env.ts file for more information on which environment variables you need to provide.

Service architecture

Job Queue

The role of the JobQueue is to perform all the smart contract and token processing in the service.

It is a priority queue that organizes all necessary work for contract ingestion and token metadata processing. Every job processed by this queue corresponds to one row in the jobs DB table, which marks its processing status and related objects to be worked on (smart contract or token).

This object essentially runs an infinite loop that follows these steps:

  1. Upon start(), it fetches a set number of job rows that are 'pending' and loads their corresponding Job objects into memory for processing, marking those rows now as 'queued'.
  2. It executes each loaded job to completion concurrently. Depending on success or failure, the job row is marked as either 'done' or 'failed'.
  3. Once all loaded jobs are done (and the queue is now empty), it goes back to step 1.

There are two env vars that can help you tune how the queue performs:

  • ENV.JOB_QUEUE_SIZE_LIMIT: The in-memory size of the queue, i.e. the number of pending jobs that are loaded from the database while they wait for execution (see step 1 above).
  • ENV.JOB_QUEUE_CONCURRENCY_LIMIT: The number of jobs that will be ran simultaneously.

This queue runs continuously and can handle an unlimited number of jobs.

Process Smart Contract Job

This job makes a contract call to the Stacks node in order to determine the total number of tokens declared by the given contract. Once determined, it creates and enqueues all of these tokens for metadata ingestion.

Process Token Job

This job fetches the metadata JSON object for a single token as well as other relevant properties depending on the token type (symbol, decimals, etc.). Once fetched, it parses and ingests this data to save it into the local database for API endpoints to return.

If a 429 (Too Many Requests) status code is returned by a hostname used to fetch metadata, the service will cease all further requests to it until a reasonable amount of time has passed or until the time specified by the host in a Retry-After response header.

Bugs and feature requests

If you encounter a bug or have a feature request, we encourage you to follow the steps below:

  1. Search for existing issues: Before submitting a new issue, please search existing and closed issues to check if a similar problem or feature request has already been reported.
  2. Open a new issue: If it hasn't been addressed, please open a new issue. Choose the appropriate issue template and provide as much detail as possible, including steps to reproduce the bug or a clear description of the requested feature.
  3. Evaluation SLA: Our team reads and evaluates all the issues and pull requests. We are avaliable Monday to Friday and we make a best effort to respond within 7 business days.

Please do not use the issue tracker for personal support requests or to ask for the status of a transaction. You'll find help at the #support Discord channel.

Contribute

Development of this product happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving the product.

Code of Conduct

Please read our Code of conduct since we expect project participants to adhere to it.

Contributing Guide

Read our contributing guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes.

Community

Join our community and stay connected with the latest updates and discussions: