Skip to content
BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
src
src
 
 
.env.example
.env.example
 
 
.gitignore
.gitignore
 
 
.npmignore
.npmignore
 
 
README.md
README.md
 
 
package.json
package.json
 
 
tsconfig.json
tsconfig.json
 
 
tsconfig.lib.json
tsconfig.lib.json
 
 

Repository files navigation

🚀 Solana NFT Indexer

A production-ready, high-performance Solana NFT transaction indexer built with TypeScript, Bun, and MongoDB

TypeScript Bun MongoDB License

Features • Architecture • Integrations • Quick Start • API Documentation • Library Usage

📖 Overview

The Solana NFT Indexer is a real-time transaction monitoring system designed specifically for tracking NFT mints and transfers on the Solana blockchain. It uses a dual-strategy indexing approach:

  • Real-time Streaming: Via Yellowstone gRPC for low-latency transaction processing
  • Historical Backfill: Via Helius API for catching up on missed transactions

The indexer stores all transaction data in MongoDB and provides a comprehensive RESTful API for querying NFT ownership, transaction history, and metadata.

✨ Features

Core Functionality

  • 🔄 Real-time Transaction Monitoring - Subscribe to specific NFT collections via Geyser gRPC
  • 📜 Historical Backfill - Catch up on missed transactions with Helius API polling
  • 💾 MongoDB Persistence - Efficient storage with compound indexes for fast queries
  • 🔍 Transaction Categorization - Automatic detection of mints vs transfers
  • 🏷️ NFT Metadata Caching - 24-hour TTL cache for reduced API calls
  • 🎯 Multi-Collection Support - Track multiple NFT collections simultaneously

API & CLI

  • 🌐 RESTful API - Comprehensive HTTP endpoints with Elysia.js
  • 💻 CLI Interface - Command-line tools for managing the indexer
  • 📊 Health Monitoring - Built-in health checks and status endpoints
  • 🔌 WebSocket Ready - Real-time updates via streaming events

Developer Experience

  • ⚡ Bun Runtime - Fast startup and low memory footprint
  • 🔒 TypeScript - Full type safety with strict mode
  • 📝 Structured Logging - Pino logger with JSON output
  • 🛡️ Error Handling - Comprehensive error recovery and retry logic
  • 📦 Importable Library - Use as a standalone app or import as a package

🏗️ Architecture

System Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                        Solana NFT Indexer System                        │
│                                                                              │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                       Application Layer                            │   │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌───────────┐ │   │
│  │  │    CLI     │  │   API      │  │ Orchestrator│  │  Service  │ │   │
│  │  │   (Elysia)  │  │  (Elysia)  │  │    (Event   │  │  Layer    │ │   │
│  │  │            │  │            │  │    Emitter) │  │           │ │   │
│  │  └─────┬──────┘  └─────┬──────┘  └──────┬──────┘  └─────┬─────┘ │   │
│  └────────┼───────────────┼───────────────────┼───────────────┼─────┘   │
│           │               │                   │                   │       │
│  ┌────────▼───────────────▼───────────────────▼───────────────────▼───────┐ │
│  │                     Service Layer (Business Logic)                │ │
│  │                                                                      │ │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐     │ │
│  │  │  Streaming   │  │   Backfill   │  │    Metadata      │     │ │
│  │  │  Service     │  │   Service    │  │    Service       │     │ │
│  │  │ (Geyser gRPC)│  │  (Helius)    │  │  (Metaplex)      │     │ │
│  │  └──────┬───────┘  └──────┬───────┘  └──────┬───────────┘     │ │
│  └─────────┼───────────────────┼───────────────────┼────────────────┘ │
│            │                   │                   │                  │
│  ┌─────────▼───────────────────▼───────────────────▼──────────┐    │
│  │              Processor Service (Transaction Pipeline)        │    │
│  │  • Categorize transactions (mint vs transfer)                   │    │
│  │  • Deduplicate and validate                                       │    │
│  │  • Update ownership records                                       │    │
│  └─────────┬──────────────────────────────────────────────────┘    │
└────────────┼───────────────────────────────────────────────────────────┘
             │
┌────────────┴─────────────────────────────────────────────────────────────┐
│                       Data Layer                                        │
│                                                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │
│  │   MongoDB    │  │   Models     │  │    Utils     │              │
│  │   (Mongoose)  │  │  (Schemas)   │  │  (Logger, etc)│              │
│  └──────────────┘  └──────────────┘  └──────────────┘              │
└───────────────────────────────────────────────────────────────────────┘

🔌 External Integrations

The indexer integrates with several third-party services to provide comprehensive NFT indexing capabilities.

Integration Overview

┌────────────────────────────────────────────────────────────────────────────┐
│                           Solana Blockchain                                 │
│                     (Source of NFT Transactions)                             │
└────────────────────────────┬───────────────────────────────────────────────┘
                             │
         ┌───────────────────┴───────────────────┐
         │                                       │
         ▼                                       ▼
┌──────────────────────┐              ┌──────────────────────┐
│  🔷 Geyser gRPC       │              │  🔵 Helius API         │
│  (Real-time Stream)   │              │  (Historical Data)    │
│                      │              │                      │
│  • SUBSCRIBE txs      │              │  • GET /assets        │
│  • Stream confirmations│              │  • GET /transactions  │
│  • Low latency        │              │  • GET /signatures    │
│  • gRPC protocol      │              │  • REST API           │
│  • Requires token     │              │  • API key auth       │
└──────────┬───────────┘              └──────────┬───────────┘
           │                                     │
           │    ┌───────────────────────────┐ │
           └────┴                           ┌─┴───────┘
                │                       │
                ▼                       ▼
    ┌──────────────────────────────────────┐
    │   Solana NFT Indexer               │
    │                                    │
    │  ┌──────────────────────────┐     │
    │  │  Metaplex JS SDK          │     │
    │  │  (NFT Metadata)           │     │
    │  │                           │     │
    │  │  • findByMint()           │     │
    │  │  • findAllByOwner()       │     │
    │  │  • parseMetadata()        │     │
    │  └──────────┬───────────────┘     │
    │             │                       │
    │             ▼                       │
    │  ┌──────────────────────────┐     │
    │  │   Transaction Processor  │     │
    │  │                           │     │
    │  │  • Parse instructions     │     │
    │  │  • Extract mint/owner     │     │
    │  │  • Detect tx type         │     │
    │  └──────────┬───────────────┘     │
    │             │                       │
    │             ▼                       │
    │  ┌──────────────────────────┐     │
    │  │      MongoDB              │     │
    │  │                           │     │
    │  │  • Transactions           │     │
    │  │  • Token Ownership        │     │
    │  │  • Collection State       │     │
    │  │  • NFT Metadata Cache     │     │
    │  └──────────────────────────┘     │
    └────────────────────────────────────┘
               │
               ▼
┌──────────────────────────────────────────────┐
│           HTTP API (Elysia.js)               │
│                                              │
│  • GET  /health                              │
│  • GET  /indexer/transactions               │
│  • GET  /indexer/ownerships                  │
│  • POST /indexer/start                       │
└──────────────────────────────────────────────┘

Third-Party Service Details

1. 🔷 Geyser (Yellowstone gRPC)

Purpose: Real-time transaction streaming

Communication Method: gRPC (bidirectional streaming)

How It Works:

Indexer ──[gRPC Subscribe]──▶ Geyser Server ◀───[Stream]──▶ Confirmed Txs
          │                                      │
          └───[Read token list]─────────────────┘

API Used:

// @triton-one/yellowstone-grpc
import Client from '@triton-one/yellowstone-grpc';

// Subscribe to transactions for specific mints
const client = new Client(endpoint, authToken);
const stream = await client.subscribe({
  transactions: {
    accountInclude: [mint1, mint2, ...]
  }
});

// Stream emits confirmed transactions
stream.on('data', (transaction) => {
  // Process in real-time
});

Data Flow:

  1. Indexer sends SubscribeRequest with mint addresses to watch
  2. Geyser maintains connection and pushes transactions as they're confirmed
  3. Each transaction includes: signature, slot, instructions, accountKeys
  4. Indexer parses SPL Token instructions to detect mints/transfers

2. 🔵 Helius API

Purpose: Historical transaction backfill and asset queries

Communication Method: HTTPS REST API

How It Works:

Indexer ──[GET /assets?collection=X]──▶ Helius ◀───[Asset List]───┘
          │                                   │
          └───[GET /signatures?address=X]──▶ Helius ◀───[Tx List]─────┘

API Used:

// HTTP REST calls
const assets = await fetch(
  `https://api.helius.xyz/v0/assets?collectionAddress=${collection}`,
  { headers: { 'Authorization': `Bearer ${apiKey}` } }
);

const transactions = await fetch(
  `https://api.helius.xyz/v0/transactions/history?address=${mint}&before=${sig}`,
  { headers: { 'Authorization': `Bearer ${apiKey}` } }
);

Key Endpoints:

  • GET /v0/assets?collectionAddress={address} - Get all NFTs in collection
  • GET /v0/transactions/history?address={address} - Get transaction history
  • POST /v0/transactions - Get transaction by signatures

Data Flow:

  1. Indexer fetches all assets in a collection
  2. For each asset, fetches transaction history since last processed signature
  3. Processes transactions in chronological order (oldest first)
  4. Handles pagination and rate limiting automatically

3. 🟣 Metaplex JS SDK

Purpose: NFT metadata fetching and parsing

Communication Method: RPC calls to Solana RPC + HTTP for off-chain metadata

How It Works:

Indexer ──[Metaplex.findByMint()]──▶ Solana RPC ◀───[On-chain Metadata Account]───┘
                                       │
                                       └───[HTTP GET metadata URI]──▶ Arweave/IPFS

API Used:

import { Umi } from '@metaplex-foundation/js';
import { publicKey } from '@solana/web3.js';

// Connect to Solana
const umi = Umi.create(process.env.RPC_ENDPOINT);

// Find NFT by mint
const nft = await findMetadataPda(umi);
const metadata = await fetchMetadata(umi, nft);

Data Flow:

  1. Indexer calls Metaplex.findByMint(mintAddress)
  2. SDK derives Metadata PDA from mint address
  3. Fetches on-chain metadata account from Solana
  4. Parses metadata structure (name, symbol, uri, creators)
  5. Optionally fetches off-chain JSON from uri field
  6. Extracts attributes/traits from metadata

4. 🟢 MongoDB (with Mongoose)

Purpose: Persistent storage for transactions, ownership, and metadata

Communication Method: MongoDB Wire Protocol (via Mongoose driver)

How It Works:

Indexer ──[Mongoose.connect()]──▶ MongoDB ◀───[Query/Insert]───┘
          │                           │
          └───[Model.create()]────────────────┘

API Used:

import mongoose from 'mongoose';

// Connect
await mongoose.connect(mongoUri);

// Define schema
const TransactionSchema = new Schema({
  signature: { type: String, unique: true },
  slot: Number,
  from_address: String,
  to_address: String,
  // ...
});

// Create document
await TransactionModel.create({
  signature: tx.signature,
  slot: tx.slot,
  // ...
});

// Query
const txs = await TransactionModel.find({ mint_address: address })
  .sort({ blockTime: -1 })
  .limit(100);

Data Models:

  • Transaction - Stores all NFT transactions
  • TokenOwnership - Current owner of each NFT
  • CollectionState - Indexing progress tracking
  • NFTMetadata - Cached metadata (24-hour TTL)

5. 🌐 Elysia.js

Purpose: HTTP API server framework

Communication Method: HTTP (serves client requests)

How It Works:

Client ──[GET /transactions]──▶ Elysia Server ◀───[Controller]───▶ MongoDB
         │                                       │
         └───[POST /start]────────────────────────┘

API Used:

import { Elysia, t } from 'elysia';

// Define schema
const TransactionSchema = t.Object({
  signature: t.String(),
  slot: t.Number(),
  from_address: t.String(),
  to_address: t.String(),
});

// Create route
app.get('/transactions', async ({ query }) => {
  const { limit = 50 } = query;
  const txs = await TransactionModel.find().limit(limit);
  return { transactions: txs, pagination: {...} };
});

Data Flow Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                        Transaction Lifecycle                               │
└─────────────────────────────────────────────────────────────────────────────┘

1. NFT Minted on Solana
   │
   ▼
2. Geyser Detects Transaction (real-time)
   └── gRPC stream → TransactionProcessor
   │
   ▼
3. Helius Backfill (if missed)
   └── HTTP GET /transactions → TransactionProcessor
   │
   ▼
4. TransactionProcessor Categorizes
   ├── SPL Token InitializeMint? → MINT
   └── SPL Token Transfer? → TRANSFER
   │
   ▼
5. Metaplex Fetches Metadata
   ├── findMetadataPda() → On-chain account
   └── fetchMetadata() → Off-chain JSON (uri field)
   │
   ▼
6. Data Stored in MongoDB
   ├── TransactionModel.create()
   ├── TokenOwnershipModel.updateOne()
   ├── NFTMetadataModel.create() (with TTL)
   └── CollectionStateModel.updateOne()
   │
   ▼
7. Queryable via HTTP API
   └── Elysia GET /indexer/transactions → MongoDB → JSON Response

🚀 Quick Start

Installation as Library

npm install solana-nft-indexer

Installation as Standalone

git clone <repo-url>
cd solana-indexer
bun install

📚 API Documentation

Endpoints

Category Endpoint Description
Health GET /health System health check
Indexer POST /indexer/start Start indexing service
Transactions GET /indexer/transactions List transactions (paginated)
Ownership GET /indexer/ownerships Get NFT ownership
Metadata GET /indexer/metadata/:mint Get NFT metadata

Example Usage

# Check health
curl http://localhost:3001/health

# Get recent transactions
curl http://localhost:3001/indexer/transactions?limit=10

# Get NFT metadata
curl http://localhost:3001/indexer/metadata/<MINT_ADDRESS>

📦 Library Usage

Import as Package

// Import services
import {
  GeyserStreamingService,
  MetadataService,
  ProcessorService
} from 'solana-nft-indexer';

// Import types
import type {
  ParsedTransaction,
  NFTMetadata,
  EnrichedTransaction
} from 'solana-nft-indexer';

// Create streaming instance
const streamer = GeyserStreamingService.getInstance();
await streamer.connect();
await streamer.subscribe(['mint1', 'mint2']);

streamer.on('transaction', (tx) => {
  console.log('New transaction:', tx);
});

// Fetch metadata
const metadataService = MetadataService.getInstance(rpcUrl);
const nft = await metadataService.getNFTMetadata(mintAddress);

Direct Instantiation

import { ProcessorService } from 'solana-nft-indexer';

// Create processor for specific collection
const processor = new ProcessorService({
  collectionAddress: 'collection_mint_address',
  rpcEndpoint: 'https://your-rpc-url'
});

// Process transaction
await processor.processTransaction(transaction, assetMap);

⚙️ Configuration

Environment Variables

Variable Required Default Description
MONGODB_URI ✅ - MongoDB connection string
YELLOWSTONE_GRPC_ENDPOINT ❌ - Geyser gRPC endpoint
HELIUS_API_KEY ✅ - Helius API key
SOLANA_RPC_ENDPOINT ✅ - Solana RPC URL
COLLECTION_ADDRESS ✅ - Collection to index

📄 License

MIT License - see LICENSE for details

Built with ❤️ by the Solana community

About

🚀 Production-ready Solana NFT indexer with real-time Geyser gRPC streaming, Helius backfill, and MongoDB persistence. Built with TypeScript + Bun.

Topics

blockchainheliusindexermetaplexnftnft-indexersolanaweb3yellowstone

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages