Config

A type-safe, namespace-aware configuration library for Go with support for multiple storage backends, built-in resilience, and OpenTelemetry instrumentation.

Features

• Multiple Storage Backends: Memory, PostgreSQL, MongoDB, SQLite, File
• Version History: Retrieve and paginate historical versions of any config key
• Multi-Store: Combine stores for caching, fallback, or replication patterns
• Namespace Isolation: Organize configuration by environment, tenant, or service
• Built-in Resilience: Internal cache ensures app works during backend outages
• Type-safe Values: Strongly typed access with automatic conversion
• Codecs: JSON, YAML, TOML encoding support
• OpenTelemetry: Tracing and metrics instrumentation
• Struct Binding: Bind configuration to Go structs with validation
• Live Binding: Auto-reload structs on config changes via polling

Design Philosophy

This library is designed for use cases like feature flags, rate limits, and dynamic configuration where eventual consistency is acceptable. The key principle is: having some configuration (even slightly stale) is better than having no configuration at all.

The library maintains an internal in-memory cache that serves as a resilience layer:

• If the backend store becomes temporarily unavailable, cached values continue to be served
• Each application instance maintains its own local cache
• Cache is automatically invalidated via the store's native change stream (MongoDB Change Streams, PostgreSQL LISTEN/NOTIFY)
• No external dependencies like Redis required for caching

Installation

go get github.com/rbaliyan/config

Quick Start

package main

import (
    "context"
    "log"

    "github.com/rbaliyan/config"
    "github.com/rbaliyan/config/memory"
)

func main() {
    ctx := context.Background()

    // Create manager with memory store
    mgr, err := config.New(
        config.WithStore(memory.NewStore()),
    )
    if err != nil {
        log.Fatal(err)
    }

    // Connect to backend
    if err := mgr.Connect(ctx); err != nil {
        log.Fatal(err)
    }
    defer mgr.Close(ctx)

    // Get configuration for a namespace (use "" for default)
    cfg := mgr.Namespace("production")

    // Set a value
    if err := cfg.Set(ctx, "app/timeout", 30); err != nil {
        log.Fatal(err)
    }

    // Get a value
    val, err := cfg.Get(ctx, "app/timeout")
    if err != nil {
        log.Fatal(err)
    }

    // Unmarshal into a typed variable
    var timeout int
    if err := val.Unmarshal(&timeout); err != nil {
        log.Fatal(err)
    }

    log.Printf("Timeout: %d", timeout)
}

Storage Backends

Memory Store

In-memory storage for testing and single-instance deployments.

import "github.com/rbaliyan/config/memory"

store := memory.NewStore()

PostgreSQL Store

Persistent storage with LISTEN/NOTIFY for real-time updates.

import (
    "database/sql"
    "github.com/lib/pq"
    "github.com/rbaliyan/config/postgres"
)

db, _ := sql.Open("postgres", "postgres://localhost/mydb")
listener := pq.NewListener(dsn, 10*time.Second, time.Minute, nil)

store := postgres.NewStore(db, listener,
    postgres.WithTable("config_entries"),
    postgres.WithNotifyChannel("config_changes"),
)

MongoDB Store

Persistent storage with change streams for real-time updates.

import (
    "go.mongodb.org/mongo-driver/v2/mongo"
    "go.mongodb.org/mongo-driver/v2/mongo/options"
    "github.com/rbaliyan/config/mongodb"
)

client, _ := mongo.Connect(options.Client().ApplyURI("mongodb://localhost:27017"))

store := mongodb.NewStore(client,
    mongodb.WithDatabase("config"),
    mongodb.WithCollection("entries"),
)

SQLite Store

Lightweight persistent storage with no external dependencies.

import "github.com/rbaliyan/config/sqlite"

store := sqlite.NewStore("config.db",
    sqlite.WithTable("config_entries"),
)

File Store (Read-Only)

Load configuration from files on disk with automatic reload via fsnotify.

import "github.com/rbaliyan/config/file"

store := file.NewStore("config/",
    file.WithCodec("yaml"),
)

Working with Values

Reading Values

val, err := cfg.Get(ctx, "database/port")
if err != nil {
    if config.IsNotFound(err) {
        // Key doesn't exist
    }
    return err
}

// Type-safe access via Value interface
port, err := val.Int64()
str, err := val.String()
flag, err := val.Bool()
num, err := val.Float64()

// Unmarshal into any type
var dbConfig DatabaseConfig
if err := val.Unmarshal(&dbConfig); err != nil {
    return err
}

// Access metadata
meta := val.Metadata()
version := meta.Version()
created := meta.CreatedAt()
updated := meta.UpdatedAt()

Writing Values

// Simple values
cfg.Set(ctx, "app/timeout", 30)
cfg.Set(ctx, "app/name", "myservice")
cfg.Set(ctx, "app/enabled", true)

// Complex values
cfg.Set(ctx, "app/servers", []string{"host1", "host2"})
cfg.Set(ctx, "app/limits", map[string]int{"max": 100, "min": 1})

// With options
cfg.Set(ctx, "app/config", value,
    config.WithType(config.TypeCustom),
    config.WithSetCodec(yamlCodec),
)

Conditional Writes

Control create/update behavior with conditional write options:

// Create only - fails if key already exists
err := cfg.Set(ctx, "feature/flag", true, config.WithIfNotExists())
if config.IsKeyExists(err) {
    // Key already existed, value not changed
}

// Update only - fails if key doesn't exist
err = cfg.Set(ctx, "feature/flag", false, config.WithIfExists())
if config.IsNotFound(err) {
    // Key didn't exist, nothing updated
}

// Default (upsert) - creates or updates
cfg.Set(ctx, "feature/flag", true) // Always succeeds

These options leverage atomic database operations:

• PostgreSQL: Uses ON CONFLICT DO NOTHING / UPDATE with row count checks
• MongoDB: Uses InsertOne / FindOneAndUpdate with upsert control

Listing Values

// List all keys with prefix using the Filter builder
limit := 100
page, err := cfg.Find(ctx, config.NewFilter().
    WithPrefix("app/database").
    WithLimit(limit).
    Build())

for key, val := range page.Results() {
    str, _ := val.String()
    fmt.Printf("%s = %s\n", key, str)
}

// Pagination: check if len(results) < limit to determine if more pages exist
if len(page.Results()) == page.Limit() {
    nextPage, _ := cfg.Find(ctx, config.NewFilter().
        WithPrefix("app/database").
        WithLimit(limit).
        WithCursor(page.NextCursor()).
        Build())
    // Process nextPage...
}

Version History

Stores that implement VersionedStore retain historical versions of config entries. Use GetVersions to retrieve them:

// Check if versioning is available
if vr, ok := cfg.(config.VersionedReader); ok {
    // Get a specific version
    page, err := vr.GetVersions(ctx, "app/timeout",
        config.NewVersionFilter().WithVersion(2).Build())

    // List all versions (newest first) with pagination
    page, err := vr.GetVersions(ctx, "app/timeout",
        config.NewVersionFilter().WithLimit(10).Build())

    for _, v := range page.Versions() {
        fmt.Printf("v%d: %v (updated %s)\n",
            v.Metadata().Version(), v, v.Metadata().UpdatedAt())
    }

    // Paginate through all versions
    if len(page.Versions()) == page.Limit() {
        nextPage, _ := vr.GetVersions(ctx, "app/timeout",
            config.NewVersionFilter().
                WithLimit(10).
                WithCursor(page.NextCursor()).
                Build())
        // ...
    }
}

The memory store supports versioning with an optional history cap:

store := memory.NewStore(memory.WithMaxHistory(100)) // Keep up to 100 versions per key

Namespaces

Namespaces provide isolation between different environments or tenants.

// Get configs for different namespaces
prodCfg := mgr.Namespace("production")
devCfg := mgr.Namespace("development")

// Same key, different values per namespace
prodCfg.Set(ctx, "timeout", 60)
devCfg.Set(ctx, "timeout", 5)

// Use "" for the default namespace
defaultCfg := mgr.Namespace("")

Namespace names may contain alphanumeric characters, underscores, dashes, dots, and colons (e.g. org.example:env.prod).

System Namespaces

The internal: prefix is reserved for system namespaces used by infrastructure components such as key rotation and service discovery. Use IsSystemNamespace() to check whether a namespace is reserved:

config.IsSystemNamespace("internal:config:crypto") // true
config.IsSystemNamespace("production")              // false

Server-side authorizers can use this to block client writes to system namespaces while allowing infrastructure components to manage them.

Context Helpers

Access configuration from context without explicit dependency injection.

// Add manager to context
ctx = config.ContextWithManager(ctx, mgr)
ctx = config.ContextWithNamespace(ctx, "production")

// Use anywhere in your application
val, err := config.Get(ctx, "app/setting")
err = config.Set(ctx, "app/setting", "value")

Multi-Store (Fallback Pattern)

Combine multiple stores for caching, fallback, or replication:

import "github.com/rbaliyan/config/multi"

// Cache + Backend pattern
cacheStore := memory.NewStore()
backendStore := postgres.NewStore(db, listener)
store := multi.NewStoreWithOptions(
    []config.Store{cacheStore, backendStore},
    []multi.Option{multi.WithStrategy(multi.StrategyReadThrough)},
)

// Primary + Backup pattern
primaryStore := postgres.NewStore(primaryDB, primaryListener)
backupStore := postgres.NewStore(backupDB, backupListener)
store := multi.NewStoreWithOptions(
    []config.Store{primaryStore, backupStore},
    []multi.Option{multi.WithStrategy(multi.StrategyFallback)},
)

mgr, _ := config.New(config.WithStore(store))

Strategies:

• StrategyFallback: Read from first available store, write to all stores
• StrategyReadThrough: Read through stores (cache miss populates earlier stores), write to all stores
• StrategyWriteThrough: Write to all stores, read from first available store

All strategies write to all stores to maintain consistency. The difference is in read behavior:

• Fallback/WriteThrough: Return first successful read
• ReadThrough: Try each store in order, populate earlier stores on cache miss

Multi-Store Diagnostics

Multi-Store supports health checks and statistics when the underlying stores implement the optional interfaces:

// Health check (if underlying stores implement HealthChecker)
if err := store.Health(ctx); err != nil {
    log.Printf("Store unhealthy: %v", err)
}

// Statistics (if underlying stores implement StatsProvider)
stats, err := store.Stats(ctx)
if err == nil {
    log.Printf("Total entries: %d", stats.TotalEntries)
}

OpenTelemetry Instrumentation

Wrap stores with tracing and metrics. Both are disabled by default and must be explicitly enabled.

import "github.com/rbaliyan/config/otel"

// Enable tracing and metrics explicitly
instrumentedStore, _ := otel.WrapStore(store,
    otel.WithServiceName("my-service"),
    otel.WithBackendName("postgres"),
    otel.WithTracesEnabled(true),   // Opt-in, disabled by default
    otel.WithMetricsEnabled(true),  // Opt-in, disabled by default
)

mgr, _ := config.New(config.WithStore(instrumentedStore))

Metrics exported:

• config.operations.total - Counter of all operations
• config.errors.total - Counter of errors by type
• config.operation.duration - Histogram of operation latency

Struct Binding

Bind configuration to Go structs with validation. Struct fields are automatically mapped to hierarchical keys using the configured struct tag (default: json).

import "github.com/rbaliyan/config/bind"

type DatabaseConfig struct {
    Host string `json:"host" validate:"required"`
    Port int    `json:"port" validate:"required,min=1,max=65535"`
}

binder := bind.New(cfg, bind.WithTagValidation())
bound := binder.Bind()

// Store a struct - creates keys: database/host, database/port
err := bound.SetStruct(ctx, "database", DatabaseConfig{Host: "localhost", Port: 5432})

// Retrieve a struct - reads keys: database/host, database/port
var dbConfig DatabaseConfig
err = bound.GetStruct(ctx, "database", &dbConfig)

Nested structs are also supported:

type AppConfig struct {
    Name  string      `json:"name"`
    Cache CacheConfig `json:"cache"`
}

type CacheConfig struct {
    TTL     int  `json:"ttl"`
    Enabled bool `json:"enabled"`
}

// SetStruct creates: app/name, app/cache/ttl, app/cache/enabled
err := bound.SetStruct(ctx, "app", AppConfig{
    Name: "myapp",
    Cache: CacheConfig{TTL: 300, Enabled: true},
})

Use nonrecursive to store a nested struct as a single JSON value instead of flattening:

type Credentials struct {
    Username string `json:"username"`
    Password string `json:"password"`
}

type AppConfig struct {
    Name  string      `json:"name"`
    Creds Credentials `json:"creds,nonrecursive"` // Store as single JSON value
}

// SetStruct creates: app/name, app/creds (not app/creds/username, app/creds/password)
// Useful when fields are tightly coupled and should be updated atomically

Live Config (Auto-Reload)

Keep a typed struct automatically synchronized with configuration using polling and atomic swap:

import "github.com/rbaliyan/config/live"

type DatabaseConfig struct {
    Host string `json:"host"`
    Port int    `json:"port"`
}

ref, err := live.New[DatabaseConfig](ctx, cfg, "database",
    live.PollInterval(10*time.Second),
    live.OnChange(func(old, new DatabaseConfig) {
        log.Printf("config changed: %s -> %s", old.Host, new.Host)
    }),
    live.OnError(func(err error) {
        log.Printf("reload error: %v", err)
    }),
)
if err != nil {
    return err
}
defer ref.Close()

// Hot path: single atomic load, zero contention
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    snap := ref.Load()
    fmt.Fprintf(w, "Host: %s, Port: %d", snap.Host, snap.Port)
})

Options:

• PollInterval(d) - Set polling interval (default: 30s)
• OnChange(fn) - Callback with old and new values on change
• OnError(fn) - Callback on reload error

Methods:

• Load() - Get current snapshot (atomic, zero-cost)
• Close() - Stop background polling
• ReloadNow(ctx) - Force immediate reload
• LastReload() - Get last reload timestamp
• LastError() - Get last error (nil if successful)
• ReloadCount() - Get total successful reload count

Codecs

Multiple encoding formats are supported.

import "github.com/rbaliyan/config/codec"

// Available codecs
jsonCodec := codec.Get("json")
yamlCodec := codec.Get("yaml")
tomlCodec := codec.Get("toml")

// Use with manager
mgr, _ := config.New(
    config.WithStore(store),
    config.WithCodec(yamlCodec),
)

Configuration Types

The library tracks value types for better type safety.

const (
    TypeInt             // int, int64
    TypeFloat           // float64
    TypeString          // string
    TypeBool            // bool
    TypeMapStringInt    // map[string]int
    TypeMapStringFloat  // map[string]float64
    TypeMapStringString // map[string]string
    TypeListInt         // []int
    TypeListFloat       // []float64
    TypeListString      // []string
    TypeCustom          // any other type
)

Error Handling

val, err := cfg.Get(ctx, "key")
if err != nil {
    switch {
    case config.IsNotFound(err):
        // Key doesn't exist
    case config.IsTypeMismatch(err):
        // Type conversion failed
    case config.IsKeyExists(err):
        // Key already exists (from WithIfNotExists)
    default:
        // Other error
    }
}

Manager Options

mgr, err := config.New(
    config.WithStore(store),           // Required: storage backend
    config.WithCodec(yamlCodec),       // Optional: default codec (default: JSON)
    config.WithLogger(slogLogger),     // Optional: custom logger
)

License

MIT License