Skip to content

21strive/redifu

Repository files navigation

                █████████████           █████████                
                ████████████          ███████████                
                ██████████          █████████████                
                ███████           ███████████████                
                █████████████████████████████████                
                ████████████████         ████████                
                ██████████████           ████████                
                ███████████              ████████                
                █████████                ████████                

redifu

Go library for a Redis-backed data layer with singleton behavior — updating an entity once automatically reflects across every collection that references it, with no data duplication.

Installation

go get github.com/21strive/redifu

Mental Model

All data is stored in two layers:

Base[T]       → key-value, one item per key (source of truth)
SortedSet     → sorted set, stores only randId as member

On every fetch, redifu always: fetches randIds from sorted set → fetches each item from Base → resolves Relations.

This means: update an item once in Base, and every collection referencing it immediately shows the latest version.

Data Structures

Base[T] — individual item

Use for get/set of a single item by ID.

postBase := redifu.NewBase[Post](redisClient, "post:%s", 7*24*time.Hour)

// Set
postBase.Set(ctx, post)

// Get
post, err := postBase.Get(ctx, randId)

// Mark as missing (avoid repeated DB hits for non-existent items)
postBase.MarkAsMissing(ctx, randId)
missing, _ := postBase.IsMissing(ctx, randId)

Timeline[T] — cursor pagination (infinite scroll)

Use for "load more" feeds. Manages page state (first/last/empty) automatically.

postTimeline := redifu.NewTimeline[Post](
    redisClient,
    postBase,
    "feed:user:%s:posts",   // key format — %s is filled by keyParams
    20,                      // items per page
    redifu.Descending,
    2*24*time.Hour,
)

// Add item
postTimeline.AddItem(ctx, post, userRandId)

// Fetch — lastRandIds is an array to tolerate items deleted from cache
output   := postTimeline.Fetch(lastRandIds).WithParams(userRandId).Exec(ctx)
items    := output.Items()
nextId   := output.ValidLastId()   // send to client, used as the next cursor
position := output.Position()      // FirstPage / MiddlePage / LastPage

// Remove item
postTimeline.RemoveItem(ctx, post, userRandId)

Sorted[T] — full sorted collection

Use for collections that are always fetched in full, or queried by score range.

commentSorted := redifu.NewSorted[Comment](
    redisClient,
    commentBase,
    "post:%s:comments",
    7*24*time.Hour,
)

// Add
commentSorted.AddItem(ctx, comment, postRandId)

// Fetch all
items, err := commentSorted.Fetch(redifu.Ascending).WithParams(postRandId).Exec(ctx)

// Fetch by score range (e.g. by timestamp)
items, err := commentSorted.Fetch(redifu.Ascending).
    WithParams(postRandId).
    WithRange(lowerUnixMilli, upperUnixMilli).
    Exec(ctx)

Page[T] — numbered pagination

Use for numbered page pagination (page 1, 2, 3).

productPage := redifu.NewPage[Product](
    redisClient,
    productBase,
    "category:%s:products",
    20,               // items per page
    redifu.Ascending,
    2*24*time.Hour,
)

// Fetch a specific page
items, err := productPage.Fetch(pageNumber).WithParams(categoryRandId).Exec(ctx)

// Check whether seeding is needed
needsSeed, _ := productPage.RequiresSeeding(ctx, pageNumber, categoryRandId)

TimeSeries[T] — query by time range

Use for historical data, charts, and reports. Tracks seeded segments and detects gaps automatically.

txTimeSeries := redifu.NewTimeSeries[Transaction](
    redisClient,
    txBase,
    "account:%s:transactions",
    7*24*time.Hour,
)

// Fetch — returns needsSeed=true if any gaps exist in the requested range
items, needsSeed, err := txTimeSeries.
    Fetch(startTime, endTime).
    WithParams(accountRandId).
    Exec(ctx)

Relation — Singleton Behavior

Relations prevent data duplication across entities that reference each other.

Field naming convention (required):

type Post struct {
    redifu.Record
    Title         string
    AuthorRandId  string          // reference: FieldName + "RandId"
    Author        account.Account // related entity field
}

Author is not stored inside Post. On fetch, redifu reads AuthorRandId, retrieves Author from Base[Account], and injects it into the Author field. Because Base[Account] is the source of truth, any update to Account is immediately reflected in all Post records.

Setup:

authorRelation, err := redifu.NewRelation[account.Account](
    account.AccountBase,
    redifu.TypeOf[Post](),
)
if err != nil {
    log.Fatal(err)
}
postTimeline.AddRelation("author", authorRelation)

Seeder

Seeders populate Redis from a SQL database when a key has no data. Standard pattern:

func GetFeed(ctx context.Context, userRandId string, lastRandIds []string) ([]Post, error) {
    needsSeed, err := postTimeline.RequiresSeeding(ctx, int64(len(lastRandIds)), userRandId)
    if err != nil {
        return nil, err
    }

    if needsSeed {
        seeder := redifu.NewTimelineSeeder[Post](redisClient, db, postBase, postTimeline)
        err = seeder.
            Seed(0, "", queryBuilder).
            WithQueryArgs(userRandId).
            WithParams(userRandId).
            Exec(ctx, scanPostRow, scanPostRows)
        if err != nil {
            return nil, err
        }
    }

    output := postTimeline.Fetch(lastRandIds).WithParams(userRandId).Exec(ctx)
    return output.Items(), output.Error()
}

Seeders are available for all structures: NewTimelineSeeder, NewSortedSeeder, NewPageSeeder, NewTimeSeriesSeeder.


Default TTL

TTL
Individual item (Base) 7 days
Sorted set / Timeline 2 days

TTL is configurable at initialization.

Limitations

  • The built-in query builder only supports PostgreSQL ($1, $2, ...). For MySQL or complex queries, write SQL directly.
  • Sorting only supports fields of type time.Time or int64.

Using with Claude Code

Redifu ships with a CLAUDE.consumer.md template that teaches Claude Code to use redifu as the default Redis data layer across your Go backend projects — without you having to explain it each time.

Setup (one-time per project)

1. Copy the template into your project:

cp path/to/redifu/CLAUDE.consumer.md your-project/CLAUDE.md

Or create a new CLAUDE.md and paste the contents of CLAUDE.consumer.md into it.

2. Customize the conventions section at the bottom of the file to match your project:

## Project conventions

- Individual item TTL: **7 days**       ← adjust as needed
- Sorted set / Timeline TTL: **2 days** ← adjust as needed
- Default `itemPerPage`: **20**         ← adjust as needed
- All redifu clients are initialized in a single `redis.go` or `cache.go` per domain
- Seeders are called at the service/handler layer, not in the repository layer

What Claude Code will do automatically

Once CLAUDE.md is in place, Claude Code will:

  • Use Base[T] whenever a new get-by-ID endpoint is added
  • Propose Timeline[T] for any feed or "load more" list
  • Propose Page[T] for numbered pagination
  • Propose TimeSeries[T] for any date-range data query
  • Wire up Relation correctly when one entity references another
  • Generate scanner functions, seeder calls, and ResetPagination handling
  • Never write redisClient.Set / redisClient.Get directly for entity data

Example prompt after setup

Add a posts feed for each user. The feed should load 20 posts at a time,
sorted by creation date descending. Posts have an author (Account entity).
The existing SQL query is:

  SELECT p.randid, p.title, p.content, p.author_randid, p.created_at
  FROM posts p
  WHERE p.user_id = $1 AND p.status = 'active'
  ORDER BY p.created_at DESC

Claude Code will generate the full integration: PostBase, PostTimeline, Relation wiring, scanner functions, seeder, and fetch handler — all using redifu patterns.

About

Unified SQL database-to-Redis solution with support for PostgreSQL and MySQL.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages