ruecache

ruecache implements a fast redis-backed self-filling model cache, built on top of rueidis with auto-pipelining.

Godoc

Features

• Self-filling: Invokes a fetch() func whenever the cache needs to be filled.
• Multi-item get and fill:: Supports getting multiple items at a time, fetching cache misses in a single pass.
• ID to cache key mapping: Caller's API deals in item ids, not cache keys.
• Cache stampede protection: Only one instance will attempt to fill the cache any given id. Implemented via SET NX PX locks.
• Stale write prevention: If the cache is invalidated during a fill, fetched data - which may now be stale - is not written to the cache. This is carefully managed using temporary version keys
• High throughput: rueidis auto-pipelining allows for a very high op/sec.

Goals

1. Work well as a transparent service wrapper. Wrap GetWidgetsById(ids []ID) map[ID]Widget-style interfaces with a cache, preserving and caching nil values and key presence.
2. Confidently prevent race conditions that could lead to stale data being returned, or worse, written to the cache.
3. Horizontally scalability, including working with Redis when sharded by hash slot.

Usage example

Instantiating the cache:

cache, err:=NewCache(
client,
ruecache.CacheOpts{
DataKey: func(widgetIdstring) string {
return"widgets:{"+widgetId+"}"
},
DataTTL: 5*time.Minute,
},
// Fetch function, invoked when widgets are needed that are not cachedfunc(ctx context.Context, widgetIds []string) (map[string][]byte, error) {
// Fetch multiple items at a time from the databasemodels, err:=db.GetWidgets(ctx, widgetIds)
iferr!=nil {
returnnil, err
}
// Serialize to bytes for persisting to the cachemodelBytes:=make(map[string][]byte)
forid, widgetModel:=rangemodels {
modelBytes[id] =widgetModel.MarshalBinary()
}
returnmodelBytes, nil
},
)
x:=&CachingWidgetService{cache: cache, db: db}

Using the cache: Example shows cache wrapping an existing service interface that's implemented by a database. Note that the GetWidgets signature matches the signature used in the Fetch function above.

func (x*CachingWidgetService) GetWidgets(ctx context.Context, widgetIds []string) (map[string]*Widget, error)
// Fetch model bytes from the cache. Will fill using Fetch function for cache misses.modelBytes, err:=x.cache.GetAndFill(ctx, widgetIds)
iferr!=nil {
returnnil, fmt.Errorf("cache: %w", err)
}
// Deserialize bytes back into Widget models.models:=make(map[string]*Widget)
forid, b:=rangemodelBytes {
models[id] =b.UnmarshalBinary()
}
returnmodels
}
func (x*CachingWidgetService) UpdateWidget(ctx context.Context, widgetIdstring, updated*Widget) error {
// Update the widget in the database, and regardless of the outcome, invalidate the cache// -- if db.UpdateWidget fails, the state of the db will be unknown.returnerrors.Join(
db.UpdateWidget(ctx, widgetId, updated),
cache.Invalidate(widgetId),
)
}

EncodedCache

A convenience wrapper, EncodedCache[T], exists to reduce serializing/deserializing boilerplate. The logic is delegated to a CacheCoder (in this case WidgetCacheCoder).

coder:=WidgetCacheCoder{}
cache, err:=NewEncodedCache[map[string]*Widget](
client,
coder,
ruecache.CacheOpts{
DataKey: func(widgetIdstring) string {
return"widgets:{"+widgetId+"}"
},
DataTTL: 5*time.Minute,
},
// NEW: Fetch function returns map[string]*Widget, not map[string][]bytefunc(ctx context.Context, widgetIds []string) (map[string]*Widget, error) {
returndb.GetWidgets(ctx, widgetIds)
},
)

func (x*CachingWidgetService) GetWidgets(ctx context.Context, widgetIds []string) (map[string]*Widget, error)
// NEW: cache.GetAndFill returns map[string]*Widget, not map[string][]bytereturnx.cache.GetAndFill(ctx, widgetIds)
}

typeWidgetCacheCoderstruct {}
func (WidgetCacheCoder) Encode(decodedmap[string]*Widget) (encodedmap[string][]byte, errerror) {
// Marshal widgets to bytes
}
func (WidgetCacheCoder) Decode(encodedmap[string][]byte) (decodedmap[string]*Widget, errerror) {
// Unmarshal bytes to widgets
}

As a further convenience, ProtoMapCoder implements this encoding logic for maps of protobufs. e.g.:

// Implements encoding to/from map[string]*Widget <-> map[string][]bytecoder:=ProtoMapCoder[*Widget]{}

FAQ

Do I have to encode from/decode to map[ID]Model?

You can use any data structure you like, as long as items have a unique identifier. For example:

cache, err:=NewEncodedCache[[]*Widget](
// ...WidgetSliceCoder{}
// NEW: Fetch function returns a slice of widgets, not a mapfunc(ctx context.Context, widgetIds []string) ([]*Widget, error) {
returndb.GetWidgets(ctx, widgetIds)
},
)
// WidgetSliceCoder deals in slices of Widgets, which are self-identifying with a WidgetId field.typeWidgetSliceCoderstruct {}
func (WidgetSliceCoder) Encode(decoded []*Widget) (encodedmap[string][]byte, errerror) {
encoded=make(map[string][]byte)
for_, w:=rangedecoded {
encoded[w.WidgetId] =w.MarshalBinary()
}
return
}
func (WidgetSliceCoder) Decode(encodedmap[string][]byte) (decoded []*Widget, errerror) {
for_, b:=rangeencoded {
w:=new(Widget)
w.UnmarshalBinary(b)
decoded=append(decoded, w)
}
return
}

Can I use this for more generalised caching, e.g. not just "models"?

Yes -- ruecache can be used for caching any kind of data or operation as long as it can be uniquely identified somehow.

This example uses ruecache to ensure an image is only resized by one server at a time, identifying: the image by its originalImageUrl:

cache, err:=NewCache(
client,
ruecache.CacheOpts{
DataKey: func(originalImageUrlstring) string {
return"resized-images:{"+md5(originalImageUrl) +"}:url"
},
DataTTL: ruecache.NoTTL,
},
// Uses FetchOne helper for single-item fillingruecache.FetchOne(func(ctx context.Context, originalImageUrlstring) ([]byte, error) {
resizedImageUrl, err:=imageutils.ResizeAndUpload(originalImageUrl, "512x512px")
iferr!=nil {
returnnil, fmt.Errorf("resizing image: %w", err)
}
return []byte(resizedImageUrl), nil
}),
)
resizedImageUrl, _:=cache.GetAndFillOne(ctx, "https://s3.amazonaws.com/images/my-image.png")

TODOs

1. Implement local optimisation for stampede protection using SingleFlight or similar.
2. Subscribe to lock releases, rather than polling with backoff.
3. Write data post-fetch even if context has been cancelled (don't throw away the fetch needlessly)

Benchmarks

Run with high (512) parallelism in order to benchmark the cache, not the RTT. Benchmarks run against redis 8.0 running locally.

goos: linux
goarch: amd64
pkg: github.com/iamcalledrob/ruecache
cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
BenchmarkCache_GetAndFill
BenchmarkCache_GetAndFill/p=512/Hit
BenchmarkCache_GetAndFill/p=512/Hit-8 627006 1812 ns/op
BenchmarkCache_GetAndFill/p=512/Miss
BenchmarkCache_GetAndFill/p=512/Miss-8 74701 17033 ns/op