Photo by Ashley McNamara, via ashleymcnamara/gophers (CC BY-NC-SA 4.0)
A flexible multi-layered caching library interacts with private (in-memory) cache and shared cache (i.e. Redis) in Go. It provides Cache-Aside strategy when dealing with both, and maintains the consistency of private cache between distributed systems by Pub-Sub pattern.
Caching is a common technique that aims to improve the performance and scalability of a system. It does this by temporarily copying frequently accessed data to fast storage close to the application. Distributed applications typically implement either or both of the following strategies when caching data:
- Using a private cache, where data is held locally on the computer that's running an instance of an application or service.
- Using a shared cache, serving as a common source that can be accessed by multiple processes and machines.
Ref: https://docs.microsoft.com/en-us/azure/architecture/best-practices/images/caching/caching3.png
Considering the flexibility, efficiency and consistency, we starts to build up our own framework.
- Easy to use : provide a friendly interface to deal with both caching mechnaism by simple configuration. Limit the size of memory on single instance (pod) as well.
- Maintain consistency : evict keys between distributed systems by
Pub-Subpattern. - Data compression : provide customized marshal and unmarshal functions.
- Fix concurrency issue : prevent data racing happened on single instance (pod).
- Metric : provide callback functions to measure the performance. (i.e. hit rate, private cache usage, ...)
sequenceDiagram
participant APP as Application
participant M as go-cache
participant L as Local Cache
participant S as Shared Cache
participant R as Resource (Microservice / DB)
APP ->> M: Cache.Get() / Cache.MGet()
alt Local Cache hit
M ->> L: Adapter.MGet()
L -->> M: {[]Value, error}
M -->> APP: return
else Local Cache miss but Shared Cache hit
M ->> L: Adapter.MGet()
L -->> M: cache miss
M ->> S: Adapter.MGet()
S -->> M: {[]Value, error}
M ->> L: Adapter.MSet()
M -->> APP: return
else All miss
M ->> L: Adapter.MGet()
L -->> M: cache miss
M ->> S: Adapter.MGet()
S -->> M: cache miss
M ->> R: OneTimeGetterFunc() / MGetterFunc()
R -->> M: return from getter
M ->> S: Adapter.MSet()
M ->> L: Adapter.MSet()
M -->> APP: return
end
sequenceDiagram
participant APP as Application
participant M as go-cache
participant L as Local Cache
participant S as Shared Cache
participant PS as PubSub
APP ->> M: Cache.Del()
M ->> S: Adapter.Del()
S -->> M: return error if necessary
M ->> L: Adapter.Del()
L -->> M: return error if necessary
M ->> PS: Pubsub.Pub() (broadcast key eviction)
M -->> APP: return nil or error
go get github.com/viney-shih/go-cacheBy adopting Singleton pattern, initialize the Factory in main.go at the beginning, and deliver it to each package or business logic.
// Initialize the Factory in main.gotinyLfu:=cache.NewTinyLFU(10000)
rds:=cache.NewRedis(redis.NewRing(&redis.RingOptions{
Addrs: map[string]string{
"server1": ":6379",
},
}))
cacheFactory:=cache.NewFactory(rds, tinyLfu)Treat it as a common key:value store like Redis. But more advanced, it coordinated the usage between multi-layered caching mechanism inside.
typeObjectstruct {
StrstringNumint
}
funcExample_setAndGetPattern() {
// We create a group of cache named "set-and-get".// It uses the shared cache only. Each key will be expired within ten seconds.c:=cacheFactory.NewCache([]cache.Setting{
{
Prefix: "set-and-get",
CacheAttributes: map[cache.Type]cache.Attribute{
cache.SharedCacheType: {TTL: 10*time.Second},
},
},
})
ctx:=context.TODO()
// set the cacheobj:=&Object{
Str: "value1",
Num: 1,
}
iferr:=c.Set(ctx, "set-and-get", "key", obj); err!=nil {
panic("not expected")
}
// read the cachecontainer:=&Object{}
iferr:=c.Get(ctx, "set-and-get", "key", container); err!=nil {
panic("not expected")
}
fmt.Println(container) // Output: Object{ Str: "value1", Num: 1}// read the cache but failediferr:=c.Get(ctx, "set-and-get", "no-such-key", container); err!=nil {
fmt.Println(err) // Output: errors.New("cache key is missing")
}
// Output:// &{value1 1}// cache key is missing
}GetByFunc() is the easier way to deal with the cache by implementing the getter function in the parameter. When the cache is missing, it is going to refill the cache automatically.
funcExampleCache_GetByFunc() {
// We create a group of cache named "get-by-func".// It uses the local cache only with TTL of ten minutes.c:=cacheFactory.NewCache([]cache.Setting{
{
Prefix: "get-by-func",
CacheAttributes: map[cache.Type]cache.Attribute{
cache.LocalCacheType: {TTL: 10*time.Minute},
},
MarshalFunc: msgpack.Marshal, // msgpack is from "github.com/vmihailenco/msgpack/v5"UnmarshalFunc: msgpack.Unmarshal,
},
})
ctx:=context.TODO()
container2:=&Object{}
iferr:=c.GetByFunc(ctx, "get-by-func", "key2", container2, func() (interface{}, error) {
// The getter is used to generate data when cache missed, and refill the cache automatically..// You can read from DB or other microservices.// Assume we read from MySQL according to the key "key2" and get the value of Object{Str: "value2", Num: 2}returnObject{Str: "value2", Num: 2}, nil
}); err!=nil {
panic("not expected")
}
fmt.Println(container2) // Object{ Str: "value2", Num: 2}// Output:// &{value2 2}
}MGetter is another approaching way to do this. Set this function durning registering the Setting.
funcExampleService_Create_mGetter() {
// We create a group of cache named "mgetter".// It uses both shared and local caches with separated TTL of one hour and ten minutes.c:=cacheFactory.NewCache([]cache.Setting{
{
Prefix: "mgetter",
CacheAttributes: map[cache.Type]cache.Attribute{
cache.SharedCacheType: {TTL: time.Hour},
cache.LocalCacheType: {TTL: 10*time.Minute},
},
MGetter: func(keys...string) (interface{}, error) {
// The MGetter is used to generate data when cache missed, and refill the cache automatically..// You can read from DB or other microservices.// Assume we read from MySQL according to the key "key3" and get the value of Object{Str: "value3", Num: 3}// HINT: remember to return as a slice, and the item order needs to consist with the keys in the parameters.return []Object{{Str: "value3", Num: 3}}, nil
},
MarshalFunc: cache.Marshal,
UnmarshalFunc: cache.Unmarshal,
},
})
ctx:=context.TODO()
container3:=&Object{}
iferr:=c.Get(ctx, "mgetter", "key3", container3); err!=nil {
panic("not expected")
}
fmt.Println(container3) // Object{ Str: "value3", Num: 3}// Output:// &{value3 3}
}- https://docs.microsoft.com/en-us/azure/architecture/best-practices/caching
- https://github.com/vmihailenco/go-cache-benchmark
- https://github.com/go-redis/cache
