Skip to content

actforgood/xconf

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Xconf

Build Status License Coverage Status Goreportcard Go Reference


Package xconf provides a configuration registry for an application.
Configurations can be extracted from a file / env / flag set / remote system.
Supported formats are json, yaml, ini, (java) properties, toml, plain.

Installation

$ go get -u github.com/actforgood/xconf

Configuration loaders

You can create your own configuration retriever implementing Loader interface. Package provides these Loaders for you:

  • EnvLoader - loads environment variables.
  • DotEnvFileLoader, DotEnvReaderLoader - loads configuration from a .env file / io.Reader.
  • JSONFileLoader, JSONReaderLoader - loads json configuration from a file / io.Reader.
  • YAMLFileLoader, YAMLReaderLoader - loads yaml configuration from a file / io.Reader.
  • IniFileLoader - loads ini configuration from a file.
  • PropertiesFileLoader, PropertiesBytesLoader - loads java style properties configuration from a file / bytes slice.
  • TOMLFileLoader, TOMLReaderLoader - loads toml configuration from a file / io.Reader.
  • ConsulLoader - loads json/yaml/plain configuration from a remote Consul KV Store.
  • EtcdLoader - loads json/yaml/plain configuration from a remote Etcd KV Store.
  • PlainLoader - explicit configuration provider.
  • FileLoader - factory for <JSON|YAML|Ini|DotEnv|Properties|TOML>FileLoaders based on file extension.
  • FlagSetLoader - extracts configuration from a flag.FlagSet.
  • MultiLoader - loads (and merges, if configured) configuration from multiple loaders.

Upon above loaders there are available decorators which can help you achieve more sophisticated outcome:

  • FilterKVLoader - filters other loader's configurations (based on keys and or their values).
    Example of applicability: I load configurations from environment, but I only want the ones prefixed with "MY_APP_" - I can apply this loader with FilterKVWhitelistFunc(FilterKeyWithPrefix("MY_APP_") filter function.
  • AlterValueLoader - changes the value for a configuration key.
    Example of applicability: I load configurations from environment and for a given key I want its value to be a slice (not a string as envs are read/stored by default) - I can apply this loader with ToStringList altering function.
  • IgnoreErrorLoader - ignores the error returned by another loader.
    Example of applicability: I load configuration from environment and from file (using a MultiLoader), but it's not mandatory for that file to exist (file it's just an auxiliary source for my configurations, that may exist) - I can use this loader to ignore "file does not exist" error.
  • FileCacheLoader - caches configuration from a [X]FileLoader until file gets modified (to be used if loader is called multiple times).
  • FlattenLoader - creates easy to access nested configuration leaf keys symlinks.
  • AliasLoader - creates aliases for other keys.

Configuration contract

The main configuration contract this package provides looks like:

type Config interface {
	Get(key string, def ...any) any
}

with a default implementation obtained with:

// NewDefaultConfig instantiates a new default config object.
// The first parameter is the loader used as a source of getting the key-value configuration map.
// The second parameter represents a list of optional functions to configure the object.
func NewDefaultConfig(loader Loader, opts ...DefaultConfigOption) (*DefaultConfig, error)

The DefaultConfig has an option of reloading configurations (interval based), if you want to retrieve updated configuration at runtime. There are 2 (proposed) ways of working with it:

  • injecting a Config reference and calling Get(key) every time you need a configuration.
  • registering your class as an observer to get notified about config changes.

Example of usage (first case) (note: code does not compile):

// cart_service.go
const (
	defaultMaxQtyCfgVal uint = 100
	maxQtyCfgKey             = "MAX_ALLOWED_QTY_TO_ORDER"
)

type CartService struct {
	config xconf.Config
}

func NewCartService(config xconf.Config) *CartService {
	return &CartService{
		config: config,
	}
}

func (cartSvc *CartService) AddProduct(sku string, qty uint) error {
	// ...
	if customerType != B2B {
		totalQty := currentQty + qty
		maxQty := cartSvc.config.Get(maxQtyCfgKey, defaultMaxQtyCfgVal).(uint)
		if totalQty > maxQty {
			return ErrMaxQtyExceeded
		}
	}
	// ...
	return nil
}

func main() {
	// somewhere in the bootstrap of your application ...
	var (
		loader  xconf.Loader // = ... your desired source(s)
		config  xconf.Config
		cartSvc *CartService
	)
	config, err := xconf.NewDefaultConfig(
		loader,
		xconf.DefaultConfigWithReloadInterval(time.Minute), // reload every minute
	)
	if err != nil {
		panic(err)
	}
	cartSvc = NewCartService(config)

	// somewhere in the application business flow ...
	_ = cartSvc.AddProduct("IPHONE", 1)

	// somewhere in the shutdown of your application ...
	if closableConfig, ok := config.(io.Closer); ok {
		_ = closableConfig.Close()
	}
}

Example of usage (second case) (note: code does not compile):

// redis_wrapper.go
const (
	RedisHostCfgKey = "REDIS_HOST"
	DefaultRedisHostCfgVal = "127.0.0.1:6379"
)

type RedisClient interface {
	Ping() error
	Get(key string) (string, error)
	Set(key string, value any, expiration time.Duration) (string, error)
	Close() error
}

type RedisClientWrapper struct {
	client *redis.Client // official client
	mu     sync.RWMutex
}

func NewRedisClientWrapper(host string) *RedisClientWrapper {
	officialClient = ...
	return &RedisClientWrapper {
		client: officialClient,
	}
}

func (wrapper *RedisClientWrapper) Get(key string) (string, error) {
	wrapper.mu.RLock()
	defer wrapper.mu.RUnlock()

	return wrapper.client.Get(key).Result()
}

func (wrapper *RedisClientWrapper) OnConfigChange(config xconf.Config, changedKeys ...string) {
	for _, changedKey := range changedKeys {
		if changedKey == RedisHostCfgKey { // or use strings.EqualFold() if you enabled DefaultConfigWithIgnoreCaseSensitivity.
			wrapper.mu.Lock()
			_ = wrapper.client.Close() // close previous client
			newClient := ... // reinitialize client based on config.Get(RedisHostCfgKey).(string)
			wrapper.client = newClient
			wrapper.mu.Unlock()
		}
	}
}

func main() {
	// somewhere in the bootstrap of your application ...
	var (
		loader      xconf.Loader // = ... your desired source(s)
		config      xconf.Config
		redisClient RedisClient
	)
	config, err := xconf.NewDefaultConfig(
		loader,
		xconf.DefaultConfigWithReloadInterval(30 * time.Second), // reload every 30 seconds
	)
	if err != nil {
		panic(err)
	}
	redisHost := config.Get(RedisHostCfgKey, DefaultRedisHostCfgVal).(string)
	redisClient = NewRedisClient(redisHost)
	config.RegisterObserver(redisClient.OnConfigChange) // register redis wrapper as an observer

	// somewhere in the application business flow ...
	_, _ = redisClient.Get("something")

	// somewhere in the shutdown of your application ...
	if closableConfig, ok := config.(io.Closer); ok {
		_ = closableConfig.Close()
	}
	_ = redisClient.Close()
}

Unmarshal configuration map to structs

This is not the subject of this package, but as a mention, you can achieve that if needed, with a package like github.com/mitchellh/mapstructure.
Example:

package main

import (
	"bytes"
	"fmt"

	"github.com/actforgood/xconf"
	"github.com/mitchellh/mapstructure"
)

type DBConfig struct {
	Host string
	Port int
	Auth Auth
}

type Auth struct {
	Username string
	Password string
}

func main() {
	var (
		jsonConfig = `{
	"db": {
		"host": "127.0.0.1",
		"port": 3306,
		"auth": {
			"username": "JohnDoe",
			"password": "verySecretPwd"
		}
	}		
}`
		dbConfig    DBConfig               // the struct to populate with configuration
		dbConfigMap map[string]any // the configuration map for "db" key
		loader      = xconf.JSONReaderLoader(bytes.NewReader([]byte(jsonConfig)))
	)

	// example using directly a Loader:
	configMap, err := loader.Load()
	if err != nil {
		panic(err)
	}
	dbConfigMap = configMap["db"].(map[string]any)
	if err := mapstructure.Decode(dbConfigMap, &dbConfig); err != nil {
		panic(err)
	}
	fmt.Printf("%+v", dbConfig)

	// example using the Config contract:
	config, err := xconf.NewDefaultConfig(loader)
	if err != nil {
		panic(err)
	}
	dbConfigMap = config.Get("db").(map[string]any)
	if err := mapstructure.Decode(dbConfigMap, &dbConfig); err != nil {
		panic(err)
	}
	fmt.Printf("%+v", dbConfig)

	// both Printf will produce: {Host:127.0.0.1 Port:3306 Auth:{Username:JohnDoe Password:verySecretPwd}}
}

TODOs

Things that can be added to package, extended:

  • Support more formats (like HCL)
  • Add also a writer/persister functionality (currently you can only read configurations) to different sources and formats (JSONFileWriter/YAMLFileWriter/EtcdWriter/ConsulWriter/...) implementing a common contract like:
type ConfigWriter interface {
	Write(configMap map[string]any) error
}
  • Add a typed struct with methods like GetString, GetInt...

Misc

  • Feel free to use this pkg if you like it and fits your needs. Check also other packages like spf13/viper ...
  • To run unit tests: make test / make cover .
  • To run integration tests: make test-integration / make cover-integration : will setup Consul and Etcd docker containers with some keys in them, run ./scripts/teardown_dockers.sh at the end to stop and remove containers).
  • To run benchmarks: make bench.
  • Project's class diagram can be found here.

License

This package is released under a MIT license. See LICENSE.
Other 3rd party packages directly used by this package are released under their own licenses.