Go Config is a pluggable dynamic config library.
Most config in applications are statically configured or include complex logic to load from multiple sources. Go Config makes this easy, pluggable and mergeable. You'll never have to deal with config in the same way again.
- Dynamic - load config on the fly as you need it
- Pluggable - choose which source to load from; file, env, consul
- Mergeable - merge and override multiple config sources
- Observable - Watch the config for changes
- Safe - specify fallback values where keys don't exist
- Source - A backend from which config is loaded
- Encoder - Handles encoding/decoding source config
- Reader - Merges multiple encoded sources as a single format
- Loader - Loader manages multiple source loading
- Config - Config abstracts away the above interfaces
- Usage - Example usage of go-config
- FAQ - General questions and answers
- TODO - TODO tasks/features
A Source is a backend from which config is loaded. Multiple sources can be used at the same time.
The following sources are supported:
- configmap - read from k8s configmap
- consul - read from consul
- etcd - read from etcd v3
- env - read from environment variables
- file - read from file
- flag - read from flags
- grpc - read from grpc server
- memory - read from memory
- microcli - read from micro cli flags
- runtimevar - read from Go Cloud runtimevar watcher
TODO:
- vault
- git url
Sources return config as a ChangeSet. This is a single internal abstraction for multiple backends.
type ChangeSet struct {
// Raw encoded config data
Data []byte
// MD5 checksum of the data
Checksum string
// Encoding format e.g json, yaml, toml, xml
Format string
// Source of the config e.g file, consul, etcd
Source string
// Time of loading or update
Timestamp time.Time
}An Encoder handles source config encoding/decoding. Backend sources may store config in many different
formats. Encoders give us the ability to handle any format. If an Encoder is not specified it defaults to json.
The following encoding formats are supported:
- json
- yaml
- toml
- xml
- hcl
A Reader represents multiple changesets as a single merged and queryable set of values.
type Reader interface {
// Merge multiple changeset into a single format
Merge(...*source.ChangeSet) (*source.ChangeSet, error)
// Return returns Go assertable values
Values(*source.ChangeSet) (Values, error)
// Name of the reader e.g a json reader
String() string
}The reader makes use of Encoders to decode changesets into map[string]interface{} then merge them into
a single changeset. It looks at the Format field to determine the Encoder. The changeset is then represented
as a set of Values with the ability to retrieve Go types and fallback where values cannot be loaded.
// Values is returned by the reader
type Values interface {
// Return raw data
Bytes() []byte
// Retrieve a value
Get(path ...string) Value
// Return values as a map
Map() map[string]interface{}
// Scan config into a Go type
Scan(v interface{}) error
}The Value interface allows casting/type asserting to Go types with fallback defaults.
type Value interface {
Bool(def bool) bool
Int(def int) int
String(def string) string
Float64(def float64) float64
Duration(def time.Duration) time.Duration
StringSlice(def []string) []string
StringMap(def map[string]string) map[string]string
Scan(val interface{}) error
Bytes() []byte
}Loader manages loading from multiple sources and representing them as a single snapshot.
// Loader manages loading sources
type Loader interface {
// Stop the loader
Close() error
// Load the sources
Load(...source.Source) error
// A Snapshot of loaded config
Snapshot() (*Snapshot, error)
// Force sync of sources
Sync() error
// Watch for changes
Watch(...string) (Watcher, error)
// Name of loader
String() string
}Config is the high level abstraction over all the underlying functionality. It provides a queryable top-level interface.
// Config is an interface abstraction for dynamic configuration
type Config interface {
// provide the reader.Values interface
reader.Values
// Stop the config loader/watcher
Close() error
// Load config sources
Load(source ...source.Source) error
// Force a source changeset sync
Sync() error
// Watch a value for changes
Watch(path ...string) (Watcher, error)
}- Sample Config
- New Config
- Load File
- Read Config
- Read Values
- Watch Path
- Multiple Sources
- Set Source Encoder
- Add Reader Encoder
A config file can be of any format as long as we have an Encoder to support it.
Example json config:
{
"hosts": {
"database": {
"address": "10.0.0.1",
"port": 3306
},
"cache": {
"address": "10.0.0.2",
"port": 6379
}
}
}Create a new config (or just make use of the default instance).
import "github.com/micro/go-config"
conf := config.NewConfig()Load config from a file source. It uses the file extension to determine config format.
import (
"github.com/micro/go-config"
"github.com/micro/go-config/source/file"
)
// Load json config file
config.Load(file.NewSource(
file.WithPath("/tmp/config.json"),
))Load a yaml, toml or xml file by specifying a file with the appropriate file extension.
// Load yaml config file
config.Load(file.NewSource(
file.WithPath("/tmp/config.yaml"),
))If an extension does not exist, specify the encoder.
enc := toml.NewEncoder()
// Load toml file with encoder
config.Load(file.NewSource(
file.WithPath("/tmp/config"),
source.WithEncoder(enc),
))Read the entire config as a map.
// retrieve map[string]interface{}
conf := config.Map()
// map[cache:map[address:10.0.0.2 port:6379] database:map[address:10.0.0.1 port:3306]]
fmt.Println(conf["hosts"])Scan the config into a struct.
type Host struct {
Address string `json:"address"`
Port int `json:"port"`
}
type Config struct{
Hosts map[string]Host `json:"hosts"`
}
var conf Config
config.Scan(&conf)
// 10.0.0.1 3306
fmt.Println(conf.Hosts["database"].Address, conf.Hosts["database"].Port)Scan a value from the config into a struct.
type Host struct {
Address string `json:"address"`
Port int `json:"port"`
}
var host Host
config.Get("hosts", "database").Scan(&host)
// 10.0.0.1 3306
fmt.Println(host.Address, host.Port)Read individual values as Go types.
// Get address. Set default to localhost as fallback
address := config.Get("hosts", "database", "address").String("localhost")
// Get port. Set default to 3000 as fallback
port := config.Get("hosts", "database", "port").Int(3000)Watch a path for changes. When the file changes the new value will be made available.
w, err := config.Watch("hosts", "database")
if err != nil {
// do something
}
// wait for next value
v, err := w.Next()
if err != nil {
// do something
}
var host Host
v.Scan(&host)Multiple sources can be loaded and merged. Merging priority is in reverse order.
config.Load(
// base config from env
env.NewSource(),
// override env with flags
flag.NewSource(),
// override flags with file
file.NewSource(
file.WithPath("/tmp/config.json"),
),
)A source requires an encoder to encode/decode data and specify the changeset format.
The default encoder is json. To change the encoder to yaml, xml, toml specify as an option.
e := yaml.NewEncoder()
s := consul.NewSource(
source.WithEncoder(e),
)The reader uses encoders to decode data from sources with different formats.
The default reader supports json, yaml, xml, toml and hcl. It represents the merged config as json.
Add a new encoder by specifying it as an option.
e := yaml.NewEncoder()
r := json.NewReader(
reader.WithEncoder(e),
)Viper and go-config are solving the same problem. Go Config provides a different interface and is part of the larger micro ecosystem of tooling.
The encoder is used by a backend source to encode/decode its data. The reader uses encoders to decode data from multiple sources with different formats, it then merges them into a single encoding format.
In the case of a file source, we use the file extension to determine the config format so the encoder is not used.
In the case of consul, etcd or similar key-value source we may load from a prefix containing multiple keys which means the source needs to understand the encoding so it can return a single changeset.
In the case of environment variables and flags we also need a way to encode the values as bytes and specify the format so it can later be merged by the reader.
In some cases source data may not actually be key-value so it's easier to represent it as bytes and defer decoding to the reader.
- Config Versioning - version the internal changeset and allow the user to specify version
- Canarying/Coordination - allow coordination of instances that subscribe to config and canary an update