QBT - Query BigTable

QBT is a small CLI application for querying Google Cloud BigTable, using the Lua programming language to express queries.

Why

cbt is a great tool for managing BigTable tables and column familes, but it's read subcomand is somewhat limited in that it doesn't allow you to provide any filters or transformations to your data.

The other option for ad-hoc queries into BigTable is to write custom scripts in your language of choice, which is quite time-consuming for exploratory queries.

Installation

Requirements

• Go 1.9+
• gcloud CLI

go get -u github.com/catkins/qbt
gcloud auth applicatiom-default login

Usage

qbt [--project=<project id>] [--instance=<instance id>] [--prefix=<row key prefix>] query <table> <query>

Example

qbt query my_table 'return row.my_cf["my_cf:haystack"] == "needle"'

# json decoding also included
qbt query my_table '
  json = require("json")
  decoded_value = json.decode(row.my_cf["my_cf:haystack"])
  return decoded_value["nested_field"] == 123
'

Configuration

Configuration can also be provided from the environment by providing the QBT_PROJECT, QBT_INSTANCE, and QBT_PREFIX environment variables, or via CLI flags.

Libraries used

• GCP client libraries for Go: cloud.google.com/go
• Cobra for CLI: github.com/spf13/cobra
• Viper for configuration: github.com/spf13/viper
• Lua implementation gopher-lua: github.com/yuin/gopher-lua
• JSON support for gopher-lua: layeh.com/gopher-json
• Errors: github.com/pkg/errors

Todo / Ideas

• Tests!
• Add result transformations
• Add other output types (eg. YAML, CSV, strings returned from scripts)
• Add debug logging
• Queries from file / STDIN
• Pre-query scripts (eg. to define functions)
• Add CLI params for specifying range / prefix queries
• Web server for submitting queries to
• Other authentication methods

Licence

Copyright 2018 Chris Atkins

MIT