Skip to content

Commit e94bc96

Browse files
committed
add README, other minor changes
- API file is now in README - bump year in LICENSE - go mod tidy
1 parent edd9297 commit e94bc96

File tree

4 files changed

+129
-49
lines changed

4 files changed

+129
-49
lines changed

API

-46
This file was deleted.

LICENSE

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
Copyright 2019, Travis Bischel.
1+
Copyright 2020, Travis Bischel.
22
All rights reserved.
33

44
Redistribution and use in source and binary forms, with or without

README.md

+121
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
kcl
2+
===
3+
4+
kcl is a complete, pure Go command line Kafka client. Think of it as your
5+
one stop shop to do anything you want to do with Kafka. Producing, consuming,
6+
transacting, administrating, and so on.
7+
8+
Unlike the small size of [kafkacat][1], this binary is ~12M compiled.
9+
It is, however, still fast, has rich consuming and producing formatting
10+
options, and a complete Kafka administration interface.
11+
12+
[1]: https://github.com/edenhill/kafkacat
13+
14+
## Configuration
15+
16+
kcl supports configuration through a config file, environment variables, and
17+
config flag overrides, with the config values being defined in that order.
18+
By default, kcl searches your OS's user config dir for a `kcl` directory and
19+
a `config.toml` file in that directory. The default path can be overridden.
20+
As well, multiple configs can easily swapped between with `kcl myconfig`.
21+
22+
The configuration supports TLS, SASL (currently PLAIN and SCRAM), seed brokers,
23+
and a timeout for requests that take timeouts.
24+
25+
## Autocompletion
26+
27+
Thanks to [cobra][2], autocompletion exists for bash, zsh, and powershell.
28+
29+
[2]: https://github.com/spf13/cobra
30+
31+
As an example of what to put in your .bashrc,
32+
33+
```bash
34+
if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
35+
. /etc/bash_completion
36+
. <(kcl misc gen-autocomplete -kbash)
37+
fi
38+
```
39+
40+
## Transactions
41+
42+
Transactions are supported by consuming a batch of records, executing a command
43+
and passing the records to the command's STDIN, reading the modified records
44+
via the command's STDOUT, and publishing those records.
45+
46+
Input to the program and output from the program is controlled through the same
47+
syntax as consuming and producing, and the `--rw` flag is a shortcut to say that
48+
the input and output will use the same format.
49+
50+
As an example, the following command:
51+
52+
```
53+
kcl transact -rw '%V{b4}%v' -dtxn -g group -t foo -x mytxn -v ./command
54+
```
55+
56+
reads topic `foo` in group `group`, executes `./command`, writes all record
57+
values to it prefixed with the four byte big endian value length, reads
58+
back records in the same format, and produces to topic `txn` all using the
59+
transactional id `mytxn`.
60+
61+
## Group Consuming
62+
63+
Group consuming is supported with the `-g` or `--group` flag to `kcl consume`
64+
or `kcl transact`. The default balancer is the cooperative-sticky balancer,
65+
which was introduced with incremental rebalancing in Kafka 2.4.0. This balancer
66+
is incompatible with the previous eager balancers (roundrobin, range, sticky),
67+
thus if you are using kcl with existing groups that have members using eager
68+
balancing strategies, be sure to specify a different balancer.
69+
70+
## API at a glance
71+
72+
Be sure to `--help` any command before using it to understand the full syntax.
73+
74+
```
75+
kcl
76+
consume
77+
produce
78+
transact
79+
misc
80+
api-versions
81+
probe-version
82+
gen-autocomplete
83+
errcode
84+
raw-req
85+
list-offsets
86+
admin
87+
elect-leaders
88+
delete-records
89+
alter-partition-assignments
90+
list-partition-reassignments
91+
alter-replica-log-dirs
92+
describe-log-dirs
93+
dtoken
94+
create
95+
renew
96+
describe
97+
expire
98+
acl
99+
create
100+
describe
101+
delete
102+
group
103+
list
104+
describe
105+
delete
106+
offset-delete
107+
topic
108+
create
109+
delete
110+
add-partitions
111+
metadata
112+
configs
113+
alter
114+
describe
115+
myconfig
116+
unlink
117+
link
118+
dump
119+
help
120+
ls
121+
```

go.sum

+7-2
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
1919
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
2020
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
2121
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
22+
github.com/frankban/quicktest v1.7.2 h1:2QxQoC1TS09S7fhCPsrvqYdvP1H5M1P1ih5ABm3BTYk=
2223
github.com/frankban/quicktest v1.7.2/go.mod h1:jaStnuzAqU1AJdCO0l53JDCJrVDKcS03DbaAcR7Ks/o=
2324
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
2425
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
@@ -37,6 +38,7 @@ github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
3738
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
3839
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
3940
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
41+
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
4042
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
4143
github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
4244
github.com/gorilla/sessions v1.2.0/go.mod h1:dk2InVEVJ0sfLlnXv9EAgkf6ecYs/i80K/zI+bUmuGM=
@@ -46,6 +48,7 @@ github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgf
4648
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
4749
github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
4850
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
51+
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
4952
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
5053
github.com/jcmturner/aescts/v2 v2.0.0/go.mod h1:AiaICIRyfYg35RUkr8yESTqvSy7csK90qZ5xfvvsoNs=
5154
github.com/jcmturner/dnsutils/v2 v2.0.0/go.mod h1:b0TnjGOvI/n42bZa+hmXL+kFJZsFT7G4t3HTlQ184QM=
@@ -61,8 +64,10 @@ github.com/klauspost/compress v1.10.10 h1:a/y8CglcM7gLGYmlbP/stPE5sR3hbhFRUjCBfd
6164
github.com/klauspost/compress v1.10.10/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
6265
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
6366
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
67+
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
6468
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
6569
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
70+
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
6671
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
6772
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
6873
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
@@ -108,8 +113,8 @@ github.com/twmb/go-rbtree v1.0.0 h1:KxN7dXJ8XaZ4cvmHV1qqXTshxX3EBvX/toG5+UR49Mg=
108113
github.com/twmb/go-rbtree v1.0.0/go.mod h1:UlIAI8gu3KRPkXSobZnmJfVwCJgEhD/liWzT5ppzIyc=
109114
github.com/twmb/go-strftime v0.0.0-20190915101236-e74f7c4fe4fa h1:eq9HJTMjHC3K/GYE7RuvhweBhIxXi9y6BYM6Aox3UbA=
110115
github.com/twmb/go-strftime v0.0.0-20190915101236-e74f7c4fe4fa/go.mod h1:k1GNMjGU8BvbV6Ej1ysLoM1cPfmybe216l51R7RKIf0=
111-
github.com/twmb/kafka-go v0.4.0 h1:jI2+cdwOAJ3zJB1RFhMZ2LWGXEy4B9SapBDU0G2HtYs=
112-
github.com/twmb/kafka-go v0.4.0/go.mod h1:wWLRH8FOgeah2itKZJnOjZovXE3GbfWTLIcgR/09GKs=
116+
github.com/twmb/kafka-go v0.4.1 h1:AL7GKnYXofXCMW4H73xRUQzZFIg63mKEcFaCO6Hox0s=
117+
github.com/twmb/kafka-go v0.4.1/go.mod h1:wWLRH8FOgeah2itKZJnOjZovXE3GbfWTLIcgR/09GKs=
113118
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
114119
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
115120
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=

0 commit comments

Comments
 (0)