Skip to content

Commit f79d021

Browse files
authored
Prepare v1.0.0 (#51)
Prepare v1.0.0 - [#52] Make shards a library to work locally and move shards_dist to a separate repo - [#53] Support named and unnamed tables - [#54] Refactor shards to use TIDs instead of names (atoms) for handling the partitions - [#55] Use the metadata ETS table for storing the partitions TIDs, instead of building atoms in runtime for table names - [#56] Create shards_group as a dynamic supervisor for creating/deleting tables dynamically as part of a supervision tree - Improve docs
1 parent 15fff93 commit f79d021

37 files changed

+3361
-5092
lines changed

.codecov.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
comment:
2+
behaviour: new
3+
require_changes: yes

.github/workflows/ci.yml

Lines changed: 32 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,43 @@ on:
88

99
jobs:
1010
test:
11-
name: make ci (OTP ${{ matrix.otp }})
1211
runs-on: ubuntu-latest
12+
name: 'Shards Test (OTP ${{ matrix.otp }})'
13+
1314
strategy:
1415
matrix:
15-
otp: [22.3, 21.3]
16+
otp: [23.1, 22.3]
17+
1618
steps:
1719
- uses: actions/checkout@v2
20+
- uses: gleam-lang/[email protected]
1821
with:
1922
otp-version: ${{ matrix.otp }}
20-
- name: Starting EPMD
21-
run: epmd -daemon
23+
2224
- name: Running Tests
23-
run: make ci
25+
run: |
26+
make test
27+
make covertool
28+
29+
- name: Running Edoc
30+
run: make docs
31+
32+
- name: Running Xref
33+
run: make xref
34+
35+
- uses: actions/cache@v1
36+
with:
37+
path: priv/plts
38+
key: ${{ runner.os }}-plt-v2-${{ hashFiles(format('{0}{1}', github.workspace, '/rebar.lock')) }}
39+
restore-keys: |
40+
${{ runner.os }}-plt-v2-
41+
42+
- name: Running Dialyzer
43+
run: make dialyzer
44+
45+
- name: Upload coverage to Codecov
46+
uses: codecov/codecov-action@v1
47+
with:
48+
file: _build/test/covertool/shards.covertool.xml
49+
flags: otp-${{ matrix.otp }}
50+
fail_ci_if_error: true

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,4 @@ priv/*.so
3333
*_plt
3434
doc
3535
docs
36+
/priv/plts

Makefile

Lines changed: 11 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,7 @@ REBAR = $(shell which rebar3)
22

33
EPMD_PROC_NUM = $(shell ps -ef | grep epmd | grep -v "grep")
44

5-
LOCAL_SUITES = "test/shards_local_SUITE,test/shards_state_SUITE,test/shards_lib_SUITE,test/shards_task_SUITE"
6-
7-
.PHONY: all check_rebar compile clean distclean dialyzer test shell doc
5+
.PHONY: all compile clean distclean test test_suite covertool dialyzer xref check shell docs
86

97
all: check_rebar compile
108

@@ -25,24 +23,24 @@ dialyzer: check_rebar
2523
xref: check_rebar
2624
$(REBAR) xref
2725

28-
ci: check_rebar check_epmd check_plt xref
26+
test: check_rebar
2927
$(REBAR) do ct, cover
3028

31-
test: check_rebar check_epmd check_plt
32-
$(REBAR) do ct, cover
29+
test_suite: check_rebar
30+
$(REBAR) do ct --suite=test/$(SUITE)_SUITE, cover
31+
32+
covertool: check_rebar
33+
$(REBAR) as test covertool generate
3334

34-
local_test: check_rebar check_epmd
35-
$(REBAR) do ct --suite=$(LOCAL_SUITES), cover
35+
docs: check_rebar
36+
$(REBAR) edoc
3637

37-
dist_test: check_rebar check_epmd
38-
$(REBAR) do ct --suite=test/shards_dist_SUITE, cover
38+
check: test covertool dialyzer xref docs
39+
@echo "OK!"
3940

4041
shell: check_rebar
4142
$(REBAR) shell
4243

43-
doc: check_rebar
44-
$(REBAR) edoc
45-
4644
check_rebar:
4745
ifeq ($(REBAR),)
4846
ifeq ($(wildcard rebar3),)
@@ -52,18 +50,6 @@ else
5250
endif
5351
endif
5452

55-
check_plt:
56-
ifeq (,$(wildcard ./*_plt))
57-
@echo " ---> Running dialyzer ..."
58-
$(REBAR) dialyzer
59-
endif
60-
61-
check_epmd:
62-
ifeq ($(EPMD_PROC_NUM),)
63-
epmd -daemon
64-
@echo " ---> Started epmd!"
65-
endif
66-
6753
define get_rebar
6854
curl -O https://s3.amazonaws.com/rebar3/rebar3
6955
chmod a+x rebar3

README.md

Lines changed: 53 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,38 @@
1-
<img src="http://38.media.tumblr.com/db32471b7c8870cbb0b2cc173af283bb/tumblr_inline_nm9x9u6u261rw7ney_540.gif" height="170" width="100%" />
1+
<h1>
2+
<img align="left" height="60px" src="guides/images/shards.png">
3+
Shards
4+
</h1>
25

3-
# Shards
46
> ### ETS tables on steroids!
5-
> Sharding support for ETS tables out-of-box.
7+
> Sharding for ETS tables out-of-box.
68
7-
[![Build Status](https://github.com/cabol/shards/workflows/CI/badge.svg)](https://github.com/cabol/shards/actions)
9+
![CI](https://github.com/cabol/shards/workflows/CI/badge.svg)
10+
[![Codecov](https://codecov.io/gh/cabol/shards/branch/master/graphs/badge.svg)](https://codecov.io/gh/cabol/shards/branch/master/graphs/badge.svg)
811
[![Hex Version](https://img.shields.io/hexpm/v/shards.svg)](https://hex.pm/packages/shards)
912

10-
11-
Why might we need **Sharding** on ETS tables? Well, the main reason is
12-
to keep the lock contention under control, in order to scale-out ETS tables
13-
(linearly) and support higher levels of concurrency without lock issues;
14-
specially write-locks, which most of the cases might cause significant
15-
performance degradation.
13+
Why might we need **Sharding/Partitioning** for the ETS tables? The main reason
14+
is to keep the lock contention under control enabling ETS tables to scale out
15+
and support higher levels of concurrency without lock issues; specially
16+
write-locks, which most of the cases might cause significant performance
17+
degradation.
1618

1719
Therefore, one of the most common and proven strategies to deal with these
1820
problems is [Sharding][sharding] or [Partitioning][partitioning]; the principle
1921
is pretty similar to [DHTs][dht].
2022

21-
This is where **Shards** comes in. **Shards** is an **Erlang/Elixir** library
22-
compatible with the current [ETS API][ets_api], which implements
23-
[Sharding][sharding] or [Partitioning][partitioning] on top of ETS tables,
24-
completely transparent and out-of-box.
25-
26-
See the [getting started][getting_started] guide
27-
and the [online documentation](https://hexdocs.pm/shards/).
23+
This is where [shards][shards] comes in. [shards][shards] is an Erlang/Elixir
24+
library fully compatible with the [ETS API][ets_api], but it implements sharding
25+
or partitioning on top of the ETS tables, completely transparent and out-of-box.
2826

29-
> [List of compatible ETS functions](https://github.com/cabol/shards/issues/1)
27+
See the **[getting started][getting_started]** guide
28+
and the **[online documentation](https://hexdocs.pm/shards/)**.
3029

3130
[ets_api]: http://erlang.org/doc/man/ets.html
3231
[sharding]: https://en.wikipedia.org/wiki/Shard_(database_architecture)
3332
[partitioning]: https://en.wikipedia.org/wiki/Partition_(database)
3433
[dht]: https://en.wikipedia.org/wiki/Distributed_hash_table
35-
[getting_started]: https://github.com/cabol/shards/blob/master/guides/getting-started.md
34+
[shards]: https://hexdocs.pm/shards/shards.html
35+
[getting_started]: guides/getting-started.md
3636

3737
## Installation
3838

@@ -42,7 +42,7 @@ In your `rebar.config`:
4242

4343
```erlang
4444
{deps, [
45-
{shards, "0.6.1"}
45+
{shards, "0.6.2"}
4646
]}.
4747
```
4848

@@ -56,27 +56,25 @@ def deps do
5656
end
5757
```
5858

59-
> Check out the [getting started][getting_started] guide to learn
60-
more about it.
59+
> For more information and examples, see the [getting started][getting_started]
60+
guide.
6161

6262
## Important links
6363

64-
* [Documentation](https://hexdocs.pm/shards) - Hex Docs.
65-
66-
* [Blog Post](http://cabol.github.io/posts/2016/04/14/sharding-support-for-ets.html) -
67-
Transparent and out-of-box sharding support for ETS tables in Erlang/Elixir.
68-
69-
* [ExShards](https://github.com/cabol/ex_shards) – Elixir wrapper for
70-
`shards`; with extra and nicer functions.
71-
72-
* [Nebulex](https://github.com/cabol/nebulex) – Distributed Caching
73-
framework for Elixir.
64+
* [Documentation](https://hexdocs.pm/shards) - Hex Docs.
7465

75-
* [KVX](https://github.com/cabol/kvx) – Simple Elixir in-memory Key/Value
76-
Store using `shards` (default adapter).
66+
* [Blog Post](http://cabol.github.io/posts/2016/04/14/sharding-support-for-ets.html) -
67+
Transparent and out-of-box sharding support for ETS tables in Erlang/Elixir.
7768

78-
* [Cacherl](https://github.com/ferigis/cacherl) Distributed Cache
79-
using `shards`.
69+
* Projects using **shards**:
70+
* [ExShards](https://github.com/cabol/ex_shards) – Elixir wrapper for
71+
`shards`; with extra and nicer functions.
72+
* [Nebulex](https://github.com/cabol/nebulex) – Distributed Caching
73+
framework for Elixir.
74+
* [KVX](https://github.com/cabol/kvx) – Simple Elixir in-memory Key/Value
75+
Store using `shards` (default adapter).
76+
* [Cacherl](https://github.com/ferigis/cacherl) Distributed Cache
77+
using `shards`.
8078

8179
## Testing
8280

@@ -91,14 +89,29 @@ You can find tests results in `_build/test/logs`, and coverage in
9189
wrapper on top of `rebar3`, therefore, you can do everything using `rebar3`
9290
directly as well (e.g.: `rebar3 do ct, cover`).
9391

94-
## Building Edoc
92+
## Generating Edoc
9593

9694
```
97-
$ make doc
95+
$ make docs
9896
```
9997

100-
> **NOTE:** Once you run the previous command, a new folder `doc` is created,
101-
and you'll have a pretty nice HTML documentation.
98+
> **NOTE:** Once you run the previous command, you will find the generated HTML
99+
documentation within `doc` folder; open `doc/index.html`.
100+
101+
## Contributing
102+
103+
Contributions to **shards** are very welcome and appreciated!
104+
105+
Use the [issue tracker](https://github.com/cabol/shards/issues) for bug reports
106+
or feature requests. Open a [pull request](https://github.com/cabol/shards/pulls)
107+
when you are ready to contribute.
108+
109+
When submitting a pull request you should not update the [CHANGELOG.md](CHANGELOG.md),
110+
and also make sure you test your changes thoroughly, include unit tests
111+
alongside new or changed code.
112+
113+
Before to submit a PR it is highly recommended to run `make check` before and
114+
ensure all checks run successfully.
102115

103116
## Copyright and License
104117

elvis.config

Lines changed: 0 additions & 29 deletions
This file was deleted.

0 commit comments

Comments
 (0)