Skip to content

Commit 47a1f3e

Browse files
authored
Integrate Plank into Transport as a module (#15)
* Move Plank code from personal repo Signed-off-by: Josh Kim <[email protected]> * Update package reference to vmware/transport-go/plank Signed-off-by: Josh Kim <[email protected]> * Add pipeline configs for Plank Signed-off-by: Josh Kim <[email protected]> * Fix broken tests Signed-off-by: Josh Kim <[email protected]> * Update Plank README Signed-off-by: Josh Kim <[email protected]> * Fix bug in REST bridge override & mux.Router concurrency Current implementation of REST bridge override had some critical issues that could lead to memory leaks and new URIs not being properly reflected or old URIs still accessible. These issues mostly originate from mux.Router not allowing full CRUD operations on the Router instance. (e.g. there was no way to effecively remove an existing route from any given router) Signed-off-by: Josh Kim <[email protected]> * chore: update Dockerfile to match latest work Signed-off-by: Josh Kim <[email protected]> * chore: Add copyright notices in src files Signed-off-by: Josh Kim <[email protected]> * Document broker sample code Signed-off-by: Josh Kim <[email protected]> * Minor changes to README & remove unnecessary files Signed-off-by: Josh Kim <[email protected]> * Add --rest-bridge-timeout to set REST bridge timeout Signed-off-by: Josh Kim <[email protected]> * Remove redundant bus and service registry init Both instances are initialized way before StartServer() is called, initialize() for example. Signed-off-by: Josh Kim <[email protected]> * Use go for build Closes vmware-archive/transport-go#19 Signed-off-by: Josh Kim <[email protected]> * Document broker_sample package code Signed-off-by: Josh Kim <[email protected]> * More documentation and gofmt and cleanup Signed-off-by: Josh Kim <[email protected]> * Break up server.go into multiple files Signed-off-by: Josh Kim <[email protected]> * Add a more complex sample Signed-off-by: Josh Kim <[email protected]> * Expose stompserver's ConnEvent and EventType These two structs are essential in notifying services about session disconnection hence opening the minimal structure for external packages to consume. Signed-off-by: Josh Kim <[email protected]> * Notify certain STOMP session events through internal channels Signed-off-by: Josh Kim <[email protected]> * Use Proxyheaders to get real IPs for proxied setups Signed-off-by: Josh Kim <[email protected]> * Prohibit STOMP clients sending directly to topic/q Signed-off-by: Josh Kim <[email protected]> * Change env vars for certain flags to avoid conflicts Signed-off-by: Josh Kim <[email protected]> * Turn requestBuilder signature into its own type Signed-off-by: Josh Kim <[email protected]> * Provide a bit more detail to request timeout Signed-off-by: Josh Kim <[email protected]> * Replace urfavecli with Cobra Closes vmware-archive/transport-go#18 Signed-off-by: Josh Kim <[email protected]>
1 parent d917abb commit 47a1f3e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+5126
-112
lines changed

.github/workflows/plank-postmerge.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Plank Post-merge pipeline
2+
3+
on:
4+
push:
5+
paths:
6+
- plank/**/*
7+
branches:
8+
- main
9+
10+
jobs:
11+
test:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Set up repo
15+
uses: actions/checkout@v2
16+
- uses: actions/setup-go@v2
17+
with:
18+
go-version: '^1.16'
19+
- run: go get ./...
20+
working-directory: plank
21+
- run: |
22+
go get github.com/axw/gocov/gocov
23+
go get github.com/AlekSi/gocov-xml
24+
go install github.com/axw/gocov/gocov
25+
go install github.com/AlekSi/gocov-xml
26+
working-directory: plank
27+
- run: |
28+
go test -v -coverprofile cover.out ./...
29+
gocov convert cover.out | gocov-xml > coverage.xml
30+
working-directory: plank
31+
# - uses: codecov/codecov-action@v1
32+
# with:
33+
# token: ${{ secrets.CODECOV_TOKEN }}
34+
# files: ./coverage.xml
35+
# flags: unittests
36+
# fail_ci_if_error: true
37+
# verbose: true
38+
# working-directory: plank

.github/workflows/plank-premerge.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: Plank pre-merge pipeline
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- plank/**/*
7+
8+
jobs:
9+
test:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Set up repo
13+
uses: actions/checkout@v2
14+
- uses: actions/setup-go@v2
15+
with:
16+
go-version: '^1.16'
17+
- run: go get ./...
18+
working-directory: plank
19+
- run: |
20+
go get github.com/axw/gocov/gocov
21+
go get github.com/AlekSi/gocov-xml
22+
go install github.com/axw/gocov/gocov
23+
go install github.com/AlekSi/gocov-xml
24+
working-directory: plank
25+
- run: |
26+
go test -v -coverprofile cover.out ./...
27+
gocov convert cover.out | gocov-xml > coverage.xml
28+
working-directory: plank

.github/workflows/transport-postmerge.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ name: Transport Post-merge pipeline
22

33
on:
44
push:
5+
paths-ignore:
6+
- plank/**/*
57
branches:
68
- main
79

.github/workflows/transport-premerge.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ name: Transport Pre-merge pipeline
22

33
on:
44
pull_request:
5+
paths-ignore:
6+
- plank/**/*
57

68
jobs:
79
test:

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,6 @@
22
/.idea
33
/bus/bus.iml
44
/bifrost
5+
6+
**/*.log
7+
**/cover.out

bus/fabric_endpoint.go

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,11 @@ import (
1414
"sync"
1515
)
1616

17+
const (
18+
TRANSPORT_INTERNAL_CHANNEL_PREFIX = "_transportInternal/"
19+
STOMP_SESSION_NOTIFY_CHANNEL = TRANSPORT_INTERNAL_CHANNEL_PREFIX + "stomp-session-notify"
20+
)
21+
1722
type EndpointConfig struct {
1823
// Prefix for public topics e.g. "/topic"
1924
TopicPrefix string
@@ -55,6 +60,11 @@ type channelMapping struct {
5560
autoCreated bool
5661
}
5762

63+
type StompSessionEvent struct {
64+
Id string
65+
EventType stompserver.StompSessionEventType
66+
}
67+
5868
type fabricEndpoint struct {
5969
server stompserver.StompServer
6070
bus EventBus
@@ -93,6 +103,18 @@ func newFabricEndpoint(bus EventBus,
93103
}
94104

95105
func (fe *fabricEndpoint) Start() {
106+
fe.server.SetConnectionEventCallback(stompserver.ConnectionStarting, func(connEvent *stompserver.ConnEvent) {
107+
busInstance.SendResponseMessage(STOMP_SESSION_NOTIFY_CHANNEL, &StompSessionEvent{
108+
Id: connEvent.ConnId,
109+
EventType: stompserver.ConnectionStarting,
110+
}, nil)
111+
})
112+
fe.server.SetConnectionEventCallback(stompserver.ConnectionClosed, func(connEvent *stompserver.ConnEvent) {
113+
busInstance.SendResponseMessage(STOMP_SESSION_NOTIFY_CHANNEL, &StompSessionEvent{
114+
Id: connEvent.ConnId,
115+
EventType: stompserver.ConnectionClosed,
116+
}, nil)
117+
})
96118
fe.server.Start()
97119
}
98120

@@ -114,6 +136,12 @@ func (fe *fabricEndpoint) addSubscription(
114136
return
115137
}
116138

139+
// if destination is a protected channel do not establish a subscription
140+
// (we don't want any clients to be sending messages to internal channels)
141+
if isProtectedDestination(channelName) {
142+
return
143+
}
144+
117145
fe.chanLock.Lock()
118146
defer fe.chanLock.Unlock()
119147

@@ -262,3 +290,10 @@ func (fe *fabricEndpoint) getChannelNameFromSubscription(destination string) (ch
262290
}
263291
return "", false
264292
}
293+
294+
// isProtectedDestination checks if the destination is protected. this utility function is used to
295+
// prevent messages being from clients to the protected destinations. such examples would be
296+
// internal bus channels prefixed with _transportInternal/
297+
func isProtectedDestination(destination string) bool {
298+
return strings.HasPrefix(destination, TRANSPORT_INTERNAL_CHANNEL_PREFIX)
299+
}

bus/fabric_endpoint_test.go

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ type MockStompServer struct {
2424
started bool
2525
sentMessages []MockStompServerMessage
2626
subscribeHandlerFunction stompserver.SubscribeHandlerFunction
27+
connectionEventCallbacks map[stompserver.StompSessionEventType]func(event *stompserver.ConnEvent)
2728
unsubscribeHandlerFunction stompserver.UnsubscribeHandlerFunction
2829
applicationRequestHandlerFunction stompserver.ApplicationRequestHandlerFunction
2930
wg *sync.WaitGroup
@@ -67,10 +68,15 @@ func(s *MockStompServer) OnSubscribeEvent(callback stompserver.SubscribeHandlerF
6768
s.subscribeHandlerFunction = callback
6869
}
6970

71+
func (s *MockStompServer) SetConnectionEventCallback(connEventType stompserver.StompSessionEventType, cb func(connEvent *stompserver.ConnEvent)) {
72+
s.connectionEventCallbacks[connEventType] = cb
73+
cb(&stompserver.ConnEvent{ConnId: "id"})
74+
}
75+
7076
func newTestFabricEndpoint(bus EventBus, config EndpointConfig) (*fabricEndpoint, *MockStompServer) {
7177

7278
fe := newFabricEndpoint(bus, nil, config).(*fabricEndpoint)
73-
ms := &MockStompServer{}
79+
ms := &MockStompServer{connectionEventCallbacks: make(map[stompserver.StompSessionEventType]func(event *stompserver.ConnEvent))}
7480

7581
fe.server = ms
7682
fe.initHandlers()
@@ -111,6 +117,7 @@ func TestFabricEndpoint_StartAndStop(t *testing.T) {
111117
func TestFabricEndpoint_SubscribeEvent(t *testing.T) {
112118

113119
bus := newTestEventBus()
120+
bus.GetChannelManager().CreateChannel(STOMP_SESSION_NOTIFY_CHANNEL) // used for internal channel protection test
114121
fe, mockServer := newTestFabricEndpoint(bus,
115122
EndpointConfig{TopicPrefix: "/topic", UserQueuePrefix:"/user/queue"})
116123

@@ -167,6 +174,11 @@ func TestFabricEndpoint_SubscribeEvent(t *testing.T) {
167174
assert.Equal(t, len(fe.chanMappings["test-service"].subs), 3)
168175
assert.Equal(t, fe.chanMappings["test-service"].subs["con1#sub3"], true)
169176

177+
// attempt to subscribe to a protected destination
178+
mockServer.subscribeHandlerFunction("con1", "sub4", "/topic/" + STOMP_SESSION_NOTIFY_CHANNEL, nil)
179+
_, chanMapCreated := fe.chanMappings[STOMP_SESSION_NOTIFY_CHANNEL]
180+
assert.False(t, chanMapCreated)
181+
170182
mockServer.wg = &sync.WaitGroup{}
171183
mockServer.wg.Add(1)
172184

plank/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
build/
2+
cert/

plank/Dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM photon:latest
2+
COPY build/plank /usr/local/bin/
3+
RUN mkdir -p /opt/plank
4+
WORKDIR /opt/plank
5+
STOPSIGNAL SIGTERM
6+
ENTRYPOINT ["plank", "start-server"]

plank/README.md

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# Plank
2+
3+
## What is Plank?
4+
Plank is just enough of a platform to build whatever you want on top. It is a small yet powerful Golang server that can serve
5+
static contents, single page applications, create and expose microservices over REST endpoints or WebSocket via a built-in
6+
STOMP broker, or even interact directly with other message brokers such as RabbitMQ. All this is done in a consistent
7+
and easy to follow manner powered by Transport Event Bus.
8+
9+
Writing a service for a Plank server is in a way similar to writing a Spring Boot `Component` or `Service`, because a lot of tedious
10+
plumbing work is already done for you such as creating an instance of a service and wiring it up with HTTP endpoints using routers etc.
11+
Just by following the API you can easily stand up a service, apply any kinds of middleware your application logic calls for, and do all these
12+
dynamically while in runtime, meaning you can conditionally apply a filter for certain REST endpoints, stand up a new service on demand, or
13+
even spawn yet another whole new instance of Plank at a different endpoint.
14+
15+
All features are cleanly exposed as public API and modules and, combined with the power of Golang's concurrency model using channels,
16+
the Transport Event Bus allows creating a clean application architecture, with straightforward and easy to follow logic.
17+
18+
Detailed tutorials and examples are currently in progress and will be made public on the [Transport GitHub page](https://vmware.github.io/transport/).
19+
Some topics that will initially be covered are:
20+
21+
- Writing a simple service and interacting with it over REST and WebSocket
22+
- Service lifecycle hooks
23+
- Middleware management (for REST bridged services)
24+
- Concept of local bus channel and galactic bus channel
25+
- Communicating between Plank instances using the built-in STOMP broker
26+
- Securing your REST and WebSocket endpoints using Auth Provider Manager
27+
28+
## Hello world
29+
### How to build Plank
30+
First things first, you'll need the latest Golang version. Plank was first written in Golang 1.13 and it still works with it, but it's advised
31+
to use the latest Golang (especially 1.16 and forward) because of some nice new packages such as `embed` that we may later employ as part of Plank codebase.
32+
Once you have the latest Golang ready, follow the commands below:
33+
34+
```bash
35+
# get all dependencies
36+
go get ./...
37+
38+
# To build against your operating system
39+
go run build.go
40+
41+
# or, to build against a specific operating system
42+
BUILD_OS=darwin|linux|windows go run build.go
43+
```
44+
45+
Once successfully built, `plank` binary will be ready under `build/`.
46+
47+
> NOTE: we acknowledge there's a lack of build script for Windows Powershell, We'll add it soon!
48+
49+
### Generate a self signed certificate
50+
Plank can run in non-HTTPS mode but it's generally a good idea to always do development in a similar environment where you'll be serving your
51+
audience in public internet (or even intranet). Plank repository comes with a handy utility script that can generate a pair of server certificate
52+
and its matching private key at `scripts/create-selfsigned-cert.sh`. Simply run it in a POSIX shell like below. (requires `openssl` library
53+
to be available):
54+
55+
```bash
56+
# generate a pair of x509 certificate and private key files
57+
./scripts/generate-selfsigned-cert.sh
58+
59+
# cert/fullchain.pem is your certificate and cert/server.key its matching private key
60+
ls -la cert/
61+
```
62+
63+
### The real Hello World part
64+
Now we're ready to start the application. To kick off the server using the demo app you have built above, type the following and you'll see something like this:
65+
66+
```bash
67+
./build/plank start-server --config-file config.json
68+
69+
______ __ ______ __ __ __ __
70+
/\ == /\ \ /\ __ \/\ "-.\ \/\ \/ /
71+
\ \ _-\ \ \___\ \ __ \ \ \-. \ \ _"-.
72+
\ \_\ \ \_____\ \_\ \_\ \_\\"\_\ \_\ \_\
73+
\/_/ \/_____/\/_/\/_/\/_/ \/_/\/_/\/_/
74+
75+
Host localhost
76+
Port 30080
77+
Fabric endpoint /ws
78+
SPA endpoint /public
79+
SPA static assets /assets
80+
Health endpoint /health
81+
Prometheus endpoint /prometheus
82+
...
83+
time="2021-08-05T21:32:50-07:00" level=info msg="Starting HTTP server at localhost:30080 with TLS" fileName=server.go goroutine=28 package=server
84+
```
85+
86+
Open your browser and navigate to https://localhost:30080, accept the self-signed certificate warning and you'll be greeted with a 404!
87+
This is an expected behavior, as the demo app does not serve anything at root `/`, but we will consider changing the default 404 screen to
88+
something that looks more informational or more appealing at least.
89+
90+
## All supported flags and usages
91+
92+
|Long flag|Short flag|Default value|Required|Description|
93+
|----|----|----|----|----|
94+
|--hostname|-n|localhost|false|Hostname where Plank is to be served. Also reads from `$PLANK_SERVER_HOSTNAME` environment variable|
95+
|--port|-p|30080|false|Port where Plank is to be served. Also reads from `$PLANK_SERVER_PORT` environment variable|
96+
|--rootdir|-r|<current directory>|false|Root directory for the server. Also reads from `$PLANK_SERVER_ROOTDIR` environment variable|
97+
|--static|-s|-|false|Path to a location where static files will be served. Can be used multiple times|
98+
|--no-fabric-broker|-|false|false|Do not start Fabric broker|
99+
|--fabric-endpoint|-|/fabric|false|Fabric broker endpoint (ignored if --no-fabric-broker is present)|
100+
|--topic-prefix|-|/topic|false|Topic prefix for Fabric broker (ignored if --no-fabric-broker is present)|
101+
|--queue-prefix|-|/queue|false|Queue prefix for Fabric broker (ignored if --no-fabric-broker is present)|
102+
|--request-prefix|-|/pub|false|Application request prefix for Fabric broker (ignored if --no-fabric-broker is present)|
103+
|--request-queue-prefix|-|/pub/queue|false|Application request queue prefix for Fabric broker (ignored if --no-fabric-broker is present)|
104+
|--shutdown-timeout|-|5|false|Graceful server shutdown timeout in minutes|
105+
|--output-log|-l|stdout|false|File to output platform logs to|
106+
|--access-log|-l|stdout|false|File to output HTTP server access logs to|
107+
|--error-log|-l|stderr|false|File to output HTTP server error logs to|
108+
|--debug|-d|false|false|Debug mode|
109+
|--no-banner|-b|false|false|Do not print Plank banner at startup|
110+
|--prometheus|-|false|false|Enable Prometheus at /prometheus for metrics|
111+
|--rest-bridge-timeout|-|1|false|Time in minutes before a REST endpoint for a service request to timeout|
112+
113+
Examples are as follows:
114+
```bash
115+
# Start a server with all options set to default values
116+
./plank start-server
117+
118+
# Start a server with a custom hostname and at port 8080 without Fabric (WebSocket) broker
119+
./plank start-server --no-fabric-broker --hostname my-app.io --port 8080
120+
121+
# Start a server with a 10 minute graceful shutdown timeout
122+
# NOTE: this is useful when you run a service that takes a significant amount of time or might even hang while shutting down
123+
./plank start-server --shutdown-timeout 10
124+
125+
# Start a server with platform server logs printing to stdout and access/error logs to their respective files
126+
./plank start-server --access-log server-access-$(date +%m%d%y).log --error-log server-error-$(date +%m%d%y).log
127+
128+
# Start a server with debug outputs enabled
129+
./plank start-server -d
130+
131+
# Start a server without splash banner
132+
./plank start-server --no-banner
133+
134+
# Start a server with Prometheus enabled at /prometheus
135+
./plank start-server --prometheus
136+
137+
# Start a server with static path served at `/static` for folder `static`
138+
./plank start-server --static static
139+
140+
# Start a server with static paths served at `/static` for folder `static` and
141+
# at `/public` for folder `public-contents`
142+
./plank start-server --static static --static public-contents:/public
143+
144+
# Start a server with a JSON configuration file
145+
./plank start-server --config-file config.json
146+
```
147+
148+
## Advanced topics (WIP. Coming soon)
149+
### OAuth2 Client
150+
Plank supports seamless out of the box OAuth 2.0 client that support a few OAuth flows. such as authorization
151+
code grant for web applications and client credentials grant for server-to-server applications.
152+
See below for a detailed guide for each flow.
153+
154+
#### Authorization Code flow
155+
You'll choose this authentication flow when the Plank server acts as an intermediary that exchanges
156+
the authorization code returned from the authorization server for an access token. During this process you will be
157+
redirected to the identity provider's page like Google where you are asked to confirm the type of 3rd party application and
158+
its scope of actions it will perform on your behalf and will be taken back to the application after successful authorization.
159+
160+
#### Client Credentials flow
161+
You'll choose this authentication flow when the Plank server needs to directly communicate with another backend service.
162+
This will not require user's consent like you would be redirected to Google's page where you confirm the type of application
163+
requesting your consent for the scope of actions it will perform on your behalf. not requiring any interactions from the user. You will need to
164+
create an OAuth 2.0 Client with `client_credentials` grant before following the steps below to implement the
165+
authentication flow.

0 commit comments

Comments
 (0)