Skip to content

Commit 7bbef03

Browse files
ciusjiciusji
authored
Merge pull request #45 from Spotrix/1.2.1-a1
1.2.1 a1
2 parents 7b158a9 + 7e84908 commit 7bbef03

File tree

13 files changed

+55306
-509
lines changed

13 files changed

+55306
-509
lines changed

‎docker/.env-non-dev

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
COMPOSE_PROJECT_NAME=spotrix
18+
19+
# database configurations (do not modify)
20+
DATABASE_DB=spotrix
21+
DATABASE_HOST=db
22+
DATABASE_PASSWORD=spotrix
23+
DATABASE_USER=spotrix
24+
25+
# database engine specific environment variables
26+
# change the below if you prefers another database engine
27+
DATABASE_PORT=5432
28+
DATABASE_DIALECT=postgresql
29+
POSTGRES_DB=spotrix
30+
POSTGRES_USER=spotrix
31+
POSTGRES_PASSWORD=spotrix
32+
#MYSQL_DATABASE=spotrix
33+
#MYSQL_USER=spotrix
34+
#MYSQL_PASSWORD=spotrix
35+
#MYSQL_RANDOM_ROOT_PASSWORD=yes
36+
37+
# Add the mapped in /app/pythonpath_docker which allows devs to override stuff
38+
PYTHONPATH=/app/pythonpath:/app/docker/pythonpath_dev
39+
REDIS_HOST=redis
40+
REDIS_PORT=6379
41+
42+
FLASK_ENV=production
43+
SUPERSET_ENV=production
44+
SUPERSET_LOAD_EXAMPLES=yes
45+
CYPRESS_CONFIG=false
46+
SPOTRIX_PORT=8088

‎docker/README.md

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Getting Started with Superset using Docker
21+
22+
Docker is an easy way to get started with Superset.
23+
24+
## Prerequisites
25+
26+
1. Docker! [link](https://www.docker.com/get-started)
27+
2. Docker-compose [link](https://docs.docker.com/compose/install/)
28+
29+
## Configuration
30+
31+
The `/app/pythonpath` folder is mounted from [`./docker/pythonpath_dev`](./pythonpath_dev)
32+
which contains a base configuration [`./docker/pythonpath_dev/superset_config.py`](./pythonpath_dev/superset_config.py)
33+
intended for use with local development.
34+
35+
### Local overrides
36+
37+
In order to override configuration settings locally, simply make a copy of [`./docker/pythonpath_dev/superset_config_local.example`](./pythonpath_dev/superset_config_local.example)
38+
into `./docker/pythonpath_dev/superset_config_docker.py` (git ignored) and fill in your overrides.
39+
40+
### Local packages
41+
42+
If you want to add Python packages in order to test things like databases locally, you can simply add a local requirements.txt (`./docker/requirements-local.txt`)
43+
and rebuild your Docker stack.
44+
45+
Steps:
46+
47+
1. Create `./docker/requirements-local.txt`
48+
2. Add your new packages
49+
3. Rebuild docker-compose
50+
1. `docker-compose down -v`
51+
2. `docker-compose up`
52+
53+
## Initializing Database
54+
55+
The database will initialize itself upon startup via the init container ([`superset-init`](./docker-init.sh)). This may take a minute.
56+
57+
## Normal Operation
58+
59+
To run the container, simply run: `docker-compose up`
60+
61+
After waiting several minutes for Superset initialization to finish, you can open a browser and view [`http://localhost:8088`](http://localhost:8088)
62+
to start your journey.
63+
64+
## Developing
65+
66+
While running, the container server will reload on modification of the Superset Python and JavaScript source code.
67+
Don't forget to reload the page to take the new frontend into account though.
68+
69+
## Production
70+
71+
It is possible to run Superset in non-development mode by using [`docker-compose-non-dev.yml`](../docker-compose-non-dev.yml). This file excludes the volumes needed for development and uses [`./docker/.env-non-dev`](./.env-non-dev) which sets the variable `SUPERSET_ENV` to `production`.
72+
73+
## Resource Constraints
74+
75+
If you are attempting to build on macOS and it exits with 137 you need to increase your Docker resources. See instructions [here](https://docs.docker.com/docker-for-mac/#advanced) (search for memory)

‎docker/docker-bootstrap.sh

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one or more
4+
# contributor license agreements. See the NOTICE file distributed with
5+
# this work for additional information regarding copyright ownership.
6+
# The ASF licenses this file to You under the Apache License, Version 2.0
7+
# (the "License"); you may not use this file except in compliance with
8+
# the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
19+
set -eo pipefail
20+
21+
REQUIREMENTS_LOCAL="/app/docker/requirements-local.txt"
22+
# If Cypress run – overwrite the password for admin and export env variables
23+
if [ "$CYPRESS_CONFIG" == "true" ]; then
24+
export SPOTRIX_CONFIG=tests.integration_tests.superset_test_config
25+
export SPOTRIX_TESTENV=true
26+
export ENABLE_REACT_CRUD_VIEWS=true
27+
export SPOTRIX__SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://superset:superset@db:5432/superset
28+
fi
29+
#
30+
# Make sure we have dev requirements installed
31+
#
32+
if [ -f "${REQUIREMENTS_LOCAL}" ]; then
33+
echo "Installing local overrides at ${REQUIREMENTS_LOCAL}"
34+
pip install -r "${REQUIREMENTS_LOCAL}"
35+
else
36+
echo "Skipping local overrides"
37+
fi
38+
39+
if [[ "${1}" == "worker" ]]; then
40+
echo "Starting Celery worker..."
41+
celery --app=superset.tasks.celery_app:app worker -Ofair -l INFO
42+
elif [[ "${1}" == "beat" ]]; then
43+
echo "Starting Celery beat..."
44+
celery --app=superset.tasks.celery_app:app beat --pidfile /tmp/celerybeat.pid -l INFO -s "${SUPERSET_HOME}"/celerybeat-schedule
45+
elif [[ "${1}" == "app" ]]; then
46+
echo "Starting web app..."
47+
flask run -p 8088 --with-threads --reload --debugger --host=0.0.0.0
48+
elif [[ "${1}" == "app-gunicorn" ]]; then
49+
echo "Starting web app..."
50+
/app/docker/docker-entrypoint.sh
51+
fi

‎docker/docker-ci.sh

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one or more
4+
# contributor license agreements. See the NOTICE file distributed with
5+
# this work for additional information regarding copyright ownership.
6+
# The ASF licenses this file to You under the Apache License, Version 2.0
7+
# (the "License"); you may not use this file except in compliance with
8+
# the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
/app/docker/docker-init.sh
19+
20+
# TODO: copy config overrides from ENV vars
21+
22+
# TODO: run celery in detached state
23+
24+
# start up the web server
25+
gunicorn \
26+
--bind "0.0.0.0:${SUPERSET_PORT}" \
27+
--access-logfile '-' \
28+
--error-logfile '-' \
29+
--workers 1 \
30+
--worker-class gthread \
31+
--threads 8 \
32+
--timeout 60 \
33+
--limit-request-line 0 \
34+
--limit-request-field_size 0 \
35+
"${FLASK_APP}"

‎docker/docker-entrypoint.sh

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one or more
4+
# contributor license agreements. See the NOTICE file distributed with
5+
# this work for additional information regarding copyright ownership.
6+
# The ASF licenses this file to You under the Apache License, Version 2.0
7+
# (the "License"); you may not use this file except in compliance with
8+
# the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
set -eo pipefail
19+
20+
if [ "${#}" -ne 0 ]; then
21+
exec "${@}"
22+
else
23+
gunicorn \
24+
--bind "0.0.0.0:${SUPERSET_PORT}" \
25+
--access-logfile '-' \
26+
--error-logfile '-' \
27+
--workers 1 \
28+
--worker-class gthread \
29+
--threads 20 \
30+
--timeout ${GUNICORN_TIMEOUT:-60} \
31+
--limit-request-line 0 \
32+
--limit-request-field_size 0 \
33+
"${FLASK_APP}"
34+
fi

‎docker/docker-frontend.sh

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one or more
4+
# contributor license agreements. See the NOTICE file distributed with
5+
# this work for additional information regarding copyright ownership.
6+
# The ASF licenses this file to You under the Apache License, Version 2.0
7+
# (the "License"); you may not use this file except in compliance with
8+
# the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
set -e
19+
20+
cd /app/superset-frontend
21+
npm install -g npm@7
22+
npm install -f --no-optional --global webpack webpack-cli
23+
npm install -f --no-optional
24+
25+
echo "Running frontend"
26+
npm run dev

‎docker/docker-init.sh

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one or more
4+
# contributor license agreements. See the NOTICE file distributed with
5+
# this work for additional information regarding copyright ownership.
6+
# The ASF licenses this file to You under the Apache License, Version 2.0
7+
# (the "License"); you may not use this file except in compliance with
8+
# the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
#
18+
set -e
19+
20+
#
21+
# Always install local overrides first
22+
#
23+
/app/docker/docker-bootstrap.sh
24+
25+
STEP_CNT=4
26+
27+
echo_step() {
28+
cat <<EOF
29+
30+
######################################################################
31+
32+
33+
Init Step ${1}/${STEP_CNT} [${2}] -- ${3}
34+
35+
36+
######################################################################
37+
38+
EOF
39+
}
40+
ADMIN_PASSWORD="admin"
41+
# If Cypress run – overwrite the password for admin and export env variables
42+
if [ "$CYPRESS_CONFIG" == "true" ]; then
43+
ADMIN_PASSWORD="general"
44+
export SUPERSET_CONFIG=tests.superset_test_config
45+
export SUPERSET_TESTENV=true
46+
export ENABLE_REACT_CRUD_VIEWS=true
47+
export SUPERSET__SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://superset:superset@db:5432/superset
48+
fi
49+
# Initialize the database
50+
echo_step "1" "Starting" "Applying DB migrations"
51+
superset db upgrade
52+
echo_step "1" "Complete" "Applying DB migrations"
53+
54+
# Create an admin user
55+
echo_step "2" "Starting" "Setting up admin user ( admin / $ADMIN_PASSWORD )"
56+
superset fab create-admin \
57+
--username admin \
58+
--firstname Superset \
59+
--lastname Admin \
60+
--email [email protected] \
61+
--password $ADMIN_PASSWORD
62+
echo_step "2" "Complete" "Setting up admin user"
63+
# Create default roles and permissions
64+
echo_step "3" "Starting" "Setting up roles and perms"
65+
superset init
66+
echo_step "3" "Complete" "Setting up roles and perms"
67+
68+
if [ "$SUPERSET_LOAD_EXAMPLES" = "yes" ]; then
69+
# Load some data to play with
70+
echo_step "4" "Starting" "Loading examples"
71+
# If Cypress run which consumes superset_test_config – load required data for tests
72+
if [ "$CYPRESS_CONFIG" == "true" ]; then
73+
superset load_test_users
74+
superset load_examples --load-test-data
75+
else
76+
superset load_examples
77+
fi
78+
echo_step "4" "Complete" "Loading examples"
79+
fi

0 commit comments

Comments
 (0)