Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: reserved system name checks when creating templates #746

Open
wants to merge 104 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
b8b6853
feat: added total results counts to list requests. This simplifies cl…
Vafilor Nov 24, 2020
f0436f0
fix: added checks to workspace/workflow templates on creation to make…
Vafilor Nov 25, 2020
f6be373
add migration for TFOD and CVAT
savan77 Nov 30, 2020
59492c0
chore: Add issue templates
rushtehrani Nov 30, 2020
446f591
Merge pull request #755 from onepanelio/chore/issue-templates
rushtehrani Nov 30, 2020
2366cbc
add migration in db and add comments
savan77 Nov 30, 2020
83812bb
update TFOD based on last migration
savan77 Nov 30, 2020
72fa2f1
update template to the previous one
savan77 Nov 30, 2020
c556e65
use latest template for up
savan77 Nov 30, 2020
1946fdd
Rename 20201130130433_update_tfod_patch.go to 20201130130433_update_t…
rushtehrani Nov 30, 2020
0639f10
Fix template name and package name
rushtehrani Nov 30, 2020
8c00552
remove namespace from path
savan77 Nov 30, 2020
dbc0081
update indentation in the template
savan77 Nov 30, 2020
762abe8
Merge pull request #756 from savan77/cvat_tfod_migration
rushtehrani Nov 30, 2020
39e9c2e
wip - move env vars to workspace creation
rushtehrani Dec 2, 2020
b246922
fix: fix existing workspace templates to add sys- prefix, including a…
Vafilor Dec 3, 2020
198926a
fix: long names for existing system workflow templates
Vafilor Dec 3, 2020
a7f1f49
fix: issue where workflow template name kept adding `sys-` prefix.
Vafilor Dec 3, 2020
7fad735
fix: changed uid to use fmt.errorf instead of errors.new - this simpl…
Vafilor Dec 3, 2020
4178efd
fix: removing sys- prefix multiple times in down migration
Vafilor Dec 3, 2020
93949bf
fix: bug adding sys name back in in down migration
Vafilor Dec 3, 2020
906e7c4
fix: issue where container env vars were not being updated
Vafilor Dec 4, 2020
6ca3d60
update method name to be more descriptive
rushtehrani Dec 4, 2020
a85f972
Merge pull request #757 from onepanelio/fix/dynamic-envs
Vafilor Dec 4, 2020
f11a36f
Merge pull request #765 from Vafilor/feat/requests.updates
Vafilor Dec 5, 2020
b4ea002
support resource limits for GPUs in Workspaces
rushtehrani Dec 7, 2020
78fde01
support resource limits for GPUs in Workflows
rushtehrani Dec 7, 2020
3abdda5
name methods more explicitly
rushtehrani Dec 8, 2020
70d604b
test: istio inject var instead of tty
Vafilor Dec 8, 2020
e29b626
fix: added timeout to k8s so it doesn't wait forever.
Vafilor Dec 8, 2020
2331e9f
Merge pull request #773 from Vafilor/fix/add.timeout.to.k8s
aleksandrmelnikov Dec 8, 2020
701d31c
feat: update kubernetes timeout to be set via environment variable
Vafilor Dec 8, 2020
28651bf
Merge pull request #774 from Vafilor/fix/add.timeout.to.k8s
aleksandrmelnikov Dec 8, 2020
e42ac38
feat: re-organized data folder for migrations
Vafilor Dec 9, 2020
b181200
update: changed tty to be under the "ONEPANEL_INTERACTIVE_SIDECAR"
Vafilor Dec 9, 2020
f937356
chore: added missing migration comments
Vafilor Dec 9, 2020
be8f157
Merge pull request #770 from rushtehrani/feat/limits
Vafilor Dec 9, 2020
6d16592
fix: yaml errors in new migration data
Vafilor Dec 9, 2020
f8a6670
chore: double quotes to single quotes in yaml
Vafilor Dec 9, 2020
f8f2345
Merge pull request #771 from Vafilor/test/istio.inject
aleksandrmelnikov Dec 9, 2020
2440940
fix: wrong migration file names
Vafilor Dec 9, 2020
16977e3
Merge pull request #775 from onepanelio/fix/wrong.migration.names
Vafilor Dec 9, 2020
f58263d
fix: wrong migration folder
Vafilor Dec 9, 2020
4d28ce4
Merge pull request #776 from onepanelio/fix/wrong.migration.names
Vafilor Dec 9, 2020
95e2516
fix: adjust curl to use specific version and fail on non 200 responses
Vafilor Dec 9, 2020
93a0770
Merge pull request #777 from Vafilor/fix/adjust.curl
aleksandrmelnikov Dec 9, 2020
ae998d0
feat: update tensorflow migration
Vafilor Dec 9, 2020
7f5d52c
chore: migration method comments
Vafilor Dec 9, 2020
cb617d0
Merge pull request #778 from Vafilor/feat/upgrade.tensorflow.workflow
aleksandrmelnikov Dec 9, 2020
92c387a
update: tf training with epochs
Vafilor Dec 9, 2020
ee7c148
Merge pull request #779 from Vafilor/feat/upgrade.tensorflow.workflow
aleksandrmelnikov Dec 9, 2020
37f1c0b
correctly add resources to script templates
rushtehrani Dec 10, 2020
1fa8242
Merge pull request #781 from rushtehrani/fix/script-resources
rushtehrani Dec 11, 2020
5d0d84c
fix: issue where workspace params (like new node pool) were not updat…
Vafilor Dec 11, 2020
50a57f0
chore: revert argo updates
Vafilor Dec 11, 2020
6f9f8ed
fix: bug where parameters query did not actually update with the set …
Vafilor Dec 11, 2020
0aee834
Merge pull request #782 from Vafilor/fix/workspace.node.pool
rushtehrani Dec 11, 2020
909a728
chore: changed newParam to be newValue to better describe the functio…
Vafilor Dec 11, 2020
8735444
Merge pull request #785 from Vafilor/fix/workspace.node.pool
Vafilor Dec 11, 2020
b6abe86
Merge pull request #783 from Vafilor/fix/revert.argo
rushtehrani Dec 11, 2020
5ce0bb1
fix: add resource limits to a main container
Vafilor Dec 12, 2020
9302853
chore: migration method documentation
Vafilor Dec 12, 2020
172c4e6
chore: clean up code for node pool resource selection
Vafilor Dec 12, 2020
f54021c
Merge pull request #788 from Vafilor/fix/gke.gpu
rushtehrani Dec 12, 2020
01cd321
fix: added migration to update jupyterlab workspace template to be ab…
Vafilor Dec 14, 2020
0b68e88
fix: wrong template name
Vafilor Dec 14, 2020
81bd32f
remove extra space
rushtehrani Dec 14, 2020
1cf0373
Merge pull request #793 from Vafilor/fix/onepanelio.core.791-gke.jupy…
rushtehrani Dec 14, 2020
01040a7
set jupyterlab workingDir to data
rushtehrani Dec 16, 2020
7ae4e13
Merge pull request #797 from rushtehrani/fix/jupyterlab-data-dir
aleksandrmelnikov Dec 16, 2020
ed54598
fix: handle case for watching workflow where a timeout occurs
Vafilor Dec 17, 2020
ec2a0f5
feat: increased default timeout for kubernetes to be 3 minutes as tes…
Vafilor Dec 17, 2020
7ec2093
fix: missing setting the initializedMigrations variable for migration…
Vafilor Dec 17, 2020
45f6d4e
feat: updated watching configmap changes so it uses its own kubeclien…
Vafilor Dec 17, 2020
2a01138
chore: codacy
Vafilor Dec 17, 2020
5bd2c96
Merge pull request #799 from Vafilor/fix/closed.web.sockets
rushtehrani Dec 17, 2020
d991898
feat: updated grpc gateway related code
Vafilor Dec 17, 2020
2931d5e
update: fixed missing generated files for grpc and moved generated fi…
Vafilor Dec 17, 2020
1257ab1
chore: generate new files from proto
Vafilor Dec 17, 2020
8b155ba
update: helper image to interact with core
Vafilor Dec 17, 2020
41f8508
update: to go packages because of grpc update
Vafilor Dec 18, 2020
19be6d4
fix: minor cleanup for helper Dockerfile
Vafilor Dec 18, 2020
5b177a1
feat: added option to Makefile to generate api via docker image
Vafilor Dec 18, 2020
4359f6f
chore: updated api to version 0.17.0
Vafilor Dec 18, 2020
bd2d98c
fix: removed non proto files for third party
Vafilor Dec 18, 2020
779cba9
chore: updated contributing guide to include docker version
Vafilor Dec 18, 2020
8dcc38c
fix: wrong docker image for make api-docker
Vafilor Dec 18, 2020
3ad5715
feat: added documentation for helper
Vafilor Dec 18, 2020
76ecdcf
chore: codacy
Vafilor Dec 19, 2020
3a5fc38
chore: codacy
Vafilor Dec 19, 2020
54580a7
Merge pull request #801 from Vafilor/feat/upgrade.grpc
Vafilor Dec 21, 2020
f196c01
update README
rushtehrani Dec 23, 2020
af6e3c8
update README
rushtehrani Dec 23, 2020
0efa00d
Merge pull request #807 from rushtehrani/chore/readme
rushtehrani Dec 23, 2020
d203b04
chore: Fix manifests repo link
rushtehrani Dec 24, 2020
7d062de
fix: added checks to workspace/workflow templates on creation to make…
Vafilor Nov 25, 2020
daa2c39
fix: fix existing workspace templates to add sys- prefix, including a…
Vafilor Dec 3, 2020
59f214d
fix: long names for existing system workflow templates
Vafilor Dec 3, 2020
9c3d55f
fix: issue where workflow template name kept adding `sys-` prefix.
Vafilor Dec 3, 2020
8580915
fix: changed uid to use fmt.errorf instead of errors.new - this simpl…
Vafilor Dec 3, 2020
b5f8d87
fix: removing sys- prefix multiple times in down migration
Vafilor Dec 3, 2020
c4e792a
fix: bug adding sys name back in in down migration
Vafilor Dec 3, 2020
643f8a5
fix: updated migration timestamp as other migrations have been create…
Vafilor Dec 24, 2020
50c75d6
Merge branch 'fix/onepanelio.core.256-reserved.names' of github.com:V…
Vafilor Dec 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**`opctl` version**
```bash
$ opctl version

CLI version: v0.16.0-rc.0
Manifest version: v0.16.0-rc.0
API version: v0.16.0-rc.0
Web UI version: v0.16.0-rc.0
```

**`opctl init` command**
You can find this at the top of your `params.yaml` file. [e.g. `opctl init --provider eks --artifact-repository-provider s3 --gpu-device-plugins nvidia`]

**Kubernetes information**
- Cloud provider: [e.g. AKS, EKS, GKE, Microk8s]
- Kubernetes version: [e.g. 1.17.13]

**Machine information**
- OS: [e.g. Ubuntu 18.04, Windows 10 19042.631, macOS 10.14.6]
- Browser: [e.g. Chrome, Firefox, Safari]

**Screenshots**
If applicable, add screenshots to help explain your problem.
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
28 changes: 26 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
## Database migrations

### Docker

Note: Up migrations are automatically executed when the application is run.

```bash
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel-helper:v1.0.0 goose -dir db/sql create <name> sql # Create migration in db/sql folder
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel-helper:v1.0.0 goose -dir db postgres "${DB_DATASOURCE_NAME}" up # Migrate the DB to the most recent version available
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel-helper:v1.0.0 goose -dir db postgres "${DB_DATASOURCE_NAME}" down # Roll back the version by 1
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel-helper:v1.0.0 goose help # See all available commands
```

### Local

Install `goose`:
```bash
go get -u github.com/pressly/goose/cmd/goose
Expand All @@ -8,13 +21,15 @@ go get -u github.com/pressly/goose/cmd/goose
Note: Up migrations are automatically executed when the application is run.

```bash
goose -dir db create <name> sql # Create migration in db folder
goose -dir db/sql create <name> sql # Create migration in db/sql folder
goose -dir db postgres "${DB_DATASOURCE_NAME}" up # Migrate the DB to the most recent version available
goose -dir db postgres "${DB_DATASOURCE_NAME}" down # Roll back the version by 1
goose help # See all available commands
```

## gRPC installation
## gRPC

### local installation

Install gRPC:
```bash
Expand Down Expand Up @@ -47,6 +62,15 @@ Make sure that your `$GOBIN` is in your `$PATH`.

## API code generation

### Docker

Generate Go and Swagger APIs:
```bash
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel-helper:v1.0.0 make api version=1.0.0
```

### Local Installation

Generate Go and Swagger APIs:
```bash
make api version=1.0.0
Expand Down
24 changes: 19 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,28 @@ jq:
protoc:
protoc -I/usr/local/include \
-Iapi/third_party/ \
-Iapi/ \
api/*.proto \
--go_out=plugins=grpc:api \
--grpc-gateway_out=logtostderr=true,allow_delete_body=true:api \
--swagger_out=allow_merge=true,fqn_for_swagger_name=true,allow_delete_body=true,logtostderr=true,simple_operation_ids=true:api
-Iapi/proto \
--go_out ./api/gen --go_opt paths=source_relative \
--go-grpc_out ./api/gen --go-grpc_opt paths=source_relative \
--go-grpc_opt paths=source_relative \
--grpc-gateway_out ./api/gen \
--grpc-gateway_opt logtostderr=true \
--grpc-gateway_opt allow_delete_body=true \
--grpc-gateway_opt paths=source_relative \
--grpc-gateway_opt generate_unbound_methods=true \
--openapiv2_out ./api \
--openapiv2_opt allow_merge=true \
--openapiv2_opt fqn_for_openapi_name=true \
--openapiv2_opt allow_delete_body=true \
--openapiv2_opt logtostderr=true \
--openapiv2_opt simple_operation_ids=true \
api/proto/*.proto

api: init protoc jq

api-docker: init
docker run --rm --mount type=bind,source="${PWD}",target=/root onepanel/helper:v1.0.0 make api version=$(version)

docker-build:
docker build -t onepanel-core .
docker tag onepanel-core:latest onepanel/core:$(COMMIT_HASH)
Expand Down
8 changes: 2 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,7 @@ Onepanel can be found on [LF AI Landscape](http://landscape.lfai.foundation/sele
## Online demo
We have created an [online demo environment](https://onepanel.typeform.com/to/kQfDX5Vf?product=github) so that you can quickly try Onepanel.

Note that this is a shared demo environment with the following restrictions:

- Data is reset every few hours
- One type of node pool (machine type) with a limit of 5 concurrent nodes
- Certain actions may be restricted
**Note:** This is a shared, read only demo environment that is reset every few hours.

## Quick start
See [quick start guide](https://docs.onepanel.ai/docs/getting-started/quickstart) to get started with the platform of your choice.
Expand All @@ -62,7 +58,7 @@ Onepanel is modular and consists of the following repositories:
[Backend](https://github.com/onepanelio/core/) (this repository) - Code base for backend (Go)\
[Frontend](https://github.com/onepanelio/core-ui/) - Code base for frontend (Angular + TypeScript)\
[CLI](https://github.com/onepanelio/cli/) - Code base for installation and management CLI (Go)\
[Manifests](https://github.com/onepanelio/core-ui/) - Kustomize manifests used by installation and management CLI (YAML)\
[Manifests](https://github.com/onepanelio/manifests/) - Kustomize manifests used by installation and management CLI (YAML)\
[Python SDK](https://github.com/onepanelio/python-sdk/) - Python SDK code and documentation (Python)\
[Templates](https://github.com/onepanelio/templates) - Various Workspace, Workflow, Task and Sidecar Templates\
[Documentation](https://github.com/onepanelio/core-docs/) - The repository for documentation site\
Expand Down
86 changes: 0 additions & 86 deletions api/api.pb.go

This file was deleted.

Loading