Skip to content

Commit

Permalink
[RND-485] Publishing to Azure Artifacts (#224)
Browse files Browse the repository at this point in the history
* Adds workflows to publish Meadowlark Packages on Azure Artifacts.

* Some progres.

* Changes on publish configuration.

* v0.4.0-dev.0

* Partially complete version management script

* Simplify

* Manage version through tagging only
Refactor for automatic triggering
Use --workspaces to capture all of them

* Need a working directory

* Need a working directory at the step level

* Revert "Need a working directory at the step level"

This reverts commit 3630667.

* Capitalization problem

* Retrieve full Git history

* Include version.js and fix lint

* Fix linting problem, then remove lint

* Use --tags flag

* Echo release version

* Only run on merge to main

* Rename file

* Set scope to @Edfi

* Standardize this file

* Update metaed references

* Security updates (#9)

* Fix the build command (#10)

* Corrections to the key handling

* Debugging

* Move the -n earlier

* Remove debugging code

* Attempting to calculate single depth

* Improve handling when there have been no other commits

* Refactor version.js and move to eng directory

* Path corrections

* Update package.json (#11)

* debugging (#12)

* Remove debug statement

* remove unnecessary comment

* Reset all version numbers

Update the workspace order for proper linking by npm install

* Fix bad workspace listing and reset version numbers

* Readme updates (#17)

* Debugging

* Rename job

* Debugging updates

* Continuing to debug

* Initial on-release file

* Package promotion process

* forgot the jobs node

* Need to check out the repo

* Remove debugging code

* Update the dev release tag

* Try getting username from secrets

* Everyone's using same version of actions/checkout

* Try using vars

* Case correction

* Remove temporary comment

* Trivial readme update

* Rename file

* Refactor bump

* Separate script for setting the version

* Typo fix

* Release management doc

---------

Co-authored-by: David Jimenez Barrantes <[email protected]>
  • Loading branch information
stephenfuqua and DavidJGapCR authored Mar 23, 2023
1 parent 9db1d62 commit 47a94fe
Show file tree
Hide file tree
Showing 36 changed files with 6,578 additions and 98 deletions.
53 changes: 53 additions & 0 deletions .github/workflows/on-merge-to-main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# SPDX-License-Identifier: Apache-2.0
# Licensed to the Ed-Fi Alliance under one or more agreements.
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

name: On Merge to Main
on:
push:
branches:
- main

env:
API_URL: https://api.github.com/repos/${{ github.repository }}
GITHUB_TOKEN: ${{ secrets.PAT_ATTACH_TO_RELEASE }}

jobs:
create-pre-releases:
name: Create Pre-Releases
runs-on: ubuntu-latest
defaults:
run:
shell: pwsh
working-directory: eng/version
steps:
- name: Checkout the Repo
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
# Need full Git history for calculating the version number
fetch-depth: 0w

- name: Create Meadowlark Pre-Release
run: |
npm install
$version = $(node ./bump.js)
if ($LASTEXITCODE -ne 0) {
throw 'Unable to retrieve version number'
}
"Creating pre-release $version" | Out-Host
$body = @{
tag_name = "$version"
target_commitish = "main"
name = $version
body = ""
draft = $false
prerelease = $true
generate_release_notes = $false
} | ConvertTo-Json
$headers = @{
Accept = "application/vnd.github+json"
Authorization = "Bearer ${{ env.GITHUB_TOKEN }}"
}
Invoke-RestMethod -Method POST -Uri ${{ env.API_URL }}/releases -Body $body -Headers $headers
71 changes: 71 additions & 0 deletions .github/workflows/on-prerelease.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# SPDX-License-Identifier: Apache-2.0
# Licensed to the Ed-Fi Alliance under one or more agreements.
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

name: On Pre-Release
on:
release:
types:
- prereleased

env:
ARTIFACTS_API_KEY: ${{ secrets.AZURE_ARTIFACTS_PERSONAL_ACCESS_TOKEN }}
GITHUB_TOKEN: ${{ secrets.PAT_ATTACH_TO_RELEASE }}

defaults:
run:
working-directory: Meadowlark-js

jobs:
azure-publish:
name: Publish to Azure Artifacts
runs-on: ubuntu-latest
outputs:
hash-code: ${{ steps.hash-code.outputs.hash-code }}
steps:
- name: Checkout repository
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0
with:
# Need full Git history for calculating the version number
fetch-depth: 0

- name: Setup Node
uses: actions/setup-node@8c91899e586c5b171469028077307d293428b516 # v3.5.1
with:
node-version: "16"
cache: "npm"
cache-dependency-path: "**/package-lock.json"

- name: Install dependencies
if: ${{ steps.modules-cache.outputs.cache-hit != 'true' }}
run: npm install

- name: Update version to match tag
working-directory: eng/version
run: |
npm install
node ./set.js ${{ github.ref_name }}
- name: Transpile to JS
run: npm run build --workspaces

- name: Create .npmrc for Azure Artifacts publishing
run: |
set +x # disable echo to shell so that the base64-encoded API key never escapes into the log.
# Create the user .npmrc file with proper settings
API_KEY=`echo -n "${{ env.ARTIFACTS_API_KEY }}" | base64`
touch $HOME/.npmrc
echo "; begin auth token" >> $HOME/.npmrc
echo //pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/:username=ed-fi-alliance >> $HOME/.npmrc
echo //pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/:_password=$API_KEY >> $HOME/.npmrc
echo //pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/:[email protected] >> $HOME/.npmrc
echo "; end auth token" >> $HOME/.npmrc
- name: Publish
run: npm publish --workspaces --tag dev

# NOTE: there are uncommitted version number changes in package.json. No
# problem! Our strategy is to rely on the tag history for version numbers,
# not on package.json.
2 changes: 1 addition & 1 deletion .github/workflows/on-pullrequest-dockerfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0

- uses: hadolint/hadolint-action@54c9adbab1582c2ef04b2016b760714a4bfde3cf # v3.1.0
name: Run Linter
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/on-pullrequest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
uses: ed-fi-alliance-oss/ed-fi-actions/.github/workflows/repository-scanner.yml@main

analyze-code:
# This Action will fail if run on anything other than a pull request
if: github.event_name == 'pull_request'
name: Analyze Code Dependencies
runs-on: ubuntu-latest
Expand Down Expand Up @@ -159,7 +160,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0

- name: Setup Node
uses: actions/setup-node@8c91899e586c5b171469028077307d293428b516 # v3.5.1
Expand Down Expand Up @@ -243,7 +244,7 @@ jobs:
# { db: "PGSQL", plugin: "@edfi/meadowlark-postgresql-backend" }, # Restore after RND-494
steps:
- name: Checkout repository
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0

- name: Setup Node
uses: actions/setup-node@8c91899e586c5b171469028077307d293428b516 # v3.5.1
Expand Down
32 changes: 32 additions & 0 deletions .github/workflows/on-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# SPDX-License-Identifier: Apache-2.0
# Licensed to the Ed-Fi Alliance under one or more agreements.
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.


name: On Release
on:
release:
types:
- released

env:
ARTIFACTS_API_KEY: ${{ secrets.AZURE_ARTIFACTS_PERSONAL_ACCESS_TOKEN }}
ARTIFACTS_USERNAME: ${{ vars.AZURE_ARTIFACTS_USER_NAME }}

jobs:
promote-Azure-artifact:
name: Promote Azure Artifact
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.3.0

- name: Promote Package
shell: pwsh
working-directory: eng
run: |
./promote-packages.ps1 `
-Version ${{ github.ref_name }} `
-Username ${{ env.ARTIFACTS_USERNAME }} `
-PersonalAccessToken ${{ env.ARTIFACTS_API_KEY}}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
.env.*
!.env.example
Meadowlark-js/.env-docker
**/node_modules/
4 changes: 2 additions & 2 deletions Meadowlark-js/.npmrc
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
registry=https://www.myget.org/F/ed-fi/npm/
always-auth=true
@edfi:registry=https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/
always-auth=false
2 changes: 2 additions & 0 deletions Meadowlark-js/backends/meadowlark-mongodb-backend/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@edfi:registry=https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/
always-auth=false
10 changes: 5 additions & 5 deletions Meadowlark-js/backends/meadowlark-mongodb-backend/package.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"name": "@edfi/meadowlark-mongodb-backend",
"main": "dist/index.js",
"version": "0.3.0-dev.0",
"version": "0.3.0-pre-36",
"description": "Meadowlark backend plugin for MongoDB",
"license": "Apache-2.0",
"publishConfig": {
"registry": "https://www.myget.org/F/ed-fi/npm/"
"registry": "https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/"
},
"files": [
"/dist",
Expand All @@ -19,9 +19,9 @@
"build:copy-non-ts": "copyfiles -u 1 -e \"**/*.ts\" \"src/**/*\" dist --verbose"
},
"dependencies": {
"@edfi/meadowlark-authz-server": "0.3.0-dev.0",
"@edfi/meadowlark-core": "0.3.0-dev.0",
"@edfi/meadowlark-utilities": "0.3.0-dev.0",
"@edfi/meadowlark-authz-server": "^v0.3.0-pre-35",
"@edfi/meadowlark-core": "^v0.3.0-pre-35",
"@edfi/meadowlark-utilities": "^v0.3.0-pre-35",
"async-retry": "^1.3.3",
"mongodb": "^4.11.0",
"ramda": "0.27.2",
Expand Down
2 changes: 2 additions & 0 deletions Meadowlark-js/backends/meadowlark-opensearch-backend/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@edfi:registry=https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/
always-auth=false
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"name": "@edfi/meadowlark-opensearch-backend",
"main": "dist/index.js",
"version": "0.3.0-dev.0",
"version": "0.3.0-pre-36",
"description": "Meadowlark backend plugin for OpenSearch",
"license": "Apache-2.0",
"publishConfig": {
"registry": "https://www.myget.org/F/ed-fi/npm/"
"registry": "https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/"
},
"files": [
"/dist",
Expand All @@ -19,9 +19,9 @@
"build:copy-non-ts": "copyfiles -u 1 -e \"**/*.ts\" \"src/**/*\" dist --verbose"
},
"dependencies": {
"@edfi/meadowlark-core": "0.3.0-dev.0",
"@edfi/meadowlark-utilities": "0.3.0-dev.0",
"@edfi/metaed-core": "3.2.1-dev.13",
"@edfi/meadowlark-core": "^v0.3.0-pre-35",
"@edfi/meadowlark-utilities": "^v0.3.0-pre-35",
"@edfi/metaed-core": "^4.0.0",
"@opensearch-project/opensearch": "^2.1.0"
},
"devDependencies": {
Expand Down
2 changes: 2 additions & 0 deletions Meadowlark-js/backends/meadowlark-postgresql-backend/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
@edfi:registry=https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/
always-auth=false
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"name": "@edfi/meadowlark-postgresql-backend",
"main": "dist/index.js",
"version": "0.3.0-dev.0",
"version": "0.3.0-pre-36",
"description": "Meadowlark backend plugin for PostgreSQL",
"license": "Apache-2.0",
"publishConfig": {
"registry": "https://www.myget.org/F/ed-fi/npm/"
"registry": "https://pkgs.dev.azure.com/ed-fi-alliance/Ed-Fi-Alliance-OSS/_packaging/EdFi/npm/registry/"
},
"files": [
"/dist",
Expand All @@ -19,8 +19,8 @@
"build:copy-non-ts": "copyfiles -u 1 -e \"**/*.ts\" \"src/**/*\" dist --verbose"
},
"dependencies": {
"@edfi/meadowlark-core": "0.3.0-dev.0",
"@edfi/meadowlark-utilities": "0.3.0-dev.0",
"@edfi/meadowlark-core": "^v0.3.0-pre-35",
"@edfi/meadowlark-utilities": "^v0.3.0-pre-35",
"pg": "^8.8.0",
"pg-format": "^1.0.4",
"ramda": "0.27.2",
Expand Down
2 changes: 1 addition & 1 deletion Meadowlark-js/lerna.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"packages": [
"packages/*"
],
"version": "0.3.0-dev.0",
"version": "v0.3.0-pre-35",
"npmClient": "npm",
"useWorkspaces": true,
"command": {
Expand Down
Loading

0 comments on commit 47a94fe

Please sign in to comment.