Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Host Mac/Linux #4141

Closed
wants to merge 67 commits into from

Conversation

aishwaryabh
Copy link
Contributor

@aishwaryabh aishwaryabh commented Nov 1, 2024

Issue describing the changes in this PR

resolves #3886

This PR creates a custom host for mac and linux. This PR does not cover signing the artifacts and simply generates them and modifies the artifact assembler.

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)

aishwaryabh and others added 30 commits September 24, 2024 12:05
* this doesn't work

* this finally works now

* default should be oop host

* added edge cases

* fixing formatting

* using determineTargetFramework

* saving tests

* adding copy step for OOP

* fixing the webhost reference

* removing test

* setting host version in ps script

* update to validate worker versions script

* update versions for worker packages

* adding ToString

* validating worker version

* validate worker versions

* updating csproj to compile

* addressing pr feedback

* updating build steps

* adding build step

* fixing build step

* trying to get this working

* reverrting target runtimes

* updating tests

* adding dotnet info step

* adding changes

* removing extra test

* trying to specify architecture

* modifying tests to see if they work

* narrowing down to tests that are failing

* trying to see if it works with nobuild flag

* addressing pr feedback

* updating tests with latest logging

* addressing comments and marking flaky tests

* updating so that we are only using net8 framework

* pushing change for branch build

* adding single quotes

* reverting quotes

* adding code mirror fiile

* updating build step

* updatinng build steps

* updating build step

* adding step for dotnet publish

* set inprochost compilation system to diff value and skip flaky test

* updating public build pipeline to trigger

* adding extra changes for pipeline

* public build yml

* updating official build

* readd net8 build artifact step

* readding space back

* addressing initial comments

* adding explicit openTelemetry dlls

* simplifying logic of startHostAction

* addressing PR feedback

* start tests

* changing some of the tests back

* reverting test back to normal

* fixing spacing for csproj

* addressing PR feedback

* adding extra variable

* adding logic for edge case scenarios

* added edge cases tests

* removing extra line in node

* addressing comments

* moving validate host runtime to its own method

* forgot to add return statement
* this doesn't work

* this finally works now

* default should be oop host

* added edge cases

* fixing formatting

* using determineTargetFramework

* saving tests

* adding copy step for OOP

* fixing the webhost reference

* removing test

* setting host version in ps script

* update to validate worker versions script

* update versions for worker packages

* adding ToString

* validating worker version

* validate worker versions

* updating csproj to compile

* addressing pr feedback

* updating build steps

* adding build step

* fixing build step

* trying to get this working

* reverrting target runtimes

* updating tests

* adding dotnet info step

* adding changes

* removing extra test

* trying to specify architecture

* modifying tests to see if they work

* narrowing down to tests that are failing

* trying to see if it works with nobuild flag

* addressing pr feedback

* updating tests with latest logging

* addressing comments and marking flaky tests

* updating so that we are only using net8 framework

* pushing change for branch build

* adding single quotes

* reverting quotes

* adding code mirror fiile

* updating build step

* updatinng build steps

* updating build step

* adding step for dotnet publish

* set inprochost compilation system to diff value and skip flaky test

* updating public build pipeline to trigger

* adding extra changes for pipeline

* public build yml

* updating official build

* readd net8 build artifact step

* readding space back

* addressing initial comments

* adding explicit openTelemetry dlls

* simplifying logic of startHostAction

* addressing PR feedback

* start tests

* changing some of the tests back

* reverting test back to normal

* fixing spacing for csproj

* addressing PR feedback

* adding extra variable

* adding logic for edge case scenarios

* added edge cases tests

* initial changes

* removing extra line in node

* removing files not needed for custom host

* adding build steps

* addressing comments

* moving validate host runtime to its own method

* forgot to add return statement

* trying out creating directory

* remove skip artifact gen

* updating build step

* updating build to include in official pipeline

* updating publishing custom host step

* code mirror

* trying to get official build working

* program cs update

* updating custom host to work with parsing local.settings.json

* console read line for stalling process

* removing changes not needed for custom host

* only including inproc6 and inproc8 changes

* addressing initial comments

* removing skip artifact gen flag

* verbose logging

* tryna add release pipeline

* windows yaml changes

* addressing comments about yml

* updating log message

* fixing spacing

* fixing spacing again

* trying spacing again

* spacing change

* tryan fix spacing this way

* spacing

* spacing

* new line

* regular indentation

* super indent

* changing spacing back to normal

* addressing comments

* fixing pipeline
* modifying nuget.config

* adding to official

* official build

* show up pls

* trying agani

* fixing this

* trying again

* see if this works

* idk man

* lmao bro

* adding nuget key with protocol

* changing pool

* remove protocol change

* removing pool change

* ignoring warning

* updating sccignore

* adding yml dependency

* adding eng repository

* moving resources to the top

* adding 1es repository and switchign to windows pool

* fixign spacing

* tryign diff formatting

* adding pool tag instead

* cleaning up branch

* adding trigger and finding pipeline utilities

* removing extra space

* trying without extra dash

* printint out current dir

* updating spacing

* i think we are in root alr

* adding pr trigger and cleaning up pr

* reverting files back to normal

* reverting code mirror back to normal

* forgot to remove file

* updating pipeline utilitites

* modifying name and pr trigger

* changes to code mirror

* removing extra directory

* removing extra code mirror step
* build steps

* trying to parallelize build steps

* Revert "build steps"

This reverts commit 9aa82fa.

* Reapply "build steps"

This reverts commit a7b7d46.

* Revert "trying to parallelize build steps"

This reverts commit d10ef5c.
…roduce the package (#3843)

* Adding artifact assembler program which collects artifacts from different sources and assemble the final package for core tools.

* Adding README

* Remove downloaded artifact directories. Minor language update to README
* adding trait for e2e tests to use in final pipeline

* test changes

* removing duplicate test

* fixing tests

* trying to filter tests again

* addressing comments

* adding extra label

* changing spacing back

* trying spacing again

* changing spacing back

* four spaces

* adding trait

* getting build to work

* removing constant

* addressing comments

* group trait
* assemble artifacts

* adding code mirror to test

* try awaiting tasks to save space

* removing --net8 dependency

* trying to delete directory one at a time

* trying again with extracting

* diff build steps for vs and cli

* forgot to include other step for var

* trying out new changes

* clean up each zip file after extracting

* trying another approach

* try to delete each directory after copying

* removing logic for creating zips rn

* including e2e test changes and add powershell file

* removing multi targeting

* pass in staging directory

* adding param

* adding logging

* printing out paths

* adding isolated e2e tests and changing verbosity for logger

* adding a new label

* comments

* adding more clarifying comments

* removing extra test and types

* adding logging statement for failing task if we fail tests

* updating tests to only run in winx64 and winx86

* target directory

* target directory

* changes to msi file

* print out directory

* resolving path

* reorg files and try to set env variable

* dont delete artifacts

* update generate msi files

* add doller sign for variable name and fix e2e test

* changing runtime version

* updating to get correct cli version

* lol forgot to have content in zip ps script

* Update src/Azure.Functions.ArtifactAssembler/ArtifactAssembler.cs

Co-authored-by: Lilian Kasem <[email protected]>

* Update src/Azure.Functions.ArtifactAssembler/ArtifactAssembler.cs

Co-authored-by: Lilian Kasem <[email protected]>

* Update src/Azure.Functions.ArtifactAssembler/ArtifactAssembler.cs

Co-authored-by: Lilian Kasem <[email protected]>

* Update src/Azure.Functions.ArtifactAssembler/ArtifactAssembler.cs

Co-authored-by: Lilian Kasem <[email protected]>

* Update src/Azure.Functions.ArtifactAssembler/ArtifactAssembler.cs

Co-authored-by: Lilian Kasem <[email protected]>

* addressing comments

* reverting programcs back to the way it was

* staging directory not correct

* merging

* Revert "merging"

This reverts commit 89cc110.

* fixing spacing

* wrong directory

* trying to free up disk space

* removing custom code mirror

* removing dollar signs

* updating build number

* adding variables back

* fix build number

* fixing candle

* fixing msi files

* reverting files back to normal

---------

Co-authored-by: Lilian Kasem <[email protected]>
* adding system.text.json

* update patch

* move to 8.0.10

* fixing indentation

* trying to change version
…ine (#3884)

* adding generate metadata file

* remove code mirror

* get correct inproc version

* print out metadata file

* fixing zip issue

* fixing release number

* fixing release number again

* removing zip artifact script and fixing code mirror
…gn func.exe and nethost.dll, and remove inproc subfolders for minified default version) (#3879)

* trying to trigger pr build

* removing pr trigger and custom path

* testing out changes

* testing out chagnes

* adding comment

* reverting changes for branch

* removing double *

* testing 1es template changes

* forgot to add 1es as a repository

* fixing publish step

* removing code mirror custom step

* adding sign step and add useDotnetTask

* minified version should not have inproc6/inproc8 subdirectories

* installing net9 and changing path

* pipeline utilities to see if that works for install netsdk

* install net9

* installing net6 as well

* move sign step before copy

* adding self-contained
@aishwaryabh aishwaryabh marked this pull request as ready for review November 6, 2024 01:20
@aishwaryabh aishwaryabh requested a review from a team as a code owner November 6, 2024 01:20
command: publish
publishWebProjects: false
zipAfterPublish: false
arguments: -c Release -r osx-x64 -o $(Build.SourcesDirectory)/pkg_output/osx/osx-x64 --self-contained
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this --self-contained argument?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so we needed it for windows so prob need for linux/mac?

@kshyju
Copy link
Member

kshyju commented Nov 6, 2024

Can you rebase this PR branch?

@aishwaryabh aishwaryabh mentioned this pull request Nov 7, 2024
5 tasks
@aishwaryabh
Copy link
Contributor Author

aishwaryabh commented Nov 7, 2024

This PR was WAY too messy with the force push of feature/oop-host to rebase. I've decided to close this PR and open this one instead. I have addressed the comments there as well.

@aishwaryabh aishwaryabh closed this Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants