Support orchestrated Azure virtual machine scale sets

[cyclelabs-ryanberger]

What feature do you want to see added?

Hi,

I am a long time user of the Azure VM Agent plugin and have recently incorporated the virtual machine scale set functionality within the plugin. When I did this, I surprisingly found out that it only support flexible virtual machine scale sets not not uniform virtual machine scale sets. We use uniform virtual machine scale sets with many other CI/CD platforms and the flow is like such:

1. Jenkinsfile is triggered - either manually or via cron timer
2. Jenkinsfile requests the label associated with the cloud template. Right now, wether using a flexible VMSS a not, an ARM template is deployed with a custom script that checks the worker into Jenkins using JNLP. Either its an individual VM, or its part of the flexible VMSS.
   - In other CI/CD platforms, it looks like this: pipeline interacts with the uniform VMSS via az cli and asks it to spin up an instance that will be our worker
3. The worker is spun up and any custom script extensions are run on it
4. These custom script extensions usually would check the worker in to Jenkins
5. The worker runs the pipeline steps as declared in the Jenkinsfile, pipeline passes or fails.
6. Retention strategy kicks in and spins down the instance depending on that configuration.
   - In other CI/CD platforms, it looks like this: pipeline interacts with the uniform VMSS via az cli and spins down the instance

I think the idea for using flexible was the dependency on the ARM template using a customScript to actually connect the worker to the controller using JNLP. BUT these flexible VMSS's do not allow custom scripts/extensions to be configured to them which is causing issues for us.

• One real example of this impacting us currently is attempting to use the Azure Monitoring Agent on our Jenkins workers to check them into a LogAnalytics Workspace. Microsoft recommends you install this as VMSS extension, but with flexible VMSS's, we cannot do that. I was trying to get it working using the initialization script within the cloud template config, using PowerShell, but I cannot get this working properly - and even if I could it won't scale well as the version changes frequently so it will always have to be updated. On all of our other CI/CD platforms, we install it with the VMSS extension and it works flawlessly without issue.

I am simply asking if there is a possibility of using either uniform OR flexible VMSS's with the plugin. Maybe there is a huge reason why only one works but the other wouldn't be possible. Just wanted to throw this out there. If any more details are needed, please feel free to ask me.

Upstream changes

No response

Are you interested in contributing this feature?

No response

[timja]

I think it was just low effort implementation, should be fine to implement.

[cyclelabs-ryanberger]

@timja Thanks! I am more than happy to help with this however I can; I probably can't write the code for the plugin to support it but I can test/help out on that front. But just to humor myself, I will have a look at the code and see if maybe it's something I could learn. I've been such a long time user, I'd like to contribute to the project if I can.

[cyclelabs-ryanberger]

@timja I wanted to add more context to this: right now with the flexible scale sets, they do not support VM Extensions due to the way the VM's are created within the scaleset, you can add an extension to the flexible VMSS, but it will not apply it to the VM's that are created within it. This poses an issue because if you want to enroll VM's within the flexible VMSS into Azure Monitoring via the Azure Monitoring agent, the only supported way of installing this now is by using their AMA VMSS Extension. It makes the enrollment process super simple and works great on uniform VMSS's.

The issue we're facing is that we are utilizing the flexible VMSS integration via the Azure VM Agent plugin to spin up our Jenkins workers, but have no way of applying VM extension to the VM's as they are spun up, to enroll them into Azure Monitoring via the Azure Monitoring Agent.

If the plugin supported using uniform VMSS's, this would not be an issue for the AMA or ANY other VM extension that anyone needed.

EDIT: I mixed up the scaleset type, uniform vs flexible.

[timja]

I've done some scaffolding here: https://github.com/jenkinsci/azure-vm-agents-plugin/compare/master...timja:azure-vm-agents-plugin:vmss-uniform?expand=1

Will see if I have time to get back to it.

Its quite a lot of work as all of the deployment / scaling needs to be managed separately.
and connection / retrieving instance ID is different

Along with not being able to pass the connection secret for inbound agent....

Not very straightforward but I'll see what I can do