[YUNIKORN-2854]Add queue maxRunningApps metrics

[kaichiachen]

What is this PR for?

Add a new metric - maxRunningApps for Prometheus pull-based monitoring. The maxapplications property is an integer value, larger than 1, which allows you to limit the number of running applications for the queue.
I fixed the resource name - "Apps" as I see maxRunningApps is a standalone metrics independent from resources.Resource. See https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/objects/queue.go#L86

type Queue struct {
	QueuePath string // Fully qualified path for the queue
	Name      string // Queue name as in the config etc.

	...
	pending             *resources.Resource       // pending resource for the apps in the queue
	allocatedResource   *resources.Resource       // allocated resource for the apps in the queue
	preemptingResource  *resources.Resource       // preempting resource for the apps in the queue
	...
	maxResource            *resources.Resource // When not set, max = nil
	guaranteedResource     *resources.Resource // When not set, Guaranteed == 0
	...
	maxRunningApps         uint64
	runningApps            uint64

What type of PR is it?

• - Bug Fix
• - Improvement
• - Feature
• - Documentation
• - Hot Fix
• - Refactoring

Todos

• - Task

What is the Jira issue?

https://issues.apache.org/jira/browse/YUNIKORN-2854?filter=-1

How should this be tested?

1. Deploy k8s cluster with kind
2. forward metrics endpoint by kubectl port-forward svc/yunikorn-service -n yunikorn 9080:9080
3. verify metrics are exposed through belowing

#> curl -s http://localhost:9080/ws/v1/metrics  | grep yunikorn_root_queue_resource
# HELP yunikorn_root_queue_resource Queue resource metrics. State of the resource includes `guaranteed`, `max`, `allocated`, `pending`, `preempting`, 'maxRunningApps'.   <---- help message is updated
# TYPE yunikorn_root_queue_resource gauge
yunikorn_root_queue_resource{resource="apps",state="maxRunningApps"} 0        <------ new added metrics
yunikorn_root_queue_resource{resource="ephemeral-storage",state="max"} 6.19061035008e+11
yunikorn_root_queue_resource{resource="memory",state="allocated"} 0
yunikorn_root_queue_resource{resource="memory",state="max"} 4.9624363008e+10
yunikorn_root_queue_resource{resource="memory",state="pending"} 0
yunikorn_root_queue_resource{resource="pods",state="allocated"} 3
yunikorn_root_queue_resource{resource="pods",state="max"} 330
yunikorn_root_queue_resource{resource="pods",state="pending"} 0
yunikorn_root_queue_resource{resource="vcore",state="allocated"} 0
yunikorn_root_queue_resource{resource="vcore",state="max"} 36000
yunikorn_root_queue_resource{resource="vcore",state="pending"} 0

4. configure maxSunningApps by

apiVersion: v1
data:
  queues.yaml: |2
    partitions:
      - name: default
        placementrules:
          - name: tag
            value: namespace
            create: true
        queues:
          - name: root
            submitacl: '*'
            maxapplications: 10
kind: ConfigMap

5. Verify metrics are reflected in endpoint

#> curl -s http://localhost:9080/ws/v1/metrics  | grep maxRunningApps           
# HELP yunikorn_queue_resource Queue resource metrics. State of the resource includes `guaranteed`, `max`, `allocated`, `pending`, `preempting`, 'maxRunningApps'.
yunikorn_queue_resource{queue="root",resource="apps",state="maxRunningApps"} 10
# HELP yunikorn_root_dev_6kdna_queue_resource Queue resource metrics. State of the resource includes `guaranteed`, `max`, `allocated`, `pending`, `preempting`, 'maxRunningApps'.
# HELP yunikorn_root_dev_ruf38_queue_resource Queue resource metrics. State of the resource includes `guaranteed`, `max`, `allocated`, `pending`, `preempting`, 'maxRunningApps'.
# HELP yunikorn_root_queue_resource Queue resource metrics. State of the resource includes `guaranteed`, `max`, `allocated`, `pending`, `preempting`, 'maxRunningApps'.
yunikorn_root_queue_resource{resource="apps",state="maxRunningApps"} 10

Screenshots (if appropriate)

Questions:

• - The licenses files need update.
• - There is breaking changes for older versions.
• - It needs documentation.

[pbacsko]

Some remarks

[pbacsko]

Please check the failed test case

[kaichiachen]

Please check the failed test case

Sure @pbacsko I fixed UT issue and updated my change

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.49%. Comparing base (7391aeb) to head (0854845).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1012   +/-   ##
=======================================
  Coverage   82.48%   82.49%           
=======================================
  Files          97       97           
  Lines       15627    15635    +8     
=======================================
+ Hits        12890    12898    +8     
  Misses       2457     2457           
  Partials      280      280           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pbacsko]

Final comments

[pbacsko]

+1