Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance testing #3400

Draft
wants to merge 26 commits into
base: main
Choose a base branch
from
+1,575 −0
Draft

Performance testing #3400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6e2b204
performance-test-for-directory-sync
bablu94 Dec 4, 2024
403ccbf
merge main
bablu94 Dec 4, 2024
70759ef
api load test
bablu94 Dec 6, 2024
7fa8128
fomratting fix for load test file
bablu94 Dec 6, 2024
d6a5755
Merge branch 'main' into performance-test
deepakprabhakara Jan 8, 2025
4f1b59e
Add saml20 dependency
niwsa Jan 15, 2025
a85a4e1
Merge branch 'main' into performance-test
niwsa Jan 20, 2025
589516a
Clean up scenarios, we only need 1 for now
niwsa Jan 20, 2025
035fc54
Refactor
niwsa Jan 20, 2025
06e43ec
Fix SAML response path test
niwsa Jan 21, 2025
78260cd
Use original target VUs
niwsa Jan 28, 2025
3fb9811
Remove vu Context and use init stage for caching global values
niwsa Feb 3, 2025
6525665
Convert ops to async and other fixes
niwsa Feb 3, 2025
d9d912b
Split load test into separate files
niwsa Feb 3, 2025
c305b7d
Merge branch 'main' into performance-test
niwsa Feb 3, 2025
5614511
Cleanup and add setuplink ops
niwsa Feb 4, 2025
f35df36
Restore lock files
niwsa Feb 4, 2025
da0eb2a
Minor fix
niwsa Feb 4, 2025
3900d0d
Add test for id fed apps
niwsa Feb 4, 2025
4999ac3
[WIP] SCIM load test
niwsa Feb 5, 2025
0058cc1
Merge branch 'main' into performance-test
niwsa Feb 6, 2025
f442cbf
Initial working commit for scim performance test
niwsa Feb 6, 2025
5f9297b
Add user operations
niwsa Feb 6, 2025
9a55192
Push group related operations
niwsa Feb 7, 2025
2368700
Cleanup and add results to README
niwsa Feb 7, 2025
8cbf7a7
Merge branch 'main' into performance-test
niwsa Feb 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 177 additions & 0 deletions performance-test/Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Steps to run k6

1. installation: sudo apt-get install k6 or refer this https://grafana.com/docs/k6/latest/set-up/install-k6/
2. for run: cd performance-test/k6
3. run command: k6 run <test-file>.js

## SSO APIs Load Test Results

✓ createSSOConnection
✓ getSSOConnection
✓ updateSSOConnection
✓ getSSOConnectionByProduct
✓ deleteSSOConnection
✓ createSetupLink
✓ getSetUpLink
✓ deleteSetUpLink
✓ processing SAML IdP Response



checks.........................: 100.00% 17065 out of 17065
data_received..................: 19 MB 235 kB/s
data_sent......................: 25 MB 310 kB/s

### Performance Metrics

| Metric | Avg | Min | Med | Max | p(90) | p(95) |
| ------------------------ | ------- | ----- | ------- | ------- | -------- | ------- |
| http_req_blocked | 5.18µs | 0s | 3µs | 1.89ms | 5µs | 7µs |
| http_req_connecting | 941ns | 0s | 0s | 1.84ms | 0s | 0s |
| http_req_duration | 66.1ms | 974µs | 13.31ms | 2.41s | 221.15ms | 241.2ms |
| http_req_receiving | 51.52µs | 6µs | 31µs | 14.87ms | 85µs | 128µs |
| http_req_sending | 21.49µs | 2µs | 11µs | 10.03ms | 30µs | 48µs |
| http_req_tls_handshaking | 0s | 0s | 0s | 0s | 0s | 0s |
| http_req_waiting | 66.03ms | 940µs | 13.24ms | 2.41s | 221.04ms | 241.1ms |

### Additional Metrics

| Metric | Value |
| ------------------- | ------------------------------------------ |
| http_reqs | 17066 (212.34/s) |
| http_req_failed | 0.00% (0 out of 17066) |
| iteration_duration | avg=1.59s, min=1.42s, med=1.53s, max=4.07s |
| iterations | 1896 (23.59/s) |
| virtual users (VUs) | min=1, max=55 |
| vus_max | 55 |

running (1m20.4s), 00/55 VUs, 1896 complete and 0 interrupted iterations
load_test ✓ [======================================] 00/55 VUs 1m20s

## Directory Sync APIs Load Test Results

✓ createDSyncSetupLink
✓ getDSyncSetupLink
✓ getDSyncLinkByProduct
✓ deleteDSyncSetupLink
✓ CreateDirectory
✓ getDirectoryByTenantAndProduct
✓ getDirectoryById
✓ getDirectoryByProduct
✓ updateDirectoryName
✓ deleteDirectory

checks.........................: 100.00% 23520 out of 23520
data_received..................: 17 MB 205 kB/s
data_sent......................: 5.7 MB 70 kB/s

### Performance Metrics

| Metric | Avg | Min | Med | Max | p(90) | p(95) |
| ------------------------ | ------- | ----- | ------- | -------- | ------- | ------- |
| http_req_blocked | 4.72µs | 0s | 2µs | 2.39ms | 5µs | 7µs |
| http_req_connecting | 987ns | 0s | 0s | 1.11ms | 0s | 0s |
| http_req_duration | 27.79ms | 922µs | 21.82ms | 187.89ms | 58.51ms | 71.24ms |
| http_req_receiving | 46.12µs | 6µs | 25µs | 15.42ms | 74µs | 119µs |
| http_req_sending | 18.66µs | 2µs | 8µs | 13.03ms | 18µs | 30µs |
| http_req_tls_handshaking | 0s | 0s | 0s | 0s | 0s | 0s |
| http_req_waiting | 27.73ms | 889µs | 21.77ms | 187.87ms | 58.44ms | 71.1ms |

### Additional Metrics

| Metric | Value |
| ------------------- | ----------------------------------------- |
| http_reqs | 23520 (290.54/s) |
| http_req_failed | 0.00% (0 out of 23520) |
| iteration_duration | avg=1.28s, min=1.02s, med=1.24s, max=1.6s |
| iterations | 2352 (29.05/s) |
| virtual users (VUs) | min=1, max=55 |
| vus_max | 55 |

running (1m20.4s), 00/55 VUs, 1896 complete and 0 interrupted iterations
load_test ✓ [======================================] 00/55 VUs 1m20s

## SCIM APIs Load Test Results

✓ createUser
✓ getUser
✓ listUsers
✓ updateUser
✓ replaceUser
✓ createGroup
✓ getGroup
✓ listGroups
✓ updateGroup
✓ addUserToGroup
✓ removeUserFromGroup
✓ deleteGroup
✓ deleteUser


checks.........................: 100.00% 23713 out of 23713
data_received..................: 38 MB 469 kB/s
data_sent......................: 7.3 MB 90 kB/s

### Performance Metrics

| Metric | Avg | Min | Med | Max | p(90) | p(95) |
| ------------------------ | ------- | ------ | ------- | -------- | ------- | ------- |
| http_req_blocked | 5.5µs | 0s | 2µs | 3.93ms | 6µs | 8µs |
| http_req_connecting | 1.15µs | 0s | 0s | 3.82ms | 0s | 0s |
| http_req_duration | 50.08ms | 1.27ms | 48.05ms | 233.12ms | 88.82ms | 96.87ms |
| http_req_receiving | 54.67µs | 6µs | 29µs | 8.83ms | 98µs | 151µs |
| http_req_sending | 15.55µs | 2µs | 9µs | 17.26ms | 20µs | 30µs |
| http_req_tls_handshaking | 0s | 0s | 0s | 0s | 0s | 0s |
| http_req_waiting | 50.01ms | 1.24ms | 47.96ms | 232.77ms | 88.78ms | 96.79ms |

### Additional Metrics

| Metric | Value |
| ------------------- | ------------------------------------------ |
| http_reqs | 23714 (291.43/s) |
| http_req_failed | 0.00% (0 out of 23714) |
| iteration_duration | avg=1.65s, min=1.05s, med=1.79s, max=1.99s |
| iterations | 1824 (22.42/s) |
| virtual users (VUs) | min=1, max=55 |
| vus_max | 55 |

running (1m21.4s), 00/55 VUs, 1824 complete and 0 interrupted iterations
load_test ✓ [======================================] 00/55 VUs 1m20s

## ID Federation Load Test Results

✓ createSAMLFederationApp
✓ getSAMLFederationApp
✓ updateSAMLFederationApp
✓ getSAMLFederationAppByProduct
✓ deleteSAMLFederationApp

checks.........................: 100.00% 13670 out of 13670
data_received..................: 11 MB 141 kB/s
data_sent......................: 5.4 MB 67 kB/s

### Performance Metrics

| Metric | Avg | Min | Med | Max | p(90) | p(95) |
| ------------------------ | ------- | ----- | ------ | -------- | ------- | ------- |
| http_req_blocked | 7.16µs | 0s | 3µs | 4.76ms | 7µs | 11µs |
| http_req_connecting | 1.82µs | 0s | 0s | 4.64ms | 0s | 0s |
| http_req_duration | 13.85ms | 150µs | 9.13ms | 127.18ms | 31.53ms | 47.21ms |
| http_req_receiving | 62.62µs | 7µs | 33µs | 15.7ms | 100µs | 163µs |
| http_req_sending | 19.53µs | 2µs | 9µs | 15.2ms | 24µs | 40µs |
| http_req_tls_handshaking | 0s | 0s | 0s | 0s | 0s | 0s |
| http_req_waiting | 13.77ms | 121µs | 9.07ms | 127.15ms | 31.46ms | 47.1ms |

### Additional Metrics

| Metric | Value |
| ------------------- | ---------------------------------------- |
| http_reqs | 19138 (237.01/s) |
| http_req_failed | 0.00% (0 out of 19138) |
| iteration_duration | avg=1.1s, min=1.01s, med=1.1s, max=1.22s |
| iterations | 2734 (33.86/s) |
| virtual users (VUs) | min=1, max=55 |
| vus_max | 55 |

running (1m20.7s), 00/55 VUs, 2734 complete and 0 interrupted iterations
load_test ✓ [======================================] 00/55 VUs 1m20s
Loading