[CI] migrate CI test steps to containers #310
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
15 times, most recently
from
6f35827 to
696827b
Compare
dpressle
requested changes
Feb 26, 2025
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
from
696827b to
9e69799
Compare
Today we use benni09 static agent to run test/gtest/valgrind steps which is unscaleable since it can only run one pipeline at a time, causing delays in builds that can be stuck waiting for hours The idea is to move these steps to containers, allowing running them in parallel as well as running multiple pipelines at the same time (depending on the capacity of the k8s cluster) Issue: HPCINFRA-3249 Signed-off-by: NirWolfer <[email protected]>
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
from
9e69799 to
1e16cae
Compare
|
bot:retest |
dpressle
requested changes
Mar 5, 2025
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
6 times, most recently
from
70a7dc5 to
9e0436b
Compare
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
2 times, most recently
from
18cb753 to
a773a22
Compare
Signed-off-by: NirWolfer <[email protected]>
NirWolfer
force-pushed
the
HPCINFRA-3249
branch
2 times, most recently
from
e52a28e to
f660eef
Compare
|
bot:retest |
dpressle
requested changes
Mar 6, 2025
|
|
||
| function do_hugepages() | ||
| { | ||
| if [[ -f /.dockerenv && ! $(grep -q hugetlbfs /proc/mounts) ]]; then |
There was a problem hiding this comment.
You should probably do a more extensive check for containterized environment like you do in gtest script
[[ -f /.dockerenv || -f /run/.containerenv || -n "${KUBERNETES_SERVICE_HOST}" ]]
| #fi | ||
| if [ ! -z "$(do_get_ip 'eth')" ]; then | ||
| test_ip_list="${test_ip_list} eth_ip4:$(do_get_ip 'eth')" | ||
| if [[ -f /.dockerenv ]] || [[ -f /run/.containerenv ]] || [[ -n "${KUBERNETES_SERVICE_HOST}" ]]; then |
There was a problem hiding this comment.
can you improve it a little bit and reduce duplication, so instead of build the ip list string in 2 places just have variables for each ip and generate the list in one place
if
ipv4=get_ipv4...
ipv6=get_ipv6...
else
ipv4=
ipv6=
fi
test_ip_list="eth_ip4:${ipv4} eth_ip6:${ipv6}"
| else | ||
| gtest_opt="--addr=$(do_get_addrs 'eth' ${opt2})" | ||
| gtest_opt_ipv6="--addr=$(do_get_addrs 'inet6' ${opt2}) -r fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" # Remote - Dummy Address | ||
| fi |
There was a problem hiding this comment.
can you add check if ips are empty (like we have in all other tests scripts)?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Today we use benni09 static agent to run test/gtest/valgrind steps which is unscalable since it can only run one pipeline at a time, causing delays in builds that can be stuck waiting for hours
The idea is to move these steps to containers, allowing running them in parallel as well as running multiple pipelines at the same time (depending on the capacity of the k8s cluster)
What
Change test/gtest/valgrind steps to run on containers on swx-k8s-spray cluster instead of benni09
Why ?
HPCINFRA-3249
Change type
What kind of change does this PR introduce?
Check list