CVM: Add boot/reboot tests #3624

trungams · 2025-02-03T03:16:35Z

Tools:
- Add tpm2 tool to support reading PCR values from the TPM
- Add bootctl tool
Tests: add boot tests for Azure Linux CVM:
- verify_encrypted_root_partition: sanity check that the root partition on an Azure Linux CVM is encrypted when deployed with DiskWithVMGuestState encryption setting.
- verify_boot_success_after_component_upgrade: check that a CVM can reboot after a boot component is upgraded.
Core: include raw version string in Posix.get_package_information return value in case the package does not follow semantic versioning (fixes Posix.get_package_information fails to parse package version if it does not follow semver #3625)

trungams · 2025-02-04T08:03:25Z

@LiliDeng @squirrelsc - This PR is ready to review. Could you help me review it when you have time? Thank you.

The query_package addition - if you are not certain about it, I can split it into another PR.

microsoft/testsuites/cvm/cvm_boot.py

LiliDeng · 2025-02-04T11:29:43Z

could you provide the full cvm mariner image name? like this format: microsoftcblmariner azure-linux-3 azure-linux-3-gen2 latest

microsoft/testsuites/cvm/cvm_boot.py

squirrelsc · 2025-02-04T18:20:42Z

microsoft/testsuites/cvm/cvm_boot.py

+        component is upgraded.
+
+        Steps:
+        1. On first boot, check current PCR values for PCR4 and PCR7


Add requirements on new environment, if it's needed.

I have defined requirement for this test case below (CvmEnabled, AZURE). Does this suffice?

Because the steps writes "on first boot". If some test coverage needs a fresh new environment, it needs a flag to make sure the test case runs first on a new deployed environment.

I see. It looks like a new feature to add to Lisa. I was only thinking of controlling it inside a runbook via keep_environment = false. Do you have any suggestions on how I should approach it?

I think it should be this one

https://github.com/microsoft/lisa/blob/main/microsoft/testsuites/core/azure_image_standard.py#L887

microsoft/testsuites/cvm/cvm_boot.py

lisa/operating_system.py

lisa/tools/tpm2.py

trungams · 2025-02-04T23:41:53Z

@LiliDeng The image I used to test this change is MicrosoftCBLMariner azure-linux-3 azure-linux-3-cvm latest

trungams · 2025-02-04T23:48:16Z

microsoft/testsuites/cvm/cvm_boot.py

+        security_profile_settings = cast(
+            SecurityProfileSettings, node.features[SecurityProfile].get_settings()
+        )
+        if not security_profile_settings.encrypt_disk:


@squirrelsc @LiliDeng Ideally I want to be able to detect this setting earlier and specify it in the requirement above. I tried adding a function to lisa/features/security_profile.py as below:

DiskEncryptionEnabled = partial( SecurityProfileSettings, encrypt_disk=True )

and then use it similarly to how CvmEnabled is used. But it doesn't skip this test case if encrypt_disk is set to false in the runbook. Rather, it turns on that value for the running VM. That's why I had to resort to VM runtime check like this. Do you know how I can implement that kind of check to add to simple_requirement?

I didn't think about this scenario. When is it needed? If a vm size supports CVM, and the test case need CVM enabled, it shouldn't be controlled by runbook. If you need to verify the CVM image on a non-CVM scenario, it needs another test case.

I mean CVM can be deployed with different os disk encryption settings:

VMGuestStateOnly: root partition is not encrypted, this equals encrypt_disk: false in runbook and is the default setting if not specified.

DiskWithVMGuestState: root partition is encrypted, this equals encrypt_disk: true in runbook.

In my test, I want to skip this test case if encrypt_disk is false. I tried to define a new function DiskEncryptionEnabled as described above and pass it to simple_requirement.supported_features but it didn't work as I expected. Rather, it just turns on disk encryption for the deployed VM even though it has been explicitly set to false in my runbook.

I'm not sure fully understand it. Please describe in below format, don't mix with other settings.

What's the settings

Expected behavior

Actual behavior

Why you think the expected behavior is right

The scenario: I want to skip a test against CVM image if the CVM is not deployed with disk encryption enabled.

In lisa/features/security_profile.py. I add the following function:

DiskEncryptionEnabled = partial( SecurityProfileSettings, encrypt_disk=True )

What's the settings:

Test case: requirement=simple_requirement(supported_features[CvmEnabled(), DiskEncryptionEnabled()])

Runbook: encrypt_disk: false

Expected behaviour: Since encrypt_disk is set to false in my runbook. I expect the CVM to be deployed without disk encryption, and the test case will then be skipped.

Actual behaviour: The CVM was deployed with disk encryption, and the test case still ran.

Why you think the expected behaviour is right: That's my understanding of requirement in Lisa - it should not alter a VM's deployment settings. A VM deployment settings should match what is specified in the runbook (or take default values if nothing is specified)

I see. Please fix it in another PR, because this one is already big enough.

It caused by the encrypt_disk is a bool, not a SetSpace. And the current logic is to set it to true, if true and false meet.

The definition should be SetSpace([True, False]). https://github.com/trungams/lisa/blob/83004c9fc5dcd09cafaa7a343c714dca13927e06/lisa/features/security_profile.py#L61

The logic should be merge SetSpace. After that, this test case should be skipped, because it selects nothing. https://github.com/trungams/lisa/blob/83004c9fc5dcd09cafaa7a343c714dca13927e06/lisa/features/security_profile.py#L80

microsoft/testsuites/cvm/cvm_boot.py

lisa/util/__init__.py

squirrelsc · 2025-02-06T06:13:50Z

lisa/operating_system.py

-    def _get_version_info_from_named_regex_match(
-        self, package_name: str, named_matches: Match[str]
-    ) -> VersionInfo:
+    def _get_version_info_from_regex(


What's the new method is for? The package_name should be the same meaning for raw_version, and the package_name is not used.

Just trying to extract similar code to a separate function, each os' _get_package_information was using similar logic, with only the regex being different

lisa/operating_system.py

squirrelsc · 2025-02-06T06:15:17Z

lisa/operating_system.py

@@ -1487,7 +1475,6 @@ class RPMDistro(Linux):

    # ex: dpdk-20.11-3.el8.x86_64 or dpdk-18.11.8-1.el7_8.x86_64
    _rpm_version_splitter_regex = re.compile(
-        r"(?P<package_name>[a-zA-Z0-9\-_]+)-"


It may bring regressions.

The new rpm query being used will no longer return package name in the result, but I agree that refactor can get a bit risky. Do you think it's possible to trigger test cases that are affected by this change to catch errors?

This it needs to be compatible with different distro versions from AzureLinux 2.0 to other RPM based Linux. So it needs to keep for old versions.

squirrelsc · 2025-02-06T06:15:44Z

lisa/operating_system.py

+    ) -> LisaVersionInfo:
+        matches = regex.search(raw_version)
+        if not matches:
+            self._node.log.warning("")


Don't use warning log in LISA, it's not recommended.

tpm2-tools package provides the toolset to interact with TPM devices Signed-off-by: Thien Trung Vuong <[email protected]>

Signed-off-by: Thien Trung Vuong <[email protected]>

- Introduce LisaVersionInfo to wrap semver.VersionInfo - Update os._get_package_information to use distro's package manager to query a package version string directly, rather than using regex to extract version string from query output - Include the raw version string by default when getting a package version, resolves parsing error when the package does not follow semantic versioning, e.g. systemd Signed-off-by: Thien Trung Vuong <[email protected]>

Implement 2 new test cases: - verify_encrypted_root_partition: check that the root partition on an Azure Linux CVM is encrypted when deployed with "DiskWithVMGuestState" encryption setting - verify_boot_success_after_component_upgrade: check that a CVM can reboot after a boot component is upgraded, and PCR values are updated correctly Signed-off-by: Thien Trung Vuong <[email protected]>

squirrelsc · 2025-02-10T03:42:20Z

microsoft/testsuites/cvm/cvm_boot.py

+                pcrs_after_reboot[4],
+                "PCR4 value is still the same even though a boot component changed",
+            ).is_not_equal_to(pcrs_before_reboot[4])
+            assert_warn(


Why is it a warning? The warning means it's not important and should be an info level bug. If it's important, make the test case fail. The middle area is not a good practice.

trungams requested review from squirrelsc and LiliDeng as code owners February 3, 2025 03:16

trungams marked this pull request as draft February 3, 2025 03:43

trungams force-pushed the tvuong/azl-cvm-boot-test branch from 39b6e6c to b900463 Compare February 4, 2025 07:46

trungams marked this pull request as ready for review February 4, 2025 07:58

LiliDeng reviewed Feb 4, 2025

View reviewed changes

microsoft/testsuites/cvm/cvm_boot.py Show resolved Hide resolved

squirrelsc reviewed Feb 4, 2025

View reviewed changes

microsoft/testsuites/cvm/cvm_boot.py Outdated Show resolved Hide resolved

squirrelsc reviewed Feb 4, 2025

View reviewed changes

microsoft/testsuites/cvm/cvm_boot.py Outdated Show resolved Hide resolved

squirrelsc reviewed Feb 4, 2025

View reviewed changes

microsoft/testsuites/cvm/cvm_boot.py Show resolved Hide resolved

squirrelsc reviewed Feb 4, 2025

View reviewed changes

lisa/operating_system.py Outdated Show resolved Hide resolved

squirrelsc reviewed Feb 4, 2025

View reviewed changes

lisa/tools/tpm2.py Outdated Show resolved Hide resolved

trungams commented Feb 4, 2025

View reviewed changes

squirrelsc reviewed Feb 6, 2025

View reviewed changes

microsoft/testsuites/cvm/cvm_boot.py Show resolved Hide resolved

squirrelsc reviewed Feb 6, 2025

View reviewed changes

lisa/util/__init__.py Outdated Show resolved Hide resolved

squirrelsc reviewed Feb 6, 2025

View reviewed changes

lisa/util/__init__.py Show resolved Hide resolved

squirrelsc reviewed Feb 6, 2025

View reviewed changes

lisa/operating_system.py Show resolved Hide resolved

squirrelsc reviewed Feb 6, 2025

View reviewed changes

trungams added 4 commits February 7, 2025 01:58

Add Tpm2 tool

6500933

tpm2-tools package provides the toolset to interact with TPM devices Signed-off-by: Thien Trung Vuong <[email protected]>

Add Bootctl tool

6a28f99

Signed-off-by: Thien Trung Vuong <[email protected]>

trungams force-pushed the tvuong/azl-cvm-boot-test branch from 5a3678c to 83004c9 Compare February 7, 2025 02:12

squirrelsc reviewed Feb 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CVM: Add boot/reboot tests #3624

CVM: Add boot/reboot tests #3624

trungams commented Feb 3, 2025 •

edited

Loading

trungams commented Feb 4, 2025

LiliDeng commented Feb 4, 2025

squirrelsc Feb 4, 2025

trungams Feb 4, 2025

squirrelsc Feb 6, 2025

trungams Feb 7, 2025

LiliDeng Feb 9, 2025

trungams commented Feb 4, 2025

trungams Feb 4, 2025

squirrelsc Feb 6, 2025

trungams Feb 6, 2025

squirrelsc Feb 7, 2025

trungams Feb 7, 2025

squirrelsc Feb 10, 2025

squirrelsc Feb 6, 2025

trungams Feb 6, 2025

squirrelsc Feb 6, 2025

trungams Feb 6, 2025

squirrelsc Feb 10, 2025

squirrelsc Feb 6, 2025

squirrelsc Feb 10, 2025

CVM: Add boot/reboot tests #3624

Are you sure you want to change the base?

CVM: Add boot/reboot tests #3624

Conversation

trungams commented Feb 3, 2025 • edited Loading

trungams commented Feb 4, 2025

LiliDeng commented Feb 4, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungams commented Feb 4, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungams commented Feb 3, 2025 •

edited

Loading