Skip to content

Tests

Jump to bottom
fanchen2 edited this page Oct 31, 2023 · 11 revisions

Test Case Design

Case List

Case ID Domain Case Name Test Purpose
1 sanity tdx_enabled Check if TDX is enabled in host and guest
2 sanity tdx_VMX_coexist Check whether TDX and VXM guests can co-exist.
3 sanity tdx_NOEPT Check when EPT=no, TDX guest should not be created.
4 sanity tdx_tsc_config Check TSC function in TDX guest
5 sanity tdx_debugon_nmi Check whether TDX guest can boot up with debug=on, and run "nmi" in qemu monitor.
6 sanity tdx_debugoff_nmi Check whether TDX guest can boot up with debug=off, and run "nmi" in qemu monitor.
7 sanity tdx_huge_resource_VM Check whether one TDX guest with half cpu and half memory of the platform
8 sanity tdx_1vcpus_10times Check whether 1 vcpu TDX guest can be created 10 times continuously
9 sanity tdx_4vcpus_1socket_10times Check whether 4 vcpu and 1 socket TDX guest can be created 10 times continuously
10 sanity tdx_4vcpus_2sockets_5times Check whether 4 vcpu and 2 socks TDX guest can be created 5 times continuously
11 sanity tdx_2TD_VM Check whether 2 TDX guest can be created together
12 tdx tdx_disabled Check if TDX can be disabled on host and guest
13 tdx tdx_8vcpus Check if TDX guest can boot up with 8 vcpus
14 tdx tdx_NO_TDVF Check if TDX guest can boot up with no TDVF
15 tdx tdx_build_kernel Build kernel on TDX guest
16 tdx tdx_debug_on Check whether TDX guest can boot up with debug=on
17 tdx tdx_debug_off Check whether TDX guest can boot up with debug=off
18 tdx tdx_debugoff_earlyprink Check whether TDX guest can boot up with debug=off and "earlyprintk" appended in guest kernel command line
19 tdx tdx_hdevents_in5sec Collect hardware events via perf during 5 seconds and check "Performance counter stats"
20 tdx tdx_VMP_cpu_onoff Check TDX guest works when some host cpu off/on
21 tdx tdx_31guests Check whether 31 TD guests can bootup together
22 tdx tdx_16G_4guests_20times Check whether 4 TDX guests with 16G memory can be created, repeat 20 times
23 tdx tdx_uefi Check whether TDX SEAM modules are loaded by UEFI
24 tdx tdx_cpuoff_pinedVMdown Boot TD guest with 64 vcpu, pin the qemu process to a host cpu, offline the host cpu, no crash on guest and host, kill the VM, repeat 20 times.
25 tdx tdx_tsc_deadline_enable Check if TSC deadline are enabled on TD guests by default
26 tdx tdx_tsc_deadline_disable Check if TSC deadline can be disabled on TD guests by qemu command
27 tdx tdx_vsock_vm_server Check iperf works with vscok: guest as server and host as client
28 tdx tdx_vsock_vm_client Check iperf works with vscok: guest as client and host as server

Case Details

TDX Sanity Test

Case ID: 001 tdx_enabled

Test Purpose: Check if TDX is enabled in host and guest. Test Steps:

  1. Boot host and do the following check. (1) rdmsr 0xfe --bitfield 15:15 → to check platform supports TDX-SEAM The expected calue is 1 Note： If get the following error, run "modprobe msr" rdmsr: open: No such file or directory (2) grep Y /sys/module/kvm_intel/parameters/tdx --------- to check if TDX is supported in KVM Note: If the value is "N", do "rmmod kvm_intel; modprobe kvm_intel tdx=1" and check again (3) Check TDX in dmesg [root@spr-2s5 ~]# dmesg | grep -E "tdx: TDX module|tdx: SEAMRR enabled" [ 97.578273] tdx: TDX module: atributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20230206, build_num 457 [ 98.522910] tdx: TDX module initialized. [root@spr-2s5 ~]# dmesg |grep -E "tdx: Successfully initialized TDX module|tdx: TDX module initialized" [ 98.522910] tdx: TDX module initialized.
  2. Create VM qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. After VM boots up, check tdx_guest on VM. (1) cat /proc/cpuinfo |grep tdx_guest (2) ls /dev/tdx_guest

Expected Result: TDX feature is enabled in host and guest

Case ID: 002 TDX_VMX_coexit

Test Purpose: Check whether TDX and VXM guests can co-exist. Test Steps:

  1. Boot host and check TDX flag. [root@spr-2s5 ~]# dmesg | grep -E "tdx: TDX module|tdx: SEAMRR enabled" [ 97.578273] tdx: TDX module: atributes 0x0, vendor_id 0x8086, major_version 1, minor_version 0, build_date 20230206, build_num 457 [ 98.522910] tdx: TDX module initialized. [root@spr-2s5 ~]# dmesg |grep -E "tdx: Successfully initialized TDX module|tdx: TDX module initialized" [ 98.522910] tdx: TDX module initialized.
  2. Create TDX guest. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Create legacy guest. qemu-system-x86_64 -accel kvm -cpu host -drive file=centos-9-embargo_intel_next_2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0,bootindex=0 -m 4096 -monitor pty -smp 1 -cpu host -device virtio-net-pci,netdev=nic0,mac=00:4d:50:4d:2b:94 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -daemonize -bios /usr/local/share/qemu/OVMF.fd

Expected Result: These two guests can boot up successfully.

Case ID: 003 TDX_VMX_NOEPT

Test Purpose: Check when EPT=no, TDX guest should not be created. **Test Steps: **

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Disable EPT in KVM. (1) rmmod kvm_intel (2) modprobe kvm_intel tdx=1 pt_mode=1 ept=0
  3. Create TDX guest. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  4. Get the error information. (1) We get the following qemu output. qemu-system-x86_64: -accel kvm: vm-type X86_TDX_VM not supported by KVM (2) We can get more detailed information in dmesg. dmesg | grep TDX ... [ 2992.482851] TDX requires mmio caching. Please enable mmio caching for TDX.

Expected Result: Fail to create TDX guest when EPT is disabled

Case ID: 004 tdx_tsc_config

Test Purpose: Check TSC function in TDX guest Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.

  2. Check TSC value on host ## dmesg | grep -i tsc [ 0.000000] tsc: Detected 1000.000 MHz processor

  3. Check TSC value on host

    (1) Check leaf 0x15 (which is related to TSC)

    # cpuid -rl 0x15 -1

    CPU 0:

    0x00000015 0x00: eax=0x00000001 ebx=0x00000028 ecx=0x017d7840 edx=0x00000000

    EAX Fixed 0x1 → 1; EDX Fixed 0x0 → 0x0; ECX Fixed 0x017D7840 → 0x017D7840; EBX As Configured: This MSR is set by TSC_FREQUENCY in TD-param and can be modified by QEMU tsc-frequency= value. (2) Calculate TSC value execute "cpuid | less" and search "Time Stamp Counter" to get below information Time Stamp Counter/Core Crystal Clock Information (0x15):

    TSC/clock ratio = 40/1 (40/1 ↔ cpuid EBX/EAX)

    nominal core crystal clock = 25000000 Hz (25000000Hz ↔ ecx: 0x017d7840) Host tsc frequency = ECX * EBX / EAX

  4. Create TDX guest qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::16391-:22 -bios /usr/local/share/qemu/OVMF.fd

  5. Check TSC value on guest

    (1) Check leaf 0x15 (which is related to TSC)

    # cpuid -rl 0x15 -1

    CPU 0:

    0x00000015 0x00: eax=0x00000001 ebx=0x00000028 ecx=0x017d7840 edx=0x00000000

    EAX Fixed 0x1 → 1; EDX Fixed 0x0 → 0x0; ECX Fixed 0x017D7840 → 0x017D7840; Guest tsc frequency = ECX * EBX / EAX Guest should have the same Guest tsc frequency as the host.

  6. Re-create TDX guest with tsc-frequency=3000000000 qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -nodefaults -daemonize -cpu host,tsc-freq=3000000000 -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::50969-:22 -bios /usr/local/share/qemu/OVMF.fd

  7. Check TSC value on guest # dmesg | grep -i tsc [ 0.000000] tsc: Detected 3000.000 MHz processor # cpuid | grep -i TSC TSC: time stamp counter = true IA32_TSC_ADJUST MSR supported = false TSC/clock ratio = 120/1 TSC frequency (Hz) = 3000000 RDTSCP = true TscInvariant = false # cpuid -rl 0x15 -1 CPU: 0x00000015 0x00: eax=0x00000001 ebx=0x00000078 ecx=0x017d7840 edx=0x00000000 Guest tsc frequency = ECX * EBX / EAX Guest tsc frequency should equal to the configured value 3000000000

Expected Result: Without tsc-frequency set, guest should have the same TSC value as the host; when set tsc-frequency, TSC value can be configured successfully for TDX guest.

Case ID: 005 tdx_debugon_nmi

Test Purpose: Check whether TDX guest can boot up with debug=on, and run "nmi" in qemu monitor. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with -object,tdx-guest,debug=on qemu-system-x86_64 -accel kvm -monitor pty -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::40920-:22 -bios /usr/local/share/qemu/OVMF.fd
  3. Login qemu monitor, and run "nmi"

Expected Result: TDX guest works with "nmi" in qemu monitor with debug=on

Case ID: 006 tdx_debugoff_nmi

Test Purpose: Check whether TDX guest can boot up with debug=off, and run "nmi" in qemu monitor. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with -object,tdx-guest,debug=off. qemu-system-x86_64 -accel kvm -monitor pty -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=off -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::37390-:22 -bios /usr/local/share/qemu/OVMF.fd
  3. Login qemu monitor, and run "nmi"

Expected Result: TDX guest works with "nmi" in qemu monitor with debug=off

Case ID: 007 tdx_huge_resource_VM

Test Purpose: Check whether one TDX guest with half cpu and half memory of the platform Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Check cpu number and memory of the platfrom grep -c processor /proc/cpuinfo free -h | grep Mem | awk '{print $4}' | awk -FG '{print $1}'
  3. Create TDX guest with half memory and half cpu numbers of the platform. qemu-system-x86_64 -accel kvm -cpu host -smp 112 -m 121G -object tdx-guest,id=tdx,debug=on -machine q35,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=121G -nographic -vga none -drive file=/share/xvs/var/tmp-img_tdx_huge_resource_VM_1690940795,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -device virtio-net-pci,netdev=nic0 -netdev user,id=nic0,hostfwd=tcp::45728-:22 -bios /usr/local/share/qemu/OVMF.fd -chardev stdio,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -monitor chardev:mux

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

Case ID: 008 tdx_1vcpus_10times

Test Purpose: Check whether 1 vcpu TDX guest can be created 10 times continuously Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with 1 vcpu. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Destroy guest
  4. Repeat step1 and step2 10 times

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

Case ID: 009 tdx_4vcpus_1socket_10times

Test Purpose: Check whether 4 vcpu and 1 socket TDX guest can be created 10 times continuously Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with 4 vcpu 1 socket qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4,sockets=1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Destroy guest
  4. Repeat step1 and step2 10 times

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

Case ID: 010 tdx_4vcpus_2sockets_5times

Test Purpose: Check whether 4 vcpu and 2 socks TDX guest can be created 5 times continuously Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with 4 vcpu and 2 socks qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4,sockets=2 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Destroy guest
  4. Repeat step1 and step2 5 times

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

Case ID: 011 tdx_2TD_VM

Test Purpose: Check whether 2 TDX guest can be created together

Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.

  2. Create 2 TDX guests on one host qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

    qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next_2.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:ee:72:58 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

TDX Functional Test

Case ID: 012 tdx_disabled Test Purpose: Check if TDX can be disabled on host and guest. Test Steps:

  1. Boot host and do the following operations. (1) reload kvm_intel with tdx=0 (2) rdmsr 0xfe --bitfield 15:15 → to check platform supports TDX-SEAM, it doesn’t ensure TDX-SEAM is not loaded and configured 1 (3) cat /sys/module/kvm_intel/parameters/tdx N
  2. Create TDX guest qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Get error information

Expected Result: When TDX is disabled in host, we will fail to create td guest

Case ID: 013 tdx_8vcpus

Test Purpose: Check if TDX guest can boot up with 8 vcpus. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with 8 vcpus. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

Expected Result: TDX guest with 8 vcpus can boot up successfully.

Case ID: 014 tdx_NO_TDVF

Test Purpose: Check if TDX guest can boot up with no TDVF. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest without OVMF.fd qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize
  3. Get error: memory_region_init_ram_gmem: error creating gmem: Invalid argument

Expected Result: TDX guest cannot boot up without TDVF

Case ID: 015 TDX_build_kernel

Test Purpose: Build kernel on TDX guest Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Login guest and build kernel

Expected Result: Kernel can be built successfully.

Case ID: 016 TDX_debug_on

Test Purpose: Check whether TDX guest can boot up with debug=on Test Steps:

  1. Boot host and check TDX flag. # dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with -object,tdx-guest,debug=on qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

Expected Result: TDX guest can boot up successfully.

Case ID: 017 TDX_debug_off

Test Purpose: Check whether TDX guest can boot up with debug=off Test Steps:

  1. Boot host and check TDX flag. # dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with -object,tdx-guest,debug=off qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=off -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

Expected Result: TDX guest can boot up successfully.

*Case ID:* 018 TDX_debugoff_earlyprink

Test Purpose: Check whether TDX guest can boot up with debug=off and "earlyprintk" appended in guest kernel command line. Test Steps:

  1. Boot host and check TDX flag. # dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.
  2. Create TDX guest with -object,tdx-guest,debug=off, and "earlyprintk" appended in guest kernel command line, for example "earlyprintk=ttyS0,115200" qemu-system-x86_64 -accel kvm -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -nodefaults -daemonize -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -kernel /boot/kvm-vmlinuz -initrd /boot/initramfs-kvm.img -append 'root=UUID=5cf6ccb0-3db3-42d2-921e-35cdd67a69bb ro console=hvc0 intel_iommu=on earlyprintk=ttyS0,115200 ' -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -bios /usr/share/qemu/OVMF.fd
  3. TDX guest boot up successfully.

Expected Result: TDX guest can boot up successfully with debug=off and "earlyprintk".

Case ID: 019 tdx_hdevents_in5sec

Test Purpose: Collect hardware events via perf during 5 seconds and check "Performance counter stats". Test Steps:

  1. Boot host and check TDX flag. # dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.

  2. Create TDX guest with pmu=on qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host,pmu=on -smp 8 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd

  3. After TDX guest boots up successfully, run the following command line to collect some hardware events during last 5 second.

    perf stat -e cache-misses -e instructions -e branch-misses -e bus-cycles -e branch-instructions -e L1-dcache-loads -a -- sleep 5

  4. Check the hardware events data, if some values are too large (say, > 1000 billions), there may be a bug.

    Performance counter stats for 'system wide':

    2,299 cache-misses 11,276,844,014 instructions 78,534 branch-misses # 0.00% of all branches 52,993,975,500 bus-cycles 2,818,253,634 branch-instructions 4,227,460,587 L1-dcache-loads

    5.336306380 seconds time elapsed

Expected Result: Hardware events are generated correctly while TDX guest is running.

Case ID: 020 tdx_VMP_cpu_onoff Test Purpose: To check TDX guest works when some cpu is on/off. Test Steps:

  1. Boot host and check TDX flag. # dmesg | grep -i tdx [ 0.240761] tdx: using preloaded SEAM module [ 3.169957] tdx: TDX initialized.
  2. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Offline some cpu on host echo 0 > /sys/devices/system/cpu/cpu17/online echo 0 > /sys/devices/system/cpu/cpu60/online echo 0 > /sys/devices/system/cpu/cpu109/online
  4. Check TD guest, ping TD guest
  5. Restore the offline cpu echo 1 > /sys/devices/system/cpu/cpu17/online echo 1 > /sys/devices/system/cpu/cpu60/online echo 1 > /sys/devices/system/cpu/cpu109/online

Expected Result: TDX works correctly with host cpu off/on

Case ID: 021 tdx_31guests

Test Purpose: Check whether 31 TD guests can bootup together. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create 1 TDX guest qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Repeat step 2, create 31 td guest in total

Expected Result: 31 TD guests can bootup together. No host kernel oops.

Case ID: 022 TDX_16G_4guests_20times

Test Purpose: Check whether 4 TD guests with 16G memory, repeat 20 times. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create 4 TDX guests with 16G memory. qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. When TDX guests boots up successfully, kill all the TD guests.
  4. Repeat 20 times.

Expected Result: TDX guests can be created and destroyed successfully. No host kernel oops.

Case ID: 023 tdx_uefi

Test Purpose: Check whether TDX SEAM modules are loaded by UEFI Test Steps:

  1. Boot host and check TDX module in dmesg. dmesg | grep -E "tdx: TDX module" [ 270.134879] tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20230420, build_num 507

Expected Result: TDX SEAM modules are loaded by UEFI successfully.

Case ID: 024 tdx_cpuoff_pinedVMdown

Test Purpose: Boot TD guest and pin the qemu process to a host cpu, offline the host cpu, check if host can work well. Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Create TD guest with 64 vcpu qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 64 -m 4096M -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. Pin Qemu to host cpu pid=ps -edf | grep qemu | grep -v grep | awk '{print $2}' taskset -pc 18 $pid
  4. Make cpu 18 offline echo 0 > /sys/devices/system/cpu/cpu18/online
  5. Execute "init 0 &" in TD guest
  6. Check host dmesg dmesg | tail -200 | grep hkid dmesg | tail -200 | grep "0x8000070100000000" No such info found in dmesg
  7. Restore cpu 18 echo 1 > /sys/devices/system/cpu/cpu18/online
  8. Repeat step 2~7 20 times

Expected Result: Host works well, no crash information found

Case ID: 025 tdx_tsc_deadline_enable

Test Purpose: Disable TSC deadline on TD guests Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Boot a TD guest qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 4 -m 4096M -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. TD VM can boot up successfully and tsc deadline is enabled by default Verify that TD guest can see tsc-deadline feature: lscpu |grep tsc_deadline_timer

Expected Result: TSC deadline can be enabled by default on TD guests.

Case ID: 026 tdx_tsc_deadline_disable

Test Purpose: Disable TSC deadline on TD guests Test Steps:

  1. Boot host and check TDX flag. ## dmesg | grep -i tdx [ 3.169957] tdx: TDX initialized.
  2. Boot a TD guest with “-tsc-deadline”, e.g., “-cpu host,-tsc-deadline” qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host,-tsc-deadline -smp 4 -m 4096M -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd
  3. TD VM can boot up successfully with -tsc-deadline. Verify that TD guest cannot see tsc-deadline feature: “lscpu |grep tsc_deadline_timer | wc -l” should output 0

Expected Result: TSC deadline can be disabled successfully on TD guests.

Case ID: 027 tdx_vsock_vm_server Test Purpose: To check iperf works with vscok: guest as server and host as client Test Steps:

  1. On host, install iperf with vsock support. (1) git clone https://github.com/stefano-garzarella/iperf-vsock (2) build iperf → cd iperf-vsock → mkdir build → cd build → ../configure (If configure fails, try running ./bootstrap.sh first) → make → cd src

  2. Boot TD guest qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd -device vhost-vsock-pci,guest-cid=3

  3. Login guest, install iperf with vsock support on guest

  4. Run iperf as vsock server on guest

    systemctl stop firewalld.service; systemctl disable firewalld.service

    setenforce 0

    ./iperf3 --vsock -s

  5. Run iperf as vsock client on host

    ./iperf3 --vsock -c 3

  6. Get the iperf data.

Expected Result: Vsock works correctly on host and guest.

Case ID: 028 tdx_vsock_vm_client Test Purpose: To check iperf works with vscok: guest as client and host as server Test Steps:

  1. On host, install iperf with vsock support. (1) git clone https://github.com/stefano-garzarella/iperf-vsock (2) build iperf → cd iperf-vsock → mkdir build → cd build → ../configure (If configure fails, try running ./bootstrap.sh first) → make → cd src

  2. Boot TD guest qemu-system-x86_64 -accel kvm -drive file=centos-9-embargo_intel_next.qcow2,if=none,id=virtio-disk0 -device virtio-blk-pci,drive=virtio-disk0 -chardev pty,id=mux,mux=on,signal=off -device virtio-serial -device virtconsole,chardev=mux -serial chardev:mux -cpu host -smp 1 -m 4096 -object tdx-guest,id=tdx,debug=on -machine q35,hpet=off,kernel_irqchip=split,memory-encryption=tdx,memory-backend=ram1 -object memory-backend-ram,id=ram1,size=4096M,private=on -nographic -vga none -device virtio-net-pci,netdev=nic0,mac=00:b8:8b:f1:04:56 -netdev tap,id=nic0,br=virbr0,helper=/usr/local/libexec/qemu-bridge-helper,vhost=on -nodefaults -daemonize -bios /usr/local/share/qemu/OVMF.fd -device vhost-vsock-pci,guest-cid=3

  3. Login guest, install iperf with vsock support on guest

  4. Run iperf as vsock server on host systemctl stop firewalld.service; systemctl disable firewalld.service

    setenforce 0

    ./iperf3 --vsock -s

  5. Run iperf as vsock client on guest

    setenforce 0

    ./iperf3 --vsock -c 2

  6. Get the iperf data.

Expected Result: Vsock works correctly on host and guest.

Clone this wiki locally