iconm-menu
English chevron-down
中文EnglishРусский
moon-outline
light

How to Compile Avocado-VT Test Cases

zhuhuankai2021-01-30virttestavocado-vt

Overview

This article uses vmtop test cases as an example to describe how to use the Avocado-VT framework to compile test cases and perform tests. If you haven't completed environment deployment, see "How to Install and Deploy the Avocado-VT Virtualization Test Framework."

Test Case Execution Process

  • Obtain parameters.

    Default parameters configured in the framework and customized parameters.

  • Prepare the environment based on the parameters.

    For example, environment for creating or deleting a VM instance.

  • Perform the test.

    Use the configured parameters to perform the test. Generally, when a test point triggers a TestFail exception, the test fails; otherwise, the test is passed.

  • Clean up the environment after the test.

    Clean up the environment based on the test result, for example, destroying the VM or deleting the image.

Cartesian Configuration

  • Keys and values

    The Avocado-VT framework uses <key> = <value> to define keys. For example, in the following code segment, test_type in line 16 is a key, and start is the value.
    Note that the Avocado-VT framework parses parameters in top-down order. Therefore, a later key overwrites the previous key.

  • Variants

    Variants are of high importance for parameter configuration. A variants stanza can contain multiple parameters of the same level.

    The output of two variants stanzas will be all possible combinations of both variant contents.

    As shown in the following code segment, the variants stanza in line 10 contains two branches: positive_test and negative_test. The two branches are independent and represent positive test cases and negative test cases, respectively. The positive_test branch contains two variants stanzas. The first one contains five elements: life_cycle, data_events, cmdline_d, multivmtop, and vcpu_hotplug. The second one contains two elements: single_vm and multi_vms. The two variants stanzas of the same level can output a maximum of 10 combinations.

    life_cycle also contains a variants stanza, which contains four elements: start, destroy, reboot, and pause_resume. Therefore, a maximum of 16 combinations can be obtained theoretically. The combination name is in the format of positive_test.single_vm.life_cycle.start.

  • Filters

    Filter statements help filter case combinations. There are two common filters: only and no. In the following code segment, the only filter in line 41 limits the variant combinations. multi_vms is combined only with life_cycle.start, and nine positive_test combinations can be obtained.

- vmtop:
    type = vmtop
    start_vm = "no"
    output_file = "vmtop_output"
    vcpu_max = 8
    vcpu_cur = 4
    vcpu_live = 6
    vm_num = 1
    only aarch64
    variants:
        - positive_test:
            variants:
                - life_cycle:
                    variants:
                        - start:
                            test_type = start
                        - destroy:
                            test_type = destroy
                        - reboot:
                            test_type = reboot
                        - pause_resume:
                            test_type = pause
                - data_events:
                    test_type = data_events
                - cmdline_d:
                    option = "d"
                    interval = 2
                - multivmtop:
                    test_type = multivmtop
                    vmtopnum = 6
                - vcpu_hotplug:
                    test_type = vcpu_hotplug
                    vcpu_placement = "static"
            variants:
                - single_vm:
                    test_type = single_vm
                    vm_num = 1
                - multi_vms:
                    test_type = multi_vms
                    vm_num = 3
                    only life_cycle.start
        - negative_test:
            variants:
                - invalid_cmdline:
                    option = "x"

Test Case Highlights

  • Dependency libraries

    Python and Avocado-VT provide a large number of encapsulated libraries. Making good use of these libraries can greatly improve the test case compilation efficiency, simplify the test case code, and improve the readability.

import os
import logging

from avocado.utils import process
from avocado.utils import path
from avocado.core import exceptions

import virttest.utils_libguestfs as lgf
from virttest.utils_test import libvirt
from virttest.libvirt_xml import vm_xml
from virttest import virsh
from virttest import data_dir
  • run Function

    The run function is an indispensable part of Avocado-VT test cases. By adding test logic to the run function using test, params, and env parameters provided by this function, we can perform a large number of operations on VM instances and design and implement test points.

    test: avocado/avocado/core/test.py
    params: avocado-vt/virttest/utils_params.py
    env: avocado-vt/virttest/utils_env.py

def run(test, params, env):
  • Parameter configuration

    Obtain VM instances and preset parameters from the CFG file. It is similar to defining and initializing variables.

vmname = params.get("main_vm", "avocado-vt-vm1")
vm = env.get_vm(vmname)

output = params.get("output_file", "output")
output_path = os.path.join(data_dir.get_tmp_dir(), output)
option = params.get("option")
interval = params.get("interval")
test_type = params.get("test_type", "")
vcpu_placement = params.get("vcpu_placement")
vcpu_cur = int(params.get("vcpu_cur"))
vcpu_max = int(params.get("vcpu_max"))
vcpu_live = int(params.get("vcpu_live"))
vm_num = int(params.get("vm_num"))
vmtopnum = params.get("vmtopnum")
  • Common library sources for VM instance operations

    libvirt: avocado-vt/virttest/libvirt_vm.py
    avocado-vt/virttest/virsh.py
    qemu: avocado-vt/virttest/qemu_vm.py

Test Points

  • Installing vmtop in the Environment

    Run the yum install -y vmtop command on the host OS to install vmtop. process.run is a function provided by Python and is used to run shell commands. If the installation fails, use the try-except syntax to report an exception.

#Install vmtop command
install_cmd = "yum install -y vmtop"
process.run(install_cmd, shell=True)
#Find vmtop command
try:
    vmtop = path.find_command("vmtop")
except path.CmdNotFoundError as info:
    raise exceptions.TestSkipError("No vmtop command found - %s" % info)
  • virsh.start

    The virsh.start function is used to start a libvirt VM instance. Note: For libvirt test cases, you need to define a VM using an XML file.

virsh.start(vmname, ignore_status=True)
  • test.fail

    Compile a vmtop_info function, obtain the return value, and check whether the return value meets the expectation. If no, the process goes to test.fail, indicating that the test case fails. This is a typical test point.

#Test vmtop when start vm
res = vmtop_info(vmtop, output_path, info_type="Domains")
if int(res) != vm_num:
    test.fail("start failed and the res = %s" % res)
  • vmtop_info

    This function uses the vmtop command and saves the vmtop output to the temporary path output_path preset in the parameter configuration phase. This capability is provided by the os library to save files generated during test case execution to avoid occupying the drive space. Instead of being tested in the run function, this function is tested separately for multiplexing. To verify other information in the vmtop output, you only need to add verification logics in the vmtop_info function.

def vmtop_info(vmtop, output_path, info_type):
    """
    Collect and return vmtop infomation
    """
    cmd = "%s -b -H -n 1 > %s" % (vmtop, output_path)
    process.run(cmd, ignore_status=True, shell=True)

    if info_type == "Domains":
        cmd = "cat %s | grep  Domains | awk '{ print $2 }'" % output_path
        res = process.run(cmd, ignore_status=True, shell=True)
    elif info_type == "Cpu":
        cmd = "cat %s | grep /KVM | awk 'END{ print NR }'" % output_path
        res = process.run(cmd, ignore_status=True, shell=True)
    elif info_type == "Data":
        cmd = "cat %s | grep  DID" % output_path
        res = process.run(cmd, ignore_status=True, shell=True)

    return res.stdout_text
  • Environment clearance

    To ensure that test cases do not affect each other, we need to clean up the environment at the end of the test. For example, destroy VM instances and remove drives.

finally:
    # Destroy vm_clone
    if test_type == "multi_vms":
        num = 1
        while num < vm_num:
            if virsh.domain_exists(vm_clone_name[num - 1]):
                if virsh.is_alive(vm_clone_name[num - 1]):
                    virsh.destroy(vm_clone_name[num - 1])
                    virsh.remove_domain(vm_clone_name[num - 1], \
                                        "--remove-all-storage", debug=True)
            num = num + 1

    if vm.is_alive():
        vm.destroy()
    vmxml.sync()

[Disclaimer] This article only represents the author's opinions, and is irrelevant to this website. This website is neutral in terms of the statements and opinions in this article, and does not provide any express or implied warranty of accuracy, reliability, or completeness of the contents contained therein. This article is for readers' reference only, and all legal responsibilities arising therefrom are borne by the reader himself.