ZStack Cloud Documentation
All
All
ZStack Cloud 4.8.10>ZStack Cloud Tutorials>ZStack Cloud Private Cloud>Smart NIC Tutorial>Scenario Practice
Get PDF

Scenario Practice

Scenario Assumption: You want to use Mellanox CX-5 smart NICs to improve the network performance of business VM instances for business requirements.

Assume that you have prepared following resources:
  • Two x86 servers, one gigabit switch, one 10-gigabit switch, and several category five cables.
  • Each x86 server is installed with a Mellanox CX-5 smart NIC.
  • Following files:
    • A latest ZStack Cloud custom ISO (h76c).
    • A latest ZStack Cloud installation package.
    • Latest Multi-mn HA suites.
    • A ZStack Cloud license (Enterprise Prepaid).
    • A latest ZStack Cloud VPC vRouter Image (KVM).
Assume that you plan your hardware networks as follows:
  • Management Network
    Table 1. Management Network
    Node Gigabit Network Interface 1 Gigabit Network Interface 2 Aggregate Network Interface Network Bridge IP Address Netmask Gateway
    Management Node1 eth0 eth1 bond0 br_bond0 172.25.15.143 255.255.255.0 172.25.0.1
    Management Node2 eth0 eth1 bond0 br_bond0 172.25.15.99 255.255.255.0 172.25.0.1
  • Business Network
    Table 2. Business Network
    Node 10-Gigabit Network Interface 1 10-Gigabit Network Interface 2 IP Address Netmask Gateway
    Management Node1 enp179s0f0 enp179s0f1 - - -
    Management Node2 enp179s0f0 enp179s0f1 - - -
Figure 1. Smart NIC Network Topology


Assume that you plan your scenario network as follows:
  • Management Network
    Table 3. Management Network
    Management Network Configuration
    NIC bond0
    VLAN ID NoVLAN
    IP Range 172.25.15.50~172.25.15.150
    Netmask 255.255.255.0
    Gateway 172.25.0.1
  • VPC Network
    Table 4. VPC Network1
    VPC Network1 Configuration
    NIC bond0
    VLAN ID 2200
    CIDR 192.167.0.1/24
    Gateway 192.167.0.1
    DHCP IP Randomly Allocated
    Table 5. VPC Network2
    VPC Network2 Configuration
    NIC enp179s0f1
    VLAN ID 2334
    CIDR 192.169.0.1/24
    Gateway 192.169.0.1
    DHCP IP Randomly Allocated
Figure 2. Smart NIC Scenario Network Topology


Detailed practice procedures are described as follows:
  1. Energize and connect hardware.
  2. Install operating systems.
  3. Configure the management network.
  4. Install HA suites.
  5. Install the license.
  6. Create a cluster with Network Acceleration Support enabled.
  7. Add the two management nodes to the cluster.
  8. Virtualize the physical NIC.
  9. Create two L2 network whose Network Acceleration Mode are respectively Smart NIC and Standard. Attach the two L2 networks to the cluster.
  10. Create a VPC vRouter and two VPC networks.
  11. Create two business VM instance and attach the two VPC networks to each VM instance.
  12. Compare the VM vNIC performance with the vDPA NIC performance.
  1. Energize and connect hardware.

    Check your hardware devices. Configure servers and network devices according to Figure 1.

  2. Install operating systems.

    Install operating system for the two servers according to following steps:

    1. Check BIOS.

      After you energize a server, manually start the server and enter its BIOS, making sure that SR-IOV Support and Intel VT-d / AMD IOMMU are enabled.

    2. Burn the custom ISO into a USB drive.
    3. Make the server start from the USB drive.
    4. Install the operating system following the wizard.

      Configure partitioning and begin to install the operating system for the server as a management node. The installation is an automatic process. During this process, set your root account and password. After the installation is finished, restart the management node and unplug the USB drive.

  3. Configure the management network.
    Run following commands on the two management nodes in turn to configure the management network according to the table Table 1 in turn.
    # Take management node1 as an example.  # Create the aggregate NIC bond0 [root@localhost ~]# zs-bond-lacp -c bond0  # Attach the two gigabit network interfaces, eth0 and eth1, of the management node1 to bond0. [root@localhost ~]# zs-nic-to-bond -a bond0 eth0  [root@localhost ~]# zs-nic-to-bond -a bond0 eth1  # After finishing the link aggregation configurations, configure the LACP aggregation on the corresponding vSwitch network interfaces.  # Create the network bridge br_bond0 and set the IP address, netmask, and gateway of the management node1. [root@localhost ~]# zs-network-setting -b bond0 172.25.15.143 255.255.255.0 172.25.0.1  # Check whether the aggregate port bond0 is created successfully. [root@localhost ~]# zs-show-network
  4. Install HA suites.

    Install HA suites on either of the two management nodes to realize Multi-MN HA.

    Take management node1 as an example.

    1. Import HA suites to management node1 and unzip the files.
    2. Install HA suites on management node1.
    3. Use the VIP to access the UI of management node1 and finish the Cloud initialization.
    4. Switch the primary and secondary management nodes. Use the VIP to access the UI of management node2 and finish the Cloud initialization.
  5. Install the license.

    Use the VIP to access the UI of either management node. Upload the Multi-MN license you prepared.

  6. Create a cluster with Network Acceleration Support enabled.

    Use the VIP to access the UI of management node1. On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Cluster. On the Cluster page, click Create Cluster. Then, the Create Cluster page is displayed.

    On the displayed page, set the following parameters:
    • Zone: The current zone is displayed.
    • Name: Enter a name for the cluster.
    • Description: Optional. Enter a description for the cluster.
    • Hypervisor: Choose KVM.
    • CPU Architecture: Choose x86_64.
    • Network Acceleration Support: Optional. Default: disabled. In this practice, enable this feature.
      Note: You can enable Network Acceleration Support for a cluster only when you create the cluster.
    • Network Configuration:
      • VDI Network: Optional. Leave it as blank in this practice.
      • Migration Network: Optional. Leave it as blank in this prectice.
    • Advanced:
      • VM CPU Model: Optional. Use the default setting in this practice.
      • Host CPU Model: Optional. Use the default setting in this practice.
    Figure 3. Create Cluster


  7. Add the two management nodes to the cluster.

    On the main menu of ZStack Cloud, choose Resource Center > Hardware > Computing Facility > Host. On the Host page, click Add Host. Then, the Add Host page is displayed. Choose Manual Addition on the displayed page.

    Take management node1 as an example. Set the following parameters:
    • Name: Enter a name for the host.
    • Description: Optional. Enter a description for the host.
    • Virtualization Technology: Choose KVM.
    • Tag: Optional. Do not set this parameter in this practice.
    • Cluster: Choose the cluster you created with Network Acceleration enabled.
    • Host Info: Choose IP Address as the addition method.
      • IP Address: Enter the host IP. In this practice, enter the IP address of management node1: 172.25.15.143.
      • SSH Port: Enter the host port. Default: 22.
      • User Name: Enter the host user name. Default: root.
      • Password: Enter the password corresponding to the user name.
    • Scan Host IOMMU Setting: Optional. Default: disabled. In this practice, enable this feature.
      Note: The setting takes effect after you restart the host.
    • Intel EPT Hardware Assist: Optional. Default: enabled.
    • Add Another Host: Select this checkbox to add management node2.
    Figure 4. Add Management Node1


  8. Virtualize the physical NIC.
    On the details page of the host, choose Associated Resource > Physical NIC, locate a Mellanox CX-5 smart NIC and click SR-IOV on its action list to virtualized the smart NIC into VF NICs of the specified number.
  9. Create two L2 network whose Network Acceleration Mode are respectively Smart NIC and Standard. Attach the two L2 networks to the cluster.
    1. Create an L2 network whose Network Acceleration Mode is Smart NIC and attach it to the cluster.

      On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L2 Network Resources > L2 Network. On the L2 Network page, click Create L2 Network. Then, the Create L2 Network page is displayed.

      On the displayed page, set the following parameter:
      • Zone: The current zone is displayed.
      • Name: Enter a name for the L2 network.
      • Description: Optional. Enter a description for the L2 network.
      • Type: Choose L2VlanNetwork.
      • Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
      • Network Acceleration Mode: Choose Smart NIC.
      • VLAN ID: Enter 2334 in this practice.
      • NIC Name: Enter the name of the L2 network device. In this practice, enter dpdkbond2.
      Figure 5. Create L2 Network ( Smart NIC)


    2. Create an L2 network whose Network Acceleration Mode is Standard and attach it to the cluster.
      On the Create L2 Network page, set the following parameters:
      • Zone: The current zone is displayed.
      • Name: Enter a name for the L2 network.
      • Description: Optional. Enter a description for the L2 network.
      • Type: Choose L2VlanNetwork.
      • Cluster: Optional. Choose the cluster you created with Network Acceleration Support enabled.
      • Network Acceleration Mode: Choose Standard.
      • VLAN ID: In this practice, enter 2200.
      • NIC Name: Enter the name of the L2 network device. In this practice, enter bond0.
      Figure 6. Create L2 Network (Standard)


  10. Create a VPC vRouter and two VPC networks.
    1. Create a VPC vRouter.

      Upload the VPC vRouter image you prepared to Cloud and create a management network and vRouter offering according to Table 3.

      On the main menu of ZStack Cloud, choose Resource Center > Network Resource > vRouter > VPC vRouter. On the VPC vRouter page, click Create VPC vRouter. Then, the Create VPC vRouter page is displayed.

      On the displayed page, set the following parameters:
      • Name: Enter a name for the VPC vRouter.
      • Description: Optional. Enter a description for the VPC vRouter.
      • vRouter Offering: Choose a vRouter Offering.
      • Cluster: Optional. Specify a cluster for the host on which the VPC vRouter is to be started.
      • Storage Allocation Policy: Specify how the Cloud allocates a primary storage. The following two policies are supported:
        • System Allocation: The Cloud allocates a primary storage according to the preconfigured policy.
        • Custom: Select a primary storage as needed.
          • Primary Storage: Select a primary storage for the VPC vRouter.
      • Host: Optional. Specify a host to start the VPC vRouter. In this practice, choose the management node1.
      • Default IPv4: Optional. Leave this parameter as blank in this practice.
      • Assign Management Network IP: Optional. Leave this parameter as blank in this practice.
      • DNS: Optional. Leave this parameter as blank in this practice.
      • CPU Pinning: Optional. Leave this parameter as blank in this practice.
      Figure 7. Create VPC vRouter


    2. Create the VPC network1.

      Create the VPC network1 according to Table 4.

      On the main menu of ZStack Cloud, choose Resource Center > Network Resource > L3 Network Resources > VPC Network. On the VPC Network page, click Create VPC Network. Then, the Create VPC Network page is displayed.

      On the displayed page, set the following parameters:
      • Name: Enter a name for the VPC network. Enter VPC Network1 in this practice.
      • Description: Optional. Enter a description for the VPC network.
      • L2 Network: Choose an L2 network (Network Acceleration Mode: Standard).
      • VPC vRouter: Optional. Choose a VPC vRouter.
      • Network Range Method: Choose CIDR.
        • CIDR: Enter 192.167.0.1/24.
        • Gateway: Optional. Enter 192.167.0.1 in this practice.
        • DHCP Service: Optional. Enabled by default.
        • IP Allocation Policy: Optional. Choose Random in this practice.
        • DHCP IP: Optional. Leave this parameter as blank in this practice.
      • DNS: Optional. Leave this parameter as blank in this practice.
      Figure 8. Create VPC Network1


    3. Create the VPC Network2.

      Create the VPC network2 according to Table 5.

      On the displayed page, set the following parameters:
      • Name: Enter a name for the VPC network. Enter VPC Network2 in this practice.
      • Description: Optional. Enter a description for the VPC network.
      • L2 Network: Choose an L2 network (Network Acceleration Mode: Smart NIC).
      • VPC vRouter: Optional. Choose a VPC vRouter.
      • Network Range Method: Choose CIDR.
        • CIDR: Enter 192.169.0.1/24.
        • Gateway: Optional. Enter 192.169.0.1 in this practice.
        • DHCP Service: Optional. Enabled by default.
        • IP Allocation Policy: Choose Random in this network.
        • DHCP: Optional. Leave this parameter as blank in this practice.
      • DNS: Optional. Leave this parameter as blank in this practice.
      Figure 9. Create VPC Network2


  11. Create two VM instances. Attach each one with the two VPC networks you created before.

    On the main menu of ZStack Cloud, choose Resource Center > Resource Pool > > Virtual Resource > VM Instance. On the VM Instance page, click Create VM Instance. Then, the Create VM Instance page is displayed. Choose Fast Creation.

    On the displayed page, set the following parameters:
    • Name: Enter a name for the VM instance.
    • Quantity: Enter 2 in this practice.
    • Tag: Optional. Do not set tags for the VM instance in this practice.
    • Offering: Choose Basic Offering.
      • Instance Offering: Choose an instance offering.
      • Image: Choose an image.
      • Data Volume: Optional. No data volume is attached by default.
    • Network Configuration:
      • Network: Attach the VPC network1 and VPC network2 to the VM instance.
      • Make Default: Make the VPC network2 as the default network.
      • Enable SR-IOV: Optional. Disable SR-IOV in this practice.
      • Assign IP: Optional. The checkbox is not selected by default. The IP is randomly allocated by DHCP.
      • MAC Address: Optional. The checkbox is not selected by default. The MAC is randomly allocated by DHCP.
      • Security Group: Optional. Do not set security groups in this practice.
      • EIP: Optional. Do not set EIPs in this practice.
    • User Data: Optional. Do not set User Data in this practice.
    Figure 10. Create VM Instance




  12. Compare the VM vNIC performance with the vDPA NIC performance.
    In this practice, use an iperf3 tool to test the network performance of the two VM instances. Compare the vNIC performance and vDPA NIC performance from 3 aspects: bandwidth, packet loss rate, and throughput.
    1. Query NIC information of the two VM instance.
      Name NIC Type IP Address
      VM Instance1 vNIC 192.167.0.166
      vDPA 192.169.0.214
      VM Instance2 vNIC 192.167.0.193
      vDPA 192.169.0.253
      Figure 11. VM NIC Information




    2. UDP small packet performance testing (packet size: 16 byte).

      Assume that the VM instance1 is the client and VM instance2 is the server.

      The testing involves two rounds.
      1. First Round: Send UDP packets from the VM instance2 (vNIC) to the VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
      2. Second Round: Send UDP packets from the VM instance2 (vDPA) to the VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.
      # Install iperf3 on both the client and the server. [root@localhost ~]# yum install iperf3  # First-Round Testing  # Run the following command on the server: [root@localhost ~]# iperf3 -s  # Run the following command on the client: [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.167.0.193 -i 1  # iperf3 commands information # -s //Run in the server mode. # -u //UDP mode. # -l //UDP small packet size. Unit: byte. # -b //Target bandwidth. Unit: bits/sec. # -t //Testing period. Unit: s. # -c //Run in the client mode and specify the server IP. # -i //Echo interval. Unit: second.  # The first-round testing lasts about 30 seconds.  # Second-Round Testing  # Run the following command on the client: [root@localhost ~]# iperf3 -u -l 16 -b 100m -t 30 -c 192.169.0.253 -i 1  # The second-round testing lasts about 30 seconds.
      Test Result
      UDP Small Packet Testing Server - Client Bandwidth (Mbits/sec) Packet Loss Rate (%) Throughput (pps)
      First Round VM Instance2 (vNIC) - VM Instance1 (VNIC) 12.1 0.46 95144.3
      Second Round VM Instance2 (vDPA) - VM Instance1 (vDPA) 19.2 0.74 151418
      Figure 12. UDP Small Packet Testing Result
      First Round

      Second Round

    3. UDP big packet performance testing (packet size: 1460 byte).

      Assume that the VM instance1 is a client and VM instance2 is a server.

      The testing involves two rounds:
      1. First Round: Send UDP packets from VM instance2 (vNIC) to VM instance1 (vNIC). View the bandwidth, packet loss rate, and throughput.
      2. Second Round: Send UDP packets from VM instance2 (vDPA) to VM instance1 (vDPA). View the bandwidth, packet loss rate, and throughput.
      # First-Round Testing  # Run the folliwng command on the client (the UDP packet size is 1460 by default): [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.167.0.193 -i 1  # The first-round testing lasts about 30 seconds.  # Second-Round Testing  # Run the following command on the client (the UDP packet size is 1460 byte by default): [root@localhost ~]# iperf3 -u -b 1G -t 30 -c 192.169.0.253 -i 1  # The second-round testing lasts about 30 seconds.
      Testing Result
      UDP Big Packet Testing Server - Client Bandwidth (Mbits/sec) Packet Loss Rate (%) Throughput (pps)
      First Round VM Instance2 (vNIC) - VM Instance1 (vNIC) 267 51 94735.1
      Second Round VM Instance2 (vDPA) - VM Instance1 (vDPA). 843 1.8 148343.5
      Figure 13. UDP Big Packet Testing Result
      First Round

      Second Round

Now, you have successfully put NICs into use.

Archives

Download Document Archives

Download

Already filled the basic info?Click here.

Enter at least 2 characters.
Invalid mobile number.
Enter at least 4 characters.
Invalid email address.
Wrong code. Try again. Send Code Resend Code (60s)

An email with a verification code will be sent to you. Make sure the address you provided is valid and correct.

同意 不同意

I have read and concur with the Site Terms 、 Privacy Policy 、 Rules and Conventions on User Management of ZStack Cloud

Download

Not filled the basic info yet? Click here.

Invalid email address or mobile number.
同意 不同意

I have read and concur with the Site Terms 、 Privacy Policy 、 Rules and Conventions on User Management of ZStack Cloud

Email Us

contact@zstack.io
ZStack Training and Certification
Enter at least 2 characters.
Invalid mobile number.
Enter at least 4 characters.
Invalid email address.
Wrong code. Try again. Send Code Resend Code (60s)

同意 不同意

I have read and concur with the Site Terms 、 Privacy Policy 、 Rules and Conventions on User Management of ZStack Cloud

Email Us

contact@zstack.io
Request Trial
Enter at least 2 characters.
Invalid mobile number.
Enter at least 4 characters.
Invalid email address.
Wrong code. Try again. Send Code Resend Code (60s)

同意 不同意

I have read and concur with the Site Terms 、 Privacy Policy 、 Rules and Conventions on User Management of ZStack Cloud

Email Us

contact@zstack.io

The download link is sent to your email address.

If you don't see it, check your spam folder, subscription folder, or AD folder. After receiving the email, click the URL to download the documentation.

The download link is sent to your email address.

If you don't see it, check your spam folder, subscription folder, or AD folder.
Or click on the URL below. (For Internet Explorer, right-click the URL and save it.)

Thank you for using ZStack products and services.

Submit successfully.

We'll connect soon.

Thank you for using ZStack products and services.