Sign up for the quarterly Flings Newsletter here!

HCIBench

Communities:

HCIBench stands for "Hyper-converged Infrastructure Benchmark". It's essentially an automation wrapper around the popular and proven VDbench open source benchmark tool that makes it easier to automate testing across a HCI cluster. HCIbench aims to simplify and accelerate customer POC performance testing in a consistent and controlled way. The tool fully automates the end-to-end process of deploying test VMs, coordinating workload runs, aggregating test results, and collecting necessary data for troubleshooting purposes.

HCIBench is not only a benchmark tool designed for vSAN, but also could be used to evaluate the performance of all kinds of Hyper-Converged Infrastructure Storage in vSphere environment.

  • Web Browser:
    IE8+, Firefox or Chrome
  • vSphere 5.5 and later environments for both HCIBench and its client VMs deployment
See the HCIBench_User_Guide.pdf for details.

Version 1.6.8.5

  • Added 2 more test cases into easy-run, 4k 100% random read and 256k 100% sequential write
  • Batch deployment will be involved if deploying more than 8 VMs to speed up deployment process
  • Allow user to choose IP prefix when using static IP
  • Optimized UI to allow user to review the results by single click
  • Fixed regression issue when placing Datacenter/Cluster in the folder

Version 1.6.8.1

  • Fixed regression when datastore is in the datastore folder
  • Avoid checking connection to host directly and use tvm deployment instead
  • Added Vdbench version check in summary script

Version 1.6.8

  • Added resource pool and VM folder fields for VMC environment
  • Fixed easy-run disk size issue
  • Enhanced pre-validation error message handling
  • Changed the names of network interface from "Public Network" to "Management Network", and "Private Network" to "VM Network"

Version 1.6.7.2

  • Enhanced write/read buffer/cache methodology
  • Fixed network ip-prefix selection issue
  • Fixed 95% percentile calculation issue

Version 1.6.7.1

  • Fixed vSAN Performance Diagnostic API call
  • Fixed network validation message not clear issue
  • Fixed setting re-use VMs as default bug in 1.6.7

Version 1.6.7

  • Enabled https instead of http
  • Added storage policy field, user can specify storage policy for the data disks. For this version, storage policy can't be assigned to existing client VMs
  • Enhanced deployment methodology
  • Enhanced vSAN Observer to avoid blow up the memory
  • Enhanced vSAN Performance Diagnostic API call with HCIBench workload configuration included
  • Added timestamp to the testing status
  • Bug fixes

Version 1.6.6.

  • Spectre & Meldown patch on both HCIBench VM and Client VM
  • Added client VM prefix field, allow running multiple HCIBench instances against single cluster
  • Attach testing log along with testing results
  • Enabled live vSAN Observer when running testing, using https://HCIBench_IP:8010
  • Updated the drop read/write cache script
  • Added more message info during the testing
  • Bug fixes

Version 1.6.5.2

  • Added case comparisons by generating an XLS file for each test folder
  • Fixed bug when there's white space in datastore name or test name

Version 1.6.5.1

  • Enhanced IP segment selection
  • Set open file limit to 4096
  • Updated vm-tools to the latest version
  • Bug fixes

Version 1.6.5

  • Enhanced 95th percentile calculation.
  • Added Curve and Multi Run calculation.
  • Added SSH Service validation.
  • Replaced DHCP Service with Static IP Service.
  • Added IP conflict check.
  • Fixed bunch of bugs.
  • Change the default client VM RAM from 4GB to 8GB

Version Version 1.6.3

  • Enhanced vSANPerformanceDiagnose function call
  • Enhanced port 443 validation
  • Enhanced results calculation
  • Added host maintenance mode validation
  • Added deployment validation

Version 1.6.2

  • Integrated with vSAN Performance Diagnostic of vSphere_6.5U1/vSAN_6.6.1.
  • Added DHCP Service validation.
  • Added Vdbench workload profile validation.
  • Removed the root password expiration policy.
  • Changed results display to show full file names.
  • Changed easy-run calculation from host basis to disk-group basis.

Version 1.6.1

  • Added network name uniqueness check
  • Changed the "disk warmup" to "Virtual disk preparation" to avoid confusion
  • Changed the pvscsi configuration, when there are more than 4 vmdks per pvscsi, more pvscsi controllers will be added and the vmdks will be evenly distributed
  • Bug fixes

Version 1.6.0.0

  • Added Clear read/write cache option for vSAN.
  • Added Easy Run feature for vSAN, Easy Run can help vSAN user to determine the VMs/VMDKs/size/Disk_Init_Methods and run testing automatically.
  • Added Re-use VMs feature, user is able to re-use the existing client VMs for more tests.
  • Added the 95th percentile of calculation into the results.
  • Had special characters issue resolved and got other bug fixes.

Version 1.5.0.5

  • Increased Timeout value of client VM disk from 30 seconds to 180 seconds.
  • Disabled client VM password expiration.
  • Disabled client VM OS disk fsck.
  • Set Observer interval to 60 seconds to shrink the size of observer data.
  • Fixed PCPU calculation.
  • Created link directory of /opt/automation/logs, user will be able to review the testing logs in http://HCIBENCH/hcibench_logs/
  • Increased the RAM of HCIBench from 4GB to 8GB to avoid running out-of-resource issue.
  • Version 1.5.0.4

    • Added the checking if user saved the configuration
    • Increased the stack size to 65536 due to "ls too many arguments issue while process PCPU usage"
    • Fixed the typo from "Netowrk" -> "Network" in the deployment page.
    • Added vCenter hostname resolve checking.
    • Fixed the client VM OS VMDK size from 15.5GB to 16GB; changed the vmdk size specification from decimal format to binary format.

    Version 1.5.0.3

    • The bug "When tested against a non-vSAN datastore and having 'Directly Deploy on Hosts' checked, the test won't go through after deployment." is fixed.
    • Enhancement: extract storage policy information when doing pre-validation and checks if the deployment size would be too aggressive to vSAN.

    Version 1.5.0.2

    • Bug "test would fail if datastores' name contains white space" fixed.
    Report a Bug