HCIBench stands for "Hyper-converged Infrastructure Benchmark". It's essentially an automation wrapper around the popular and proven open source benchmark tools: Vdbench and Fio that make it easier to automate testing across a HCI cluster. HCIBench aims to simplify and accelerate customer POC performance testing in a consistent and controlled way. The tool fully automates the end-to-end process of deploying test VMs, coordinating workload runs, aggregating test results, performance analysis and collecting necessary data for troubleshooting purposes.
HCIBench is not only a benchmark tool designed for vSAN, but also could be used to evaluate the performance of all kinds of Hyper-Converged Infrastructure Storage in vSphere environment.
- Web Browser:
IE8+, Firefox or Chrome
- vSphere 5.5 and later environments for both HCIBench and its client VMs deployment
- Switched UI to dark theme
- Redesigned VMDK preparation methodology, which can complete much faster using RANDOM on deduped storage
- Added VMDK preparation process update
- Added Graphite port check into prevalidation
- Added vCenter/Host password obfuscation
- Added "Delete Guest VM" button
- Fixed Grafana display issue
- Fixed FIO blank results issue
- Bug fixes
- MD5 checksum of HCIBench_2.1.ova: d37e6f164ed962a6e7ccbe104ba9eaec
- Added fio as an alternative workload generator
- Added Grafana for workload live monitoring
- Switched UI to clarity
- Allow user to select one to four cases while using easy-run
- Bug fixes
- MD5 checksum of HCIBench_2.0.ova: ba3c2b06b8c27fb41a1bb68baedb325f
- Enhanced easy-run, put original 4k,70% read as the first test case, then 4k, 100% read and 256k, 100% write
- Enhanced tvm deployment validation
- Added Checksum into easy-run consideration
- Updated guest VM template with increased ring_pages and disk scheduler
- Added DNS configuration guidance into welcome message
- Added 2 more test cases into easy-run, 4k 100% random read and 256k 100% sequential write
- Batch deployment will be involved if deploying more than 8 VMs to speed up deployment process
- Allow user to choose IP prefix when using static IP
- Optimized UI to allow user to review the results by single click
- Fixed regression issue when placing Datacenter/Cluster in the folder
- Fixed regression when datastore is in the datastore folder
- Avoid checking connection to host directly and use tvm deployment instead
- Added Vdbench version check in summary script
- Added resource pool and VM folder fields for VMC environment
- Fixed easy-run disk size issue
- Enhanced pre-validation error message handling
- Changed the names of network interface from "Public Network" to "Management Network", and "Private Network" to "VM Network"
- Enhanced write/read buffer/cache methodology
- Fixed network ip-prefix selection issue
- Fixed 95% percentile calculation issue
- Fixed vSAN Performance Diagnostic API call
- Fixed network validation message not clear issue
- Fixed setting re-use VMs as default bug in 1.6.7
- Enabled https instead of http
- Added storage policy field, user can specify storage policy for the data disks. For this version, storage policy can't be assigned to existing client VMs
- Enhanced deployment methodology
- Enhanced vSAN Observer to avoid blow up the memory
- Enhanced vSAN Performance Diagnostic API call with HCIBench workload configuration included
- Added timestamp to the testing status
- Bug fixes
- Spectre & Meldown patch on both HCIBench VM and Client VM
- Added client VM prefix field, allow running multiple HCIBench instances against single cluster
- Attach testing log along with testing results
- Enabled live vSAN Observer when running testing, using https://HCIBench_IP:8010
- Updated the drop read/write cache script
- Added more message info during the testing
- Bug fixes
- Added case comparisons by generating an XLS file for each test folder
- Fixed bug when there's white space in datastore name or test name
- Enhanced IP segment selection
- Set open file limit to 4096
- Updated vm-tools to the latest version
- Bug fixes
- Enhanced 95th percentile calculation.
- Added Curve and Multi Run calculation.
- Added SSH Service validation.
- Replaced DHCP Service with Static IP Service.
- Added IP conflict check.
- Fixed bunch of bugs.
- Change the default client VM RAM from 4GB to 8GB
Version Version 1.6.3
- Enhanced vSANPerformanceDiagnose function call
- Enhanced port 443 validation
- Enhanced results calculation
- Added host maintenance mode validation
- Added deployment validation
- Integrated with vSAN Performance Diagnostic of vSphere_6.5U1/vSAN_6.6.1.
- Added DHCP Service validation.
- Added Vdbench workload profile validation.
- Removed the root password expiration policy.
- Changed results display to show full file names.
- Changed easy-run calculation from host basis to disk-group basis.
- Added network name uniqueness check
- Changed the "disk warmup" to "Virtual disk preparation" to avoid confusion
- Changed the pvscsi configuration, when there are more than 4 vmdks per pvscsi, more pvscsi controllers will be added and the vmdks will be evenly distributed
- Bug fixes
- Added Clear read/write cache option for vSAN.
- Added Easy Run feature for vSAN, Easy Run can help vSAN user to determine the VMs/VMDKs/size/Disk_Init_Methods and run testing automatically.
- Added Re-use VMs feature, user is able to re-use the existing client VMs for more tests.
- Added the 95th percentile of calculation into the results.
- Had special characters issue resolved and got other bug fixes.
- Added the checking if user saved the configuration
- Increased the stack size to 65536 due to "ls too many arguments issue while process PCPU usage"
- Fixed the typo from "Netowrk" -> "Network" in the deployment page.
- Added vCenter hostname resolve checking.
- Fixed the client VM OS VMDK size from 15.5GB to 16GB; changed the vmdk size specification from decimal format to binary format.
- The bug "When tested against a non-vSAN datastore and having 'Directly Deploy on Hosts' checked, the test won't go through after deployment." is fixed.
- Enhancement: extract storage policy information when doing pre-validation and checks if the deployment size would be too aggressive to vSAN.
- Bug "test would fail if datastores' name contains white space" fixed.