Summary
Guest Reclaim reclaims dead space from NTFS volumes hosted on a thin provisioned SCSI disk. The tool can also reclaim space from full disks and partitions, thereby wiping off the file systems on it. As the tool deals with active data, please take all precautionary measures understanding the SCSI UNMAP framework and backing up important data.
Features
- Reclaim space from Simple FAT/NTFS volumes
- Works on WindowsXP to Windows7
- Can reclaim space from flat partitions and flat disks
- Can work in virtual as well as physical machines
Whats a Thin provisioned (TP) SCSI disks?
In a thin provisioned LUN/Disk, physical storage space is allocated on demand. That is, the storage system allocates space as and when a client (example a file system/database) writes data to the storage medium. One primary goal of thin provisioning is to allow for storage overcommit.
A thin provisioned disk can be a virtual disk, or a physical LUN/disk exposed from a storage array that supports TP. Virtual disks created as thin disks are exposed as TP disks, starting with virtual Hardware Version 9.
For more information on this please refer http://en.wikipedia.org/wiki/Thin_provisioning.
What is Dead Space Reclamation?
Deleting files frees up space on the file system volume. This freed space sticks with the LUN/Disk, until it is released and reclaimed by the underlying storage layer. Free space reclamation allows the lower level storage layer (for example a storage array, or any hypervisor) to repurpose the freed space from one client for some other storage allocation request.
For example:
- A storage array that supports thin provisioning can repurpose the reclaimed space to satisfy allocation
requests for some other thin provisioned LUN within the same array. - A hypervisor file system can repurpose the reclaimed space from one virtual disk for satisfying allocation needs of some other virtual disk within the same data store.
GuestReclaim allows transparent reclamation of dead space from NTFS volumes.
For more information and detailed instructions, view the Guest Reclaim ReadMe (pdf)
System Requirements
Thin provisioned (TP) SCSI disk
Space can be reclaimed on SCSI disks that advertise themselves as thin provisioned (TP) devices.
GuestReclaim queries the device for its TP status using standard SCSI primitives like reading the vital product data page (B0 vpd). GuestReclaim will issue SCSI Unmap commands to the underlying storage for reclaiming dead space.
Supported Operating Systems
- Desktop: XP onwards up to Windows 7
- Server: Until Windows 2008
Administrative Privileges
The tool needs to be executed with Administrator privileges.
Supported Filesystem NTFS only.
Instructions
The tool is provided as a standalone binary executable. Unzip the package, and it will contain a program. The tool needs to be executed with Administrator privileges.
Run GuestReclaim.exe from the command prompt. Use the --list option to list available thin provisioned disks on the system. If none show up, it means that the first 16 drives are not thin provisioned. Export an environment variable RECLAIM_DEBUG to see verbose output of the TP querying results.
For detailed instructions, you'll get the pdf in the download, but you can also view them here in the pdf.
Video
Change Log
Engineers
 | Faraz ShaikhWorks in the VMFS team group. | |
 | Prasanna AithalWorks in the VMFS team group. |
My question is when do partition, I see one host do not inquiry block limits/B0h on one model SAS HDD, but do inquiry B0h
on another model, my question is:
It should depend on different HDD family,Why does it happen?
Now, post release of vSphere 5.1, this tool seems to work on vmdks as well…
I have tested it on a windows 2008 R2 standard (without SP1) guest VM and it works like a charm.
Although, it does not seem to work with other versions of windows server like
Windows Server 2008 R2 Standard SP1
Windows Server 2008 R2 Enterprise SP!
Windows Server 2008 R2 Enterprise
Windows Server 2008 Standard
Anyone has a clue why is it so?
I am unable to see any physical disks when I hit the below command:
guestreclaim.exe –list
So found the solution to the problem.
parameters:
ctkEnabled
scsi0:0.ctkEnabled
had to be changed to “FALSE”.
Backup Software for virtual had made changes to the VMs configuration parameters.
Space reclamation for windows virtual machines with hardware version 9 and updated VM tools and running on vSphere 5.1 works perfectly fine!
Kushal,
Yes putting any filter driver on top of base VMFS files, will make disks not show in the –list command.
Filter driver would include cbt, redo snapshots etc.
Filter layer needs to implement unmap functionality.
Space efficient sparse disk *may* show up though, But … SESparse disk have their own reclaim method please use that.
-Faraz
I’ve upgraded my ESXi test host to 5.1, upgraded the VMFS partition, upgraded tools and hardware version to 9 and tried to run the tool on my Windows 2008 R2 dev box.
I now see the two drives if I use –list but trying to run the actual reclaim I just get:
TrimFileSize = 512MB
99 %FSCTL_MARK_HANDLE failed with 5
Number Of holes punched 435
Reclamation SUCCESS status = 0
Checking the size of my VMDK shows that it’s actually increased by a few GB and not shrunk by the 5oGB that it could have.
Any ideas?
Alex,
Thanks for trying this tool and providing feedback
The tool reclaims dead space, so we need to calculate the dead space to begin with.
Example,
Vmdk logical size = 50G (should be thin to begin with, thick vmdks are not reported as unmap able. Since you see the disk in the list it is thin)
Disk consumption from guest = 20G (seen from windows as disk used space)
Vmdk physical used space = 30G (seen by du *-flat.vmdk (or thru vi client) )
So
reclaimable space = Vmdk physical space – disk consumption from guest = 30G – 20G = 10G (max savings)
DO NOT run space reclaimation if there isn’t substantial reclaimable space(in order of gigabytes).
This is because reclaimation process itself has some overhead, and so
Total saving = reclaimable space – overhead
If overhead head is higher you’ll see space bloat instead of savings( which is what may have happened in your case).
Overhead is high if
-ntfs is fragmented
-unmap granularity is high(1M for VMFS)
Above behavior is also explained in “Suggested space reclaimation workflow” section of the guest reclaim readme(linked above).
If you want to test the tool in action attach a thin vmdk, format it with NTFS, copy a big iso/DVD image on the volume,delete the iso. Now run the tool to reclaim the space. Again HW version 9 and vmfs only.
Once again,thanks for trying this tool and providing feedback. You should also try SE Spare a new Vmdk format that is supposed to have a configurable block size. Using a smaller block size and some additional logic, the reclaimation overhead in SE sparse is far less
http://blogs.vmware.com/vsphere/2012/08/vsphere-5-1-new-storage-features-2.html#more-3932
Thanks
Faraz
Faraz,
Thanks for coming back to me. This is an area of VMware that causes a great deal of pain so I’m hoping for great things from this fling.
I read through all the documentation before I started and I think the disk I’m using is an excellent candidate for a test.
It’s a 300GB thin-provisioned disk that has 135GB used and is using just over 200GB of physical space on the VMFS partition (both via du -h and the datastore browser).
Using the calculation above I should be able to recover approx 65GB of space but nothing happens when I run the fling (as an admin in an elevated cmd prompt), I get the error above and the disk vmdk never shrinks.
I’ve not tried de-fragmenting the drive first but this is Windows 2008 R2 so it shouldn’t really need it.
This VMFS partition has been upgraded from VMFS 3 to VMFS 5.58 so the block size is 2MB, could this be causing the issue?
Sadly I’m not using view so SE sparse disks arn;t available to me yet.
Kind Regards,
Alex
Alex,
I tried this on a fresh Esx 5.1, VMFS5, WinXPSP2, Buslogic adapter and it seems to work at my end. So VMWare’s implementation on UNMAP for vmfs files in esx 5.1 stands verified. Now for the issue at your end, i can think of 2 things.
a. the upgraded vmfs-3/5 2MB block size could be causing unmaps not to go through, due to misalignment.
b. Windows 7 didn’t let user mode apps like guestReclaim send UNMAPS to live volumes. May be a recent patch to 2008 is disallowing UNMAPs.
To check if UNMAP works at all in your VM. I suggest you add a *NEW* disk to your virtual machine, format and copy some stuff on it and then DESTROY THIS NEW DISK by fully unmapping it using the –disk option
c:\>guestReclaim.exe –disk \\.\PhysicalDrive1
(UNDERSTAND THAT THE CONTENTS OF THE WHOLE DISK WILL BE UNMAPPED. so run it against the right disk !)
If basic unmap works, then try the –volumefreespace option after de-fragmenting.
Hope this helps.
Sample run.
1. 10G thin vmdk, freshly formatted with NTFS, takes around 59 blocks i.e 59MB disk space.
vmkfstools -D ./data-flat.vmdk
….
nb 59
…
2. After copy over a 700MB iso file, Disk Block usage goes up to 755 blocks i.e 755MB
vmkfstools -D ./data-flat.vmdk
….
nb 755
3. After deleting the iso. Disk block usage remain the same.
vmkfstools -D ./data-flat.vmdk
nb 755
4. After running reclaim in guest. it reclaimes ~500MB.
vmkfstools -D ./data-flat.vmdk
nb 260
C:\GuestReclaimNew>guestReclaim.exe –volumefreespace E:\
DiskName : \\.\PhysicalDrive1
VolumeName : \\.\E:
TrimFileName : E:\VMW1.tmp
****************************************
Storage Adapter Properties for Disk \\.\PhysicalDrive1
V…
****************************************
Storage Device Properties for Disk \\.\PhysicalDrive1
..
****************************************
Block Limits VPD Page [B0] for device \\.\PhysicalDrive1
reserved = 0
optimalTransLenGran = 0
maxTransLen = 0
optimalTransLen = 0
maxPrefetchTransLen = 0
maxUnmapLbaCount = 800
maxUnmapDescriptorCount = 1
optimalUnmapGranularity = 800
unmapAlignment = 0
ugavalid = 0
SCSI Address:
PortNumber = 2
PathId = 0
TargetId = 1
Lun = 0
TrimFileSize = 512MB
99 %
Number Of holes punched 9456
Just to make sure I’m understanding this properly… This will send SCSI unmap commands to VMFS, but will VMFS send these commands further down the line (eg. to the storage device)? For example, I have thin provisioned volumes presented to my hosts from EqualLogic storage, to which I subsequently present TP guest disks to VMs. I run this tool in the VM, it will free up white space as seen by VMFS… But will those unmap commands by passed on to the iSCSI volume itself? In other words, this only frees up white space ‘one level up?’.
I only ask because EqualLogic happens to be supporting scsi unmap in their upcoming v6 firmware, so if VMFS is passing on unmap commands to an iSCSI volume, this will also free up white space on the EQL…
Scott,
AFAIK, unmaps is 2 step process.
1. Guest freeing up space to VMFS.
2. VMFS passing down the unmap to the array.
1 and 2 are not connected. VMFS may collect free blocks from multiple guests and then send down bulk unmaps to back end array.
Our 2k3 VM does not report any thin provisioned disks using the –list command. Using the volumespacefree option:
“D:\GuestReclaim>guestReclaim.exe –fullvolume E:\
DiskName : \\.\PhysicalDrive1
VolumeName : \\.\E:
TrimFileName :
****************************************
Storage Adapter Properties for Disk \\.\PhysicalDrive1
Version = 32l
Size = 32l
MaximumTransferLength = 16777215l
MaximumPhysicalPages = 65l
AlignmentMask = 0l
AdapterUsesPio = FALSE
AdapterScansDown = FALSE
CommandQueueing = TRUE
AcceleratedTransfer = TRUE
BusType = SCSI
BusMajorVersion = 2
BusMinorVersion = 0
****************************************
Version = 40
Size = 107
DeviceType = 0
DeviceTypeModifier = 0
RemovableMedia = FALSE
CommandQueueing = TRUE
VendorIdOffset = NA
ProductIdOffset = NA
ProductRevisionOffset = NA
SerialNumberOffset = NA
BusType = SCSI
INQUIRY B0 VPD failed failed SCSI_STAT 2
Failed to get Initialize SCSI Backend TP Device \\.\PhysicalDrive1 status 50
Reclamation FAILED status = 50″
VMware Tools build 341836
VMware Version 7
ESX 4.1, Build 433742
I have replicated this problem on two different 2k3 machines.
Philip/Stefan,
This is for “Windows Machine running on ESX on VMFS data stores” and unmap support for virtual disk on ESX will be coming up in some upcoming VMWare VSphere release.
Workstation products don’t need to use this as equivalent functionality is provided under the “Disk Shrink” and umbrella options.
BTW … INQUIRY B0 VPD failed failed SCSI_STAT 2
Means that the underlying device is not supporting thin provisioning, which is true for the workstation platform.
-Faraz
Hello
This not Working on Mac Fusion Tech Preview 2012 (Windows 2008R2 VM with Hardware Version9)
C:\Users\root\Desktop\GuestReclaimNew>guestReclaim.exe –volumefreespace c:\
DiskName : \\.\PhysicalDrive0
VolumeName : \\.\c:
TrimFileName : c:\VMW558E.tmp
****************************************
Storage Adapter Properties for Disk \\.\PhysicalDrive0
Version = 32l
Size = 32l
MaximumTransferLength = 16777215l
MaximumPhysicalPages = 257l
AlignmentMask = 0l
AdapterUsesPio = TRUE
AdapterScansDown = FALSE
CommandQueueing = TRUE
AcceleratedTransfer = TRUE
BusType = UNKNOWN
BusMajorVersion = 2
BusMinorVersion = 0
****************************************
Storage Device Properties for Disk \\.\PhysicalDrive0
Version = 40
Size = 328
DeviceType = 0
DeviceTypeModifier = 0
RemovableMedia = FALSE
CommandQueueing = TRUE
VendorIdOffset = NA
ProductIdOffset = NA
ProductRevisionOffset = NA
SerialNumberOffset = (
BusType = UNKNOWN
INQUIRY B0 VPD failed failed SCSI_STAT 2
Failed to initialize SCSI Backend for TP Device \\.\PhysicalDrive0 status 50
Reclamation FAILED status = 50
Hi!
I tried this utility and get the below result.
What is the problem with my VM disk?
c:\GuestReclaimNew>guestReclaim.exe –list
DiskName :
VolumeName :
TrimFileName :
Unmap Options Available on this system
Scanning uptil max 16 drives
****************************************
Storage Adapter Properties for Disk \\.\PhysicalDrive0
Version = 32l
Size = 32l
MaximumTransferLength = 524288l
MaximumPhysicalPages = 128l
AlignmentMask = 3l
AdapterUsesPio = FALSE
AdapterScansDown = FALSE
CommandQueueing = TRUE
AcceleratedTransfer = TRUE
BusType = UNKNOWN
BusMajorVersion = 2
BusMinorVersion = 0
****************************************
Storage Device Properties for Disk \\.\PhysicalDrive0
Version = 40
Size = 328
DeviceType = 0
DeviceTypeModifier = 0
RemovableMedia = FALSE
CommandQueueing = TRUE
VendorIdOffset = NA
ProductIdOffset = NA
ProductRevisionOffset = NA
SerialNumberOffset = NA
BusType = UNKNOWN
INQUIRY B0 VPD failed failed SCSI_STAT 2
CreateFileA failed for drive \\.\PhysicalDrive1 2
CreateFileA failed for drive \\.\PhysicalDrive2 2
CreateFileA failed for drive \\.\PhysicalDrive3 2
CreateFileA failed for drive \\.\PhysicalDrive4 2
CreateFileA failed for drive \\.\PhysicalDrive5 2
CreateFileA failed for drive \\.\PhysicalDrive6 2
CreateFileA failed for drive \\.\PhysicalDrive7 2
CreateFileA failed for drive \\.\PhysicalDrive8 2
CreateFileA failed for drive \\.\PhysicalDrive9 2
CreateFileA failed for drive \\.\PhysicalDrive10 2
CreateFileA failed for drive \\.\PhysicalDrive11 2
CreateFileA failed for drive \\.\PhysicalDrive12 2
CreateFileA failed for drive \\.\PhysicalDrive13 2
CreateFileA failed for drive \\.\PhysicalDrive14 2
CreateFileA failed for drive \\.\PhysicalDrive15 2
Win32U_CreateFile failed partition \\.\A: status 2.
Win32U_CreateFile failed partition \\.\B: status 2.
IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS failed partition \\.\D: status 1.
Win32U_CreateFile failed partition \\.\E: status 2.
Win32U_CreateFile failed partition \\.\F: status 2.
Win32U_CreateFile failed partition \\.\G: status 2.
Win32U_CreateFile failed partition \\.\H: status 2.
Win32U_CreateFile failed partition \\.\I: status 2.
Win32U_CreateFile failed partition \\.\J: status 2.
Win32U_CreateFile failed partition \\.\K: status 2.
Win32U_CreateFile failed partition \\.\L: status 2.
Win32U_CreateFile failed partition \\.\M: status 2.
Win32U_CreateFile failed partition \\.\N: status 2.
Win32U_CreateFile failed partition \\.\O: status 2.
Win32U_CreateFile failed partition \\.\P: status 2.
Win32U_CreateFile failed partition \\.\Q: status 2.
Win32U_CreateFile failed partition \\.\R: status 2.
Win32U_CreateFile failed partition \\.\S: status 2.
Win32U_CreateFile failed partition \\.\T: status 2.
Win32U_CreateFile failed partition \\.\U: status 2.
Win32U_CreateFile failed partition \\.\V: status 2.
Win32U_CreateFile failed partition \\.\W: status 2.
Win32U_CreateFile failed partition \\.\X: status 2.
Win32U_CreateFile failed partition \\.\Y: status 2.
Win32U_CreateFile failed partition \\.\Z: status 2.
Drive Name Mapped Volumes
*********************************************
Reclamation SUCCESS status = 0
Is this possible to make this tool workable on NFS datastores ?
Thanks for sharing your work
Rapheal,
This tools primarily focuses on “ESX+VMFS+SCSI disk emulation” environments.
Moreover, full ESX unmap support for the above in a virtual disk environment is yet to come.
Currently it works only for RDMs, in the future I will keep posting about any good hyper visor/s that come up with official support for TP virtual disks.
NFS would be the next logical step, but it needs support from the hypervisor + nfs server.
Bottom line is if the hyper visor exposes virtual disks as thin provisioned the tools will reclaim space from it using standard SCSI UNMAP commands. Handling these UNMAP commands is the job of backend storage.
-Faraz
Mike,
The hypervisor needs to expose virtual disks as thin provisioned.
Tool won’t pickup disks that don’t show up as thin provisioned disks.
Currently ESX 5.0 doesn’t expose SCSI virtual disks as disks as thin provisioned.
About when will this supported on ESX ? please check official sources for features in upcoming vSphere 5 release if any .
Documentation section and feature section is misleading, I will clarify the documents to state this explicitly.
Please see updates to section 6
http://download3.vmware.com/software/vmw-tools/guestreclaim/GuestReclaimReadMe.pdf
Faraz Shaikh
Will this work to reclaim space for a thin provisioned VMDK for a VM for ESXi5? After reading the documentation it seems like it should but after I tried it in my lab I cannot get it to work. When I run guestReclaim.exe –list it does not list any volumes. Am I doing something wrong or is my understanding of the capabilities of the utility incorrect?
No, It will not.
The hypervisor does not yet report disks as thin provisioned to the guest OS, when using VMDKs.
For now this tool will only work on RDMs using physical compatibility mode.