Imagine a vSphere 4.0 environment, running short on space with a dozen or so small datastores and most VMs using thin-provisioned disks. A decision is made to consolidate to fewer/larger datastores and convert the VMs to thick to make it easier for the distributed administration team to realize when a datastore was low on space and not consider it for new VMs, and to also prevent any catastrophes from over-subscribing a datastore with thin-provisioned VMs that eventually fill up and overflow the datastore.
Before doing this, we need an understanding of the different vmdk storage formats. Luckily, EMC guru Chad Sakac outlines these disk types in his article Thin on Thin? Where should you do Thin Provisioning – vSphere 4.0 or Array-Level?.
- Thin: A VM’s thin-provisioned vmdk only consumes as much space on the datastore as the guest OS has written to its volumes inside the VM (the hypervisor quickly zeroes out the next available block and then commits the VM’s write IO). This format allows provisioning ahead of time an amount of storage the VM is expected to need over its lifetime and without the need to expand it in the future (though it can be expanded, if necessary). This also allows for over-subscribing a datastore by creating multiple VMs where the total sum of their thin-provisioned disks is larger than the datastore is actually capable of holding. This is OK while there is a lot of free space, but when free space is low and used space starts creeping up close to the total size of the datastore, extra caution must be taken to prevent the datastore from filling up entirely, causing a major headache. Simple things can easily push it over the edge such as VM swap files, snapshots, or other normal usages.
- ZeroedThick: A VM’s zeroedthick-provisioned vmdk appears to the datastore filesystem to be as large as the original provisioned size and actively using all that space. However, from the perspective of the storage array hosting the datastore LUN, only the amount of data written by the Guest OS to its volumes is actually written to the vmdk on the datastore. The array sees this small usage and not the larger usage of the entire vmdk that the datastore reports for the vmdk. This is important when using array-based thin-provisioning. LUNs with zeroedthick-provisioned VMs do not consume any more space than thin-provisioned VMs, so thin-provisioning on the array is still useful. When a VM needs to write a new block, no zeroing is required since the vmdk already lays claim to blocks on the filesystem that it needs, and space isn’t wasted on the array, and a datastore cannot overflow or be over-provisioned.
- EagerZeroedThick: A VM’s eagerzeroedthick-provisioned vmdk actually does consume the entire amount of provisioned space both on the datastore and on the underlying LUN. Any space remaining in the vmdk after the Guest OS has written what it needs is filled up by blocks of zeros. Consequently, these types of vmdks take awhile to create because the hypervisor inflates the vmdk with these blocks of zeros which must all be written to the storage array which also has to commit space. Not only Guest OS data but also zeroed blocks that the hypervisor has written out at disk creation that haven’t been overwritten by guest data. However, if the array supports a feature such as ‘zero-based page reclaim’ in a thin-provisioned pool, it can scan for these zeroed blocks and reclaim them as free space since it recognizes that no data is actually stored there. Note that the VMware feature Fault Tolerance (FT) requires eagerzeroedthick-provisioned disks, as does VMs participating in Microsoft Clustering (MSCS). This requirement ensures the space is there to support the critical availability requirement inherent in FT and MSCS VMs.
The concern is that by converting from thin-provisioned to thick-provisioned VMs during storage vMotion to new/larger datastores, lots of additional space would be used on the datastores and disk array. The important part to know is that while the datastore appears to be using more space with all VMs being thick-provisioned, it actually isn’t using all that space on the disk array because the VM disks are converted to zeroedthick-provisioned during storage vMotion. Not eagerzereodthick.