VMFS3HeapSize Problems

A Customer called me yesterday and told me that he had certain problems with his VMFS datastores.

He explained the following behavior:

  • Cannot migrate VMs from one host to another host
  • Cannot power on or off VMs
  • VMDK files shows 0gb

We done a remote session and I went through the vmkernel.log. I found the following error messages:

I checked the datastores and saw that there are several stores with more than 2tb (1 with 10tb, 1 with 3tb and 2-4 with 2 tb).

He told me that the complete storage box is about 20tb.

I found a KB article which describes the problem. After reading Heap Size in this article I could remember that I read about this problem some month ago when Michael Webster blogged about the topic The Case for Larger Than 2TB Virtual Disks and The Gotcha with VMFS.

After I know the problem and how to fix it, I wanted to know which impact it would have when Heap Size runs out of space. I found a blog post at VirtuallyHyper from Karim Elatov which describes how to get the currently used Heap Size.

There are two ways to get the used and configured Heap Size. According to VirtuallyHyper you can use the following:

At the ESXi Console type:

After this inputs you get a similar output:

The output shows that the configured maximum Heap Size is 640MB, which can also be checked by typing esxcfg-advcfg -g /VMFS3/MaxHeapSizeMB at the ESXi console. I’m using a fresh installed ESXi and as you can see VMware changed the default MaxHeapSizeMB to it’s maximum of 640MB. In the output you can also see the utilization of the heap size.

There are a second methode of showing the heap size. You can type

and the the following output

Now I know how much Heap Size the ESX is using but I still didn’t now what will happen if the Heap Size runs out of memory.

I decided to twitter to Cormag Hogan (@vmwarestorage) to ask for an advice and only few seconds later he sent me a post from his blog with the required information.

Shame on me I haven’t read this post before. 😉

Leave a Reply