The default maximum size for data transmission on Ethernet is 1500 bytes for legacy compatibility reasons, unfortunately for newer high-speed networks breaking up data into 1500 byte chunks called frames creates too much overhead and prevents one from utilizing the full capabilities of the network hardware. For example on a 10Gbit link you are only able to transmit about 3-4Gbit of data between machines using 1500 byte frames due to the overhead of processing so many small frames of data while we were able to easily saturate 10Gbit when using jumbo frames. Modern full-featured network equipment can transmit larger frame sizes if configured to do so, usually up to 9000 bytes, allowing for full utilization of the available resources. Jumbo frames are usually only used in a controlled and segmented environment, for example an iSCSI SAN, as they can cause additional overhead from the need for packet fragmentation at the point when converting between a 9000 byte frame network and a standard 1500 byte frame network.
Configuring Jumbo Frames on an ESX VM server
Support for jumbo frames is included in ESX and ESXi but is not exposed in the GUI for the default vSwitch implementation, in addition remote configuration for vSwitches is not enabled on ESXi using the free license from the API using PowerCLI or VMware CLI.
Example for vSwitch:
esxcfg-vswitch --list esxcfg-vswitch --mtu 9000 vSwitch0
Example for VMkernel interface (for vMotion or iSCSI):
esxcfg-vmknic --list esxcfg-vmknic --mtu 9000 "VMkernel Portgroup Name"
Note: When configuring a VMkernel interface for Jumbo Frames make sure that the underlying vSwitch is also configured for Jumbo Frames.
Configuring Jumbo Frames on ESX VM guest
The default emulated Intel E1000 network device presented to guests in VMware ESX does not support Jumbo Frames, one must install VMware Tools and enable a VMXNET3 interface to take advantage.