24.03.2019 - update for ESXi version 6.7, please scroll down

Introduction

Recently we had to prepare the lab for executing test scenarios for VMware’s ESXi and iSCSI. You can use laptop for that. Here is a how-to on running ESXi 6.0 under QEMU/KVM.

Requirements

• Modern laptop - with some sort of virtualization acceleration technology included (Intel i7-4600U in my case; 8GB of memory may not be enough to run complex setup)
• VMware vSphere ESXi iso (you can download it from my.vmware.com)
• QEMU-KVM installed

Prepare host (your laptop)

Enable nested virtualization. This allows You to run a virtual machine (VM) inside another VM while still using hardware acceleration from the host.

Check if already enabled:

# cat /sys/module/kvm_intel/parameters/nested
N

If no, use boot parameter kvm-intel.nested=1 or configure module parameter (options kvm-intel nested=y in /etc/modprobe.d/kvm-intel.conf) and reload kvm-intel module.

Same procedure applies to AMD CPUs. Use kvm-amd instead.

Create VM for ESXi

Create a new VM for ESXi:

• at least 2 vCPUs
• at least 4GB of memory
• set vCPU model to Westmere
• set Chipset model to Q35
• use e1000 network device model
• use SATA disks
• use SATA CDROMs

Not all QEMU/KVM GUIs allows to enable nested virtualization option, so add it manually - virsh edit your-vm-name and put <kvm><hidden state='on'/></kvm> and <feature policy='require' name='vmx' />.

  <features>
    .
    .
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
.
.
.
  <cpu mode='custom' match='exact'>
    .
    .
    <feature policy='require' name='vmx'/>
  </cpu>

Boot ESXi VM from ESXi iso

Creating VM inside ESXi

ESXi is up and running at your command. When creating VM inside ESXi remember to put addition parameter vmx.allowNested = "TRUE"

PS. We recommend FreeNAS when testing different storage solutions for ESXi - iSCSI, NFS and such. It’s easy to setup and maintain and offers wide variety of options.

24.03.2019 - ESXi version 6.7

~# virsh dumpxml esxi67
<domain type='kvm' id='15'>
  <name>esxi67</name>
  <uuid>e148bff6-b548-4eb3-98b2-49e6bbd33c33</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-noTSX-IBRS</model>
    <vendor>Intel</vendor>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='umip'/>
    <feature policy='require' name='ssbd'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='abm'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/michal/vms/esxi67.qcow2'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <boot order='1'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:68:95:1b'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='e1000'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/5'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/5'>
      <source path='/dev/pts/5'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich6'>
      <alias name='sound0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir0'/>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir1'/>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <stats period='5'/>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+77:+77</label>
    <imagelabel>+77:+77</imagelabel>
  </seclabel>
</domain>