VirtualBox allows us to use a disk device directly, rather than using a file as a virtual volume. For me, since I have two SSDs in my laptop, it meant I could tinker with virtual machines without risking my Windows 7 partition, while also being able to boot the VMs on real hardware if I wanted.
I never fully got my head around VMDK formats for RAW devices when using multiple independent partitions (e.g. one for each VM) on the disk – it’s hard to experiment on physical disks without risking disaster.
The knowledge that I was missing is that the partition layout of the disk is sort of irrelevant. What we define in a virtual disk is the virtual sector list, known as ‘extents’ and, for each extent, where the real sector data resides (filesystem, raw disk sectors, or simply zeros). VBoxManage uses the partitioning information when creating raw disks to determine the sectors required when creating the VMDK file using the CLI.
I say ‘sort of’ irrelevant, because while VirtualBox does just use the disk sector ranges to map to the virtual disk, it obtains an exclusive lock on the partition from the OS.
A worked example of my VMDK extents.
Part or all of a disk can be used as a RAW partition in a virtual machine. The VMDK file will define the sector layout of your virtual disk, as seen by the VM. This is defined as a list of ‘extents’ with access mode, size, type, physical device, and starting offset.
Below is the partitioning I have defined in the host. I use the whole disk for VMs, so none of these partitions are actually used for host filesystems. There is around 100GB of unpartitioned space on this 240GB disk.
wmic:root\cli>partition get Name,NumberOfBlocks,StartingOffset Name NumberOfBlocks StartingOffset Disk #0, Partition #0 125829120 1048576 Disk #0, Partition #1 125829120 64425558016 Disk #0, Partition #2 33554432 128850067456
wmic output above shows the partitions, though gives the starting offset in bytes, so we need to divide these numbers by 512 to get the starting block offset (which gives 2048, 125831168, and 251660288 respectively).
Below is an snippet from one of my virtual disks – it’s physically primary partition 2 on the disk, exactly 60GiB in size. It has no access to any other sectors on the disk.
The virtual disk is backed by a 1MiB file (2048 sectors at physical offset 0) and a chunk of PhysicalDrive0 (125829120 sectors at physical offset 125831168), so the extent descriptions effectively concatenate the file and the physical disk sectors into a single virtual disk as seen by the VM.
# Extent description RW 2048 FLAT "Raw_Sectors_1MiB" 0 RW 125829120 FLAT "\\.\PhysicalDrive0" 125831168
In the lines above, taken from my
vmdk file, line 2 defines a read-write area of the virtual disk that is 2048 sectors in length, starting at sector zero. It contains flat data, in other words raw sectors, and the sectors are stored in the file
./Raw_Sectors_1MiB. This allows a virtual disk to have its own partitioning and bootloader without messing up the contents of the main physical partition sectors (e.g. without it, the underlying partition would end up with an MBR/GPT etc., which would only make sense to a VM).
An empty file can be created with the following command, which in this case creates a 1MiB file to hold the 2048 sectors at the start of my virtual disk: –
fsutil file createnew Raw_Sectors_1MiB 1048576
In line 3 of the extent descriptions, I define the next 125829120 sectors in the virtual drive as being backed by PhysicalDrive0 sectors starting at physical sector offset of 125831168. Since we are accessing the raw physical drive, we are paying no regard to the actual partitioning of the physical drive, so it’s important to make sure the physical sector offset and sector size are correct.
The information below shows what our virtual disk looks like to the guest, in this case booting from a live-cd.
From the above fdisk output on the VM, we can see that the disk size is 125831168 sectors. This is exactly the sum of the sector sizes in the two extents defined above (2048 + 125829120).
Note that the disk size of 125831168 is coincidentally the same as the sector offset into the physical disk in the VMDK. This is because there happens to be a similar 60GiB partition preceding the one we’re using, which has a physical 2048 sectors preceding it (for partition alignment to a 1MiB boundary, as recommended). In short, the offset to our 1MiB + 60GiB partition is at the end of a 1MiB + 60GiB chunk of physical disk!
So, that was a short explanation of what it means to define a virtual disk from raw sectors on a host drive. This is the information I would have benefited from when reading the references below.
Breaking up the partition.
In this example, I will use the same partition for two virtual machines. Note that, due to the exclusive access rights of the guest’s process on the partition, only one VM defined like this can run at any one time. Therefore, this is done just for example, though it certainly has some use-cases.
So, suppose I decide that 60GiB is too much for the VM I was creating, I decided to use half the partition for one VM’s disk, and the other half for another. I didn’t modify the disk’s partition table at all, I just allocated half the physical sectors to one virtual disk, and the other half to the another virtual disk. The extents were defined as follows.
# Extents for the first virtual disk on P2 RW 2048 FLAT "Raw_Sectors_1MiB" 0 RW 62914560 FLAT "\.\PhysicalDrive0" 125831168
# Extents for the second virtual disk on P2 RW 2048 FLAT "Raw_Sectors_1MiB" 0 RW 62914560 FLAT "\\.\PhysicalDrive0" 188745728
These are defined in different directories so each has its own
Raw_Sectors_1MiB file for the first 2048 sectors. The second extent in both virtual disks is 62914560 sectors (30GiB), one backed by sectors at the start of partition 2 (125831168), and the other backed by sectors halfway through partition 2 (188745728).
I have deliberately not given the full files here because none of the other lines in a VMDK file are relevant outside of my system. You have to follow the official docs when messing around with physical disks, but hopefully this will help clarify things if you have the same unanswered questions that I did.
The VirtualBox documentation gives a lot of detail on how to configure RAW disk. You can read it here: http://www.virtualbox.org/manual/ch09.html#rawdisk
The libvmdk project has a detailed document that describes the VMDK file format. Not all the information will apply to the VirtualBox implementation, but what VirtualBox uses is documented clearly here: https://github.com/libyal/libvmdk/blob/master/documentation/VMWare%20Virtual%20Disk%20Format%20(VMDK).asciidoc