Ubuntu Create Hard Drive Image

Image with MBR

• Smallest useful for newer large BIOS flashing
• Larger are normal Hard/SSD disk or partition images

To make anything over 16Mb you are better off using images with an MBR which means you will need a partition table.

Create the blank image

The best thing to do is stick to simple geometries/sizes So I am going to stick with the ~max CHS 'standard' of 255 heads with 63 sectors/track = 255×63 = 16065 sectors/track. (This is also fdisk etc. choose as the default geometries for a empty image)

Lets pick a small disk to hold the bios file and the flash utils. 16Mb disk = 16x1024x1024 = 16777216 bytes at the standard 512 bytes/sector = 16777216/512 = 32768 sectors.

Now I need to find the number of tracks

I want 32768 sectors / 16065 sectors per track = 2.0397 tracks.

Clearly a track has to be a whole number, so taking the next largest 3. (I realise its all virtual, but to understand what is going on!)

3 tracks x 16065 sectors / track = 48195 sectors

512 bytes / sector = 48195 x 512 = 24675840 bytes = 24675840/(1024x1024) = 23.53Mb which is ample for the BIOS.

Create the disk, we want 48195 sectors at 512 bytes/sector, so create it in the /tmp/ folder

$ dd if=/dev/zero of=/tmp/test.ima bs=512 count=48195
48195+0 records in
48195+0 records out
24675840 bytes (25 MB) copied, 0.0458234 s, 538 MB/s

Quick Sizing table

tracks	* sec/track	= sectors (count)	* bytes/sec (bs)	= bytes	= Megabytes (Mb)
1	16065	16065	512	8225280	7.84
2	16065	32130	512	16450560	15.68
3	16065	48195	512	24675840	23.53
13	16065	208845	512	106928640	101.97
26	16065	417690	512	213857280	203.95
39	16065	626535	512	320785920	305.92
51	16065	819315	512	419489280	400.05
64	16065	1028160	512	526417920	502.03
82	16065	1317330	512	674472960	643.22
100	16065	1606500	512	822528000	784.42
128	16065	2056320	512	1052835840	1004

Syslinux

Partition blank image

Now we can partition it with fdisk, I set one new primary partition, accepting the defaults for start and end sector. Set partition 1 bootable (a), changed the partition type (t) to 04 (FAT16) and wrote it out (w)

Everything but DOS can cope with a start sector of 2048 which is useful for correct disk alignment with 4k sector HDD's or SSD's if your image is ending up on a real disk.

$ fdisk /tmp/test.ima
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x39d2aac8.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 1
First sector (2048-48194, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-48194, default 48194): 
Using default value 48194

Command (m for help): a
Partition number (1-4): 1

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 04
Changed system type of partition 1 to 4 (FAT16 <32M)

Command (m for help): w
The partition table has been altered!


WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.

Check its correct (note the heads sectors/track are defaulted to 255/63, if you choose something else, you will need to tell fdisk with -H and -S)

$ fdisk -l /tmp/test.ima 

Disk /tmp/test.ima: 24 MB, 24675840 bytes
255 heads, 63 sectors/track, 3 cylinders, total 48195 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x39d2aac8

        Device Boot      Start         End      Blocks   Id  System
/tmp/test.ima1   *        2048       48194       23073+   4  FAT16 <32M

$ file /tmp/test.ima
/tmp/test.ima: x86 boot sector; partition 1: ID=0x4, active, starthead 32, startsector 2048, 46147 sectors, code offset 0x0

hexedit /tmp/test.ima

This is the raw partition table

000001B0   00 00 00 00  00 00 00 00  D0 7D CA 1B  00 00 80 20  21 00 04 FE  3F 02 00 08  .........}..... !...?...
000001C8   00 00 43 B4  00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  ..C.....................

Put an appropriate MBR on it

Contrary to what you may think this won't overwrite/alter the partition table. Although the partition table is part of the MBR, when people say copy/overwrite the MBR it doesn't mean the partition table code.

Using syslinux MBR code

Note, DONT user cat otherwise you will overwrite the whole image, cat only works when correctly mounted on a block device with no offset, so you are writing to the beginning of the image

e.g. This will overwrite the image! $ cat /usr/lib/syslinux/mbr.bin > /tmp/test.ima

e.g. Assuming mounted above with offset, this will put the MBR in the filesystem, not at the beginning! $ cat /usr/lib/syslinux/mbr.bin > /dev/loop0

Use dd instead so you are less likely to get in a muddle, be very careful to make sure 'if' and 'of' are the right way around and the byte size (bs) and count are right!

If you forget conv=notrunc it will also overwrite the whole file! (=dont truncate the output file)

This is writing 440 bytes of the mbr.bin file to the start of the image file, once. (The partition table starts after that) Note the MBR data is actually only 440 bytes.

$ dd if=/usr/lib/syslinux/mbr.bin of=/tmp/test.ima bs=440 count=1 conv=notrunc
0+1 records in
0+1 records out
440 bytes (440 B) copied, 2.8356e-05 s, 15.5 MB/s

Get image offset to mount on loop device

You can mount an MBR image easily in userspace, first you need to find the offset, the distance between the start of the image and the start of the volume (think of it as the distance between /dev/sdd and /dev/sdd1)

Here parted is printing out the partition table in units B = bytes. (Fdisk wont show it in bytes)

$ parted /tmp/test.ima unit B print
WARNING: You are not superuser.  Watch out for permissions.
Model:  (file)
Disk /tmp/test.ima: 24675840B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start     End        Size       Type     File system  Flags
 1      1048576B  24675839B  23627264B  primary               boot

This shows 1048576 bytes offset. (You can work this out from file but more prone to mistakes, e.g. 2048 sectors x 512 bytes/sector = 1048576)

Loop mount image file

Now you need to mount it (you need to have your user in disk group to not need sudo, "sudo usermod -a -G disk yourusername" ), unfortunately losetup wont error if you get the offset wrong!

$ losetup -o 1048576 /dev/loop0 /tmp/test.ima

Note, this isn't mounting any filesystem, just the image file as a block device.

Format the new image

Not much to say, formats the partition just mounted as FAT16, pick a sensible name :)

$ mkfs.msdos -F 16 -n "richud.com" /dev/loop0
mkfs.msdos 3.0.9 (31 Jan 2010)
Loop device does not match a floppy size, using default hd params

Note, as soon as it completes udev? will notify the system that a new disk is present and if you are using Unity an icon will appear. If you go into Nautilus you should now see a drive called "richud.com" appear. Also it will probably auto mount it (my automount is turned off)

Have a look with mount to see if it automounted, if so it will be like this...

Mount the new image

$ mount
<snip>
/dev/loop0 on /media/richud.com type vfat (rw,nosuid,nodev,uid=1000,gid=1000,shortname=mixed,dmask=0077,utf8=1,showexec,flush,uhelper=udisks)

If not do this to manually mount it. It will mount using the name you specified when you created the filesystem.

$ udisks --mount /dev/loop0
Mounted /org/freedesktop/UDisks/devices/loop0 at /media/richud.com

Boot sector and system files

For syslinux use the ready made boot sector installer and boot program copier. This installs the syslinux boot sector to /dev/loop0 which due to being mounted offset is the first partition (not the start of the disk) and also copies ldlinux.sys which is the bootloader file (equivalent of DOS's system files).

$ syslinux --install /dev/loop0
$ ls -ln /media/richud.com
total 32
-r--r--r-- 1 1000 1000  32256 2012-01-15 16:52 ldlinux.sys

You will also need to create a menu file, syslinux.cfg to do anything useful with it and maybe copy some other syslinux .c32 files and probably memdisk to it. If you just want to test it you dont need this and you should see it boot and just moan it cant find a config file.

Unmount the filesystem with udisks.

$ udisks --unmount /dev/loop0

Fix the boot sector

When formatting with mkfs.msdos , it cannot see the partition table (as its loop mounted at the time, so it only sees the partition not anything before) so it cannot set the right values (although it doesnt anyway), the same goes for ms-sys. This means they need manually fixing.

The vitalness of this depends on the BIOS and how/what you are booting from. Getting it right should assure it to work on as many things as possible. This is vital for certain machines e.g. Intel 965 boards.

• Note in the examples below the "OEM name" part like "MSWIN4.1" may not look like yours, it depends what you used to format and/or added a boot sector, with Syslinux, via ms-sys for MSDOS etc.

FAT16

The following parts of the boot sector need fixing to match the partition settings.

These offsets are from the beginning of the boot sector, but as you only have partition mounted on /dev/loop0, they will be effectively at the beginning anyway. (If you mounted /tmp/test.ima later they would be 0x100000 into the image, with example settings of 2048 start sector e.g. 0x1C would be 0x10001C from image start)

In this example we are using (C/H/S) 3/255/63 and starting sector of 2048.

• offset 15 Media Descriptor = F8 (hard drive, should be already right)
• offset 18 Sectors/track (63) = 3F
• offset 1A Heads (255) = FF
• offset 1C-1D Hidden Sectors in Partition (2048) = 0800 BUT it is in little endian so = 00 08
• offset 24 Logical Drive Number of Partition = 80 (hard drive)

Dont forget you are editing the start of the partition on /dev/loop0 not the actual disk.

$ hexedit /dev/loop0

Original

00000000   EB 3C 90 4D  53 57 49 4E  34 2E 31 00  02 04 04 00  02 00 02 43  B4 F8 30 00  .<.MSWIN4.1........C..0.
00000018   20 00 40 00  00 00 00 00  00 00 00 00  00 00 29 C8  6D 9B DC 72  69 63 68 75   .@...........).m..richu
00000030   64 2E 63 6F  6D 20 46 41  54 31 36 20  20 20 FA 33  C9 8E D1 BC  FC 7B 16 07  d.com FAT16   .3.....{..

Corrected,

00000000   EB 3C 90 4D  53 57 49 4E  34 2E 31 00  02 04 04 00  02 00 02 43  B4 F8 30 00  .<.MSWIN4.1........C..0.
00000018   3F 00 FF 00  00 08 00 00  00 00 00 00  00 08 29 C8  6D 9B DC 72  69 63 68 75  ?.............).m..richu
00000030   64 2E 63 6F  6D 20 46 41  54 31 36 20  20 20 FA 33  C9 8E D1 BC  FC 7B 16 07  d.com FAT16   .3.....{..

FAT12

Keep

• C/H/S (3/255/63) and start sector (2048)

Change

• fdisk set type to 01
• mkfs.msdos -F 12 -n "richud.com" /dev/loop0
• Same offsets as FAT16 to fix.

FAT32

Make 2x larger using 6 cylinders, this equates to about 47Mb (16065x6x512/(1024x1024)) (Cant make a FAT32 one too small as wont be enough clusters)

Change

• C/H/S (6/255/63), but keep start sector (2048)
• dd if=/dev/zero of=/tmp/test.ima bs=512 count=96390
• fdisk set type to 0b (w95 FAT32)
• mkfs.msdos -F 32 -n "richud.com" /dev/loop0

First 4 offsets are same as FAT12/16

• offset 15 Media Descriptor = F8 (hard drive, should be already right)
• offset 18 Sectors/track (63) = 3F
• offset 1A Heads (255) = FF
• offset 1C-1D Hidden Sectors in Partition (2048) = 0800 BUT it is in little endian so = 00 08

NOTE!!! This offset is changed from 24 to 40!

• offset 40 (not 24 as in FAT 12/16) Logical Drive Number of Partition = 80 (hard drive)

Corrected

00000000   EB 58 90 4D  53 57 49 4E  34 2E 31 00  02 01 20 00  02 00 00 00  00 F8 00 00  .X.MSWIN4.1... .........
00000018   3F 00 FF 00  00 08 00 00  86 70 01 00  D6 02 00 00  00 00 00 00  02 00 00 00  ?........p..............
00000030   01 00 06 00  00 00 00 00  00 00 00 00  00 00 00 00  80 00 29 2B  D9 9E 44 72  ..................)+..Dr
00000048   69 63 68 75  64 2E 63 6F  6D 20 46 41  54 33 32 20  20 20 FA 33  C9 8E D1 BC  ichud.com FAT32   .3....

Unmount loop mount

Finally unmount the image file with losetup.

$ losetup -d /dev/loop0

Testing

This is running with Qemu, note there was no added config menu or anthing else, hence error message.

DOS

Partition blank image

Note, DOS cant handle a non 63 start sector (although it is ok if chainloaded e.g. via PXE booting)

Same as for syslinux, but need extra 'c' to turn on DOS compatability, which then starts default sector at 63 not 2048.

$ fdisk /tmp/test.ima
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x5a484409.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): c
DOS Compatibility flag is set (DEPRECATED!)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 1
First sector (63-48194, default 63): 
Using default value 63
Last sector, +sectors or +size{K,M,G} (63-48194, default 48194): 
Using default value 48194

Put an MBR on it

As above with dd, See here

dd if=/usr/lib/syslinux/mbr.bin of=/tmp/test.ima bs=440 count=1 conv=notrunc

Get image offset to mount on loop device

If you are making a DOS compatible one you will end up with an offset of 32256

$ parted /tmp/test.ima unit B print
WARNING: You are not superuser.  Watch out for permissions.
Model:  (file)
Disk /tmp/test.ima: 24675840B
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End        Size       Type     File system  Flags
 1      32256B  24675839B  24643584B  primary               boot

Loop mount image file

$ losetup -o 32256 /dev/loop0 /tmp/test.ima

Format new image

As above with mkfs.msdos, See here

mkfs.msdos -F 16 -n "richud.com" /dev/loop0

Mount new image

As above with udisks, See here

udisks --mount /dev/loop0

Copy system files

For a bootable DOS floppy image, copy these system files (I took them from a Win98 boot floppy), I will assume you are familiar with DOS!

These are from Win98SE 4.10.2222, (I didn't have any joy with the MSDOS6.22 ones on the first few attempts)

$ ls -ln /media/richud.com/
total 310
-rwxr-xr-x 1 1000 1000  93890 1999-04-23 23:22 COMMAND.COM
-r--r--r-- 1 1000 1000 222390 1999-04-23 23:22 IO.SYS
-r--r--r-- 1 1000 1000      9 1999-07-18 22:05 MSDOS.SYS

Unmount the filesystem with udisks.

$ udisks --unmount /dev/loop0

Write bootable boot record

Use ms-sys to write a proper boot record for FAT16 (option -6)

You will need to force it as it writing to what it thinks is a floppy (no MBR) as only the actual partition is mounted on loop0 (so thats all it knows exists)

(Assuming you just did a make when building it, the executable will be in ./ms-sys-2.2.1/bin/ms-sys)

$ ./ms-sys -6 -f /dev/loop0
FAT16 boot record successfully written to /dev/loop0
$ ./ms-sys /dev/loop0
/dev/loop0 has a FAT16 file system.
/dev/loop0 has an x86 boot sector,
it is exactly the kind of FAT16 DOS boot record this program
would create with the switch -6 on a FAT16 partition.

Fix Boot sector

As above with hexedit, See here

hexedit /dev/loop0

In this example we are using (C/H/S) 3/255/63 and starting sector of 63.

• offset 15 Media Descriptor = F8 (hard drive, should be already right)
• offset 18 Sectors/track (63) = 3F
• offset 1A Heads (255) = FF
• offset 1C-1D Hidden Sectors in Partition (63) = 00 3F , BUT it is in little endian so = 3F 00
• offset 24 Logical Drive Number of Partition = 80 (hard drive)

Unfixed

00000000   EB 3C 90 4D  53 57 49 4E  34 2E 31 00  02 04 04 00  02 00 02 04  BC F8 30 00  .<.MSWIN4.1...........0.
00000018   20 00 40 00  00 00 00 00  00 00 00 00  00 00 29 6C  B3 39 28 72  69 63 68 75   .@...........)l.9(richu
00000030   64 2E 63 6F  6D 20 46 41  54 31 36 20  20 20 FA 33  C9 8E D1 BC  FC 7B 16 07  d.com FAT16   .3.....{..

Fixed

00000000   EB 3C 90 4D  53 57 49 4E  34 2E 31 00  02 04 04 00  02 00 02 04  BC F8 30 00  .<.MSWIN4.1...........0.
00000018   3F 00 FF 00  3F 00 00 00  00 00 00 00  80 00 29 6C  B3 39 28 72  69 63 68 75  ?...?.........)l.9(richu
00000030   64 2E 63 6F  6D 20 46 41  54 31 36 20  20 20 FA 33  C9 8E D1 BC  FC 7B 16 07  d.com FAT16   .3.....{..

Unmount loop mount

Finally unmount the image file with losetup.

$ losetup -d /dev/loop0

Testing

Testing with qemu

qemu -hda test.ima

I put the image on my PXE server and served it up to a PXE booting Virtualbox (loaded via memdisk, also I gzipped it so its tiny.).

Changing the MBR

I haven't had issue with using Syslinux's for everything but as it is distributed with other versions for specific issues it may be worth noting how to change it.

If you have the mbr as a file (mbr.bin as with syslinux) use dd as explained already and write directly to the image. It should be 440 bytes.

If you want to use a Windows one ms-sys is probably the easiest way. the below will write a Windows 95B/98/98SE/ME MBR to the image (you have to force it)

$ ./ms-sys-2.2.1/bin/ms-sys -f -9 /tmp/test.ima
Windows 95B/98/98SE/ME master boot record successfully written to /tmp/test.ima

Things to check if it doesn't work

Be careful about wether you are editing the image file /tmp/test.ima or the block device /dev/loop0 Be careful not to edit the image file when its mounted as a block device

• MBR present (tend to always use syslinux), if it stops at blank screen probably means the MBR is missing.
• Fdisk is correct C/H/S , partition type, (Fat 16, type 04), set active.
• Start sector must be 63 for DOS (default 2048 for everything else, but can also be 63)
• You formatted it correctly with mkfs.msdos specifying the correct filesystem you partitioned it with (FAT16, option -F 16)
• Boot sector has been put on after formatting, should match partition type (FAT16, option -6)
• You have correctly fixed it, any setting wrong here will most likely stop it working, this is the most likely place that is wrong. You will probably get 'Invalid system disk' (error coming from the boot sector code)
• Be careful with what you are doing when and how its mounted. If the offset is wrong when its mounted you wont be looking at the start of the partition.