path: root/etc/etc.i386/INSTALL.linux

$OpenBSD: INSTALL.linux,v 1.15 2010/02/22 10:44:07 giovanni Exp $

   Linux + OpenBSD: it's possible

   by Marc Espie --  espie@OpenBSD.org
   recent information by Tim Kornau -- opti@openbsd.de

It is perfectly possible to have Linux and OpenBSD on the same disk.
Both can read and write other partitions.

You can even install OpenBSD from an ext2fs partition (choose install from
disk... ext2fs does not appear in the choices, but `default' it is).

If you are starting from scratch, it is better to install Linux first.
Since you are going to use several OSes, you need a way to multi-boot.
If you keep Windows NT (or XP) on the disk, its multi-booter can deal
with OpenBSD (see the FAQ). Otherwise Linux's lilo fits the bill fine.
Recent versions of GRUB can also multiboot OpenBSD.

IMPORTANT: don't forget about lilo. If you use lilo, you can't uninstall 
linux from this disk without *first* restoring the MBR to an 
un-liloed state and making *dead* sure OpenBSD boots as a default.

If you want to grab space from an older Windows/DOS partition, use fips.
Fips20 knows all about FAT32, so windows 95 is no longer a problem.
Or use the commercial offering Partition Magic.

Other sources of information, especially concerning other BSD systems,
must be taken with a healthy dose of skepticism. OpenBSD definitely
differs:
- disklabels can hold up to 16 partitions,
- type is A6, not A5,
- the `special' partitions in the disklabel are only a (root), b (swap),
and c (whole disk).

Planning & Good advice
----------------------
If you are starting on a new machine, be prepared to throw your
installation away. It is generally the case that you will install the
machine, play with it for a week/a month, and find out that you don't like
the setup, and then start over.

Write down any interesting information you find out during your first
installation.  Don't do too many things to your box during the first month,
as you will lose these while reinstalling, unless you can do backups
conveniently.

Do you really need to have a dual-boot machines ? Most people don't need
both Linux and OpenBSD.  Once you're satisfied with OpenBSD, you may find
out you just want to erase Linux...

Try to find out what your precise needs are, locate partitions whose size
may change next to each other, as far as possible... Put partitions whose
contents are unimportant (or whose backups are always up-to-date) next to
the frontier between OpenBSD and Linux. For instance, it's usually a good
idea to locate the swap area such that you can grow or shrink it. Keep in
mind that exceptional usage (very large, temporary swaps) can use a
temporary file instead of a partition, under both OpenBSD (svnd) and Linux.

First principles
----------------
OpenBSD doesn't only use the MBR partitions (the ones mapped in the Master
Boot Record) for booting. Afterwards, it trusts some bsd specific
information called the disklabel, which is another completely distinct
description of your hard disk. It does not even have to be consistent with
the usual DOS partitions information.
[OpenBSD requires a primary MBR partition for booting, anything else is
officially unsupported.]

Throughout this document, we will distinguish between MBR partitions and
disklabel partitions whenever this is necessary.

The safest way to deal with things is to allocate one primary MBR partition for
OpenBSD, type A6, that you will partition further with disklabel.

If you can, it is even safer to devote a full disk to OpenBSD: this limits
the number of mistakes you can do. Admittedly there are some cases where
this isn't a option (my machine is a laptop... I have to cope with the
hard disk I have), or where this can be slower (SCSI disk setups will
yield better throughput with swap interleaved among the disks).

Prime rule: *ALWAYS* use the disk partitioning tools that go with the OS.
They know more about its internal workings than you do. So use linux fdisk
for linux partitions, don't let it touch the OpenBSD disklabel, and
reciprocally.

Mapping your disk
-----------------
Starting from Linux, get a grasp of your partitions. Use df to check which
is what, then fdisk to get the actual setup of the disk.

Write down the setup onto a bit of paper, in case you make a mistake further
down.  It can come in very handy.

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders
Units = cylinders of 8064 * 512 bytes

   Device Boot    Start      End   Blocks   Id  System
/dev/hda1             1      211   850720+   6  DOS 16-bit >=32M
/dev/hda2           212      273   249984   83  Linux native
/dev/hda3           274      992  2899008   a6  OpenBSD

The + at the end of the DOS line is because linux fdisk is brain-damaged
and wants to write output in 1024-sized chunks, so this stands for
`850720 blocks and a half'

As you can see, my linux setup is very small. I have enough to check how
things such as gcc work on linux, but my machine is definitely an
OpenBSD developer's box.

Get the display to sectors with u, and jot down the corresponding
information as well:

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders
Units = sectors of 1 * 512 bytes

   Device Boot    Start      End   Blocks   Id  System
/dev/hda1            63  1701503   850720+   6  DOS 16-bit >=32M
/dev/hda2       1701504  2201471   249984   83  Linux native
/dev/hda3       2201472  7999487  2899008   a6  OpenBSD
--
Okay, finally switch to expert mode, and write the corresponding data.

Disk /dev/hda: 128 heads, 63 sectors, 993 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl   Start    Size ID
 1 00   1   1    0 127  63  210      63 1701441 06
 2 00   0   1  211 127  63  272 1701504  499968 83
 3 00   0   1  273 127  63  991 2201472 5798016 a6
 4 00   0   0    0   0   0    0       0       0 00


Note that this is STILL the same data. The good point about this last
display is that it is almost what you're going to see in OpenBSD fdisk,
usually ! (I think there may be some very weird cases where this won't
match)

There are some differences though, mostly because Linux fdisk has made
some rather confusing choices:
- in simple mode it starts numbering cylinders at 1... whereas
everything else starts from 0.
- in simple mode it shows blocks of 1024 bytes, which makes for half-blocks
(marked with a +) and sizes halved from the real block size.
- in expert mode it shows extended partitions offset from the start
of the extended partition.
- the hd/sec/cyl is a confusing order, as the sector number is computed
from cyl/hd/sec, in that order.
- it never shows and doesn't care about the real disk geometry.

You will notice that I don't have a linux swap partition visible. My
linux setup currently uses the OpenBSD swap area.

Before starting to install OpenBSD, now would be a good time to check the
INSTALL.pt document...  Especially note the alignment restriction of
partitions (first sector of a partition must be at head 0, sector 1 of a
cylinder). This is enforced by Linux' fdisk.

The other point to note is that extended partitions are actually linked
lists. This will show up in OpenBSD's fdisk.

Your clock and OpenBSD
----------------------
OpenBSD expects your hardware clock to be in universal time, and uses
time zones to give you local time. With Linux, this depends...
most distributions use a small program called hwclock to set up the
system time from the hardware clock when booting... there is a --utc
option if your hardware clock is in universal time, but this is not
always what happens by default.

For instance, on a redhat system, up to 5.2, this happens in
/etc/rc.d/rc.sysinit which loads an /etc/sysconfig/clock that defines a
variable called UTC, and then proceeds calling hwclock.
- ensure UTC is set to true,
- adjust your hardware clock from the system time if necessary, e.g.,
hwclock --systohc --utc.

Normally, this is one of the choices that the Linux installation program
lets you do: set your hardware clock to GMT.

The OpenBSD installation
------------------------
If you've got the space, you can install from your ext2fs partitions. This
is what I did, a long time in the past, as I had a slip connection to the
rest of the world, and the OpenBSD install floppy does not include slip.

REMEMBER TO BACKUP ALL IMPORTANT DATA ON YOUR DISK BEFORE DOING THE
INSTALLATION !!!

So you cp floppy*.fs /dev/fd0, then reboot from the floppy.

First, the booter loads, then there is a boot prompt. Five seconds later,
the kernel and the ram disk image are loaded, and the kernel is run.
After a while, through a few more prompts, you get to fdisk and you can
enter the new partition into the MBR.

This is what the fdisk dump looked like after my changes:
Disk: wd0       geometry: 992/128/63 [7999488 sectors]
Offset: 0       Signatures: 0xAA55,0x0
         Starting        Ending
 #: id  cyl  hd sec -  cyl  hd sec [     start -       size]
-------------------------------------------------------------------------
 0: 06    0   1   1 -  210 127  63 [        63 -    1701441] DOS > 32MB
 1: 83  211   0   1 -  272 127  63 [   1701504 -     499968] Linux files*
 2: A6  273   0   1 -  991 127  63 [   2201472 -    5798016] OpenBSD
 3: 00    0   0   0 -    0   0   0 [         0 -          0] unused

fdisk geometry tells me that I needn't worry about cylinder 1024.
This matches very  closely with what linux fdisk saw.

If you had extended partitions, it would be a trifle bit harder:
you just follow the extended partition links using select, jot down
whatever you need, add the OpenBSD partition into the MBR to look like
you want it to, and save everything.

The * at the end of partition #1 means that the system normally boots under
Linux. In reality, lilo takes control and disregards this completely.

After you leave fdisk, you get to the interesting part: the disklabel
itself. If all goes well, OpenBSD synthesizes a nice disklabel out of what
it can deduce from the disk, including the ext2fs partitions.

There are only a few subtleties to take care of:
- initially, you can ONLY edit the disklabel part that matches the
OpenBSD partition that was declared in the MBR (what you just entered in
fdisk, the `slice' from FreeBSD lingo). Most simple installation don't
need to edit more than that, but you can use b 0 * to unlock the whole
disk (this is a bad idea in most cases).
- your real disk geometry becomes more relevant. The Berkeley fast file system
can't use partial cylinder groups, hence BSD partitions should start
on cylinder boundaries, as any remaining sectors  will be lost anyway.
(Actually, what's important is the disk geometry that disklabel gives you.
Trust it on that). In my case, sectors/cylinder=1008 and bytes/sector=512,
so the granularity of disklabel partitions is 504 Kb.
- units for size and offset can be given as sectors (default) or cylinders.

After editing, this is what my disklabel looks like:
# editing

# using MBR partition 2: type A6 off 2201472 (0x219780) size 5798016 (0x587880)
# /dev/rwd0c:
type: ESDI
disk:
label: TOSHIBA MK4006M
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 7944
total sectors: 8007552
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0

16 partitions:
#        size   offset    fstype   [fsize bsize   cpg]
  a:   104832  2201472    4.2BSD     1024  8192    16   # (Cyl. 2184 - 2287)
  b:   274176  2306304      swap                        # (Cyl. 2288 - 2559)
  c:  8007552        0    unused        0     0         # (Cyl.    0 - 7943)
  d:   613872  2580480    4.2BSD     1024  8192    16   # (Cyl. 2560 - 3168)
  e:   920304  4846464    4.2BSD     1024  8192    16   # (Cyl. 4808 - 5720)
  f:   628992  4217472    4.2BSD     1024  8192    16   # (Cyl. 4184 - 4807)
  g:   204624  5766768    4.2BSD     1024  8192    16   # (Cyl. 5721 - 5923)
  h:  1073520  5971392    4.2BSD     1024  8192    16   # (Cyl. 5924 - 6988)
  i:   962640  7044912    4.2BSD     1024  8192    16   # (Cyl. 6989 - 7943)
  j:  1701441       63     MSDOS                        # (Cyl.    0*- 1687)
  k:   499968  1701504    ext2fs                        # (Cyl. 1688 - 2183)
  l:  1023120  3194352    4.2BSD     1024  8192    16   # (Cyl. 3169 - 4183)

Things that check, more or less automatically:
- this disklabel is saved in MBR partition #2 (basic DOS partition 2),
as expected.
- all the BSD partitions proper are aligned on a cylinder boundary (ie no '*').
the root partition begins at the precise same offset the corresponding DOS
partition begins, and it extends for the same length.
- the ext2fs partitions have the exact same layout under the OpenBSD
disklabel.
- the swap partition is very large. It gets used as mfs for my tmp
directories.

And here is the corresponding /etc/fstab:
/dev/wd0a       /          ffs    rw    1       1
/dev/wd0d       /usr       ffs    ro    1       2
/dev/wd0l       /usr/local ffs    ro    1       2
/dev/wd0e       /home      ffs    rw    1       2
/dev/wd0g       /var       ffs    rw    1       2
/dev/wd0h       /big       ffs    ro    1       2
/dev/wd0f       /usr/obj   ffs    rw    1       2
/dev/wd0i       /vbig      ffs    rw    1       2
/dev/wd0j       /dos       msdos  rw    1       2
/dev/wd0k       /linux     ext2fs rw
/dev/wd0b       /tmp       mfs    rw

One point that is somewhat laborious is that the disklabel -E mode
(which you are currently using) tends to move partitions around to ensure
that ALL defined partitions are contiguous. For that reason, it is better
if you don't have to use b 0 *, otherwise partitions will be moved around to
remove holes, without regard for the rigid MBR partitioning.
ext2fs and DOS partitions  should be recognized and positioned
automatically if all goes well.

Once the disklabel is written to disk, the installation proceeds as usual.

ext2fs partitions are perfectly usable from OpenBSD.

Booting with GRUB
-----------------
Here is a sample configuration for a linux 2.4, linux 2.6, OpenBSD 3.6,
WindowsXP
                                                                                
timeout 30                                                                      
default 0                                                                       
fallback 1                                                                      
                                                                                
title OpenBSD                                                                   
rootnoverify (hd0,3)                                                            
makeactive                                                                      
chainloader +1                                                                  
                                                                                
title WinOS                                                                     
rootnoverify (hd0,0)                                                            
chainloader +1                                                                  
                                                                                
title           Debian GNU/Linux, kernel                                        
root            (hd0,2)                                                         
kernel          /boot/vmlinuz root=/dev/ide/host0/bus0/target0/lun0/part3 ro    
savedefault                                                                     
boot                                                                            

Booting with GRUB2
-----------------
Here is a sample configuration for OpenBSD and Windows 7

menuentry "OpenBSD" {
	set root=(hd0,3,a)
	chainloader +1
}

menuentry "Windows 7" {
	insmod ntfs
	set root=(hd0,1)
	chainloader +1
}

Booting with lilo
-----------------
First time I booted my system back, I did not get into OpenBSD as expected...
I plain forgot I had installed lilo in the master boot block, and lilo
does not heed the active partition flag. The fix was rather simple: from
the Linux system, I just had to edit lilo.conf to add the OpenBSD entry:

boot=/dev/hda
map=/boot/map
install=/boot/boot.b
prompt
timeout=500
other=/dev/hda3
        label=bsd
        table=/dev/hda
image=/boot/vmlinuz-2.2.3
        label=linux
        vga=4
        root=/dev/hda2
        read-only
other=/dev/hda1
        label=dos
        table=/dev/hda

Rerun lilo (DON'T FORGET THAT STEP), and voila, OpenBSD is able to boot!

Linux and OpenBSD partitions
----------------------------
You will probably need to reconfigure and rebuild your linux kernel 
to recognize BSD disklabels... Here is how it shows up
on my box:

Partition check:
 hda: hda1 hda2 hda3! < hda5 hda6 hda7 hda8 hda9 hda10 hda11 hda12 hda13 hda14
hda15 >

- the disklabel is detected early, but handled later.
- disklabel handling should remove duplicates: all partitions that are present
as both MBR and disklabel partition should get removed silently. (this does
not seem to work as advertised presently).
- the remaining partitions are checked for consistency.

and here is my linux fstab:

/dev/hda2               /                       ext2    defaults        1 1
/dev/hda6               swap                    swap    defaults        0 0
/dev/hda5               /bsd                    ufs     ufstype=44bsd   1 2
/dev/hda7               /bsd/usr                ufs     ufstype=44bsd   1 2
/dev/hda8               /home                   ufs     ufstype=44bsd   1 2
/dev/hda9               /bsd/usr/obj            ufs     ufstype=44bsd   1 2
/dev/hda1               /dos                    vfat    defaults        1 2
/dev/fd0                /mnt/floppy             ext2    noauto          0 0
/dev/cdrom              /mnt/cdrom              iso9660 noauto,ro

linux kernels also include a working UFS, though you may run into problems when
writing to ufs partitions.   Note the ufstype=44bsd. If you forget that
in your mounts, it will fail. Depending upon your installation, you may
get a failure message, or you will have to dig through /var/log/ to find
it.

Running Linux binaries under OpenBSD
------------------------------------
You just have to recompile your BSD kernel with COMPAT_LINUX, and set up
/emul/linux as explained in compat_linux(1). If you run GENERIC, you don't
have to recompile anything, as this is the default setup.

It's a good idea to mount your Linux file system under another point, then
make symbolic links so that you can control what gets used precisely.

As you have a complete linux system, don't bother with the ports
emul/linux_lib entry: it's only a set of Linux libraries for people who
don't have a Linux system running.

A small detail that may cause problems: uname still says `OpenBSD', even
under Linux compatibility.  The reason behind that is that we don't want
netscape to tell it was run from a Linux box, when it is used under
OpenBSD.

Some programs, for instance maple, do depend on uname answering `Linux'.
For maple, this is straightforward: you just have to fudge
/usr/local/maple/bin/maple.system.type to check OpenBSD in the same
class with Linux.

Similar shell scripts are easy to fix.  Binary programs that don't run
suid can be coerced by using LD_PRELOAD.

As a rule, this should be achieved on a program-by-program basis.
The more networking programs that do tell they're running under OpenBSD,
the merrier !

A word of caution: brain-damaged linux installations
----------------------------------------------------
I just reinstalled the linux side of my laptop using redhat 5.2.  This
CD does insist on having TWO linux partitions: one root partition, and one
swap partition (even though I have 32 Mb of memory, largely enough for
the installation). Since it uses a 2.0.36 kernel, it does NOT handle BSD
disklabels, so I couldn't tell it to use my swap area (I have this bad
feeling that distributed 2.2 boot kernels won't include BSD disklabel
handling anyway). Instead, I had to back my last OpenBSD partition up,
fiddle with my fdisk setup to feed the last cylinder as a swap partition
to their brain dead install CD. Then fetch the latest kernel source to the
linux side, and recompile to get a fully working linux setup. Finally,
reset the fdisk/disklabel to its normal state, and get the backed-up
partition to its normal location.

Another word of caution: getting enough rope to hang yourself
-------------------------------------------------------------
One previous version of this document got into much nastier details, and
gave installation instructions that were thoroughly dangerous.
- various tools may yield distinct views of your disk. These will match,
most often, but not always. Don't get confused if various fdisk, disklabel
utilities don't yield the same information. Generally, sector sizes and
offsets should match.
- try to keep various OSes segregated to their areas. Don't depend on
OpenBSD information to be correct for your linux setup and vice-versa.
Some weird problem with the brain-damaged PC architecture may force us to
tweak things so that OpenBSD will work seamlessly everywhere. Compatibility
with weird tricks is not a priority.

This being said, you will have noticed by now that the OpenBSD disklabel is
just another description of your hard-disk. It is almost completely
independent from the MBR description (it just has to be on the right sector
for the boot process to find it). You can get into trouble if things don't
match. Let it live within its MBR partition, unless you're completely sure
you know what you are doing, and don't expect there will always be someone
to get you out of trouble. If your setup is really too weird, no-one can help.

As far as the boot process goes, I think lilo allows you to boot from ANY
partition recorded in the MBR, including extended partitions. 

Several bsd on the same disk MAY be possible, but will be harder to manage:
- it is better if disklabels match,
- linux will obey the first disklabel it finds, try to ensure this is
OpenBSD disklabel, it can describe more partitions than the others,
- other BSD may get confused with each other data. Normally, the A5/A6
split ensures that Net/Free won't get mixed up with OpenBSD,
- FreeBSD and NetBSD will probably get confused with each other,

Finally, how much disk space do you have anyway ?  Do you really need to
cram that many OSes on the same disk ? Put them on separate disks rather.
If you reach that stage, you'd better be ready to hack at the linux kernel
to recognize several disklabels, for instance, or generally know what
you're doing.