Blog

CentOS root on ZFS

Nov 7, 2016 by klump. Filed under Server. Tagged LinuxZFS.

I have bought some Infiniband cards to link the fileserver and the hypervisor together. I chose Infiniband over 10GbE because it is cheaper and offers me more bandwidth, but more on that in a later post. I have traditionally run FreeBSD for my fileserver because it has had ZFS for quite a while and I want to have a proven implementation of my filesystem on my fileserver.
Rabbit meets beetroot

Jan 31, 2016 by klump. Filed under fun. Tagged rabbit.

When peeling the beetroot for a soup, I got the urge to color our white/gray-ish rabbit with it. This is how it looks:
How to create a working CA

Dec 10, 2013 by klump. Filed under SSL. Tagged CA.

We recently had a lab in school, where we had to create a certificate authority using openssl, which includes a index file containing all certificates issued by the CA. More information about the process can be found here.
Centralized Authentication with OpenBSD and OpenLDAP

Nov 24, 2013 by klump. Filed under OpenBSD. Tagged LDAPAuthentication.

If you have a lot of servers and a lot of users, you do not want to manage them via editing passwd files. Using LDAP is one way out of the dilemma. Searching the web resulted in a lot of outdated information, but after a while I found something helpful on a mailing list, so thanks Gary ;)
Collection of Useful Oneliners

Sep 23, 2013 by klump. Filed under Linux. Tagged oneliners.

Here comes my personal collection of oneliners. I hope it will grow with time.

Stuff

Gentoo install speed through

The instructions in the “Gentoo handbook”:http://www.gentoo.org/doc/en/handbook/handbook are very good if you are doing the setup for the first time. Now I just open it to not forget anything and scroll over the most, so today I decided to make my own check list.

Set up partitions
Make filesystems

Unpack the stage3 archive

links ftp://ftp.sunet.se/pub/Linux/distributions/gentoo/releases/amd64/current-stage3/
sha512sum stage3-amd64-<release>.tar.bz2
tar xvjpf stage3-*.tar.bz2

Edit /etc/portage/make.conf

Chroot

cp -L /etc/resolv.conf /mnt/gentoo/etc/
mount -t proc none /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
chroot /mnt/gentoo /bin/bash

Sync portage
```
mkdir /usr/portage
emerge-webrsync
```
Set some system settings
Profile and useflags

Timezone

ln -s /usr/share/zoneinfo/Europe/Stockholm /etc/localtime
echo `Europe/Stockholm` > /etc/timezone

Locale settings

vim /etc/locale.gen
locale-gen
vim /etc/locale.conf

Hostname
```
echo `HOSTNAME` > /etc/hostname
```
KERNELTIME!

Don’t forget the bootloader

emerge -av syslinux
mkdir -p /boot/extlinux
extlinux -i /boot/extlinux
dd bs=440 count=1 conv=notrunc if=/usr/share/syslinux/gptmbr.bin of=/dev/sda
cp /usr/share/syslinux/{menu,vesamenu}.c32 /boot/extlinux
vim /boot/extlinux/extlinux.conf

set root password
install stuff you need
- dhcpcd
- cryptsetup
- …
reboot and hope you did not forget anything ;)

Manatee<!DOCTYPE html> Summer project: Manatee
- home
- blog
- technical details
- about me
Summer project: Manatee

Some days ago Max and I restarted working on a project we both started some time ago. After a short while of poking in the code, we both decided that it would be much faster and cleaner to just drop the old code and start from scratch. So we did. Right now it is looking good and are making good progress. Lets hope we can finish the project before it gets boring and starts to rot away in the project folder like so many other projects…

Manatee is a ruby on rails application which allows you to keep track of your expenses. The code can be found at GitHub.

The logo was drawn by my little brother, thanks Joni :D

Infomational stuff

Jul 31, 2013

by klump

Filed under projects

Page generated at Nov 8, 2020 11:55 UTC
Rails<!DOCTYPE html> Summer project: Manatee
- home
- blog
- technical details
- about me
Summer project: Manatee

Some days ago Max and I restarted working on a project we both started some time ago. After a short while of poking in the code, we both decided that it would be much faster and cleaner to just drop the old code and start from scratch. So we did. Right now it is looking good and are making good progress. Lets hope we can finish the project before it gets boring and starts to rot away in the project folder like so many other projects…

Manatee is a ruby on rails application which allows you to keep track of your expenses. The code can be found at GitHub.

The logo was drawn by my little brother, thanks Joni :D

Infomational stuff

Jul 31, 2013

by klump

Filed under projects

Page generated at Nov 8, 2020 11:55 UTC
Ruby<!DOCTYPE html> Summer project: Manatee
- home
- blog
- technical details
- about me
Summer project: Manatee

Some days ago Max and I restarted working on a project we both started some time ago. After a short while of poking in the code, we both decided that it would be much faster and cleaner to just drop the old code and start from scratch. So we did. Right now it is looking good and are making good progress. Lets hope we can finish the project before it gets boring and starts to rot away in the project folder like so many other projects…

Manatee is a ruby on rails application which allows you to keep track of your expenses. The code can be found at GitHub.

The logo was drawn by my little brother, thanks Joni :D

Infomational stuff

Jul 31, 2013

by klump

Filed under projects

Page generated at Nov 8, 2020 11:55 UTC
KVM<!DOCTYPE html> How to resize kvm raw image with GPT partitions
- home
- blog
- technical details
- about me
How to resize kvm raw image with GPT partitions

I am running several VMs with kvm. After a little while I noticed that I was a little to stingy with the amount of space assigned to each VM.

These are the steps I used to reassign more space to the VMs:
1. Stop the VM
2. Make a backup!
3. Get the difference between the size at the moment and the wished size (in my case 4 GiB)
4. Create a image with this size consisting of zeros
  # copy 4096 times 1 MB of zeros into raw.img dd if=/dev/zero of=raw.img bs=1M count=4096
5. Join the image and the zeros into a new file:
  cat vm.img raw.img > new_vm.img
6. Get the starting sector of the partition and delete the partition
  gdisk new_vm.img p (remember the starting sector!) d (use the number of the partition) w (write changes to disk, let gdisk correct the GPT header which is not at the end)
7. Recreate the partition
  gdisk new_vm.img n (use the number and the first sector from above) [you might want to set the attributes (like bootable) as well] w
8. Start the VM and then on (in) the VM and run fsck on the partition
  fsck /dev/mypartition # or for root fs: touch /forcefsck && reboot
9. Grow the partition inside the VM
  resize2fs /dev/mypartition
Update 2017-08-15

It is possible to use sparse files for the update as well:
# create sparse file truncate -s 16G new_vm.img # copy old image dd if=vm.img of=new_vm.img conv=notrunc
Then continue with deleting and recreating the partition you want to resize.
Infomational stuff

Jul 31, 2013

by klump

Filed under Linux
Page generated at Nov 8, 2020 11:55 UTC
GPT<!DOCTYPE html> How to resize kvm raw image with GPT partitions
- home
- blog
- technical details
- about me
How to resize kvm raw image with GPT partitions

I am running several VMs with kvm. After a little while I noticed that I was a little to stingy with the amount of space assigned to each VM.

These are the steps I used to reassign more space to the VMs:
1. Stop the VM
2. Make a backup!
3. Get the difference between the size at the moment and the wished size (in my case 4 GiB)
4. Create a image with this size consisting of zeros
  # copy 4096 times 1 MB of zeros into raw.img dd if=/dev/zero of=raw.img bs=1M count=4096
5. Join the image and the zeros into a new file:
  cat vm.img raw.img > new_vm.img
6. Get the starting sector of the partition and delete the partition
  gdisk new_vm.img p (remember the starting sector!) d (use the number of the partition) w (write changes to disk, let gdisk correct the GPT header which is not at the end)
7. Recreate the partition
  gdisk new_vm.img n (use the number and the first sector from above) [you might want to set the attributes (like bootable) as well] w
8. Start the VM and then on (in) the VM and run fsck on the partition
  fsck /dev/mypartition # or for root fs: touch /forcefsck && reboot
9. Grow the partition inside the VM
  resize2fs /dev/mypartition
Update 2017-08-15

It is possible to use sparse files for the update as well:
# create sparse file truncate -s 16G new_vm.img # copy old image dd if=vm.img of=new_vm.img conv=notrunc
Then continue with deleting and recreating the partition you want to resize.
Infomational stuff

Jul 31, 2013

by klump

Filed under Linux
Page generated at Nov 8, 2020 11:55 UTC
lmb<!DOCTYPE html> Summer project: LMB
- home
- blog
- technical details
- about me
Summer project: LMB

And for another time I picked up work on a almost forgotten project: The lawn mower bot (lmb). Together with my girlfriends dad I started this project quite a while ago. He wanted a automatic lawn mower, but we both considered the prices unreasonable high. So we decided to give it a try and started building our own mower. The first part was very easy: take some DC motors from China, mount them on a piece of wood. For the mowing part we used replacement part from a real automated mower. An arduino should become the heart of the lmb.

So far so good, but how do you control the 24V DC motors with the arduino? We needed an H-bridge. As buying finished stuff is boring, we worked out a h bridge on our own. The soldering took quite a while (yes, I am lazy). Yesterday we gave it a try, and I think we fried it pretty well. Somehow I even managed to fry my arduino, as it does no longer talk to any computer. Seems like it did not take 12V on the Vin pin very well…

After this setback I am tired of soldering h bridges (without a method to create real PCBs, it is a real pain with protoboard and all the cables). I am really thinking about buying some H-bridges from China, as they are not really that expensive. It is not a cool as building your own, but at least you can be sure it works.

To be continued…

Infomational stuff

Aug 12, 2013

by klump

Filed under projects

Page generated at Nov 8, 2020 11:55 UTC
oneliners<!DOCTYPE html> Collection of Useful Oneliners
- home
- blog
- technical details
- about me
Collection of Useful Oneliners

Here comes my personal collection of oneliners. I hope it will grow with time.
- Download all files recursively in a directory listing with wget:
  wget -m --no-parent http://example.com/path/to/files/
- Find all files and open them in Vim
  find /path/to/directory -name '*.rb' -exec vim '{}' +
- Create sparse files
  dd if=/dev/zero of=sparse.file bs=1 count=0 seek=4G # or truncate -s 4G sparse.file
Infomational stuff

Sep 23, 2013

by klump

Filed under Linux
Page generated at Nov 8, 2020 11:55 UTC

LDAP<!DOCTYPE html> Centralized Authentication with OpenBSD and OpenLDAP

Centralized Authentication with OpenBSD and OpenLDAP

If you have a lot of servers and a lot of users, you do not want to manage them via editing passwd files. Using LDAP is one way out of the dilemma. Searching the web resulted in a lot of outdated information, but after a while I found something helpful on a mailing list, so thanks Gary ;)

Here are my steps to success:

Install login_ldap which is used to provide the authentication via LDAP, e.g. logins and stuff.
```
pkg_add -iv login_ldap
```

Append the following to the /etc/login.conf.

ldap:\
  :auth=-ldap:\
  :x-ldap-server=ldap.devrandom.se:\
  :x-ldap-uscope=sub:\
  :x-ldap-basedn=ou=users,dc=devrandom,dc=se\
  :x-ldap-filter=(&(objectclass=posixAccount)(uid=%u)):\
  :tc=default:

Ypldap is used to fetch additional information from LDAP, like the home directories of the users or group information. My /etc/ypldap.conf:

# : ypldap.conf,v 1.4 2012/04/30 12:16:43 ajacoutot Exp 

domain    "devrandom.se"
interval  60
provide map "passwd.byname"
provide map "passwd.byuid"
provide map "group.byname"
provide map "group.bygid"

directory "ldap.server.devrandom.se" {
  basedn "dc=devrandom,dc=se"

  # passwd maps configuration (RFC 2307 posixAccount object class)
  passwd filter "(objectClass=posixAccount)"

  attribute name maps to "uid"
  fixed attribute passwd "*"
  attribute uid maps to "uidNumber"
  attribute gid maps to "gidNumber"
  attribute gecos maps to "cn"
  attribute home maps to "homeDirectory"
  attribute shell maps to "loginShell"
  fixed attribute change "0"
  fixed attribute expire "0"
  fixed attribute class "ldap"

  # group maps configuration (RFC 2307 posixGroup object class)
  group filter "(objectClass=posixGroup)"

  attribute groupname maps to "cn"
  fixed attribute grouppasswd "*"
  attribute groupgid maps to "gidNumber"
  # memberUid returns multiple group members
  list groupmembers maps to "memberUid"
}

It is very important to set the fixed attribute class "ldap" line, as indicates that this user should authenticate using login_ldap.

Now we need to configure the domain.

domainname devrandom.se
echo devrandom.se > /etc/defaultdomain

Next step is to tell the system that additional users and groups are supplied by YP. Appending a line to the /etc/passwd and /etc/group should do the job. It is advisable to use vipw to edit those files. Append /etc/passwd:
```
+:::::::::/bin/ksh
```
Append to /etc/group:
```
+:::
```
Enable the services in the /etc/rc.conf.local and reboot.
```
portmap_flags=""
ypldap_flags=""
ypbind_flags=""
```

Authentication<!DOCTYPE html> Centralized Authentication with OpenBSD and OpenLDAP

Centralized Authentication with OpenBSD and OpenLDAP

Here are my steps to success:

Install login_ldap which is used to provide the authentication via LDAP, e.g. logins and stuff.
```
pkg_add -iv login_ldap
```

Append the following to the /etc/login.conf.

ldap:\
  :auth=-ldap:\
  :x-ldap-server=ldap.devrandom.se:\
  :x-ldap-uscope=sub:\
  :x-ldap-basedn=ou=users,dc=devrandom,dc=se\
  :x-ldap-filter=(&(objectclass=posixAccount)(uid=%u)):\
  :tc=default:

Ypldap is used to fetch additional information from LDAP, like the home directories of the users or group information. My /etc/ypldap.conf:

# : ypldap.conf,v 1.4 2012/04/30 12:16:43 ajacoutot Exp 

domain    "devrandom.se"
interval  60
provide map "passwd.byname"
provide map "passwd.byuid"
provide map "group.byname"
provide map "group.bygid"

directory "ldap.server.devrandom.se" {
  basedn "dc=devrandom,dc=se"

  # passwd maps configuration (RFC 2307 posixAccount object class)
  passwd filter "(objectClass=posixAccount)"

  attribute name maps to "uid"
  fixed attribute passwd "*"
  attribute uid maps to "uidNumber"
  attribute gid maps to "gidNumber"
  attribute gecos maps to "cn"
  attribute home maps to "homeDirectory"
  attribute shell maps to "loginShell"
  fixed attribute change "0"
  fixed attribute expire "0"
  fixed attribute class "ldap"

  # group maps configuration (RFC 2307 posixGroup object class)
  group filter "(objectClass=posixGroup)"

  attribute groupname maps to "cn"
  fixed attribute grouppasswd "*"
  attribute groupgid maps to "gidNumber"
  # memberUid returns multiple group members
  list groupmembers maps to "memberUid"
}

It is very important to set the fixed attribute class "ldap" line, as indicates that this user should authenticate using login_ldap.

Now we need to configure the domain.

domainname devrandom.se
echo devrandom.se > /etc/defaultdomain

Next step is to tell the system that additional users and groups are supplied by YP. Appending a line to the /etc/passwd and /etc/group should do the job. It is advisable to use vipw to edit those files. Append /etc/passwd:
```
+:::::::::/bin/ksh
```
Append to /etc/group:
```
+:::
```
Enable the services in the /etc/rc.conf.local and reboot.
```
portmap_flags=""
ypldap_flags=""
ypbind_flags=""
```

CA<!DOCTYPE html> How to create a working CA
- home
- blog
- technical details
- about me
How to create a working CA

We recently had a lab in school, where we had to create a certificate authority using openssl, which includes a index file containing all certificates issued by the CA. More information about the process can be found here.
1. First create the directory:
  mkdir /CA cd /CA
2. Copy and adopt the openssl.cnf existing on your system.
  cp /etc/ssl/openssl.cnf /CA/openssl.cnf
3. Change at least the lines as indicated below:
  [CA_default] dir = /CA certificate = /YOUR-CA.crt # change the filename to something appropriate private_key = /private/YOUR-CA.key # change the filename to something appropriate [policy_match] countryName = match # change to supplied if you want to sign certs for other countries stateOrProviceName = match # change to supplied if you want to sign certs for other regions organizationName = match # change to supplied if you want to sign certs for other organizations
4. Now you need to create some folders and files…
  mkdir private newcerts certs touch index.txt echo 01 > serial
5. With the basics in place, you can now create a private key for your CA. Change the filename to the same value you used in the openssl.cnf. If you want you can restrict the permissions on this key, as it should remain secret.
  openssl genrsa -aes256 -out private/YOUR-CA.key 8192
6. The private key is now used to create the certificate for your CA. This certificate can be distributed, and should be imported into the browsers…
  openssl req -new -x509 -days 3650 -sha512 \ -config /CA/openssl.cnf \ -key private/YOUR-CA.key \ -out YOUR-CA.crt
7. Don’t forget to change the filenames to the right values. This root CA remains valid for 3650 days (10 years). Now you should have a working CA.
Time to sign some requests!
1. If you need a certificate, you will first have to create a certificate signing request. The following command will create a private key, so run it on the machine that will use the certificate. If you want to use the certificate for a website, it is important to use the full domain name as CN.
  openssl req -newkey rsa:4096 -sha512 \ -config /etc/ssl/openssl.cnf \ -keyout /etc/ssl/private/MY-SITE.key \ -out MY-SITE.req
2. This request (MY-SITE.req) should transfered to the CA machine, e.g. via scp. The last step is to sign the request.
  openssl ca -config /CA/openssl.cnf \ -infiles MY-SITE.req \ -out MY-SITE.crt
3. The resulting MY-SITE.crt file is the certificate which can now be used to secure communication with your machine.
Infomational stuff

Dec 10, 2013

by klump

Filed under SSL
Page generated at Nov 8, 2020 11:55 UTC
rabbit<!DOCTYPE html> Rabbit meets beetroot
- home
- blog
- technical details
- about me
Rabbit meets beetroot

When peeling the beetroot for a soup, I got the urge to color our white/gray-ish rabbit with it. This is how it looks:

Infomational stuff

Jan 31, 2016

by klump

Filed under fun

Page generated at Nov 8, 2020 11:55 UTC
Linux<!DOCTYPE html> CentOS root on ZFS
- home
- blog
- technical details
- about me
CentOS root on ZFS

I have bought some Infiniband cards to link the fileserver and the hypervisor together. I chose Infiniband over 10GbE because it is cheaper and offers me more bandwidth, but more on that in a later post. I have traditionally run FreeBSD for my fileserver because it has had ZFS for quite a while and I want to have a proven implementation of my filesystem on my fileserver.

However, when running iSCSI over Infiniband, there are two ways: IP over Infiniband or an iSCSI extension called iSER. The first method imposes quite some overhead, somebody reported 20× the bandwidth and double in IOPS when running with iSER instead of IPoIB. The problem: targets with iSER are not (yet) supported on FreeBSD 11, however acting as an initiator works. So I needed to switch to ZFS on Linux as LIO support iSER targets.

As I use CentOS on all the other servers, I wanted to run my fileserver on CentOS as well. I really liked the Root on ZFS install that is integrated with FreeBSD, I wanted to replicated it with CentOS 7. The original tutorial did not really work out for me, I liked the Ubuntu one a bit better - so I cherrypicked from both and here is the result.

My root zpool is called zroot and is a mirror over two SSDs. Start by CentOS to another medium and installing zfs and zfs-dracut as described in the guide. I opted to use mbr and to ditch the UEFI stuff. As I wanted to create a swap partition on each SSD, I created a 8GB partition in the end and used the rest for ZFS.
1. My partition table:
  sfdisk -d /dev/sda
  
  # partition table of /dev/sda unit: sectors /dev/sda1 : start= 2048, size=217662383, Id=bf, bootable /dev/sda2 : start= 0, size= 0, Id= 0 /dev/sda3 : start= 0, size= 0, Id= 0 /dev/sda4 : start=217664431, size= 16777217, Id=82
2. First create the zpool:
  zpool create -d \ -o feature@async_destroy=enabled \ -o feature@empty_bpobj=enabled \ -o feature@lz4_compress=enabled \ -o ashift=12 -O atime=off -O canmount=off \ -O mountpoint=/ -R /media -O compression=lz4 \ zroot mirror /dev/sda1 /dev/sdb1
  
  Note: It is important to create the pool with -d, otherwise grub2-probe cannot detect the filesystem correctly as ZFS.
3. Check that pool was created:
  zpool status zroot
  
  pool: zroot state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: none requested config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sda1 ONLINE 0 0 0 sdb1 ONLINE 0 0 0 errors: No known data errors
4. Lazy variant to get good permanent device names:
  zpool export zroot zpool import -o altroot=/media -d /dev/disk/by-path/ zroot
5. Lets create and mount some ZFS filesystems for different parts of the system
  zfs create -o canmount=off -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ zroot/ROOT/centos zfs create -o setuid=off zroot/home zfs create -o mountpoint=/root zroot/home/root zfs create -o canmount=off -o setuid=off -o exec=off zroot/var zfs create zroot/var/cache zfs create zroot/var/log zfs create zroot/var/spool zfs create -o exec=on zroot/var/tmp zfs create zroot/var/mail zfs mount -a chmod 1777 /media/var/tmp
6. Everything that is not stored in one of the filesystems, is in zroot/ROOT/centos. Making going back and forth between snapshots easier as all the logfiles and stuff will be kept. If someone want to have another root (try something), one can simply switch the mountpoints and be good to go…
7. Now it is time to copy the system from the original install to the zroot.
  mount --bind / /mnt/ rsync -avPX /mnt/. /media/. # Next two steps only if you have /boot and /home on a different partition rsync -avPX /boot/. /media/boot/ rsync -avPX /home/. /media/home/. umount /mnt/tmp/
8. Final system adoption in the new system, but first prepare the chroot environment.
  mount --rbind /dev /media/dev/ mount --rbind /sys /media/sys/ mount --rbind /proc /media/proc/ chroot /media
9. Edit fstab, remove all entries and add swap partitions.
  vi /etc/fstab
10. Update grub config /etc/default/grub and add/ update the following lines:
  GRUB_CMDLINE_LINUX="nofb splash=quiet crashkernel=auto rhgb quiet boot=zfs rpool=zroot bootfs=zroot/ROOT/centos" GRUB_PRELOAD_MODULES="part_msdos zfs"
11. Make an ugly hack because apparently the grub utilities cannot handle persistent device names very well…
  pushd /dev ln -s /dev/disk/by-path/* . # Use right directory for you zpool device names popd
12. Rebuild and install grub on the root SSDs:
  grub2-mkconfig -o /boot/grub2/grub.cfg grub2-install /dev/sda grub2-install /dev/sdb
13. Enable the zfs-mount service otherwise, CentOS will only mount the zroot/ROOT/centos filesystem.
  systemctl enable zfs-mount.service
14. Update dracut config in /etc/dracut.conf
  add_dracutmodules+="zfs"
15. Rebuild initramfs
  dracut -f -v /boot/initramfs-$(uname -r).img $(uname -r)
16. Do anything else you want, then reboot. And hope for the best. If things should go south, boot from the original medium and troubleshoot or start over by destroying the zpool.
Infomational stuff

Nov 7, 2016

by klump

Filed under Server
Page generated at Nov 8, 2020 11:55 UTC
ZFS<!DOCTYPE html> CentOS root on ZFS
- home
- blog
- technical details
- about me
CentOS root on ZFS

I have bought some Infiniband cards to link the fileserver and the hypervisor together. I chose Infiniband over 10GbE because it is cheaper and offers me more bandwidth, but more on that in a later post. I have traditionally run FreeBSD for my fileserver because it has had ZFS for quite a while and I want to have a proven implementation of my filesystem on my fileserver.

However, when running iSCSI over Infiniband, there are two ways: IP over Infiniband or an iSCSI extension called iSER. The first method imposes quite some overhead, somebody reported 20× the bandwidth and double in IOPS when running with iSER instead of IPoIB. The problem: targets with iSER are not (yet) supported on FreeBSD 11, however acting as an initiator works. So I needed to switch to ZFS on Linux as LIO support iSER targets.

As I use CentOS on all the other servers, I wanted to run my fileserver on CentOS as well. I really liked the Root on ZFS install that is integrated with FreeBSD, I wanted to replicated it with CentOS 7. The original tutorial did not really work out for me, I liked the Ubuntu one a bit better - so I cherrypicked from both and here is the result.

My root zpool is called zroot and is a mirror over two SSDs. Start by CentOS to another medium and installing zfs and zfs-dracut as described in the guide. I opted to use mbr and to ditch the UEFI stuff. As I wanted to create a swap partition on each SSD, I created a 8GB partition in the end and used the rest for ZFS.
1. My partition table:
  sfdisk -d /dev/sda
  
  # partition table of /dev/sda unit: sectors /dev/sda1 : start= 2048, size=217662383, Id=bf, bootable /dev/sda2 : start= 0, size= 0, Id= 0 /dev/sda3 : start= 0, size= 0, Id= 0 /dev/sda4 : start=217664431, size= 16777217, Id=82
2. First create the zpool:
  zpool create -d \ -o feature@async_destroy=enabled \ -o feature@empty_bpobj=enabled \ -o feature@lz4_compress=enabled \ -o ashift=12 -O atime=off -O canmount=off \ -O mountpoint=/ -R /media -O compression=lz4 \ zroot mirror /dev/sda1 /dev/sdb1
  
  Note: It is important to create the pool with -d, otherwise grub2-probe cannot detect the filesystem correctly as ZFS.
3. Check that pool was created:
  zpool status zroot
  
  pool: zroot state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: none requested config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sda1 ONLINE 0 0 0 sdb1 ONLINE 0 0 0 errors: No known data errors
4. Lazy variant to get good permanent device names:
  zpool export zroot zpool import -o altroot=/media -d /dev/disk/by-path/ zroot
5. Lets create and mount some ZFS filesystems for different parts of the system
  zfs create -o canmount=off -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ zroot/ROOT/centos zfs create -o setuid=off zroot/home zfs create -o mountpoint=/root zroot/home/root zfs create -o canmount=off -o setuid=off -o exec=off zroot/var zfs create zroot/var/cache zfs create zroot/var/log zfs create zroot/var/spool zfs create -o exec=on zroot/var/tmp zfs create zroot/var/mail zfs mount -a chmod 1777 /media/var/tmp
6. Everything that is not stored in one of the filesystems, is in zroot/ROOT/centos. Making going back and forth between snapshots easier as all the logfiles and stuff will be kept. If someone want to have another root (try something), one can simply switch the mountpoints and be good to go…
7. Now it is time to copy the system from the original install to the zroot.
  mount --bind / /mnt/ rsync -avPX /mnt/. /media/. # Next two steps only if you have /boot and /home on a different partition rsync -avPX /boot/. /media/boot/ rsync -avPX /home/. /media/home/. umount /mnt/tmp/
8. Final system adoption in the new system, but first prepare the chroot environment.
  mount --rbind /dev /media/dev/ mount --rbind /sys /media/sys/ mount --rbind /proc /media/proc/ chroot /media
9. Edit fstab, remove all entries and add swap partitions.
  vi /etc/fstab
10. Update grub config /etc/default/grub and add/ update the following lines:
  GRUB_CMDLINE_LINUX="nofb splash=quiet crashkernel=auto rhgb quiet boot=zfs rpool=zroot bootfs=zroot/ROOT/centos" GRUB_PRELOAD_MODULES="part_msdos zfs"
11. Make an ugly hack because apparently the grub utilities cannot handle persistent device names very well…
  pushd /dev ln -s /dev/disk/by-path/* . # Use right directory for you zpool device names popd
12. Rebuild and install grub on the root SSDs:
  grub2-mkconfig -o /boot/grub2/grub.cfg grub2-install /dev/sda grub2-install /dev/sdb
13. Enable the zfs-mount service otherwise, CentOS will only mount the zroot/ROOT/centos filesystem.
  systemctl enable zfs-mount.service
14. Update dracut config in /etc/dracut.conf
  add_dracutmodules+="zfs"
15. Rebuild initramfs
  dracut -f -v /boot/initramfs-$(uname -r).img $(uname -r)
16. Do anything else you want, then reboot. And hope for the best. If things should go south, boot from the original medium and troubleshoot or start over by destroying the zpool.
Infomational stuff

Nov 7, 2016

by klump

Filed under Server
Page generated at Nov 8, 2020 11:55 UTC