FreeBSD: Home server

2019-03-27 (modified 2021-02-18)

Introduction

This is my home server, more or less. I use this page to remember what I’ve done and as a reference for less-often used commands, in case I need it later on.

Maybe some of it might be of help to others, so I published it.

I got tired of my Synology DS1812+ and went back to the glorious FreeBSD, ZFS and jails setup.

Upon first boot

SSH access

Copy public part of a pre-generated SSH key from your client machine for easy SSH administration.

From client machine:

$ ssh-copy-id username@ip.of.server.host
$ ssh username@ip.of.your.server

Remove password authentication and rely solely on pubkeys:

# vim /etc/ssh/sshd_config
ChallengeResponseAuthentication no
PasswordAuthentication no
PubkeyAuthentication yes

Make sure the SSHd is enabled on startup in /etc/rc.conf:

sshd_enable="YES"

Restart the SSH service:

# service sshd restart

Updating the OS

# freebsd-update fetch
# freebsd-update install

Useful tools

$ su -
# pkg install sudo vim-tiny tmux lsof rsync git

Run rehash if you are using csh to update $PATH.

Run visudo and uncomment “%wheel” line.

Vim config

This is my tiny vimrc for some sane defaults, including saving a keystroke for ex commands by switching the use of : and ;.

~/.vimrc:

set nocompatible hidden nobackup visualbell
filetype indent on

set modelines=0 nomodeline
set clipboard+=unnamed
set wildchar=<Tab> wildmenu wildmode=list:longest,full
set path+=**
set laststatus=2
set tabstop=4 softtabstop=4 shiftwidth=0 expandtab shiftround
set smartcase
set backspace=2

let mapleader = ","
nnoremap ; :
nnoremap : ;
nnoremap Q :.!$SHELL<CR>
nnoremap <silent> <leader>l :set list!<CR>
set listchars=tab:▸\ ,eol:¬,trail:·

tmux

I rarely bother with reconfiguring. I like using defaults. With that said, I am bothered by always trying tmux attach upon login and not finding any active session.

This little script in ~/bin checks if there is a running session already and then attaches to that, otherwise it will create one.

#!/bin/sh

tmux has
if [ $? == 1 ]; then
    tmux
else
    tmux attach
fi

I call it ta and chmod +x ~/bin/ta for an easy tmux jump start.

HDD monitoring

# pkg install smartmontools
# echo 'smartd_enable="YES"' >> /etc/rc.conf

Create and edit /usr/local/etc/smartd.conf as suggested.

Comment DEVICESCAN.
Uncomment/create:
/dev/ada0 -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/ada1 -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/ada2 -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/ada3 -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/ada4 -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/ada5 -a -o on -S on -s (S/../.././02|L/../../6/03)
...

# service smartd start
# smartd -q onecheck
# smartctl -tshort /dev/ada0
...
# smartctl -a /dev/ada0

Message of the day

Get rid of all that welcome stuff, generate something nice with figlet and edit /etc/motd:

 _____              ____ ____  ____
|  ___| __ ___  ___| __ ) ___||  _ \
| |_ | '__/ _ \/ _ \  _ \___ \| | | |
|  _|| | |  __/  __/ |_) |__) | |_| |
|_|  |_|  \___|\___|____/____/|____/

Ports

# portsnap fetch extract

For ports updates from time to time:

# portsnap fetch update

NTP

Set NTP to sync time on boot (if you only installed ntpd and not ntpdate):

# echo 'ntpd_sync_on_start="YES"' >> /etc/rc.conf

Edit /etc/ntp.conf if needed. NTP pools are pretty clever and try to use the closest one anyway…

Get rid of any leap second fetching errors:

# pkg install ca_root_nss && service ntpd onefetch

Disable syslogd network

I do not gather logs from other sources and I do not send my logs away. Disable network binding by starting with double s flags, as in -ss. Add to /etc/rc.conf:

syslogd_flags="-ss"

User/group administration

Show user:

# id <user>

Add group and add members:

# pw groupadd <groupname>
# pw groupmod <groupname> -m <user>

ZFS

Show status:

# zpool status
# zfs list
# df -h

Create a new pool and file systems:

# zpool create -f -m /data data raidz2 /dev/... /dev/...
# zpool status
# zfs create [options] poolname/whatever

…or import an existing pool from a previous installation:

# zpool import <poolname>

Replacing zfs-on-root mirror

The new disk MUST be of the same size or bigger than the one still in the pool.

# zpool status
# zpool detach zroot ada0p3 // failed drive/partition (first drive in this case)
// Replace physical drive
# gpart create -s GPT ada0
# gpart add -b 40 -l gptboot0 -s 512K -t freebsd-boot ada0 // get the -l (label) flag right
# gpart add -s 2G -l swap0 -t freebsd-swap ada0 // or whatever swap space used
# gpart add -t freebsd-zfs -l zfs0 ada0
# zpool status
# zpool attach zroot ada1p3 ada0p3 // zpool attach <pool> <existing> <new>

jails

Taken from https://clinta.github.io/freebsd-jails-the-hard-way/ for manual managing of jails, since a lot of third-party tools are badly maintained. And the magic ain’t that magical.

Base jail and a skeleton template

Create a zfs for all jails:

# zfs create -o mountpoint=/usr/local/jails zroot/jails

Set up a template base installation with ports inside:

# zfs create -p /usr/local/jails/templates/base
# fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/12.2-RELEASE/base.txz -o /tmp/base.txz
# tar -xvf /tmp/base.txz -C /usr/local/jails/templates/base
# fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/12.2-RELEASE/ports.txz -o /tmp/ports.txz
# tar -xvf /tmp/ports.txz -C /usr/local/jails/templates/base
# cp /etc/resolv.conf /usr/local/jails/templates/base/etc/resolv.conf
# cp /etc/localtime /usr/local/jails/templates/base/etc/localtime

Update template:

# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/templates/base fetch install
# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/templates/base IDS

Create a skeleton template that each jail will be copied from:

# zfs create -p zroot/jails/templates/skeleton

Copy variable skeleton content from base template:

# mkdir -p /usr/local/jails/templates/skeleton/usr/ports/distfiles /usr/local/jails/templates/skeleton/home /usr/local/jails/templates/skeleton/portsbuild
# mv /usr/local/jails/templates/base/etc /usr/local/jails/templates/skeleton/etc
# mv /usr/local/jails/templates/base/usr/local /usr/local/jails/templates/skeleton/usr/local
# mv /usr/local/jails/templates/base/tmp /usr/local/jails/templates/skeleton/tmp
# mv /usr/local/jails/templates/base/var /usr/local/jails/templates/skeleton/var
# mv /usr/local/jails/templates/base/root /usr/local/jails/templates/skeleton/root

Create symlinks in base template to the skeleton:

# cd /usr/local/jails/templates/base
# mkdir skeleton mnt
# ln -s skeleton/etc etc
# ln -s skeleton/home home
# ln -s skeleton/root root
# ln -s skeleton/tmp tmp
# ln -s skeleton/var var
# cd usr/ports/
# ln -s ../../skeleton/usr/ports/distfiles distfiles
# cd ../../usr/
# ln -s ../skeleton/usr/local local

Edit make.conf so portbuild work dir is inside skeleton:

# echo "WRKDIRPREFIX?=  /skeleton/portbuild" >> /usr/local/jails/templates/skeleton/etc/make.conf

Snapshot of the skeleton and a place for the thin jails:

# zfs snapshot zroot/jails/templates/skeleton@skeleton
# zfs create zroot/jails/thinjails

Create /etc/jail.conf with global settings:

# /etc/jail.conf

# Global settings

interface = "em0";
host.hostname = "$name.vkbox";
path = "/usr/local/jails/$name";
ip4.addr = 10.0.1.$ip;
mount.fstab = "/usr/local/jails/$name.fstab";

exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.poststop = "sleep 2";
exec.clean;
mount.devfs;

Update all jails base installation and ports:

# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/templates/base fetch install
# portsnap -p /usr/local/jails/templates/base/usr/ports auto

Simply backup /usr/local/jails/thinjails/ to save the configuration for each jail.

Start jails on boot:

# echo 'jail_enable="YES"' >> /etc/rc.conf
# service jail start

Create a new jail

New jail (called httpd):

# zfs clone zroot/jails/templates/skeleton@skeleton zroot/jails/thinjails/httpd
# echo hostname=\"httpd\" > /usr/local/jails/thinjails/httpd/etc/rc.conf
# mkdir -p /usr/local/jails/httpd

Update and append /etc/jail.conf:

httpd {
    $ip = 91;
}

Create jail fstab /usr/local/jails/httpd.fstab:

/usr/local/jails/templates/base  /usr/local/jails/httpd/ nullfs   ro          0 0
/usr/local/jails/thinjails/httpd     /usr/local/jails/httpd/skeleton nullfs  rw  0 0

Start/stop jail:

# jail -c httpd
# jail -r httpd

# jexec httpd

Do whatever you want with your new jailed environment. Like, changing root password:

# passwd

Shell script for new jails

I have put the above commands in a shell script in ~/bin for easy creation of new jails:

#!/bin/sh

if [ `whoami` != "root" ]; then
    echo Must be root.
    exit 1
fi

_NAME=$1
_IP=$2

if [ -z "$_NAME" ] || [ -z "$_IP" ]; then
    echo "usage: $0 <name> <last octet in IP>"
    exit 1
fi

echo Cloning zfs skeleton.
zfs clone zroot/jails/templates/skeleton@skeleton zroot/jails/thinjails/$_NAME
echo Settings hostname and rc.conf stuff in new jail.
echo "hostname=\"$_NAME\"
sendmail_enable=\"NONE\"
syslogd_flags=\"-ss\"
ntpd_enable=\"NO\"
" > /usr/local/jails/thinjails/$_NAME/etc/rc.conf
echo Creating a mountpoint for new jail.
mkdir -p /usr/local/jails/$_NAME

echo Updating /etc/jail.conf.
echo "
$_NAME {
    \$ip = $_IP;
}
" >> /etc/jail.conf

echo Creating a fstab for new jail.
echo "/usr/local/jails/templates/base  /usr/local/jails/$_NAME/ nullfs   ro          0 0
/usr/local/jails/thinjails/$_NAME     /usr/local/jails/$_NAME/skeleton nullfs  rw  0 0
" >> "/usr/local/jails/$_NAME.fstab"

echo Start the new jail with 'jail -c $_NAME'.
echo Do not forget to change root password inside jail.

I have named it ~/bin/jcreate and run it as such:

$ sudo jcreate httpd 91

List jails

See current active jails:

# jls

Remove a jail

Stop the jail, delete the entry in /etc/jail.conf and then remove the mountpoint, fstab and zfs dataset:

# jail -r <name>
# rm -rf /usr/local/jails/<name>
# rm -rf /usr/local/jails/<name>.fstab
# zfs destroy zroot/jails/thinjails/<name>

Shell script for removing jails

#!/bin/sh

if [ `whoami` != "root" ]; then
    echo Must be root.
    exit 1
fi

_NAME=$1

if [ -z "$_NAME" ]; then
    echo "usage: $0 <name>"
    exit 1
fi

echo Stopping jail.
jail -r $_NAME

echo Removing files and folders.
rm -rfv /usr/local/jails/$_NAME
rm -rfv /usr/local/jails/$_NAME.fstab

echo Destroying zfs dataset.
zfs destroy zroot/jails/thinjails/$_NAME

echo Updating /etc/jail.conf.
awk "!/^$_NAME/" RS= ORS="\n\n" /etc/jail.conf > /etc/jail.conf.new
mv /etc/jail.conf /etc/jail.conf.bkp
mv /etc/jail.conf.new /etc/jail.conf

echo $_NAME has been removed.

I have named it ~/bin/jremove and run it as such:

$ sudo jremove httpd

Update jails

Update all jails:

# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/templates/base fetch install
# portsnap -p /usr/local/jails/templates/base/usr/ports auto

Backup jails

Every jail customization is stored in /usr/local/jails/thinjails, so just backup that one however you like.

UTF-8

/etc/login.conf:

    :setenv=... // append ",LC_COLLATE=C
    // At the end of the default list, append
    :charset=UTF-8:\
    :lang=en_US.UTF-8: // or whatever from `locale -a`

Update:

# cap_mkdb /etc/login.conf