I’ve recently (re)discovered how beneficial tmux is when managing my small FreeBSD-based home server (running FreeBSD 14.0-RELEASE-p5 at time of this post). According to the FreeBSD man pages, tmux is a terminal multiplexer that it enables a number of terminals to be created, accessed, and controlled from a single screen. tmux may be detached from a screen and continue in the background, then later reattached.
My FreeBSD servers run headless so I typically administer them from the command line via SSH connections. Thanks to tmux, I can have one or more terminal sessions operating simultaneously over a single SSH connection. But what I appreciate even more is that should my connection to a server fail for any reason, my terminal session will be maintained by tmux, allowing me to pick right back up where I left off when the SSH connection can be reestablished.
I’m using FreeBSD binary packages instead of ports these days so to install tmux (version 3.3a_1 at the time of this post) I first update the package repository on the system then install tmux as the user root:
1
2
pkg update
pkg install tmux
Once installed I logged into my system through SSH and start tmux:
1
tmux
Optionally I can name my session by passing the parameter -s [your-session-name] to tmux when I start it. For example:
1
tmux new-smy-new-session
When tmux is started it creates new terminal session within a single window. A green status line appears at the bottom of the screen. This is used to show information about the session as well as for entering commands. Now that I’m connected to tmux, I can run any commands or programs I normally would.
Let’s test to make sure tmux is working as as it should by running the the table of processes or “top” command to display information on your system’s CPU(s) and running processes.
1
top
Now that top is running, I can detach from tmux using the command Ctrl+b followed by d, leaving top running in active in memory. Now I’ll end my SSH session and reconnect. In order to get back to my running instance of top I need only reattach to tmux.
1
tmux attach
And I now see top running just as I had left it. You can see the value of tmux in this example. If I were to lose SSH connection to a server for whatever reason, any work I was doing would still be running under tmux when I reestablish my connection.
I can also run two or more terminal sessions simultaneously in tmux. To do this I create a new window using the command Ctrl+b followed by c. Each of these windows is assigned an index number (e.g., 0,1,.. ) and those are shown in the green status line. I can now navigate between these windows using Ctrl+b followed by the number assigned to them , or by using Ctrl+b followed by n (next) or p (previous).
Another handy command is Ctrl+b followed by w, which opens a panel allowing me to chose a window interactively.
Here is a list of the keybindings I find useful. You can view a list of all of the keybindings that tmux offers by using the command Ctrl+b followed by ? or by consulting the tmux man pages.
Ctrl+b ? View all keybindings. Press q to exit. Ctrl+b d Detach from the current session. Ctrl+b c Create a new window. Ctrl+b x Close window. Ctrl+b n or p Move to the next or previous window. Ctrl+b 0 (1,2…) Move to a specific window by number. Ctrl+b w Open a panel to navigate across windows in multiple sessions.
Conclusion
I’ve really enjoyed getting back into using tmux. It’s made managing my servers more productive as well giving me piece of mind knowing that I can safely return to what I was doing should I lose my SSH connection.
Here’s a very short post on the FreeBSD top command, mostly because I was using it the other day and it was um…, top of mind (sorry).
The table of processes or “top” command can be used to display information on your FreeBSD system’s CPU(s) and running processes. By default top lists the processes based on their CPU usage, and will update the list every second. Here are a few of the top options I find handy:
-h Summarizes the options available for top
-a Displays the command name and its full path, rather than just its executable name.
-H Display each thread for a multi-threaded process individually. By default a single summary line is displayed for each process.
-I Toggles the display of idle processes. By default, top displays both active and idle processes.
-m Display statistics based on either I/O or CPU usage. The default is CPU.
-S Display system processes. Normally, system processes such as pagedaemon and clock are not shown.
Here’s a simple little Bourne shell-compatible script I wrote to create and delete rolling ZFS snapshots on a small home server running the FreeBSD operating system. Sure there’s ports/packages available in FreeBSD to accomplish this, but there’s something to be said for keeping it simple and having one less piece of software to update and maintain.
As currently configured, the script recursively creates a daily snapshot of /pool_0/dataset_0. The name of each snapshot consists of the word “snap,” followed by the date and time the snapshot was taken (e.g., snap-202002102300). Snapshots are retained in pool_0/dataset_0/.zfs/snapshots. The script will then destroy any snapshot older that 90 days and log what its done to the file cronlog located in my home directory. A typical ~/cronlog entry looks like the following:
To use the script, I save it as zfssnap.sh in ~/bin, where I keep most of my helper scripts, and make it executable:
1
chmod+x~/bin/zfssnap.sh
Then I delegate some ZFS permissions to user iceflatline so snapshots can be created and destroyed without becoming the root user. However, this command is issued as the user root:
Note that I’ve set permissions at the zpool level, which means that all datasets under pool_0 will inherit these settings. Alternatively I could have applied the permissions just to dataset_0.
Then I make the snapshots retained in pool_0/dataset_0/.zfs/snapshots. Here too, the command must be issued as the user root:
1
zfs set snapdir=visible pool_0
Here again, I’ve applied applied this setting at the zpool level so when additional datasets are created in pool_0 they will inherit this setting. And here too, I could have applied this setting just to dataset_0.
Finally I configure a cronjob in the crontab for user iceflatline so that the script runs daily at 23:00:
1
2
3
4
### DAILY
# Run backup scripts every day at 2300
023***/home/iceflatline/bin/zfssnap.sh
If you’d like to use the script as is then simply modify the variable src_0 to reflect the name of your zpool and dataset. Else, modify the script to suite your needs.
Prefer to run the script more frequently than daily? Modify your cronjob. Here are couple of examples for running the script more frequently:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
### HOURLY
# Run snapshot script every 15 minutes
#*/18 * * * * /home/iceflatine/bin/zfssnap.sh
# Run snapshot script every 30 minutes
#*/33 * * * * /home/iceflatine/bin/zfssnap.sh
# Run snapshot script at the top of every hour
#3 * * * * /home/iceflatine/bin/zfssnap.sh
# Run snapshot script every three hours at the top of each hour
#3 */3 * * * /home/iceflatine/bin/zfssnap.sh
# Run snapshot script every six hours at the top of each hour
#3 */6 * * * /home/iceflatine/bin/zfssnap.sh
Don’t like the snapshot naming convention. No problem, just modify the variable snap_prefix.
Prefer to have more or less than 90 snapshots? Modify the retention variable.
Want to add other zpools and/or datasets? Add them to another variable (e.g., src_1), then modify the remainder of the script to recognize and take action on them.
Prefer to have the script log its output to a different location? Modify the log variable.
You get the idea.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
#!/bin/sh
### BEGIN INFO
# PROVIDE:
# REQUIRE:
# KEYWORD:
# Description:
# This script is used to create zfs snapshots and remove older snapshots.
# The number of snapshots to retain is defined in the variable retention.
# Author: iceflatline <iceflatline@gmail.com>
#
# OPTIONS:
# -v: Be verbose
### END INFO
### START OF SCRIPT
# These variables are named first because they are nested in other variables.
snap_prefix=snap
retention=90
# Full paths to these utilities are needed when running the script from cron.
In a previous post I described how to install and configure Tobi Oetiker’s MRTG (Multi Router Traffic Grapher) on a Ubuntu server. In this post I will describe how to install and configure it on FreeBSD. Once configured, you’ll be able to use MRTG to monitor the traffic in and out of your network using the SNMP capability in your network’s gateway/router. MRTG generates static HTML pages containing PNG images which provide a visual representation of this traffic. MRTG typically produces daily, weekly, monthly, and yearly graphs. MRTG is written in perl and works on Unix/Linux as well as Windows. MRTG is free software licensed under the GNU GPL.
Software versions used in this post were as follows:
apache24 2.4.23
FreeBSD 11.0-RELEASE
mrtg-2.17.4
The steps discussed assume that the FreeBSD Ports Collection is installed. If not, you can install it using the following command:
1
portsnap fetch extract
If the Ports Collection is already installed, make sure to update it:
1
portsnap fetch update
Okay, let’s get started. All commands are issued as user root. When building the various ports you should accept the default configuration options.
Install a http server
MRTG requires an http server to be installed and operating correctly. In our example, we’ll install and use the Apache http server. Navigate to the Apache port and build it:
1
2
cd/usr/ports/www/apache24
make config-recursive install distclean
Once Apache has been successfully installed, Use the sysrc command to add the following line to /etc/rc.conf so that the Apache server will start automatically at system boot:
1
sysrc apache24_enable="YES"
Now let’s start Apache to make sure it works:
1
service apache24 start
Point your web browser to the host name or IP address of the FreeBSD host you’ve installed Apache on and you should see the venerable “It works!”
Install and configure MRTG
Now that we have an http server up and running let’s install MRTG:
1
2
cd/usr/ports/net-mgmt/mrtg
make install clean
What does the MRTG port install and where is that stuff located?
1
2
3
4
5
6
7
8
9
10
# find / -name 'mrtg'
/var/run/mrtg
/var/mail/mrtg
/usr/ports/net-mgmt/mrtg
/usr/local/share/doc/mrtg
/usr/local/share/examples/mrtg
/usr/local/bin/mrtg
/usr/local/etc/mrtg
/usr/local/bin/cfgmaker
/usr/local/bin/indexmaker
MRTG provides the example configuration file /usr/local/etc/mrtg/mrtg.cfg.sample that describes global configuration parameters as well as various configuration options for the SNMP targets you want to monitor. If you already have some experience with MRTG and SNMP you can simply copy or move this file to /usr/local/etc/mrtg/mrtg.cfg then modify it to meet your requirements. In our example, however, we’re going to create the requisite mrtg.cfg file from scratch.
MRTG includes the script cfgmaker that will create and populate a basic mrtg.cfg file with information obtained from your gateway/router. So, before running /usr/local/bin/cfgmaker, you should activiate and configure the SNMP service in your gateway/router. This typically involves logging into the device and enabling SNMP. The default SNMP community name is typically “public.” If you change the SNMP community name to something else, make note of it. Now, let’s run cfgmaker, substituting your SNMP community name if you’ve changed it, and adding the IP address of your gateway/router:
If you would like to add more than one device to mrtg.cfg simply append the additional URL(s) to the same mrtg.cfg file. Then, when you build the web page using the indexmaker command described below, graphs associated with each device will be displayed on the same HTML page:
Next, open /usr/local/etc/mrtg/mrtg.cfg and, under Global Config Options, uncomment the line WorkDir: /home/http/mrtg and change it to WorkDir: /usr/local/www/apache24/data/mrtg. This is the directory from which the Apache http server will server the MRTG html pages. If you’re using something other than Apache as your http server then you’ll need to change this path.
Next, uncomment the line Options[_]: growright, bits. By default MRTG graphs grow to the left, so the option growright specifies that the direction of the traffic visible in MRTG’s graphs flips causing the current time to be at the right edge of the graph and the history values to the left. The option bits specifies that the monitored traffic values obtained from your device is multiplied by 8 and displayed bits per second instead of bytes per second.
MRTG includes the script indexmaker. This is what we’ll use to create the pages used to display the MRTG graphs. First, let’s create the directory from which Apache http server will serve up the pages:
1
mkdir/usr/local/www/apache24/data/mrtg
Then use indexmaker combined with our mrtg.cfg file to create and populate an index.html file in that directory:
Now we need to add an Alias and a Directory directive to Apache’s configuration file to support MRTG. Open /usr/local/etc/apache24/httpd.conf and add the following lines in the section containing similar Directory directives, or it can simply be appended to the bottom of the file:
1
2
3
4
5
6
7
Alias/mrtg"/usr/local/www/apache24/data/mrtg"
<Directory"/usr/local/www/apache24/data/mrtg">
Options None
AllowOverride None
Require all granted
</Directory>
And change the user and group for the following directories to mrtg:
1
2
chown-Rmrtg:mrtg/usr/local/etc/mrtg
chown-Rmrtg:mrtg/usr/local/www/apache24/data/mrtg
Finally, let’s restart the http server:
1
service apache24 restart
Starting MRTG
Okay, now that MRTG has been installed and configured let’s start it up and see what it displays. Use the sysrc command to add following line to /etc/rc.conf:
1
sysrc mrtg_daemon_enable="YES"
Then start the MRTG daemon:
1
service mrtg_daemon start
The MRTG daemon will now run automatically each time FreeBSD starts.
Now point your browser to http://your-http-server-address/mrtg and you should see a page that resembles Figure 1. You may have more or less graphs depending on the number of interfaces reported by your devices(s).
Figure 1
You’ll see the graph starting to “grow” to the right as the traffic is monitored over time, and the Y axis displayed as bits per second. If you click on any one of these graphs you’ll be taken to differnt page showing individual graphs for 30 minute, two hour, and daily averages, along with the maximum, average, and current bit rate in and out of that particular interface. By default, these graphs will update every 5 minutes.
Only interested in displaying one particular interface? Want to graph other SNMP data? Now that you that you have a basic mrtg.cfg file created you can modify it or incorporate some of the global and target parameter examples contained in the file /usr/local/etc/mrtg/mrtg.cfg.sample to further customize your configuration. Just remember to run indexmaker again to update the MRTG index.html file.
Conclusion
This concludes the post on how to install and configure MRTG on FreeBSD. As you can see, MRTG isn’t terribly complicated and proves to be a really nice port for monitoring and graphing traffic in and out your gateway/router. For a full list of all the configuration options and other information I encourage you to visit the MRTG web site.
In a recent post I described how I improved the reliability of my file system backups by using the data replication capabilities inherent in the FreeBSD Zettabyte File System (ZFS). In this post I will explore Tarsnap, another tool I recently started to use to perform secure offsite backups of my most important files.
The versions for the software used in this post are as follows:
FreeBSD 11.0-RELEASE
tarsnap 1.0.37
The steps discussed assume that the FreeBSD Ports Collection is installed. If not, you can install it using the following command:
1
portsnap fetch extract
If the Ports Collection is already installed, make sure to update it:
1
portsnap fetch update
Okay, let’s get started. Okay, let’s get started. All commands are issued as the user root. While building the various ports you should accept all default configuration options unless otherwise instructed.
Create a Tarsnap account
Before installing Tarsnap I visited the Tarsnap registration page and created an account. Tarsnap operates on a prepaid basis, so you have to add some money to your account before you can start using it. The minimum amount is $5.00. Money will be deducted from your pre-paid amount based on the actual number of bytes stored and bandwidth used (after compression and data deduplication). Tarsnap prices are currently $.25 per Gigabyte per month for storage, and $.25 per byte for bandwidth.
Install Tarsnap
After creating an account it was time to install Tarsnap. First I made sure the Ports Collection was up to date:
1
portsnap fetch update
Then proceeded with the install, accepting all default configuration options:
1
2
cd/usr/ports/sysutils/tarsnap
make config-recursive install distclean
Next, I ran tarsnap-keygen, a utility which registers my machine with the Tarsnap server and generates a key that is used to encrypt and sign the archives that I create. I needed to have the e-mail address and password I used to create my Tarsnap account handy when running this command. In following example I’ve registered a machine with the host name tarsnap-test:
Note that if I had multiple machines containing files I wished to backup to Tarsnap, I would want create a separate key file for each machine.
By default tarsnap-keygen will create the key file /root/tarsnap.key. This can be changed by adding the option keyfile to specify a different location and/or key name. In the example above I’ve changed the name of my key file to tarsnap-test.key to help disambiguate keys in case I add additional machines to my Tarsnap account in the future.
Tarsnap creates the file /usr/local/etc/tarsnap.conf when installed. This config file is read by the tarsnap utility and specifies a number of default options, all of which will be ignored if the options in question are specified at the command line. Since I change the name of default key file, I revised the value for the option keyfile in /usr/local/etc/tarsnap.conf:
1
keyfile/root/tarsnap-test.key
Note that you should store a copy of this key someplace safe. If you lose your Tarsnap key file(s), you will not be able to create new archives or access your archived data.
Using Tarsnap
After installing Tarsnap I was ready to create and backup my first archive. Tarsnap commands follow a syntax similar to the venerable tar utility. The -c option creates a new archive containing the specified files. The -f option specifies which file to write the archive to:
Performing subsequent backups of these files will go faster since Tarsnap’s deduplication feature will avoid sending data which was previously stored.
If I want to list all archives stored with Tarsnap I can use the following command:
1
tarsnap--list-archives|sort
Adding one or more instance of the -v option to this command will make the output more verbose. For example, if -v is specified one or more times, the creation time of each archive is printed; if it is specified two or more times, the command line with which Tarsnap was invoked to create each archive is also printed.
If I want to list the files contained within a single archive I can use the following command:
1
tarsnap-tvf backup-20150729|more
The -t option is used to print to stdout the files stored in the specified archive; the -v option of course makes the output a little more verbose.
If I wanted to delete one or more archives I can use the -d option:
1
tarsnap-df backup-20150729
When the time comes to restore one or more files from Tarsnap I have a couple of options. For example, I can recover all files contained in a particular archive using the following command. In this example, I’ve extracted all files contained in the archive backup-20150729 to /tmp where I can recover one of more files:
1
2
cd/tmp
tarsnap-xf backup-20150729
Or I can extract just one of the directories in this archive:
Note here that you must exclude the leading / from the directory you’re restoring from. So in this case, instead of /pool_0/dataset_0/some-directory/, it should be pool_0/dataset_0/backup/some-directory.
Or regress even further into the archive to recover a single file if desired:
Finally, if for whatever reason I no longer wish to use Tarsnap on this machine I can invoke the nuke option, which will delete all of the archives stored:
1
tarsnap--nuke
To make sure you’re really serious, Tarsnap will ask you to type the text “No Tomorrow” when using this command.
Okay, after getting comfortable with the Tarsnap commands and backing up files manually for a couple of days, I created this ugly little script that creates a daily archive of a specified directory; looks for any archives older than 30 days and deletes them; and, logs its output to the file /home/iceflatline/cronlog:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/bin/sh
### BEGIN INFO
# PROVIDE:
# REQUIRE:
# KEYWORD:
# Description:
# This script is used to perform backups using the tarsnap utility. The number of backups to retain is defined in the variable retention.
# Author: iceflatline <iceflatline@gmail.com>
#
# OPTIONS:
# -c: Create an archive containing the specified files
# -d: Delete the specified archive
# -f: Archive name
### END INFO
### START OF SCRIPT
backup_prefix=backup
retention=30
# Full paths to the following utilities are needed when running the script from cron.
echo"Could not find any backups to destroy.">>$log
fi
# Mark the end of the script with a delimiter.
echo"**********">>$log
# END OF SCRIPT
I wrote the script to /home/iceflatline/bin/tarsnap.sh where I maintain some other scipts and made it executable:
1
chmod+x/home/iceflatline/tarsnap.sh
Then added the following cron job to the crontab under user root. The script runs every day at 0800 local time:
1
2
# Run tarsnap backup script everyday at 0800
08***/home/iceflatline/bin/tarsnap.sh
Conclusion
Well, that’s it. A short post describing my experiences using Tarsnap, an easy, secure and inexpensive solution for performing offsite backups of my most important files.
