138 lines
5.3 KiB
Markdown
138 lines
5.3 KiB
Markdown
# Admin Scripts
|
|
|
|
This repo contains some administration scripts for administrering a debian machine.
|
|
The covered tasks range from file change tracking via http/ssh monitoring to zfs health checking.
|
|
|
|
|
|
## Installation
|
|
|
|
The scripts in this repo ***must*** be checked out into `/root/scripts`.
|
|
|
|
```bash
|
|
cd /root
|
|
git clone ssh://git@git.mneun.ch:2222/radioelephant/admin-scripts.git scripts
|
|
mkdir -p /root/logs
|
|
```
|
|
|
|
If you are unable to clone via ssh, try via https.
|
|
|
|
```bash
|
|
cd /root
|
|
git clone https://git.mneun.ch/radioelephant/admin-scripts.git scripts
|
|
mkdir -p /root/logs
|
|
```
|
|
|
|
### Config Files
|
|
|
|
For each script there is a `.script_name_env.EXAMPLE` file, which you must copy (remove `.EXAMPLE` part) and edit while providing your own information.
|
|
|
|
```bash
|
|
SCRIPT_NAME=zfs_health_check
|
|
cp /root/scripts/.${SCRIPT_NAME}_env.EXAMPLE /root/scripts/.${SCRIPT_NAME}_env
|
|
vim /root/scripts/.${SCRIPT_NAME}_env
|
|
```
|
|
|
|
If you want to use the example configuration, you can symbolic link the files.
|
|
This really only makes sense for `.system_health_check`.
|
|
The command for this is:
|
|
|
|
```bash
|
|
cd /root/scripts
|
|
ln -s .system_health_check.EXAMPLE .system_health_check
|
|
```
|
|
|
|
### Shutdown Notification
|
|
|
|
To be notified immediately before sever shutdown the contained `run-before-shutdown.service` can be installed as follows.
|
|
This service file will send a telegram notification informing about the pending shutdown.
|
|
|
|
```bash
|
|
ln -s /root/scripts/run-before-shutdown.service /etc/systemd/system/shutdown.target.wants/run-before-shutdown.service
|
|
systemctl daemon-reload
|
|
```
|
|
|
|
### Startup Helper
|
|
|
|
To install the startup helper script into the regular user home use the following commands, while supplying your username on the first line.
|
|
|
|
```bash
|
|
USRNAME=radioelephant
|
|
ln -s /root/scripts/post_startup.sh /home/$USRNAME/post_startup
|
|
cp /root/scripts/.post_startup_env.EXAMPLE /home/$USRNAME/.post_startup_env
|
|
vim /home/$USRNAME/.post_startup_env
|
|
```
|
|
|
|
*Just shutdown your computer if you don't know how to exit VIM at this point.*
|
|
|
|
### Updating
|
|
|
|
You can update the admin scripts by pulling from the git remote.
|
|
|
|
```bash
|
|
cd /root/scripts
|
|
git pull
|
|
```
|
|
|
|
|
|
## Usage
|
|
|
|
The check and monitoring scripts in this repo can be run periodically be run and if any problems are detected, they produce output.
|
|
|
|
The output of these scripts can be redirected and used however you like.
|
|
Typically I redirect the output to the `telegram_notification.sh` script which notifies me of any noisy scripts.
|
|
|
|
Regardless of any problems each script also logs its executions in `/root/logs`.
|
|
Make sure you created this folder during installation.
|
|
|
|
### Crontab
|
|
|
|
You can schedule regular execution of these scripts with "cron".
|
|
To configure "cron" use the `crontab -e` command, which opens VIM, *our favourite text editor*, containing the cron schedule.
|
|
If you are unsure about the cron schedule, use [Crontab Guru](https://crontab.guru).
|
|
|
|
My current crontab looks like this:
|
|
|
|
```crontab
|
|
* * * * * bash -c '/root/scripts/file_monitor.sh | /root/scripts/telegram_notification.sh'
|
|
*/2 * * * * bash -c '/root/scripts/monitoring.sh | /root/scripts/telegram_notification.sh'
|
|
*/4 * * * * bash -c '/root/scripts/dyndns.sh | /root/scripts/telegram_notification.sh'
|
|
*/3 * * * * bash -c '/root/scripts/system_health_check.sh | /root/scripts/telegram_notification.sh'
|
|
15 * * * * bash -c '/root/scripts/docker_health_check.sh | /root/scripts/telegram_notification.sh'
|
|
*/15 * * * * bash -c '/root/scripts/zfs_health_check.sh | /root/scripts/telegram_notification.sh'
|
|
|
|
@reboot sleep 10 && /root/scripts/telegram_notification.sh '[STARTUP] System just booted'
|
|
@reboot sleep 30 && bash -c '/root/scripts/zfs_health_check.sh | /root/scripts/telegram_notification.sh'
|
|
```
|
|
|
|
Adapt this to your needs, you might also implement other checks and only use the `telegram_notification.sh` script from this repo.
|
|
Or you might implement your own notification script to notify you via another service.
|
|
The `telegram_notification.sh` can easily be adapted (just remove comment) to forward all notifications to `STDOUT` which typically makes cron send a mail.
|
|
|
|
### Startup
|
|
|
|
The `post_startup.sh` script is a helper script which should be run as soon as possible after a system boot.
|
|
It will check for encrypted datasets with user provided passwords which are not mounted yet.
|
|
After decrypting the datasets this script starts the configured docker containers which are depending on the encrypted datasets.
|
|
Because my containers struggle with DNS which depends on a docker container itself, the script also restarts some configured docker containers at this point.
|
|
|
|
To complete the post startup tasks use the following command after logging in as your regular user:
|
|
|
|
```bash
|
|
./post_startup.sh
|
|
```
|
|
|
|
You can permit your user to run all of the commands in the script with out having to enter the users password by adapting the "sudoers" file.
|
|
Beware that this is a security concern, as the regular user can thus manage all docker containers, which can hijack your host network, amongst other things.
|
|
But as I connect to my servers via SSH keys which are stored reasonably securely on Yubikeys, I waive this security measure.
|
|
To edit the "sudoers" file use the `visudo` command as root, which also opens VIM, *our favourite text editor*.
|
|
My relevant part of the "sudoers" file looks as follows:
|
|
|
|
```sudoers
|
|
radioelephant ALL=NOPASSWD: /usr/bin/zfs mount -a -l
|
|
radioelephant ALL=(docker) NOPASSWD: /usr/bin/docker
|
|
```
|
|
|
|
## Contributors
|
|
|
|
- Robin Meier (robin@meier.si)
|