Server Maintenance

H&D's VPS usually run on GNU/Linux Debian distribution.

Updating the system once a week (or every two weeks) is kinda good. It's good to cross-check the Debian mailing-list (debian-lts-announce@lists.debian.org) beforehand to see if anything major needs a security update or particular care.

To upgrade the system, your unix user needs to be in the sudo group. Following the basic process:

sudo apt update , upgrade local db packages
sudo apt list --upgradable , show upgradable packages
sudo apt upgrade , upgrade packages

To list all installed packages, run:

apt list --installed

Disk space

As of 2023-04-24, the main sources of disk space usage are:

Chatty Pub
Nextcloud
Etherpad
System logs

Chatty Pub

Currently we are copying the whole Zulip files folder from the VPS where Zulip runs, to the VPS where Chatty Pub runs. The files folder amount to around 4.6 GB.

It would be useful to allocate some time to the project again and look up if Zulip in the meanwhile extended their APIs to allow to fetch files, and eventually if this solves the problem we ran into 2 years ago when trying to do that.

Another alternative option would be to update the codebase of Chatty Pub to detect which files needs to be copied over, instead of mirroring the whole files folder. This though seems harder than the first option.

Nextcloud

We need to outline and agree on a set of rules on which files and especially how long do they live on Nextcloud. For instance:

are we using NC for long-term access storage?
can NC be used for temporarily accessing files online and at some later point remove them (eventually with an archive option)?

if we want to use NC for long-term storage (for eg replacing Google Drive, Dropbox, etc), then it would be wiser to setup another VPS for it, and use that disk space primarily for NC and other sharing / archive purposes.

Using the occ command with nextcloud:

sudo -u www-data php7.4 /var/www/nc/occ

If you get Memcache errors:

sudo -u www-data php7.4 --define apc.enable_cli=1 /var/www/nc/occ

Etherpad

The Etherpad database in MySQL is around 1.3 GB. This is mostly due to how Etherpad uses MySQL (it dumps each pad's JSON entry as-is into a MysSQL column, and probably this does not allow MySQL to optimise disk space...).

A possible idea on how to work around this constant growing size are, is to setup an archive system. For instance:

each pad "lives" for ~ 1 year, after which it is exported as HTML and removed from the MySQL database; see etherpump
- this also means that the pad's welcome text will display the 1 year duration as a reminder, and will share a link to where the archived pad will be found
- we also have to deal with the __NOINDEX__ magic word and the private / public implication of this: if the pad is private, should it be archive after 1 year or not?

MySQL

Handy command to check the size of each database:

mysql db, check disk size:
    SELECT table_schema "database name", sum( data_length + index_length ) / 1024 / 1024 "database size in MB", sum( data_free )/ 1024 / 1024 "free space in MB" FROM information_schema.TABLES GROUP BY table_schema;

You can also do:

ls -lh /var/lib/mysql

System Logs

We set the system logs to be pruned (or deleted) every 6 day and growing for a maximum of 300 MB:

journalctl --vacuum-time=6d --vacuum-size=300M

See ref. Nginx logs were also growing up to 68 MB. Not as big as the 800 MB of the system journal, but still much more than the few KB of other programs. You can check and change the logrotate rules for Nginx, for instance, by doing:

vi /etc/logrotate.d/nginx

Change nginx with any other available log file. See here for more.