Thu Sep 29 13:03:32 +07 2022
Some stuff related VMware and SSL certificates
Thu Sep 29 12:47:49 +07 2022
chown error in slapd start
Wed Jun 15 11:20:14 +07 2022
Rebuilding the firewall
After the power failure of May 10th, 2022, the firewall would not restart.
In this report, I want to summarize the steps that I have taken to get the firewall working again.
Hard disk failure
One of the hard disk had failed. Because the disks are in a RAID 1 array, it was only a matter of replacing the failed disk with one saved from a other machine. The RAID will take care of the mirroring of the nee disk.
The steps to restore the file systems
When trying to boot the firewall, it would hang at a message about Rebuilding LDAP database. My diagnostic was that some file had been corrupted by the power failure and needed to be restored.
In order to restore the file-systems from Amanda, I need to boot from a live system that offers:
- mounting the Linux ext partitions;
- connecting to Amanda server with ssh;
- extracting the files with tar.
Booting the machine from a USB key
That is the part that took me the longest to solve.
The server would boot from a Freebased installation USB, I could launch a live shell, but the partitions on the hard disk were faulty and if the live shell can mount an ext partition, it is missing the fsck for that type of partition.
To mount an ext partition:
mount -t ext2fs /dev/something /somewhere
Any other type of USB key would not work. I tried Ubuntu, UBCD, Hiren, Vyatta...
I finally resolved to install Ventoy (ventoy.net) on a USB key; and on that Ventoy key I copied Hiren's BootCD 12.0. Note that Ventoy needs a machine under Windows to first create the USB key.
All the files that I used are under /home/pc-application/ZeroShell
For the next step, I also copied ZeroShell 3.9.5 on the Ventoy key.
Restore the file systems
Booting Ventoy key, I could start Hiren BooCD and choose Mini Linux.
In that mini Linux environment, I could fsck the damaged partitions and restore the file systems.
There are 3 partitions on the disk:
- /dev/sda1 10 GB, that is mounted on /boot
- /dev/sda2 50 GB, that is mounted on /cdrom
- /dev/sda3 75 GB, that is mounted on /DB
Note that / is a RAM disk.
Restoring is just a matter of recovering the data from Amanda, mounting each partition to /mnt on the mini Linux and issuing commands like:
ssh root@192.41.170.11 tar cf - the recovered partition|(cd /mnt; tar xfBp -)
Note that root must be allowed to connect to Amanda.
Rebooting the firewall
When the 3 partitions had been restored, I tried to reboot the firewall, but the error about rebuilding LDAP persisted.
At the end of my rope, I tried to launch ZeroShell 3.9.5 from the Ventoy key. I was not sure of what to expect, but I knew that ZeroShell could run without being copied to the hard disk. That newer version of ZeroShell managed to rebuild LDAP and finished booting. It was just a matter of rebooting the firewall from its original disk.
So far, I don't know why that Dell PowerEdge R200 is so temperamental about the type of USB it can boot from.
I am not sure what LDAP is being used for by ZeroShell because U think I have disabled it in the configuration. I have added a script that dumps LDAP database every two hours.
It seems that the restoring of the file systems has messed up some of the group ownership of some files and directories. The only real effect I could see is in Amanda in /Database/Links/opt/libexec/amanda/.
When booted from the USB key in ZeroShell 3.9.5, the firewall did work properly, but I had no access through SSH or the web interface. That is why I reverted to the previous version.
Wed Jun 8 16:06:02 +07 2022
Some things to install Amanda on FreeBSD
Recently I ran into a couple of problems when installing or upgrading Amanda on FreeBSD. The problems may have been present earlier, but I did not saw them then.
Modify /usr/local/etc/pkgtools.confAdd the following in the file /usr/local/etc/pkgtools.conf so that Amanda is being built with the proper configuration:
MAKE_ARGS = { 'misc/amanda-server' => 'AMANDA_USER=amanda AMANDA_GROUP=amanda AMANDA_CONFIG=normal AMANDA_WITHOUTIPV6=yes', 'misc/amanda-client' => 'AMANDA_USER=amanda AMANDA_GROUP=amanda AMANDA_CONFIG=normal AMANDA_GNUTAR_LISTDIR=/usr/local/var/amanda/gnutar-lists AMANDA_WITHOUTIPV6=yes' }
On a client machine, only the amanda-client part is needed.
Most of the configuration choices come from very old installation (pre-FreeBSD) but this is how it works for CSIM.
Modify the location of amanda-security.confOn the server, the security file for Amanda client is located in /usr/local/etc/amanda-client/amanda-security.conf. If kept in the default place, it will conflict with the ownership and mode of the default directory.
This change can only be made by modifying the Makefile file in /usr/ports/misc/amanda-server. Note that even if that configuration file only pertains to Amanda client, the makefile is located in Amanda server.
Around the 33rd line of the makefile, change the value of --with-security-file to reflect:
--with-security-file=/usr/local/etc/amanda-client/amanda-security.confAlways build or upgrade the server after building the client
Building Amanda client will change the contents of the file /usr/local/lib/perl5/site_perl/Amanda/Constants.pm to contain the line:
$AMANDA_COMPONENTS = " client amrecover ndmp";
Only after building or upgrading the server will the line be reverted to the correct:
$AMANDA_COMPONENTS = " server restore client amrecover ndmp";
Failing to have the proper line will result of the following error message in Amanda summary email:
taper: FATAL Can't locate object method "new" via package "Amanda::Xfer::Dest::Taper::Splitter" (perhaps you forgot to load "Amanda::Xfer::Dest::Taper::Splitter"?) at /usr/local/lib/perl5/site_perl/Amanda/Taper/Scribe.pm line 764.
Thu May 19 11:08:40 +07 2022
Upgrade FreeBSD source with git
- Install git from the ports.
- Run it once to create the proper environment:
sudo git clone --branch releng/13.1 https://git.freebsd.org/src.git /usr/src
- Updates should be run with the command
cd /usr/src sudo git pull
- To change to a new release
cd /usr/src sudo git checkout releng/13.2
Fri Feb 18 15:03:42 +07 2022
Amanda looses connection to certain Ubuntu servers
It seems that from time to time amanda will loose connection to some servers running Ubuntu.
In the daily amanda mail report, there will be lines like:
puffer1000 / lev 0 FAILED [Request to puffer1000 failed: timeout waiting for ACK]
and an amcheck -c on that host will return an error.
This can be cleared by restarting inetd on the affected servers:
sudo service inetd restartor xinetd on firewall:
sudo service xinetd restart
Wed Feb 16 13:26:20 +07 2022
Naming virtual machines
When a virtual machine (VM) is created to be used/managed by a user, it should not persist when the use leaves AIT.
Proper naming of the VMs will allow the scripts that manage the user accounts to automatically back-up and delete the obsolete VMs.
The name of the VMs should be of the form:
- In the simplest form, the name of the VM is the username, for example st12345
- If one user needs more than one VM, the VM name can start with the username, followed by a space or - or _ and some number or index; for example st123456-2 or st123456 banana
- Alternatively, it is possible to set the name of the owner of the
VM in the VM notes; the line should be of the form
owner=st123456
Note that the check is very loose on this form, if the string owner=user matches, whatever follows that string is accepted.
Any VM that follow these conventions will be automatically backed-up in the user's homedirectory and deleted from the virtual server when the user's account expires.
Mon Feb 14 13:53:07 +07 2022
Managing eduroam profile on enterprise-wifi.net
enterprise-wifi.net is a very strong tool to create automatic configuration for eduroam. Once the eduroam profile has been created, auto-configuration scripts can be downloaded for Windows, Mac-OS and Linux (so far, no script is available for Android).
Safely configuring an eduroam profile by hand can be tricky as it requests very specific steps to be executed in a given order, eterprise-wifi.net is a great time and headache saver.
But it comes with some caveats.
Managing the administrators
Only Google accounts can create and manage profiles on enterprise-wifi.net. Furthermore, I could not find a way to assign the master administrator role to someone else: the use who first created the profile is the only one that can add administrators. When that user becomes invalid in Google, a totally new profile must be created from scratch.
The profile used for AIT/CSIM has been created by nicoleo@ait.asia
Managing the certificates
For a PEAP-MSCHAPv2 the profile must include a root certificate authority, even when you use a properly certified, and the infrastructure is able to provide all CA, root and intermediates.
As a consequence, if the CA included to the profile expire, the profile need to be generated anew and every device must install the new profile.
As of February 2022, we are using the following:- Let's Encrypt intermediate CA, expires in September 2025;
- ISRG X1 intermediate CA, expire in September 2024;
- ISRG X1 self signed root CA, expires in June 2035.
The profile will have to be updated by September 2024.
The intermediate CAs are provided by Let's Encrypt when signing or renewing a certificate, the are in the file ca.cert; the root CA, I found it on Let's Encrypt web page about the chain of trust.
Fri Feb 11 11:25:04 +07 2022
Installing ESXi 6.x on Dell R740
I used the following method to install the hypervisor.
- Create the virtual disk on the RAID controller;
- Download the proper driver from VMware Compatibility Guide;
- Connect a spare disk via USB or use an USB key, this will be used to do a temporary installation.
Note: it is very important that the temporary disk or the USB key are smaller than the RAID volume created at step 1;
- Install ESXi on the temporary disk or the USB key;
- Install the driver for the RAID controller; it should be done with a command like:
esxcli software vib install -d filename.zip
- Boot the newly installed ESXi from the temporary disk or USB key; you should be able to go to the storage and see the RAID/PERC H750 volume;
Note: it is very important that you create a different temporary install for each server (steps 4 and 5); if you clone one single image to several ESXi servers, you will have some UUID conflicts later;
- Reboot the system with a live Linux;
- Copy the system from the temporary disk or USB key to the RAID virtual disk, use a command like:
dd if=/dev/sdx of=/dev/sdy
- Remove the temporary disk or USB and reboot into the newly installed ESXi.
Other solution would be to install PowerCLI to prepare a new installation media that include the RAID driver.
Fri Jan 21 11:38:12 +07 2022
Create new collection in DSpace
At the beginning of each years, new collections must be added to DSpace. The collections are called: CS Dissertations, CS Resarch reports, CS Theses followed by the year, same for IM and DSAI. For consistency, care must be take for the name of the collections, so they are displayed in the corresct alphabetical order.
1 | Select the proper community and create a new collction. | |
2 | Then you need to assign the roles. Click on Create... for Submitters. You will define who is allowed to submit new documents. | ![]() |
3 | Enter the group name CSIM_LDAP and click on Groups... | ![]() |
4 | Click on Add. | ![]() |
5 |
Click on Save. Any user autenticated to CSIM LDAP server will be allowed to submit a new document. | ![]() |
6 |
Note that the Submitters has changed to COLLECTION iii SUBMIT. Click on Create... for the Accept/Reject Step. Repeat the steps 3, for and 5 with the group Office instead of CSIM_LDAP. That will make sure that the users beloging to the group Office (ie. our secretaries) can accept the documents submitted. | ![]() |
7 |
Note that the Accept/Reject Step has changed to COLLECTION iii WORKFLOW STEP-1. Click on Edit authirization policies directly. | ![]() |
8 |
This page lists the autorizations that have been defined. By AIT policy, the contents of the thesis and dissertations must not be publicly accessible. Click on DEFAULT_BITSTREAM_READ. | ![]() |
9 |
In the pull down menu, select AIT. That encomprise users connected inside the Institue and users authenticated to CSIM LDAP. Any of these users can read the contents of the document. Click Save. | ![]() |
10 |
Note that the DEFAULT BITSTREAM READ has been changed to AIT. At this point, you can click on Edit near the COLLECTION iii SUBMIT and COLLECTION iii WORKFLOW SETP 1 and check that the correct groups are defined: CSIM_LDAP and Office. When you are done, click on Return. The new collection has been created and configured, students can start adding thesis and reports to the new collection. Reppeat for all the new collections that need to be created at the beginning of each year. | ![]() |