Tue Nov 3 10:59:36 ICT 2015

How-to power-off CSIM servers from remote

There is a definite order of action to properly power-off CSIM system from a remote access.
  1. Remote connect to bazooka.cs.ait.ac.th using ssh, the following connections will have to be initiated from bazooka only;
  2. Choose a time for the shutdown; the time should be well enough in the future so you can complete all the following tasks before it expires, a good practice is 30 minutes.
    VMware ESXi servers may take a substential anount of time to shutdown, depending of the number of virtual machine running on the server. A god rule of thumb is to consider 2 minutes for each vitual machines, plus 2 minutes for the server itself.

    When choosing the time to shutdown, allow enough time for the shutdown to properly complete before the general shutdown of the main electric power, a good practice is 30 minutes.

    So if a general failure of the main power is planned for 11:00, you should plan the shutdown time at 10:30 and start working at 10:00. In the following examples, we use 10:30 for the shutdown time.

  3. Start the shutdown sequence on the router: ssh to the account vyatta on the router and enter the command:
    sudo /sbin/shutdown -h 10:30 &
    
    and disconnect from the router;
  4. Start the shutdown sequence on the fireall: ssh to the account admin on the firewall and enter the command:
    shutdown -h 10:30 &
    
    and disconnect from the firewall.
    For the router and the firewall, you can reconnect and issue a shutdown -c to cancel the shutdown sequence;
  5. Start the shutdown sequence on amanda: ssh to amanda and enter the command:
    sudo shutdown -p 10:30 &
    
    and disconnect from amanda.
    For amanda, you cannot cancel the shutdown sequence;
  6. Start the shutdown sequence on VMware ESXi servers; the following has to be repeated for each of the servers: ssh to the account root on the VMware ESXi server and enter the command:
    /vmfs/volumes/datastore1/shutdown 635 &
    
    and disconnect from the server.
    To shutdown a VMware ESXi server, you don't use the time of shutdown as a parameter, but instead you use the number of seconds remaining until the time of shutdown: if the wall clock is 10:19:25 and you want to shutdown at 10:30, that makes it 10 minutes and 35 seconds, that is, 635 seconds.
    For the ESXi server, you can reconnect and issue a ps -c | grep shutdown to find the process ID of the shell script and kill the script to cancel the shutdown sequence.
    Every virtual machine on the ESXi server will be gracefully shutdown provided that:
    • VMware tools are installed on the virtual machine;
    • The virtual machine is configured to do a guest shutdown (default is a plain power off) in vSphere client Configuration/Virtual Machine Startup-Shutdown.

      On that note, you can also configre the startup to Continue immediately if VMware Tools start.


Posted by Olivier | Permanent link | File under: administration, firewall, vmware, freebsd, router