Learn to be a Linux Sysadmin

I stumbled across a great post on Reddit not long ago, which detailed “what to do” if you want to learn to be a Linux SysAdmin. It’s a bit dated, it looks like – around 2 years old. But it still looks to me like a pretty solid start, and I’m at a point where I feel like I need a long-term project to really kick this whole system administrator thing off. So I think I’m going to give it a shot, though with a few minor tweaks to the original.

  1. Setup a KVM hypervisor.
  2. Install a Spacewalk server inside of that hypervisor. Use CentOS 7 as the distro for all following works. Set up errata importation on the CentOS channels, so you can properly see security update advisory information.
  3. Create a VM to provide named and dhcpd service to your entire environment. Setup the dhcp daemon to use the Spacewalk server as the pxeboot machine, thus allowing you to use Cobbler to do unattended OS installs. Ensure that every forward zone has a reverse zone associated with it. Use something like “internal.virtnet” (but not “.local”) as your internal DNS zone.
  4. Use that Spacewalk server to automatically (without touching it) install a new pair of OS instances, which which you will then create a Master/Master pair of LDAP servers. Make sure they register with the Spacewalk server. Do not allow anonymous bind; do not use unencrypted LDAP.
  5. Reconfigure all 3 servers to use LDAP authentication.
  6. Create two new VMs which will then be Postgresql VMs. Use pgpool-ll to setup Master/Master replication between them. Export the database from your Spacewalk server and import it into the new pgsql cluster. Reconfigure your Spacewalk instance to run off of that server.
  7. Set up a Puppet Master. Plug it into the Spacewalk server for identifying the inventory it will need to work with. Cheat and use Ansible for deployment purposes, again plugging into the Spacewalk server.
  8. Deploy another VM. Install iscsitgt and nfs-kernel-server on it. Export a LUN and an NFS share.
  9. Deploy another VM. Install bakula on it, using the postgresql cluster to store its database. Register each machine on it, storing to flatfile. Store the bakula VM’s image on the iscsi LUN, and every other machine on the NFS share.
  10. Deploy two more VMs. These will have httpd (Apache2) on them. Leave essentially default for now.
  11. Deploy two more VMs. These will have tomcat on them. Use Jboss Cache to replicate the session caches between them. Use the httpd servers as the frontends for this. The application you will run is JBoss Wiki.
  12. Deploy another VM. This will do iptables-based NAT/round-robin loadbalancing between the two httpd servers.
  13. Deploy another VM. Install postfix. Set it up to use a gmail account to allow you to have it send emails, and receive messages only from your internal network.
  14. Deploy another VM. Set up a Nagios server. Have it use snmp to monitor the communication state of every relevant service involved above. This means doing a “is the right port open” check, and a “I got the right kind of response” check, and “we still have file system space free” check.
  15. Deploy another VM. On this VM, set up a syslog daemon to listen to every other server’s input. Reconfigure each other server to send their logging output to various files on the syslog server. Setup logstash or kibana or greylog to parse those logs.
  16. Document every last step you did in getting to this point in your brand new Wiki.
  17. Go back and create Puppet Manifests to ensure that every last one of these machines is authenticating to the LDAP servers, registered to the Spacewalk server, and backed up by the bakula server.
  18. Go back, reference your documents, and set up a Puppet Razor profile that hooks into each of these things to allow you to recreate, from scratch, each individual server.
  19. Destroy every secondary machine you’ve created and use the above profile to recreate them, joining them to the clusters as needed.
  20. Dreate three more VMs. A CentOS 5, 6, and 7 machine. On each of these machines, set them up to allow you to create custom RPMs and import them into the Spacewalk server instance. Ensure your Puppet configurations work for all three and produce like-for-like behaviors.

I’ll be honest. I’m familiar with some of this, but there are some things in here that I’ve never touched, and a few that I’ve never even heard of. I look forward to getting started!

#how-to, #linux, #sysadmin