I’ve been playing with more clustering as I prepare for a RedHat class in August and figured I would write about ClusterIt. I was looking to run a few commands on about six servers and went looking for a simple solution. I believe ClusterIt provides an elegant solution for very little work.
Here is a list of commands and their description from their respective manpages.
dsh – Run a command on a cluster of machines as defined in the CLUSTER environmental variable.
dshbak – Takes input from the dsh command and formats it to look nicer for the user.
run – Run a command on a machine at random.
rseq ñ Run a command on a sequence of machines or cluster.
pcp – Copy a file to a number of machines.
pdf – Display free disk space across a number of machines, can be for a single filesystem or the entire machine.
prm – Delete a file, directory or list of files on a number of machines.
rvt – Remote terminal emulator.
clustersed ñ Quickly dissect cluster files, used to cut individual groups out of a cluster file.
dtop – Used to remotely monitor and display top information, this program segfaulted on my system.
There are also some more involved commands, the daemons for these must be set up on the remote machines.
barrier ñ Used to synchronize execution of commands on slower and faster machines. When a barrier is set, the process is not released until all of the nodes or processes have met the barrier condition.
barrierd ñ The daemon portion of barrier that accepts connections from the client program barrier.
jsh ñ Run scheduled commands on remote machines.
jsd ñ A simple command scheduling daemon for remote execution.
The first thing you need to do is make sure you have ssh password-less login set up. I went to our network management server and added a couple of the servers that needed to be able to run commands remotely.
In case you are doing this from scratch, here is the sequence of commands. Generate private/public keys on your management server A.
ssh-keygen -t dsa press enter when it asks for the filename press enter when it asks for the passphrase (yes, a blank passphrase)
This will generate two files: ~/.ssh/id_dsa and ~/.ssh/id_dsa.pub. You now want to allow access from this server (A) to the remote server (B) by putting the contents of ~/.ssh/id_dsa.pub from A into ~/.ssh/authorized_keys2 on B.
cat ~/.ssh/id_dsa.pub | ssh B 'cat >> ~/.ssh/authorized_keys2'
Make sure permissions are correct and are not writable or readable except by the owner. Do this on both server A and B.
chmod a-x,go-w,o-r ~/.ssh/*
And to verify it works.
ssh B ls -la
Now it’s time to install ClusterIt. I like to have a suite of programs installed in a common directory but don’t want to modify my MANPATH or worry about other nonsense. This is how I installed ClusterIt.
./configure --bindir=/usr/local/clusterit make make install cd /usr/local/clusterit/ ls
If you read the manpage for dsh or one of the other program in ClusterIt you can see a number of environmental variables and how to set up the ClusterIt environmental variables and files. A snippet of the manpage for dsh.
ENVIRONMENT dsh utilizes the following environment variables. CLUSTER Contains a filename, which is a newline separated list of nodes in the cluster. RCMD_CMD Command to use to connect to remote machines. The command chosen must be able to connect with no password to the remote host. Defaults to rsh ...removed for brevity... FILES The file pointed to by the CLUSTER environment variable has the following format: pollux castor GROUP:alpha rigel kent GROUP:sparc alshain altair LUMP:alphasparc alpha sparc This example would have pollux and castor a member of no groups, rigel and kent a member of group 'alpha', and alshain and altair a member of group 'sparc'. Note the format of the GROUP command, it is in all capital letters, followed by a colon, and the group name. There can be no spaces following the GROUP command, or in the name of the group.
As a result I set up my .bashrc with the following options for ClusterIt.
CLUSTER=/etc/clusterit/servers export CLUSTER RCMD_CMD=/usr/bin/ssh export RCMD_CMD PATH=$PATH:/usr/local/clusterit export PATH
Make sure you re-source your .bashrc.
And I have a simple /etc/clusterit/servers file:
cat /etc/clusterit/servers B C D
Now to test.
dsh uptime B: 17:44:26 up 24 days, 6:32, 5 users, load average: 0.02, 0.01, 0.00 C: 17:46:56 up 443 days, 9:53, 2 users, load average: 0.00, 0.00, 0.00 D: 17:46:56 up 443 days, 9:52, 1 user, load average: 0.00, 0.01, 0.00
And finally run some commands.
man pcp pcp /usr/local/bin/script.sh /usr/local/bin/script.sh dsh /usr/local/bin/script.sh -d /tmp dsh scp /tmp/output.txt user@A:/tmp/
That last command you must have password-less login from the ClusterIt servers back to your management server.