I’ve been playing with more clustering as I prepare for a RedHat class in August and figured I would write about ClusterIt. I was looking to run a few commands on about six servers and went looking for a simple solution. I believe ClusterIt provides an elegant solution for very little work.

Here is a list of commands and their description from their respective manpages.
dsh – Run a command on a cluster of machines as defined in the CLUSTER environmental variable.
dshbak – Takes input from the dsh command and formats it to look nicer for the user.
run – Run a command on a machine at random.
rseq ñ Run a command on a sequence of machines or cluster.
pcp – Copy a file to a number of machines.
pdf – Display free disk space across a number of machines, can be for a single filesystem or the entire machine.
prm – Delete a file, directory or list of files on a number of machines.
rvt – Remote terminal emulator.
clustersed ñ Quickly dissect cluster files, used to cut individual groups out of a cluster file.
dtop – Used to remotely monitor and display top information, this program segfaulted on my system.

There are also some more involved commands, the daemons for these must be set up on the remote machines.
barrier ñ Used to synchronize execution of commands on slower and faster machines. When a barrier is set, the process is not released until all of the nodes or processes have met the barrier condition.
barrierd ñ The daemon portion of barrier that accepts connections from the client program barrier.
jsh ñ Run scheduled commands on remote machines.
jsd ñ A simple command scheduling daemon for remote execution.

The first thing you need to do is make sure you have ssh password-less login set up. I went to our network management server and added a couple of the servers that needed to be able to run commands remotely.

In case you are doing this from scratch, here is the sequence of commands. Generate private/public keys on your management server A.

ssh-keygen -t dsa
press enter when it asks for the filename
press enter when it asks for the passphrase (yes, a blank passphrase)

This will generate two files: ~/.ssh/id_dsa and ~/.ssh/ You now want to allow access from this server (A) to the remote server (B) by putting the contents of ~/.ssh/ from A into ~/.ssh/authorized_keys2 on B.

cat ~/.ssh/ | ssh B 'cat >> ~/.ssh/authorized_keys2'

Make sure permissions are correct and are not writable or readable except by the owner. Do this on both server A and B.

chmod a-x,go-w,o-r ~/.ssh/*

And to verify it works.

ssh B ls -la

Now it’s time to install ClusterIt. I like to have a suite of programs installed in a common directory but don’t want to modify my MANPATH or worry about other nonsense. This is how I installed ClusterIt.

./configure --bindir=/usr/local/clusterit
make install
cd /usr/local/clusterit/

If you read the manpage for dsh or one of the other program in ClusterIt you can see a number of environmental variables and how to set up the ClusterIt environmental variables and files. A snippet of the manpage for dsh.

dsh utilizes the following environment variables.

CLUSTER            Contains a filename, which is a newline separated 
list of nodes in the cluster.

RCMD_CMD           Command to use to connect to remote machines.  
The command chosen must be able to connect with no password to 
the remote host.  Defaults to rsh

 ...removed for brevity...

The file pointed to by the CLUSTER environment variable has the 
following format:

This example would have pollux and castor a member of no groups, 
rigel and kent a member of group 'alpha', and alshain and altair a 
member of group 'sparc'.  Note the format of the GROUP command, 
it is in all capital letters, followed by a colon, and the group name.
There can be no spaces following the GROUP command, or in the 
name of the group.

As a result I set up my .bashrc with the following options for ClusterIt.

export CLUSTER

export RCMD_CMD

export PATH

Make sure you re-source your .bashrc.

source ~/.bashrc

And I have a simple /etc/clusterit/servers file:

cat /etc/clusterit/servers

Now to test.

dsh uptime
B:  17:44:26 up 24 days,  6:32,  5 users,  load average: 0.02, 0.01, 0.00
C:  17:46:56 up 443 days,  9:53,  2 users,  load average: 0.00, 0.00, 0.00
D:  17:46:56 up 443 days,  9:52,  1 user,  load average: 0.00, 0.01, 0.00

And finally run some commands.

man pcp
pcp /usr/local/bin/ /usr/local/bin/ 
dsh /usr/local/bin/ -d /tmp
dsh scp /tmp/output.txt user@A:/tmp/

That last command you must have password-less login from the ClusterIt servers back to your management server.

This entry was posted in Linux. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s