The FILES variable should be populated with a whitespace separated list of files, directories, and block devices to track.

The DB_ABCD variables should be populated with appropriate credentials to talk to a mysql server.

The actual script looks something like this:

#!/bin/bash

FILES='/var/lib/mysql/ibdata1 /var/lib/mysql/db/table.ibd /dev/sda1 /var/log/mysql'

LOCAL=`hostname -s`
DB_HOST='aaa'
DB_USER='bbb'
DB_PASS='ccc'

function insert {
    FILE=$1
    SIZE=$2
    QUERY="replace into metrics.storage_usage values( now(), '$LOCAL', '$FILE', $SIZE )"
    mysql --host=${DB_HOST} --user=${DB_USER} --password="${DB_PASS}" -e "${QUERY}"
}

for FILE in $FILES
do
    if [ -d $FILE ]; then
        BASE=$FILE
        SIZE=`du -ks $FILE/ | awk '{print $1}'`
    elif [ -b $FILE ]; then
        TMP=`df -k -P $FILE | tail -n1 | awk '{print $3 " " $6}'`
        SIZE=`echo $TMP | awk '{print $1}'`
        BASE=`echo $TMP | awk '{print $2}'`
    else
        BASE=`basename $FILE`
        SIZE=`du -k $FILE | awk '{print $1}'`
    fi

    echo "$BASE = $SIZE"
    insert $BASE $SIZE
done

I am putting my data into a table called “storage_usage” in a database called “metrics”:

CREATE TABLE `storage_usage` (
  `ts` date NOT NULL,
  `host` varchar(25) NOT NULL,
  `file` varchar(64) NOT NULL,
  `size` int(10) unsigned NOT NULL COMMENT 'in kbytes',
  PRIMARY KEY (`ts`,`host`,`file`)
)

Obviously, this could be tweaked in any different number of ways, based on your needs. One tweak you might want to consider if you’re running it in a daily cron is to remove the echo so you don’t get an email report of every run. Also, if you might want to record more than one snapshot per file per host per day – in the which case you probably need to change the type of the timestamp column to a datetime. Or there might be cases where you want to change the replace to an insert or… whatever

It works under the assumption that your server daemon has a unique name and only ever runs a single instance – this also means that the binary and the init.d script cannot share a name – otherwise strange things happen

Actual invocation logic may need to be updated on a per-service basis and chkconfig style headers would have to be added manually, but it works well for what it is.

#!/bin/bash

DIR=''	# path to the daemon executable
CMD=''	# name of the command itself
ARG=''	# optional. any arguments to pass when starting
NAM=''	# descriptive name of the daemon so it shows up pretty

function get_ps {
	ps --no-header -C${CMD}
}

function do_start {
	echo -n "Starting ${NAM}... "
	cd ${DIR}
	nohup ./${CMD} ${ARG} &
	SUCC=`get_ps | wc -l`
	if [ "1" == "$SUCC" ]; then
		echo "[SUCCESS]"
	else
		echo "[FAILURE]"
	fi
}

function do_stop {
	echo -n "Stopping ${NAM}... "
	PID=`get_ps | awk '{print $1}'`
	kill $PID
	SUCC=`get_ps | wc -l`
	if [ "0" == "$SUCC" ]; then
		echo "[SUCCESS]"
	else
		echo "[FAILURE]"
	fi
}

case "${1:-''}" in
	'start')
		do_start
		;;
	'stop')
		do_stop
		;;
	'restart')
		do_stop
		do_start
		;;
	*)
		#echo "Usage: $SELF start|stop|restart|reload|force-reload|status"
		echo "Usage: $SELF start|stop|restart"
		exit 1
		;;
esac

What I want:

ammon@hermes:~/repo$ svn info --get-revision .
1234

But of course, nothing like this exists.

Thankfully, svn info’s output IS easy enough to parse. You just have to do it your self.

ammon@hermes:~/repo$ svn info | grep Revision | awk -- '{print $2}'
1234

Will give you the revision of your current checkout without the network hit of a call to svn log.

To get the current version of the repo itself (hits the network), add “-r HEAD” to the svn info call:

ammon@hermes:~/repo$ svn info -r HEAD | grep Revision | awk -- '{print $2}'
1280

Of course, svn info also supports outputting info as xml, so you could use that to parse things in a more advanced environment but one where you’re still not using the svn api bindings.

And now on a related note…

Yesterday during a code review, the svn:keywords property was discussed, and the idea proposed to start using it to embed magical versioning information into our builds (for bug reporting reasons and such). Well, I decided to jump on the idea this afternoon. After a bit of fiddling, we now have a little component that can be used to display the revision number of the current application build – all w/o anyone having to remember to do it.

The SVN Book explains how to use the property in greater detail.

I created a simple xml file (it could be any text file, really), that I then dropped the svn keyword tags into it.

<?xml version="1.0" encoding="utf-8"?>
<version>
	<date>$Date$</date>
	<rev>$Rev$</rev>
</version>

I then set the svn:keywords property on the file to “Date Rev” (ie, the names of the keywords i want replaced – see the full docs for a list of valid keywords).

Upon commit, the file is updated on the server to now look more like this:

<?xml version="1.0" encoding="utf-8"?>
<version>
	<date>$Date: 2006-12-19 14:30:02 -0700 (Tue, 19 Dec 2006) $</date>
	<rev>$Rev: 376 $</rev>
</version>

Unfortunately… there’s a major hitch in what I wanted to do. These labels actually only update when the file in question is updated. Ie, it displays the last time the source file was changed, not the last time the entire repository was changed. So, this trick is useful for updating header comments or perhaps an internal version constant string (if you’re doing server-side post-commit builds or something). But… it’s not what I want in this case

In the end, I wound up having to do a bit of old fashioned sed manipulation on the server side post-commit… but that doesn’t change the fact that subversion still allows me to do this kind of magic. It’s just not quite as magical as I’d originally hoped it could be.

My post-commit hook now looks something like this:

#!/bin/bash
REPO=$1
REV=$2
DATE=`date`

cd /var/www/$REPO
svn up .
sed -i "s/\\\\\\\$Rev.*\\\\\\\$/\\\\\\\$Rev: $REV \\\\\\\$/g" version.xml
sed -i "s/\\\\\\\$Date.*\\\\\\\$/\\\\\\\$Date: $DATE \\\\\\\$/g" version.xml
ant
svn revert version.xml

I’m keeping updated copies of a part of the repository in a web accessible directory, and am using Ant to build deployable zip files and such. The sed commands are performing a search and replace on the svn keywords before the build and I then revert the changes (just in case somebody actually changes the real contents of version.xml – which would result in a conflict if we kept these local changes around).

The file I’m producing now looks like this:

<?xml version="1.0" encoding="utf-8"?>
<cmykversion>
	<date>$Date: Tue Dec 19 15:43:07 MST 2006 $</date>
	<rev>$Rev: 378 $</rev>
</cmykversion>

If I really wanted to, I could generate the same exact date format as SVN uses, but I prefer this one for its brevity.

I landed on a French anime wallpaper site. I’d not been there before, and I started looking at a decently sized collection of images that I’d also never before encountered. Naturally, I decided to acquire them – but the sheer scope of the site lended a non-trivial amount of difficulty to this. At least, it did until I realized that their images were stored in sequentially numbered files in sequentially numbered directories. No zero padding or anything.

This led to the following script, which is currently quite happily churning its way through the 36th directory – whatever show that may actually happen to be

I am making one assumption here, that they haven’t divided things up into more than 500 directories. I could just as easily have done the sort of while loop as I did on the inside on the outside, but this is also meant as demonstrative code for future reference.

#!/usr/bin/bash
for DIR in `seq 0 500`
do
        mkdir $DIR
        cd $DIR
        IMG = -1
        while [ $? != 1 ]
        do
                ((IMG += 1))
                wget -c host/$DIR/$IMG.jpg
        done
        cd ..
        rmdir $DIR

        if [ $? == 0 ]
        then
                echo "removed $DIR, assuming we're done"
                exit
        fi
done

Why not just use wget’s spider functionality? I have in the past, but not for something this fun. Besides, this way I don’t have any cleanup to do (other than identifying shows and renaming their folders when all is done).

It’s working on directory number 45 now