Tag Archives: bash

report-storage.sh

In the grand tradition of my publishing little building-block shell scripts of interest, here goes another one. This is a simple cron job that I run daily on a number of hosts to generate storage usage growth. (This is in addition to Cacti and Nagios which poll some of this data already but for different reasons and with different granularity).

The FILES variable should be populated with a whitespace separated list of files, directories, and block devices to track.

The DB_ABCD variables should be populated with appropriate credentials to talk to a mysql server.

The actual script looks something like this:

I am putting my data into a table called “storage_usage” in a database called “metrics”:

Obviously, this could be tweaked in any different number of ways, based on your needs. One tweak you might want to consider if you’re running it in a daily cron is to remove the echo so you don’t get an email report of every run. Also, if you might want to record more than one snapshot per file per host per day – in the which case you probably need to change the type of the timestamp column to a datetime. Or there might be cases where you want to change the replace to an insert or… whatever 😉

init.d template

This is a rudimentary template that I’ve been using for very quick and dirty /etc/init.d scripts recently.

It works under the assumption that your server daemon has a unique name and only ever runs a single instance – this also means that the binary and the init.d script cannot share a name – otherwise strange things happen 😉

Actual invocation logic may need to be updated on a per-service basis and chkconfig style headers would have to be added manually, but it works well for what it is.

svn get revision

One of the more annoying things about svn is that (to my knowledge), there exists no single simple command to retrieve the revision number from a shell.

What I want:
[code]
ammon@hermes:~/repo$ svn info –get-revision .
1234
[/code]

But of course, nothing like this exists.

Thankfully, svn info’s output IS easy enough to parse. You just have to do it your self.

[code]
ammon@hermes:~/repo$ svn info | grep Revision | awk — ‘{print $2}’
1234
[/code]

Will give you the revision of your current checkout without the network hit of a call to svn log.

To get the current version of the repo itself (hits the network), add “-r HEAD” to the svn info call:

[code]
ammon@hermes:~/repo$ svn info -r HEAD | grep Revision | awk — ‘{print $2}’
1280
[/code]

Of course, svn info also supports outputting info as xml, so you could use that to parse things in a more advanced environment but one where you’re still not using the svn api bindings.

svn keyword magic

Subversion is officially the coolest thing ever 😉

And now on a related note…

Yesterday during a code review, the svn:keywords property was discussed, and the idea proposed to start using it to embed magical versioning information into our builds (for bug reporting reasons and such). Well, I decided to jump on the idea this afternoon. After a bit of fiddling, we now have a little component that can be used to display the revision number of the current application build – all w/o anyone having to remember to do it.

The SVN Book explains how to use the property in greater detail.

I created a simple xml file (it could be any text file, really), that I then dropped the svn keyword tags into it.

I then set the svn:keywords property on the file to “Date Rev” (ie, the names of the keywords i want replaced – see the full docs for a list of valid keywords).

Upon commit, the file is updated on the server to now look more like this:

Unfortunately… there’s a major hitch in what I wanted to do. These labels actually only update when the file in question is updated. Ie, it displays the last time the source file was changed, not the last time the entire repository was changed. So, this trick is useful for updating header comments or perhaps an internal version constant string (if you’re doing server-side post-commit builds or something). But… it’s not what I want in this case 😉

In the end, I wound up having to do a bit of old fashioned sed manipulation on the server side post-commit… but that doesn’t change the fact that subversion still allows me to do this kind of magic. It’s just not quite as magical as I’d originally hoped it could be.

My post-commit hook now looks something like this:

I’m keeping updated copies of a part of the repository in a web accessible directory, and am using Ant to build deployable zip files and such. The sed commands are performing a search and replace on the svn keywords before the build and I then revert the changes (just in case somebody actually changes the real contents of version.xml – which would result in a conflict if we kept these local changes around).

The file I’m producing now looks like this:

If I really wanted to, I could generate the same exact date format as SVN uses, but I prefer this one for its brevity.

wallpaper yoinkage

Ok, so it’s 4am and I’m awake. Nothing new there. I was scrounging sites and looking for some bit of information obscure enough that I’d actually forgotten what my original goal was. Thus, it was only natural that I get sidetracked by whatever shiny object floated past me as I clicked URL’s at random.

I landed on a French anime wallpaper site. I’d not been there before, and I started looking at a decently sized collection of images that I’d also never before encountered. Naturally, I decided to acquire them – but the sheer scope of the site lended a non-trivial amount of difficulty to this. At least, it did until I realized that their images were stored in sequentially numbered files in sequentially numbered directories. No zero padding or anything.

This led to the following script, which is currently quite happily churning its way through the 36th directory – whatever show that may actually happen to be 😉

I am making one assumption here, that they haven’t divided things up into more than 500 directories. I could just as easily have done the sort of while loop as I did on the inside on the outside, but this is also meant as demonstrative code for future reference.

Why not just use wget’s spider functionality? I have in the past, but not for something this fun. Besides, this way I don’t have any cleanup to do (other than identifying shows and renaming their folders when all is done).

It’s working on directory number 45 now 😉