Why you should care

There are quite a few reasons why you would want to back-up your website. For one thing, in the case of some kind of security breach, you don't want to lose the files on the server. Even if someone broke in, with a backup you could just restore it and you'd be back in a jiff. Otherwise, maybe you just want full control of your files, and knowing that they sit on a server somewhere remote doesn't make you feel as good as knowing they are right on your local disk. Whatever the reason, the following method is well suited to Wordpress sites, but general enough to apply to just about any website.

However, the following method enables you to transfer files in both direction, it's equally ideal for deployment. It makes no difference if you're uploading or downloading, we cover both bases.

How it works

Okay, that was the sales pitch. The script was written to allow for fast deployment of files on a server. Using Wordpress as an example, if you're hacking on your theme and you want to upload that one file you changed and see the result, you can do that quickly and painlessly with rsync. It's really the best way to transfer one file when you know none of the other files have changed. rsync synchronizes two locations, transferring only what has changed.

The files are transferred with rsync over ssh, so you need shell access on the server for this.

In a typical example where you have an account on a web server, this is how your file structure is at the root level (your homedir):

$ ls ~
.bashrc
.htaccess
.ssh
bin/
etc/
mail/
public_ftp/
public_html/
=> cgi-bin/
=> images/
=> => picture.jpg
=> index.html
tmp/

The files in bold are the ones you want to synchronize with your local disk and keep up-to-date. But there will generally be a lot of other files you're not interested in, generated in your homedir automatically, like raw web traffic logs, mail spam etc. (If the item is a directory, you want all the files and dirs it contains to be synchronized.)

So the issue is to selectively pick the items you want. But there may also be certain types of files inside these dirs you don't want, like for instance I ignore cgi-bin. So you want a way to exclude certain files/dirs from being transferred.

How to

Now that you know what's happening, it's time to set it up. You fill in the variables at the top of the script. local_path is where you want the files on disk. remote_path is where they are located on the server (in most cases ~ or /home/username). locations is the list of top level directories/files you want to synchronize. And finally exclusions are patterns you want to exclude (so if it contains cgi-bin, then that directory and all the files in it will be excluded from the synchronization).

Once that's done, you just run

$ sync.sh down

to download the files on the server to your local dir, and

$ sync.sh up

to transfer your local changes to the website. Finally,

$ sync.sh

alone will log you into your server with ssh.

Time to synchronize full local/remote tree for matusiak.eu (5470 files) when no changes were made: 4.4 seconds. ;)

A small note about security

Note that this script does not violate or subvert how you access your server. It uses ssh as the underlying security context. You can easily synchronize up/down with public key authentication, in which case you'll never have to type in your password when running sync.sh, and it's actually more secure as well. :)

#!/bin/bash
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 2.


# server setup
hostname="matusiak.eu"
username=""
ssh_port="22"

# local setup
local_path="/local/path"

# remote setup
remote_path="~"
locations="bin backups public_html"

exclusions="cgi-bin *.swp *~" #.swp are vim swap files


## EDIT BELOW THIS LINE IF YOU KNOW WHAT YOU'RE DOING

# rsync options
rsync_options="--archive --verbose --stats --progress"

# switch priority
nice="nice -n 10"


inc_list=""
function inclusion_list() {
	for i in $exclusions; do
		inc_list="${inc_list}--filter='- $i' "
	done
	for i in $locations; do
		inc_list="${inc_list}--filter='+ /$i' "
	done
	inc_list="${inc_list} --filter='- /*'"
}

function shell() {
	 ssh -C ${username}@${hostname} -p ${ssh_port}
}

function sync_up() {
	inclusion_list
	cmd="${nice} rsync ${rsync_options} -e \"ssh -p ${ssh_port}\" \
	${inc_list} \
	${local_path}/* \
	${username}@${hostname}:${remote_path} "
	echo "$cmd"
	sh -c "$cmd"
}

function sync_down() {
	inclusion_list
	mkdir -p ${local_path}
	cmd="${nice} rsync ${rsync_options} -e \"ssh -p ${ssh_port}\" \
	${inc_list} \
	${username}@${hostname}:${remote_path}/* \
	${local_path} "
	echo "$cmd"
	sh -c "$cmd"
}


if [ -z "$1" ]; then
	shell
elif [ "$1" = "down" ]; then
	sync_down
elif [ "$1" = "up" ]; then
	sync_up
else
	echo "$0 [down|up]"	
fi

Å bestille bonusreiser på Internett er gratis

Da vi innfører et servicegebyr for å bestille bonusreiser via Medlemsservice den 21. mai, lønner det seg å bestille via våre nettsider. Basic-medlemmer må fra 21. mai betale et servicegebyr på 150 kr. Se linken for mer informasjon.

How bout that. SAS Braathens has my email address, because I book all my flights online. Periodically they send me this spam, which is "great offers" included in "important membership information". Today they sent me an update on my bonus miles (along with a host of "great offers", of course), including this paragraph, which I find completely incomprehensible.

Booking flights online using air miles is free

Since we are about to introduce a service charge for booking flights out of air miles through our Membership Service, from May 21, we encourage you to book them on our website. Basic-members will incur a charge of 150kr from May 21.

Well that's an interesting way of telling you that 5 days from now you'll be paying money for booking flights from air miles you've saved up. So something that's not supposed to cost you anything now will, meanwhile you can avoid paying the service charge if you book online. Which is exactly what you expected to hear based on the headline, isn't it?

It almost reminds me of Orwell's 1984. First the weekly quotum for chocolate is 30 grams. So first they announce it is lowered to 15g, and the a week later, as if nothing happened, the "increase it" to 20g, to demonstrate how generous they are.

The latest from back home:

Yes, that's right, yet another fire. Trondheim chock full of wooden houses has been a feast to fires in the past couple of years. The biggest one destroyed a whole block:

The city decided to reinvent itself by putting up an ugly ass new building:

And that's just a prospect from the architects, believe me it looks crap up close. But the fun didn't stop there. A restaurant smack in the middle of town caught fire. The place is so central that it's practically a landmark.

I don't know if you shop at Hennes&Mauritz much, but their building burnt to the ground a few years ago:

And apparently, last year in April while I was here in Holland, yet another fire.

Come see Trondheim, a city that enjoys a good bonfire. Here's a full timeline, but unfortunately doesn't list upcoming events. That's just the big ones, there are bound to be plenty of smaller fires as well, like one covered in the past.

On the upshot, I don't think any people were hurt in any of these fires. "But", you say, "that's good, Trondheim center is packed with these old wooden houses that are all the same and the city badly needs urban development. At least this way, new buildings can come up." Well, that's true, but it would be nice if this could come about through controlled change and some proper innovative thinking, not clumsy accidents followed by damage control. As it is, the city is hell bent on preserving these "cultural treasures" instead of actually building something that is worthwhile. Conservativism to the fullest. The cathedral is one of those landmarks that really stand for something, the rest of the city center is practically expendable architecturally. Well, it's little old Trondheim, with all its close minded provincial mindset, don't expect urban progress.

Oh, and happy Constitution Day.

Normally I'd just read these political articles and move on, but one that I found today is so succinct and logical that I think it deserves a note in the margin. Incompetent Design, the premise of how Bush and his buddies manage to make their dealings seem incompetent whereby they're point by point achieving exactly what they want to achieve.

It's a sobering perspective on the situation, certainly similar views have been expressed in the past, but I haven't come across a theorem equally elegant, logical and complete.

It's interesting how the assumption of "things aren't going the way we want them to" has been made into fact without actually being established anywhere. Meanwhile, sock puppets and figureheads like Bush, Rumsfeld, Gonzales and Rice come out and say the most outlandish things and the only response is mockery. Fine way to deflect real reactions. Underestimating idiots sure can be dangerous.

As with a lot of things, the theory of music intervals is complicated enough to require a little bit of maturity. The division of the octave into different tunings has evolved with the times, and while the concept is simple, there are many nuances.

An easy way to illustrate is by observing the piano keyboard. The most fundamental scale is C-D-E-F-G-A-B-C, and that's an octave, of course. Each of the scales (or collections) built on these successive pitches has its own fancy Greek name, so if you want to understand what the heck people are saying, you better know these as well.

The problem is a typical memorization exercise, tie 7 different pitches to 7 Greek names. There is no obvious way to do this, match one set of numbers with another set of names. Except when you're a football supporter. We can memorize lineups and shirt numbers like nobody's business, linking them to faces and positions on the pitch. So why not leverage this ability.

There are 7 players, so we're going to play a 2-2-2 formation. The numbering is standard, ascending from right to left. C is our fundamental pitch, so that's where we start counting.

You're going to look at this formation, and the next time someone mentions Lydian mode you're going to think "Lydian... right midfield, shirt number 4, so that's... C, D, E, F. The mode is then F-G-A-B-C-D-E-F".

If only we could get this team into Football Manager, everyone would know this.