Open source projects understandably face a lot of challenges. One of those challenges is to communicate effectively with the user base. It is my claim that many of the common channels of communication are ineffective.

If you're a developer, put yourself in a user's place. Users have questions. Sometimes they have requests. Some projects go the extra mile to establish a channel for quick, uncommitted communication. But most projects don't do this, they rely on the classical communication channels.

Mailing lists are the way most projects exchange ideas between developers. They are archived, which makes them good for keeping track of what has been covered. But in order to post a single question to a mailing list, you first have to subscribe to it. Think about this for a minute. You're not a developer, you just want to ask one question, but in order to do that, you have to commit yourself to receive all the emails exchanged on the list. I have faced this problem countless times. Currently I'm on the cmake list, because I needed to ask a question about a month ago and I thought why not just remain there in case I have something else to ask in the near future.

This is completely pointless. Even if I just want to ask one question, I have to subscribe to the list, validate my email address, sometimes even await moderator approval (for some reason or another, joining a mailing lists always seems to take half a day between subscribing and being subscribed), post the question, wait for the answer, receive all kinds of irrelevant emails, see if I'm satisfied or I ask a follow-up, get more irrelevant emails, then finally unsubscribe. In the past couple of years, I've been through the subscribe-unsubscribe loop at least 20 times, usually with no lasting interest in the project. Not that it's a big problem, my gmail account is basically unlimited storage, and with the filters I don't have to look at these emails at all (which I don't). But cmake's mail server still keeps sending them even though I don't want them. It's a waste.

Naturally, there are reasons why it works like this. Lists have to protect themselves from spammers and trolls, but it doesn't make the user role any more fun. And even though mail archives are nice for developers to keep track of their ideas, more often than not they suck for finding information. A lot of mail archives don't have a search function. Of course, that begs the question why even have an archive then? In one or two cases I actually had to download all the monthly archives and grep them locally. And that's when the archive actually was downloadable.

Another common channel is irc. Great for irc people, less amazing for the rest of us. Irc demands that you find yourself at the channel at same time that the person who can answer your question is also there. If not, you can hang around all day, and pay attention to the channel, so that you don't miss it when the person is there and can help you. Some people probably have that kind of patience, but I don't.

Again, irc can be archived, but it's a very noisy medium. In between actual technical questions, there tends to be a lot of chatter. Nothing wrong with that, but it makes it less than ideal for effective communication.

An increasingly popular channel is the forum. Forums work really well, if they get enough traffic. Unlike irc, they aren't real time so it doesn't matter when you're there. And they are far more organized than mailing lists too. Again unlike mailing list, the sign-up procedure tends to be speedy. And again unlike mailing lists, it's non-committed: you can post a question today, get the answer, and never visit the website again. Your account remains, but there's no stream of emails coming after you. And forums are searchable.

Forums have their own problems. If they aren't sufficiently active to get your questions answered, they are useless. And they require more moderation too.

A non-interactive, yet still highly effective channel is the wiki. Wikis are fantastic for ad hoc, unofficial information. And their informal, editing-oriented feel encourage updates. Superb for all kinds of tips and frequently asked questions.

But, unless unrestricted (and thus open to spam), not useful for informal interaction.

This isn't to say that existing, well established channels are obsolete. Gmane certainly gives you more out of mailing lists, with a threaded view and searching. But the fact remains that many of the tools in use today are not optimal, and in some cases, very ineffective.

I didn't mention bug trackers, because they are quite effective at what they set out to do, and because they aren't quite general purpose either.

I get the feeling that pet owners just don't get enough of human stupidity. They crave more. So what better than to own an animal that's dumber than any human that ever lived.

Personally I feel humans are stupid enough. Don't get me wrong, we have the capacity to act intelligently, but all too often we have bad judgment or we just decide to be stupid. Granted, that's a little sad. But I find that human stupidity, my own included, is already filling my need for unintelligent behavior. I don't need an animal to supply me with a constant source of dumbness.

Pet owners seem to have a different view on this. In fact, they seem altogether thrilled with any kind of accomplishment an animal can aspire to, however simple. Omg I threw the stick and you brought it back to me. And we only rehearsed this 47 times. I mean how long would it take you to train your kid to do this? Try it once and your job is done. And what's so amazing about fetching thrown objects? Is this some kind of useful skill that society is in desperate need of? Or the whole dog training program. Teach it to sit up, to roll around, to fetch, to behave. Wooow, your dog is soo smart. What kind of low threshold for achievement do you have to be impressed by these things? Parents send their kids to school to be engineers, accountants, lawyers, advertisers, realtors etc. But when the dog claws on the door when you get home it's omg my dog is sooo clever, it missed me. Of course it did, you *feed* it. Oh my cat it so smart, all it does is sit there all day.

And what's so great about being around pets anyway? They're dirty, smelly, noisy, unruly, unpredictable (in spite of those top shelf training videos) and just plain ugly too, much of the time. And not a shred of intelligence. You know a park is a place where you want to take your kids and just let them play, it's one of the few safe places for them to be if you live in a city, a place with no traffic. A park is a place where you're supposed to be able to run around, play football with dad, roll around in the grass and do whatever you want to do, with impunity. But then dog owners think it's a great idea to take the dog for a walk in the park, and suddenly it's transformed into a minefield. Now you have to watch your every step. Great way to ruin it for everyone. In Holland they even make it explicit, with signs that mark this area is a dog toilet.

I think pet owners actually wish they lived on a farm, but since they don't they want pets. It's like a bad compromise, a pet in the city instead of animals on a farm.

Animals are nice as mascots, as characters in children's shows, as teddy bears, as illustrations in books. Cuddly, cute, compassionate, thoughtful, clean, and loving. We all like good fiction.

If you want a living thing to love, to talk to, to take care of, get a plant. Or take that plunge and get a human. I hear kids are great.

If you've not had the pleasure of sampling authentic Dutch bread, let me tell you that you are in for an experience you didn't see coming.

If you're a conservative/romantic when it comes to bread, you probably slice your bread with a knife. This means that a) it's hard enough to be sliced and b) it's a loaf, not pre-sliced. Well, if you come to Holland, both of those terribly reasonable assumptions are unapologetically cast aside.

When you buy bread here, it's a soft sponge. And it's pre-sliced. In fact, natives are so terrified of bread that isn't sliced that if you actually find it in the supermarket, a rarity in itself, it has a big label saying not sliced. :scared: Yeah, take your chances, buddy.

This presents a real practical problem, because the bread gets deformed before you get home. If you apply the classic 7-loaves-to-a-plastic-bag formula, you'll notice that you can fit many more loaves into a bag, just like you can fill a bucket with untold amounts of cotton. But the result is that the bread doesn't restore its original form, as a sponge would. It's permanently deformed. So now instead of putting bread at the bottom, you have to handle it like the most fragile piece of fruit, silk gloves treatment.

But that's not all. The bread here just isn't very good. Although there is plenty of choice, it's a choice of mediocre products. Sure enough, they may appear to have all kinds of different flavors, and different grains, and embellished packaging, but when you actually try it, you often realize that the most fancy looking products have an aftertaste you can't stand. They also smell funny. On the whole, the bread gives the impression of being manufactured, rather than baked.

Predictably, it's not just the bread. A rich culture for bread products tends to show in all kinds of pastry. One that Holland apparently doesn't have. And the pastry here is truly bad. There really isn't anything you would buy to say serve to guests, or for a snack. For instance, you'll find croissants, but they're far from the real thing.

In Poland the bread is certainly a lot more authentic, although it isn't that great. But not this mushy spongy stuff. Meanwhile, the pastry is very good, there's lots of choice and tradition.

In Norway the bread culture is very high, and just about any bread you can find is better than anything you'll find around here. The pastry side of things is less prosperous, there are a few trademark traditional products, but not too many of them. Then there's a tradition of imported products like Wiener-rolls, which is a cornerstone in Norwegian pastry. Not native perhaps, but embraced as one's own.

The patient sustained multiple severe stab wounds to the upper chest and abdomen. The consumer didn't stop until he was able to retrieve the product.

They basically use this packaging for anything they can get away with. In some stores half the products are packaged this way, in what has to be the most consumer hostile packaging known to man. There are three big problems with it:

1. It's a huge pain to open, unless you have a mechanical lab on hand.
2. Sometimes it's almost impossible to open without damaging the product.
3. There is no semblance of putting it back together the way it was, you open it you bought it

I would take the cheapest, ugliest brown carton box with the technical specifications printed on the side over this plastic. And it would be cheaper. No pictures either, just open the box and see what it looks like. And if you don't want the customer to open the box for some reason, just seal it with duck tape. And leave one box open so they can check out the merchandise.

Why you should care

There are quite a few reasons why you would want to back-up your website. For one thing, in the case of some kind of security breach, you don't want to lose the files on the server. Even if someone broke in, with a backup you could just restore it and you'd be back in a jiff. Otherwise, maybe you just want full control of your files, and knowing that they sit on a server somewhere remote doesn't make you feel as good as knowing they are right on your local disk. Whatever the reason, the following method is well suited to Wordpress sites, but general enough to apply to just about any website.

However, the following method enables you to transfer files in both direction, it's equally ideal for deployment. It makes no difference if you're uploading or downloading, we cover both bases.

How it works

Okay, that was the sales pitch. The script was written to allow for fast deployment of files on a server. Using Wordpress as an example, if you're hacking on your theme and you want to upload that one file you changed and see the result, you can do that quickly and painlessly with rsync. It's really the best way to transfer one file when you know none of the other files have changed. rsync synchronizes two locations, transferring only what has changed.

The files are transferred with rsync over ssh, so you need shell access on the server for this.

In a typical example where you have an account on a web server, this is how your file structure is at the root level (your homedir):

$ ls ~
.bashrc
.htaccess
.ssh
bin/
etc/
mail/
public_ftp/
public_html/
=> cgi-bin/
=> images/
=> => picture.jpg
=> index.html
tmp/

The files in bold are the ones you want to synchronize with your local disk and keep up-to-date. But there will generally be a lot of other files you're not interested in, generated in your homedir automatically, like raw web traffic logs, mail spam etc. (If the item is a directory, you want all the files and dirs it contains to be synchronized.)

So the issue is to selectively pick the items you want. But there may also be certain types of files inside these dirs you don't want, like for instance I ignore cgi-bin. So you want a way to exclude certain files/dirs from being transferred.

How to

Now that you know what's happening, it's time to set it up. You fill in the variables at the top of the script. local_path is where you want the files on disk. remote_path is where they are located on the server (in most cases ~ or /home/username). locations is the list of top level directories/files you want to synchronize. And finally exclusions are patterns you want to exclude (so if it contains cgi-bin, then that directory and all the files in it will be excluded from the synchronization).

Once that's done, you just run

$ sync.sh down

to download the files on the server to your local dir, and

$ sync.sh up

to transfer your local changes to the website. Finally,

$ sync.sh

alone will log you into your server with ssh.

Time to synchronize full local/remote tree for matusiak.eu (5470 files) when no changes were made: 4.4 seconds. ;)

A small note about security

Note that this script does not violate or subvert how you access your server. It uses ssh as the underlying security context. You can easily synchronize up/down with public key authentication, in which case you'll never have to type in your password when running sync.sh, and it's actually more secure as well. :)

#!/bin/bash
#
# Author: Martin Matusiak <numerodix@gmail.com>
# Licensed under the GNU Public License, version 2.


# server setup
hostname="matusiak.eu"
username=""
ssh_port="22"

# local setup
local_path="/local/path"

# remote setup
remote_path="~"
locations="bin backups public_html"

exclusions="cgi-bin *.swp *~" #.swp are vim swap files


## EDIT BELOW THIS LINE IF YOU KNOW WHAT YOU'RE DOING

# rsync options
rsync_options="--archive --verbose --stats --progress"

# switch priority
nice="nice -n 10"


inc_list=""
function inclusion_list() {
	for i in $exclusions; do
		inc_list="${inc_list}--filter='- $i' "
	done
	for i in $locations; do
		inc_list="${inc_list}--filter='+ /$i' "
	done
	inc_list="${inc_list} --filter='- /*'"
}

function shell() {
	 ssh -C ${username}@${hostname} -p ${ssh_port}
}

function sync_up() {
	inclusion_list
	cmd="${nice} rsync ${rsync_options} -e \"ssh -p ${ssh_port}\" \
	${inc_list} \
	${local_path}/* \
	${username}@${hostname}:${remote_path} "
	echo "$cmd"
	sh -c "$cmd"
}

function sync_down() {
	inclusion_list
	mkdir -p ${local_path}
	cmd="${nice} rsync ${rsync_options} -e \"ssh -p ${ssh_port}\" \
	${inc_list} \
	${username}@${hostname}:${remote_path}/* \
	${local_path} "
	echo "$cmd"
	sh -c "$cmd"
}


if [ -z "$1" ]; then
	shell
elif [ "$1" = "down" ]; then
	sync_down
elif [ "$1" = "up" ]; then
	sync_up
else
	echo "$0 [down|up]"	
fi