(Ouch! Looks like WordPress update to 3.1.3 wiped all the modifications I made to the default theme. Admittedly I should've seen that coming.)
What I want to do is basically attach a key-value pair to an EC2 instance when launching it in AWS Management Console and read the value inside the instance when it's running. To be more specific, I use this to to set a key called environment that can have values like dev, stage and prod so that the Django config can decide which database to connect to etc. while starting up. I suspect that in Boto the current instance can somehow be referenced in a more direct fashion but this works as well.
First, append the following to /etc/profile:
# See: http://stackoverflow.com/questions/625644/find-out-the-instance-id-from-within-an-ec2-machine
export EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id || die \"wget instance-id has failed: $?\"`"
test -n "$EC2_INSTANCE_ID" || die 'cannot obtain instance-id'
export EC2_AVAIL_ZONE="`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone || die \"wget availability-zone has failed: $?\"`"
test -n "$EC2_AVAIL_ZONE" || die 'cannot obtain availability-zone'
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\\([0-9][0-9]*\\)[a-z]*\\$:\\\\1:'`"
Now we know the region and instance ID. Next, install Boto by running the following commands:
wget "http://boto.googlecode.com/files/boto-2.0b4.tar.gz"
zcat boto-2.0b4.tar.gz | tar xfv -
cd boto-2.0b4
python ./setup.py install
#!/usr/bin/env python
import os
from boto import ec2
ec2_instance_id = os.environ.get('EC2_INSTANCE_ID')
ec2_region = os.environ.get('EC2_REGION')
conn = ec2.connect_to_region(ec2_region)
reservations = conn.get_all_instances()
instances = [i for r in reservations for i in r.instances]
for instance in instances:
if instance.__dict__['id'] == ec2_instance_id:
print instance.__dict__['tags']['environment']
Messages sent by Amazon Simple Notification Service are signed, and checking that any received message is indeed from AWS and not from some douche trying to outsmart you is not very hard (nor should it be optional, for that matter):
sns-verify.php
The verify_sns() function expects the message in JSON format, plus region (e.g. "eu-west-1"), numerical account ID without dashes and an array containing the topics you're interested in. The code will verify both SubscriptionConfirmation and Notification messages. It loads the certificate from the address in SigningCertURL field to check against for each message separately because the certificate changes over time, as described here. It is also checked that the host where the certificate is loaded from is in the amazonaws.com domain.
Example usage where subscriptions are automatically confirmed:
Plan to use an in-house box to run a XenServer to host XP instances (I need multiple Windows desktops for "testing" purposes if anyone asks) had to be scrapped because the box was simply too loud and I couldn't get the wireless bridge to work – not that the latter would have helped anyway because like I said the box really is loud and relocating it anywhere inside our flat just wouldn't lower the noise level enough for it to not disturb sleep.
Which brings us here: launching a Windows Server 2008 R2 instance on Amazon EC2 and setting up Remote Desktop Services to enable multiple simultaneous client sessions. Below we can see Alice, Bob, Charlie and Dave each happily running their own Remote Desktop session at the same time:
The whole thing runs "tolerably" smoothly even on the severely memory-limited Micro Instace:
At $0.035 per hour this can be considered cheap. And, the server can be shut down when it's not needed in which case the only charge will be for the admittedly humonguos (35 gigabytes) Windows root partition. And of course those clients would need Client Access Licenses which adds a one time cost of roughly $100 per client. Now, to directly compare this kind of setup with having an actual physical server would indicate poor judgement as both have their strong and weak points but costs can be compared. So here we have an estimate of what the total cost of running a server like this for a three-year period would be, sans CALs:
On-Demand EC2
Reserved EC2
(1-year Contract)
Reserved EC2
(3-year Contract)
One-time costs
$0.00
$54.00
$82.00
Compute
$922.32
$421.56
$421.56
Storage (35 GB)
$138.60
$138.60
$138.60
I/O (10 IOPS)
$103.00
$103.00
$103.00
Transfer In (1 GB/m)
$3.60
$3.60
$3.60
Transfer Out (10 GB/m)
$48.60
$48.60
$48.60
Total Cost (Euros)
849.69 €
613.00 €
557.11 €
Per Month (Euros)
23.60 €
17.03 €
15.48 €
Source: http://calculator.s3.amazonaws.com/calc5.html
Then again, that 600 € would get you two HP Proliant MicroServers. Yet, then again, that price does not include Windows licenses and they would need a physical location, electricity, an Internet connection – an so on.
Small AP is small – and has a built-in antenna, too. I got two of these (for 19,90€ per piece – not A-link list price…) and set up a bridge so I could relocate my noisy Xen box from living room to kitchen to keep the box running 24/7 and sleep. (Turns out that in the end even this didn't help because the box remained loud enough to disturb sleep no matter what settings were selected in BIOS thermal management.) Initially it looked like the bridge worked just fine, except my testing revealed that the transmission speed was nowhere near the advertised "IEEE 802.11n (draft 2.0) / 150Mb":
XenCenter.iso 100% 44MB 1.5MB/s 00:29
After trying different cryptos from WPA2 to plain text and fiddling with various other settings I came to the conclusion that the slow speed was a feature of the device. Anyway, this was not really any kind of concern as I was more interested in latency, which was low enough (a few milliseconds). Put all this together and my opinion is that it's good enough for an access point that is about the size of a deck of cards and costs twenty euros.
What did turn out to be a problem is that at times the AP's would somehow manage get a broadcast storm going on, which of course took the wired network down with it very quickly. I wasn't really able to get to the root of this but from what I observed I can tell that the broadcast storm would happen even when one AP was connected to the primary wired segment and the AP at the other end was just "floating" there, with nothing connected to its' Ethernet ports. Also, while after enabling STP in the devices I could, using tcpdump, observe the STP config packets doing their thing and reconfiguring after for example dropping and then reconnecting either end of the bridge, this (STP) did nothing to prevent the broadcast storm from happening. I should also note for the record that I was using the "WDS", not "AP+WDS" mode.
Verdict: the devices just aren't suitable for this application, i.e. they are buggy and do not fully work as advertised but given their relatively compact size and ability to function as clients on a WLAN, I'll keep these.
While looking for a virtualization solution in order to make computational matters more flexible, efficient and manageable (et cetera, et cetera) here at home, various offerings that are listed below were tested. To be honest, to say that I "tested" these would be twisting the truth quite a bit as the methodology used was not very scientific and things were guided more by hunch than strict reason. But then again as I would be the only one who would get hurt if things went horribly wrong it wouldn't really matter that much if the "wrong" solution was chosen. So far, after a few days, it's looking like the choice I made was right. The following took part in our non-scientific non-review:
At first Eucalyptus sounded like an awesome choise, given for example its Amazon EC2 API compatibility, but in practice it turned out that while the idea of having a private cloud at one's disposal is great, having this much flexibility brings with it a much higher level of complexity in managing the system, which pretty much makes the whole idea of having a cloud a moot point. And as I have just one host machine, running Eucalytus wasn't as straightforward as it could be. And also, what I'm really looking for is virtualization of a couple of servers that I like to have around, not a pool of cloud computing resources which can these days be bought at very reasonable rates (or for free, even) if needed. Nice offering, though, which I bet we will see gaining more and more ground in the future. Oddly, I'm unable to find a single service provider offering a service similar to EC2, but built on top of Eucalyptus. Perhaps the tools that would facilitate selling an Eucalyptus-based cloud service do not yet exist?
As for Parallels, I think it's debatable just how "bare metal" their hypervisor really is. It may be so that I have let myself be enchanted by marketers to believe that this bare metal thing is something radically different. The other possibility is that Parallels themselves are bending the meaning of the term here and are selling their system as "bare metal" when in fact it's not that bare. At least to me it looked like a full host operating system was installed and I can't see how this makes things that much different from having a regular server and running the hypervisor on it. Of course one difference is that you don't need for example a Windows server license to run the software but there's still a regular operating system involved that's running the show. Don't get me wrong, I use Parallels products almost on a daily basis (for example the illustration image on top, of the XenCenter management tool, is running on Windows XP installation inside a Parallels Desktop for Mac virtual machine) and I have nothing against them, it's just that this personal experience I have with their server offering wasn't that super. Their management tools are cross-platform (all three Windows, Mac, Linux) which is a plus but they want $500 for a license per server which I'm not going to pay them. Also, Parallels Server could be considered somewhat obscure in comparison to the others so this may very well turn out to not be a good choice in the long run.
VMWare's offer just wouldn't work for some reason, perhaps my hardware was somehow incompatible. Or something, I don't know. VMWare being such a traditional virtualization house, this would've been the "correct" choice in a similar way to "No manager ever got fired for buying IBM". But as no one was going to fire me for whatever choice I made here I gave up and moved on.
Last on the table was XenServer from Citrix. I went with the default installation and just used one whole disk and let the installer set things up the default way, i.e. a few gigabytes for all the Xen stuff and the rest for LVM storage. Like rest, the system can be managed from command line (local console or over SSH) but as my primary aim here is to get things done and having a point and click interface makes learning curve that much less steep, I went and installed the XenCenter management console on a Windows XP virtual machine (which was of course not hosted on this machine).
Making a paravirtual Ubuntu guest did not require any kind of wizardy, I just followed steps 3–5 in Installing Ubuntu Server 10.04 (32bit and 64bit) LTS (steps 1 and 2 were not necessary as the Ubuntu 10.04 64-bit template was already there after a fresh install). After I had one machine set up I turned that in to a virtual machine template and using this template it's super-fast to start new servers when needed. Also, these (para)virtual machines don't seem to be taking much of a performance hit and all and all I'm really pleased with the results.
The only thing missing here is lm-sensors or something similar so that I could at least see CPU and motherboard temperatures of a running system but I suppose this can be arranged.
Update: A Windows XP guest, with paravirtual device drivers, was also easy enough to install. And following the instructions in chapter 3.4. "Preparing to clone a Windows VM", working with Redmond is greatly simplified as an XP template can be prepared and new virtual machines invoked on demand and disposed of after use – this way one doesn't have to worry if for example installing a software package for testing purposes will mess the system up somehow.
