I realised when auditing my delicious bookmarks recently how much I rely on Wikipedia to look things up and today donated for the first time.

I had previously moaned about seeing Jimmy Wales’s face every time I logged in, like this and laughed my head off at this piss take from The Daily What:

I found today at Information Is Beautiful the following demonstration of just how effective the campaign has been though.

Wikimedia have done some nice analysis of the campaign on the Meta Wiki if you are interested.

Give to Wikipedia here

I remembered seeing Simon Long’s comment on twitter a few weeks ago and it was rattling around in the back of my mind.

Will #VMware Load-Based Teaming remove the need for #Cisco EtherChannel? Discuss….

I long ago investigated NIC Teaming algorithms and settled on IP Hash with Cisco Etherchannels for most environments, only really using something else if the client happened not have stacked switches. Thanks to Scott Lowe for this superb article on the matter.

When vSphere 4.1 came out with Load Based Teaming, I was pleased that at last we had an algorithm that would have a go at proper load balancing and not just load distribution but had not got round to investigating much more.

At Forward we have just updated to 4.1, Enterprise Plus and have bought some shiny new Extreme Summit X650 Series 10G switches; so Simon’s comment was particularly apropos.

I had decided I wanted to try and use LBT but was unsure if I should port-channel the uplink ports. It turns out you can’t. I thought maybe you should to be honest, it does not mention in the dvSwitch guide as far as I can see but the ESX host requirements for link aggregation KB (updated today) is very clear

• The switch must be set to perform 802.3ad link aggregation in static mode ON and the virtual switch must have its load balancing method set to Route based on IP hash.

• Enabling either Route based on IP hash without 802.3ad aggregation or vice-versa disrupts networking

ie you need both IP Hash and EtherChannel and neither will work without the other.

In answer to Simon’s question, my feeling is you may still get better performance from EtherChannel and IP based hash for some workloads but would guess “usually” LBT wins. I think the case where you may get better utilisation is when certain VMs have very high bandwidth requirements to different IPs. As described here IP Hash is the only way to allow traffic from one vNIC to leave over different pNICs at the same time.

It is interesting that even with LBT bandwidth is still limited to the maximum bandwidth a single pNIC can provide for individual VMs / vmkernels, also IP hash will not get higher than a single pNIC for a vMotion or other point to point connections. So 10G is going to perform better for these operations than 10x1G, however you team them.

We recently bought the Hyper9 capacity planning, reporting and monitoring solution for our VMware infrastructure and I quite soon made use of it to troubleshoot some problems reported to us like backups taking longer and databases being slower than normal.

In the 3par storage I could see that IO was unusually high of late.

Then I looked at the top-n datastores by IOPS and graphed them A huge jump for sharedstorage8, so I looked at its VMs and found the culprit VM.

Here it is against our big “Superhero” database and the vCenter server with the DB.

A lot of IO from a machine the owner thought was doing nothing!

Hyper9 is a pretty good tool for reporting, alerting and troubleshooting your VMware infrastructure, the query language is lucene based and this gives you lots of options in creating custom views and alerts.

Here are two Wordle visualisations, first the words used in my blog. I like reading obviously but it seems to be quite heavy on stuff from the single article I wrote on Rhipe - unusual words I suppose.

The second is my del.icio.us tags, when is later going to happen?

With delicious closing I am looking at alternatives, send recommendations if you have any.

Well, I guess I need to say something about my company Christmas present to all of their ~150 staff, a three day trip to Vegas.

We flew out last Thursday and stayed for 3 nights at the Wynn, which is a great hotel.

Thursday: Arrive and sleep (forgive me I only just got back from Egypt after flying to Manchester instead of London and having to sit all night on a freezing cold coach!).

Friday: Flew in a helicopter into the Grand Canyon which was rather awesome.

Sat: Went to see Zumanity by Cirque du Soleil, it was amazing, I must see them again.

Sunday: A few of us went shooting, An M16

An H&K; MP5

A Mac-10 not firing it gangster style unfortunately.

A Tommygun

A Desert Eagle

And I did a Shotgun

And I’ve got the T-shirt to prove it Peace Through Superior Firepower indeed

Pics On Flickr

Vids On Youtube

I has been a crazy few weeks, after being in the UK only 2 days after the Egypt trip I went to Las Vegas with Forward and I am just getting my head back together now.

You know you are a huge geek when you take as much space for books as clothes.

The trip was ace, a week on the M/S Hamees, Stopping at: Valley of the Kings Temple of Queen Hatshepsut The Colossi of Memnon Edfu Temple Kom Ombo, which I had not heard of but has some extraordinarily vibrant original colour remaining Abu Simbel Karnak Temple Luxor Temple (we had the place to ourselves at night, it was amazing)

5 Nights in Movenpick El Gouna on the Red Sea and 2 nights in the Nile Palace

It was incredibly relaxing, nice to be incommunicado for 2 weeks and catch up on some reading and just mince around the beach. I managed to read Godel Escher Bach at last and also ticked off The Ode Less Travelled from my 101 Goals. It was weird not having my phone to distract me in spare moments, making me daydream more and think about people and events I have not thought about it ages, I should do it more often.

Pictures are on flickr

I am off to Egypt for 2 weeks, a one week Nile cruise with Voyages Joules Verne starting at Luxor and working down to the Aswan High Dam via The Valley Of The Kings and ending in the Moevenpick Hotel in El Gouna. No Cairo, Pyramids or the Cairo Museum this trip, another time though.

Im particularly excited about Abu Simbel,, The Red Chapel of Hatshepsut, Temple of Amenhotep III among tons of others.

Also I am very much looking forward to No Phone and No Computers for the whole trip, the longest I will have been without either for for a good few years. I am taking the chance to catch up on some reading and take a break, I will have my Kindle with loads of books on it but am taking The Dots-and-boxes Game: Sophisticated Child’s Play (one of my 101 goals is to make a dots and boxes program), Gödel, Escher, Bach (another 101, started 3 times and never finished and it has intrigued me since my first year in university) and Winning Ways for your Mathematical Plays. Pen’n’paper geekery for the win!

I spent a while this week getting Rhipe, a java package that integrates the R environment with Hadoop, to work. Forward are pretty heavy users of Hadoop and it’s supporting ecosystem so R will be another way for the devs to interrogate the huge (and rapidly growing!) datasets we have.

Installing R

Adding the repositry

Create a new file at /etc/sources.list.d/R.list

#R repositry
deb http://rh-mirror.linux.iastate.edu/CRAN/bin/linux/ubuntu hardy/

(we are still using hardy, with the Cloudera packages)

Add the gpg keys for the repository

gpg --keyserver pgp.mit.edu --recv-key E2A11821
gpg -a --export E2A11821 | sudo apt-key add -

Install and update R

Easy:

$ sudo apt-get install r-base r-base-dev pkg-config littler
$ sudo R
> update.packages()

Set environment variables for Rhipe

Add to bottom of /etc/environment

HADOOP=/usr

create it for current session

$ export HADOOP=/usr

install protobuff

# wget http://protobuf.googlecode.com/files/protobuf-2.3.0.tar.bz2
# tar jxf protobuf-2.3.0.tar.bz2
# cd protobuf-2.3.0
# ./configure
# make
# make install
# ldconfig

install Rhipe

# wget http://www.stat.purdue.edu/~sguha/rhipe/dn/Rhipe_0.64.tar.gz
# R CMD INSTALL Rhipe_0.64.tar.gz

So all is well except that the test code here is a bit off.

For me today

> library(Rhipe)

Only works as root. It seems that

> rhwrite(list(1,2,3),"/tmp/x")

should be:

> rhwrite(list(1,2,3),"/tmp/x",1)

then

> rhread("/tmp/x")

works properly.

Also in the longer example

map <- expression({
  lapply(seq_along(map.values),function(r){
    x <- runif(map.values[[r]])
    rhcollect(map.keys[[r]],c(n=map.values[[r]],mean=mean(x),sd=sd(x)))
  })
})

## Create a job object
z <- rhmr(map, ofolder="/tmp/test", inout=c('lapply','sequence'),
          N=10,mapred=list(mapred.reduce.tasks=0),jobname='test')

## Submit the job
rhex(z)

## Read the results
res <- rhread('/tmp/test/p*')
colres  <- do.call('rbind', lapply(res,"[[",2))

colres
       n      mean        sd
 [1,]  1 0.4983786        NA
 [2,]  2 0.7683017 0.2937688
 [3,]  3 0.5936899 0.3425441
 [4,]  4 0.3699087 0.2666379
 [5,]  5 0.5179839 0.4060244
 [6,]  6 0.6278925 0.2952608
 [7,]  7 0.4920088 0.2785893
 [8,]  8 0.4592598 0.2674592
 [9,]  9 0.5734197 0.1928496
[10,] 10 0.4942676 0.2989538

Where line 16 has been changed from the original

res <- rhread('/tmp/test')

Thanks to Saptarshi Guha, the author of Rhipe for so quickly responding to my query in the group and also the authors of this discussion on setting up R in Ubuntu

I was reading SICP over lunch and found this lovely footnote on probabilistic methods for deciding if a number is prime. (it is #47)

Numbers that fool the Fermat test are called Carmichael numbers, and little is known about them other than that they are extremely rare. There are 255 Carmichael numbers below 100,000,000. The smallest few are 561, 1105, 1729, 2465, 2821, and 6601. In testing primality of very large numbers chosen at random, the chance of stumbling upon a value that fools the Fermat test is less than the chance that cosmic radiation will cause the computer to make an error in carrying out a “correct’’ algorithm. Considering an algorithm to be inadequate for the first reason but not for the second illustrates the difference between mathematics and engineering.

Well, it is day 300 of my 101 goals in 1001 days. So as per meta-goal 101 “Do 100 day updates” here is a quick report on my progress.

Completed 66 - Via Feratta in Italy 85 - Visit Pergamon Museum 1 - Teetotalitarianism for 3 months 2 - Cheeseless for 3 months 11 -Reread all Dennett books 15 -Proofread for Project Guttenburg 53 - Make Jam 86 - Give Carrie a British Museum Tour 48 - Create a Backblaze storage pod 100 Set success criteria / progression metrics for each goal

Changing 18 - Read epic literature A bit vague (in spite of meta-goal 100) and I have loads of reading goals so im changing it to “Read Joyce”, in particular:

• Dubliners

• A Portrait of the Artist as a Young Man

• Ulysses

• Finnegans Wake

26 -Scuba My ears are bad, I worry this may destroy them, not settled on a replacement goal yet.

Sustained Effort - On Track 19 - Blog on average once a week 50 - Move 10 people to FreeAgent 68 - Complete Pimsleur Spanish 88 - Go to the theatre on average once a month 95 - Pay off all credit cards 96 - Let loans run course and dont get any more 101 - Do 100 day updates 5 - Lose 2 stone 13 - Release 303 books on bookcrossing.com 76 -Do on average 1 Project Euler problem per week

Sustained Effort - Behind 8 - Read all the VSIs 12 - Read all PG Wodehouse 60 - Hike on average once a month 79 - Raise £5005 for charity 81 - Watch all TTC Art history DVDs 90 - See all world heritage sites in the UK

In Progress 91 - Memorise 10 poems 92 - Read “An Ode Less Travelled”, do the exercises (but not share them!) 72 - Read “Winning Ways” 75 - Watch SICP, do exercises from book