I really enjoy the work of Jorge Luis Borges. If you have not heard of him, check out In Our Time - Borges.

I found some audio of some wonderful lectures on poetry he gave at Harvard in the late 60s. The interesting thing about them is that he composes them from memory as he was blind by that point. He demonstrates the breadth of his reading with examples from Middle German, Old Norse, Spanish and English, tracing down the sources of some of the quotes was quite a challenge as they come thick and fast during the lectures. I have listened to them a lot and he has become like an old friend, I can hear his voice when I read his essays or stories.

My two favorite quotes are:

Awake! for Morning in the Bowl of Night
Has flung the Stone that puts the Stars to Flight:
And Lo! the Hunter of the East has caught
The Sultan’s Turret in a Noose of Light.

Dreaming when Dawn’s Left Hand was in the Sky
I heard a Voice within the Tavern cry,
“Awake, my Little ones, and fill the Cup
Before Life’s Liquor in its Cup be dry.”

From The Rubaiyat Of Omar Khayyam, translated by Edward Fitzgerald and

Great wine like blood from Burgundy,
Cloaks like the clouds from Tyre,
And marble like solid moonlight,
And gold like frozen fire.

Smells that a man might swill in a cup,
Stones that a man might eat,
And the great smooth women like ivory
That the Turks sell in the street.”

from G. K. Chesterton’s The Ballad Of The White Horse (which was actually the first poem of this length I have ever read, I got it on my kindle from Project Gutenberg)

I am reading through SICP with some of the guys in work and got to a bit that fascinated me and reminded me of my previous life as a mathematician.

The basic data structure in Scheme is the cons cell, essentially representing a ordered pair. So

> (cons 2 3)
(2 . 3)

creates the pair (2 . 3)

Lets call that a,

> (define a (cons 2 3))

There are 2 other functions that act on these cons cells, car returns the first, cdr returns the second

> (car a)
2

> (cdr a)
3

You can build lists by chaining these cons cells and unpick the contents by using car and cdr

> (define b (cons 3 a))
> b
(3 2 . 3)

> (car b)
3

> (cdr b)
(2 . 3)

> (cdr (cdr b))
3

The interesting bit is at p91 of the book where it breaks down the distinction between procedures and data

(define (cons x y)
  (define (dispatch m)
    (cond ((= m 0) x)
          ((= m 1) y)
          (else (error "arg no 0 or 1" m))))
  dispatch)

So here cons is a function of x and y as you would expect. The trick is that its return value is a function (internally called dispatch) of one argument that returns x if passed 0 and y if passed 1, otherwise it throws an error.

Now car and cdr can be defined

(define (car z) (z 0))

(define (cdr z) (z 1))

So car takes z as an argument and tries to apply z to 0, while cdr applys it to 1. While car and cdr will accept any function as an argument it makes sense if z is the cons of something as before.

As an example

(car (cons 1 2))

car will pass 0 to (cons 1 2), which returns 1, similarly

(cdr (cons 1 2))

returns 2 as cdr will pass 1 to (cons 1 2)

This is not how the interpreter actually works, just a nice example of how the special forms don’t have to be all that special.

Now for twice, it reminded me of some Set Theory, in particular forming the Natual Numbers in terms of sets.

We define 0 to be the empty set $0 = {\emptyset}$.

Then we can define

(this has 1 element - we say has cardinality 1)

Then

Notice this has cardinality 2 and is formed as the set containing all the previous numbers.

So we have all positive whole numbers. You can then define arithmetic and number theory in terms of these sets.

See this site for a nice introduction to set theory (it also begins by defining ordered pairs)

Loving SICP, got to do some Induction for the first time in ages last night!

A few weeks ago I went to see Krapp’s Last Tape at the Duchess Theatre in London with some friends. I use “with” in a weak sense as as ever I was running late and had to get a jog on to arrive something like on time, just arriving as it started and pleading to be let in. I came in, trying not to pant from running, to a very oppressive silence; the entire audience watching Michael Gambon slumped in a chair and just sat at the first free seat I saw. The Silence went on for quite some time and really built up a lot of tension, so much in me that I did not take my coat or jacket off throughout the entire performance. I loved the premise: a (senile?) old man listening to tapes of his younger self that he recorded on his birthday, as there is only one actor he is engaged in dialogue with himself via the tapes he listens to. I wont pretend to have any more insight than the Wikipedia article I’ve already linked to but the other thing that resonated with me was the shifting accent of Krapp, sometimes sounding quite RP, sometimes Irish and sometimes just a lilt. A wonderful production, go see it if you can.

I was looking at this article (The Genuine Sieve of Eratosthenes / PDF) about a common functional prime number generator that is mistakenly called the Sieve of Eratosthenes.

It is quite complicated, section 2 deals with the performance of the naive algorithm and 3 implements the sieve properly. I could not quite grok the Haskell but the following quote helped me know what was going on

Whereas the original algorithm crosses off all multiples of a prime at once, we perform these “crossings off” in a lazier way: crossing off just-in-time. For this purpose, we will store a table in which, for each prime p that we have discovered so far, there is an “iterator” holding the next multiple of p to cross off. Thus, instead of crossing off all the multiples of, say, 17, at once (impossible, since there are infinitely many for our limit-free algorithm), we will store the first one (at 17 × 17; i.e., 289) in our table of upcoming composite numbers. When we come to consider whether 289 is prime, we will check our composites table and discover that it is a known composite with 17 as a factor, remove 289 from the table, and insert 306 (i.e., 289+17). In essence, we are storing “iterators” in a table keyed by the current value of each iterator.

Here it is using a dictionary for each composite number with a list of increments it was reached by:

from itertools import count

def gen_primes():
    yield 2
    def update_composites(number,increment):
        try:
            composites[number] += [increment]
        except:
            composites[number] = [increment]

    composites = {4:[2]}

    c = count(3)
    while 1:
        #print composites
        next = c.next()
        smallest = min(composites.keys())
        incs = composites[smallest]
        del composites[smallest]
        for inc in incs:
            update_composites(smallest+inc,inc)
        if next < smallest:
            yield next
            c.next()
            update_composites(next**2,next)
                        
g=gen_primes()
    
for i in range(100):
    print g.next()

(On Github)

They mention a speedup moving to a heapqueue as the data structure (as we just take the lowest each lookup) so I implemented that too, here is the code

from itertools import count
from heapq import heappush,heappop

def gen_primes():
    yield 2
    
    def update_composites(number,increment):
        heappush(composites,(number,increment))
    composites = [(4,2)]

    c = count(3)
    
    while 1:
        #print composites
        smallest,increment = heappop(composites)
        update_composites(smallest+increment,increment)
        if composites[0][0] == smallest:
            continue
        next = c.next()
        if next < smallest:
            yield next
            c.next()
            update_composites(next**2,next)

g=gen_primes()
    
for i in range(100):
    print g.next() 

(On github)

I have been meaning to learn ruby for a while and the place I am working now uses a lot so I had another look at it. I read Learn To Program, a simple but good book and found the bit on blocks and procs etc pretty good and wanted to see if I could do the stuff in Python as well. Python has anonymous “lambda” functions but they are limited to one line a subset of the syntax which is a bit annoying sometimes. My worry with methods in Ruby is that they are not first class, I think because you can omit parenthesis and so you have no way of referring to them without invoking them.

I remembered this while reading the SICP book, the question was about the difference between this program in applicative and normal order evaluation

(define (p) (p))

(define (test x y)
  (if (= x 0)
      0
      y))

It rang a bell as (define (p) p) does not go into an infinite loop if you invoke p. In lisp (p) calls the procedure p with no arguments whereas p is just a reference to the function. In python someinstance.method refers to the method, someinstance.method() calls it, Ruby seems to need Proc objects to get around this (IMHO as a beginner)

I redid all the examples from the book in Python

Eg 1

Ruby

def maybe_do some_proc 
  if rand(2) == 0
    some_proc.call
  end 
end

def twice_do some_proc 
  some_proc.call 
  some_proc.call
end

wink = Proc.new do 
  puts '<wink>'
end

glance = Proc.new do 
  puts '<glance>'
end

Python

import random

def maybe_do(some_proc):
    if random.choice(range(2)) == 0:
        some_proc()

def twice_do(some_proc):
    some_proc()
    some_proc()

def wink():
    print 'wink'

def glance():
    print 'glance'

for i in range(5):
    print 'running for i=',i
    maybe_do(wink)

Eg2

Ruby

def do_until_false first_input, some_proc 
  input = first_input 
  output = first_input
  while output 
    input = output 
    output = some_proc.call input
  end
  input
end

build_array_of_squares = Proc.new do |array| 
  last_number = array.last 
  if last_number <= 0
    false 
  else
    # Take off the last number...
    array.pop
    # ...and replace it with its square...
    array.push last_number*last_number
    # ...followed by the next smaller number.
    array.push last_number-1
  end 
end

always_false = Proc.new do |just_ignore_me| 
  false
end

puts do_until_false([5], build_array_of_squares).inspect

yum = 'lemonade with a hint of orange blossom water' 
puts do_until_false(yum, always_false)

Python

def do_untill_false(first_input, some_proc):
    input = first_input
    output = first_input
    while output:
        input = output
        output = some_proc(input)
    return input

def build_array_of_squares(array):
    last_number = array.pop()
    if last_number <= 0:
        return False
    else:
        array.append(last_number * last_number)
        array.append(last_number - 1)
        return array

def always_false(just_ignore_me):
    return False

def just_ignore_me():
    pass

print do_untill_false([5], build_array_of_squares)
yum = 'lemonade with a hint of orange blossom water'
print do_untill_false(yum, always_false)

Eg3

Ruby

def compose proc1, proc2 
  Proc.new do |x|
    proc2.call(proc1.call(x))
  end 
end

square_it = Proc.new do |x| 
  x*x
end

double_it = Proc.new do |x| 
  x+x
end

double_then_square = compose double_it, square_it 

square_then_double = compose square_it, double_it

puts double_then_square.call(5) puts square_then_double.call(5)

Python

def compose(proc1,proc2):
    def composed(x):
        return proc2(proc1(x))
    return composed

def square_it(x):
    return x**2

def double_it(x):
    return x*2

double_then_square = compose(double_it,square_it)
square_then_double = compose(square_it,double_it)

print double_then_square(5)
print square_then_double(5)

Eg4

class Array
  def each_even(&was_a_block__now_a_proc) 
    # We start with "true" because 
    # arrays start with 0, which is even. 
    is_even = true
    self.each do |object| 
      if is_even
        was_a_block__now_a_proc.call object
      end
      # Toggle from even to odd, or odd to even.
      is_even = !is_even
    end 
  end
end

fruits = ['apple', 'bad apple', 'cherry', 'durian'] 
fruits.each_even do |fruit|
  puts "Yum! I just love #{fruit} pies, don't you?" 
end

[1, 2, 3, 4, 5].each_even do |odd_ball|
  puts "#{odd_ball} is NOT an even number!" 
end

Python

class MyArray(list):
    def each_even(self):
        for i in range(len(self)):
            if i % 2 == 0:
                yield self[i]

fruits = MyArray(['apple', 'bad apple', 'cherry', 'durian'])

for fruit in fruits.each_even():
    print 'yum! I love %s pies, dont you?' % fruit

for odd_ball in MyArray([1,2,3,4,5]).each_even():
    print '%s is NOT an even number' % odd_ball

Eg5

Ruby

def profile block_description, &block 
  start_time = Time.new 
  block.call 
  duration = Time.new - start_time 
  puts "#{block_description}: #{duration} seconds"
end

profile '25000 doublings' do 
  number = 1
  25000.times do 
    number = number + number
  end

  puts "#{number.to_s.length} digits"
  # That's the number of digits in this HUGE number.
end

profile 'count to a million' do 
  number = 0 1000000.times do
    number = number + 1
  end 
end

Python

def profile(description, function):
    import time
    start_time = time.time()
    function()
    duration = time.time() - start_time
    print '%s: %s seconds' % (description, duration)
    print function.__name__
    print 'see, "function.__name__" can be used in place of description in python'

def count_to_a_million():
    number = 0
    for i in range(1000000):
        number = number+1

profile('count to a million', count_to_a_million)

def profiled(function):
    def new_function(*args, **kwargs):
        import time 
        start_time = time.time()
        result = function(*args, **kwargs)
        print function.__name__, 'took', time.time() - start_time, 'secs'
        return result
    return new_function

@profiled
def count_to_a_million_again():
    number = 0
    for i in range(1000000):
        number = number + 1

count_to_a_million_again()

This uses decorators, a nice Python feature that uses higher order functions (and the fact functions are first class in python).

In Conclusion

IMHO, at this point in my experience of Ruby, with all the disclaimers about my non expert status etc. Like:

• No restriction on complexity of anonymous functions

Dont Like:

• Methods being different from Procs/Blocs, non-uniform syntax
• Leaving out parenthesis (though I await DSL goodness later!)
• “end” everywhere (I know the indentation thing in python is contentious!)

Here is a little trick I keep re-using to output csv files of data from vCenter.

[powershell]$reportArray = @()

foreach($cluster in get-cluster){ foreach($vm in get-vm -Location $cluster){ foreach($ds in get-DataStore -VM $vm.name){ $reportArray += New-Object PSObject -Property @{ ClusterName = $cluster.Name VMName = $vm.Name DatastoreName = $ds.name FreeSpaceMB = $ds.FreeSpaceMB CapacityMB = $ds.CapacityMB } echo $cluster.name $vm.name $ds.name $ds.FreeSpaceMB $ds.CapacityMB } } }

The important bit is [powershell]
$reportArray += New-Object PSObject -Property @{ ClusterName = $cluster.Name VMName = $vm.Name DatastoreName = $ds.name FreeSpaceMB = $ds.FreeSpaceMB CapacityMB = $ds.CapacityMB } [/powershell] I cant remember where I saw it but it creates an object with properties with the information I want to use that I can then sort, filter and send places.

I used it again today: [powershell] $reportArray = @()

Get-VM | % {$vmguest = Get-VMGuest -VM $_ ; $reportArray += New-Object PSObject -Property @{ VMName = $.name GuestOS = $vmguest.OSFullName IPAddress = [string]::Join(“ “, $vmguest.IPAddress) HOSTNAME = $.guest.hostname } }

$reportArray | Out-GridView [/powershell] Using the lovely Out-GridView commandlet

UPDATE Turns out you need this update for the New-Object PSObject -Property bit to work.

Last weekend Petra and I went to Berlin, it is a great city. She arranged the trip for my birthday because I am always banging on about the Pergamon Museum and the Ishtar Gate (seeing it was one of the 101 goals). Berlin has a lovely Museum Island that has just itself become a UNESCO World Heritage Site (and is in need of some work as it looks to have been neglected for years). We also saw the Nefertiti bust at the Altes Museum which was also spectacular, alongside some very interesting painting from the Amarna period, very unlike other Egyptian art.

We also went to see Checkpoint Charlie museum, a museum of international nonviolent protest (obviously focused mainly on the GDR and the wall it backed onto for years). I was very moved by it, particularly the thoughts and aspirations of normal people, it is far too simplistic to just think of Germany at that time of just millions of Nazis who just lost a war. I love to think I would have the physical bravery to resist such a regime but have never been tested. The mayor of Berlin calling out to the allied nations during the Berlin air lift

“You peoples of the world, you people of America, of England, of France, look on this city, and recognize that this city, this people, must not be abandoned — cannot be abandoned!”

Seeing how people tried to escape, risking their lives to try and get to the west and live our kind of lifestyle (which is not without its problems) was interesting. There was exhibits on Ghandi and the rest of Eastern Europe and the fight against opression and it was clear everywhere that people improve their situation and that of others by orgainising and demanding change. As lots of my friends are now from Eastern Europe I am fascinated about how they experienced the fall of communism in their countries. I am a bit quick to dismiss the G8 protesters, people protesting the Iraq war or whatever sometimes, I think I use the fact the world is complex and situations subtle as an excuse for inaction and came away with a desire to direct some of my skills and energies to making even a small difference. I am unsure what it will be though, toying with a few ideas at the moment.

Another highlight was the Reichstag, in particular taking the trip up to the dome. The queue takes too long but the once you are up it is the best view of Berlin and it is wonderful to be so close to the Norman Foster designed cupola.

As an example of Powershell vs the Perl toolkit in VMware I want to show you triggering a storage rescan on a host in both. [perl] #!/usr/bin/perl -w

use strict; my @hbas = /usr/sbin/esxcfg-info \| grep vmkernel -i \| grep hba \| awk -F\. \{\'print \$29\'\};

foreach my $hba (@hbas) { system(“/usr/sbin/esxcfg-rescan $hba”); }[/perl] (From XtraVirt with comments and iSCSI stuff removed.)

It’s not awful, using grep, awk, sed etc has a long history and loads of people are good at it, but I will never be convinced it is intuitive and am far from a wizard myself.

The equivalent in Powershell is [powershell]Get-VMHost | Get-VMHostStorage -Refresh -RescanAllHba -RescanVmfs[/powershell] (once you have run Connect-VIServer, note this actually does it for all hosts in the vcenter too) Maybe you just want to do one cluster [powershell]Get-Cluster -Name CLUSTERNAME | Get-VMHost | Get-VMHostStorage -Refresh -RescanAllHba -RescanVmfs[/powershell] The key advantage for Powershell is the richness of the object being passed around. For example if I run [powershell]Get-VMHost | Get-ScsiLun [/powershell] What is returned is an iterator over all the LUNs on all the hosts. What I see if I run it in the console is a representation of the attributes in a table [code] CanonicalN ConsoleDeviceName LunType CapacityMB MultipathPolicy ame ———- —————– ——- ———- ————— mpx.vmh… /vmfs/devices/genscsi/mpx.v… cdrom RoundRobin naa.500… /vmfs/devices/disks/naa.500… disk 2096128 RoundRobin [/code] The great thing with powershell is I can filter, select, sort etc based on any of these and output however I want.

For example [powershell]Get-VMHost | Get-ScsiLun | Format-Table CanonicalName,CapacityMB[/powershell] returns [code] CanonicalName CapacityMB ————- ———- mpx.vmhba3:C0:T0:L0 naa.50002ac285210609 2096128 naa.50002ac285070609 2096128[/code]

A more fun example to whet your appetite would be [powershell] Get-VMHost | Get-ScsiLun | where {$_.LunType -match ‘disk’} | Select CanonicalName,CapacityMB | Sort-Object -Property CapacityMB | Export-Csv “C:\LUNsBySize.csv” -NoTypeInformation [/powershell] In words

Loop through all hosts and their LUNs Pick out the ‘disk’s Select just CanonicalName and CapacityMB Sort by the CapacityMB Export it to a CSV

The file looks like [code] CanonicalName,CapacityMB mpx.vmhba0:C0:T0:L0,69973 mpx.vmhba0:C0:T0:L0,69973 mpx.vmhba0:C0:T0:L0,69973 mpx.vmhba0:C0:T0:L0,139979 mpx.vmhba0:C0:T0:L0,139979 naa.50002ac285440609,2096128 naa.50002ac0a09a0609,2096128 naa.50002ac2853d0609,2096128 [/code] Without “-NoTypeInformation” it puts #TYPE System.Management.Automation.PSCustomObject at the top of the file.

PowerShell is great, check out this guide to learn about tricks using the pipeline.

I use powershell all the time now to manage VMware, with the PowerCLI commandlets you can achieve a lot and it matches Alan Kay’s motto

Simple things should be simple, complex things should be possible.

The simple: I want to go and set all the LUNs in VMware to use “RoundRobin” as the MultipathPolicy

My first attempt was,

[powershell]Get-VMHost | Get-ScsiLun | Set-ScsiLun -MultipathPolicy “RoundRobin”[/powershell]

Loop through the hosts, loop though each of its LUNs, set MultipathPolicy to “RoundRobin”

Which works, but was taking ages. I was about to make separate scripts for each server when I realised I was not quite doing what I wanted with that script. I really wanted to:

Loop through the hosts, loop though each of its LUNs,
if it was not already RoundRobin, set MultipathPolicy to “RoundRobin”

[powershell]Get-VMHost | Get-ScsiLun | where {$_.MultipathPolicy -ne “RoundRobin”} | Set-ScsiLun -MultipathPolicy “RoundRobin”[/powershell]

Which ran loads faster as the “set” command still ran and took a while even if you were setting it to what it already was.

The Complex:

I took the wonderful vcheck script and made it run every night. Great email overview of the virtual infrastructure each morning.

Well, as per my meta-goal 101 – Do 100 day updates, here is the Day 200 update.

Not much movement really….

The Good:- 66 – Via Ferrata in Italy I go on Friday, cant wait.

82 – Visit Egypt I go in October for 2 weeks, 1 week cruise - 1 week sun and as much archaeology as I can get!

94 – Go to Edinburgh festival Will fly over for a weekend in Aug, let me know if you are interested in joining

28 – Drive Offroad Going back to Dubai in December when it cools off, will be driving a mates Hummer in the desert.

75 – Watch SICP, do exercises from book Did half when visiting Germany from Den Haag a few weeks ago, mind bending but wonderful stuff.

72 – Read “Winning Ways” Inspired by Martin Gardners death, I did half of the first book when I had some downtime in Manchester. First bit of new Maths in ages and it felt great. See Games of No Chance for more fun.

8 – Read all the VSIs Read Vikings, Human Prehistory, Memory, Archaeology, Evolution, maybe more - i need to get tactical and create a master list.

13 – Release 303 books on bookcrossing.com Tasked my long suffering mother with labeling them all, need to teach her how to register them then release them. Still need to sort out the Crossing Zone in Liverpool too.

48 – Create a backblaze storage pod They are getting made soon, due on the 28th

The Bad:- 5 – Lose 2 stone Got half way Jan/Feb, might have taken a step back living away again… Correctable though!

19 – Blog on average once a week Way off, need to blog furiously to catch up.

The Almost Had:- 22 – Spend 3 months in another country Spent two in Holland (Den Haag / The Hague) working for ING.