We stayed at the Wynn again.

Beautiful view from my room

On the first day we went carting.

The final day I went to do a skyjump off the Stratosphere (have a DVD I’ll upload when I find it)

Between that some drinking and gambling…

Random head coming out of the lake in the Wynn

Then home!

Share on Facebook

84 – Revisit Met Museum
Went while I was at Hadoop World (pic is actually the Natural History museum but what the hell)

During my recent Africa Trip:
24 – Swim with sharks

33 – Safari
Visited Addo Elephant Park during my recent South Africa Trip

34 – Vinyard tour
I visited the boekenhoutskloof vinyard in Franschhoek South Africa

I’ll do a full write-up of the Africa Trip soon, it was amazing.

So now out of the 101 I have done 24 with 18 on track and only 300 days to go! Need to get a shift on and tick off as much as I can.

The dayzeroproject site is back up, I am on there as thattommyhall

Share on Facebook

The archive is available and I thought I would share my highlights:

• Vilayanur Ramachandran: The Emerging Mind was the first one I ever listened to, great peek into how the brain works
• Edward Said: The representation of the Intellectual
• John Searle: Minds Brains And Science. I dont agree with Searle but find him very engaging and thought provoking.
• Bertrand Russell: Authority and the Individual. The first Reith lecture was by a personal hero.

Share on Facebook

(picture from Wikipedia)

We did it ultralight, using kit I have blogged about before. My mate Ben got well into expedition planning mode and prepared an optimal food mix for the trip and introduced us to SCROGIN (Sultanas Chocolate Raisons Orange Ginger Imagination Nuts) and ANZAC biscuits (his lovely other half is a kiwi).

I was pleased to fit it all in a 30L sack, made the walking much easier than it might have been.

As I had just been in Lisbon for a stag do the weekend before I was not feeling 100% when we got the sleeper to Inverness on the Monday night but we arrived somewhat fresh and started walking immediatly. The sleeper is really nice and I would deffinatly recommend it over flying if you need an early start in Scotland, see ScotRail. By the end of Tuesday we had got most of the way to Invermoriston (nearly 30 miles) but were all exhausted. We wildcamped with some stunning views.

The Wednesday we walked to Fort Augustus and decided to take a B&B for the night as non of us had slept well and our legs and feet were killing. We were fortunate enough to stay at Old Pier House which was lovely and we got moving again on the Thursday with much more enthusiasm than we ended the day before.

Thursday night we got past laggan and camped at a campsite on the north of Loch Lochy.

Friday was an epic day, taking in the 2 munros ( Meall na Teanga and Sròn a’ Choire Ghairbh and walking about 25 miles then (we thought) finishing the walk.

We had actually just reached Neptune’s Staircase and we wound up bivvying at the start line of Maggies Monster Bike and Hike. We must have looked quite odd…

We spent the first few hours of the Saturday finishing it off and arriving at Fort William where we ate the biggest amount of food we could.

A great hike with 2 great guys and as it is a UK long distance path it is another of my 101 goals in 1001 days days ticked off

Share on Facebook

As all the talks will be online soon I will just mention a few things that I enjoyed.

The two keynotes were excellent, Doug Cutting on the history of Hadoop and Ted Dunning on the future. Both were very interesting and had a great feel for the community aspect of Open Source software. Ted works for MapR technologies but the talk was not a sales pitch. Ted spoke about how Hadoop fails currently to get the most out of the components and what we might get if we could. MapR are used by EMC for their new Hadoop distro, among other things I think they have reimplemented HDFS. An interesting number of companies had got some pretty big amounts of funding to build front-ends to Hadoop, DataMeer have an excel-like web frontend that looks interesting.

Talks I enjoyed were:

NODE.JS FOR HEAVY I/O
A superb intro to Node.js, with an example small enough to fit on a slide but not completely trivial.

TIME SERIES OR CAUSAL ANALYSIS WITHOUT LIMITS!
Shivek was awesome, engaging and enthusiastic. The topic itself was fascinating, using
Pi Calculus to reason about and design map/reduce algorithms. He made the point that most Hadoop jobs are datacentric but showed how to do some more mathscentric algorithms like FFTs

OH LEONHARD, WHERE ART THOU?
Jim Webber on graph databases in general and Neo4J in particular. Quite a nice reference to Euler in the title. If your data is a graph, why not have a database that is too?

REALTIME BIG DATA AT FACEBOOK WITH HADOOP AND HBASE
From Jonathan Gray, this talk was really interesting – amazing the throughput they are getting from HBase. I think Forward are more like Facebook than Google (more freedom within teams, choice of tech/roll your own vs Google wanting everything on BigTable. I cringed a bit at the thought of loads of servers running random C++ apps all over the place though…)

NEWER DEVELOPMENTS IN LARGE DATA TECHNIQUES
Joseph Turian from MetaOptimise gave a great overview of recent academic work on Machine Learning and Natural LAnguage Processing, buzzwords to look out for are: Deep Learning, Semantic Hashing and Semantic Parsing. Also look at GraphLab, Machine Learning on graph databases

DIGITISED DUTCH CULTURAL HERITAGE, MAHOUT & HADOOP
COMPOSING MAHOUT CLUSTERING JOBS
Two good talks on using Mahout, the first is on a Dutch Gov project, Images for the future to archive and categorise AV heritage resources. The second had a nice demo of categorising stack-overflow.

Lightning Talks:
The Lustre filesystem from Eric Barton of Whamcloud talked about how his company are developing Lustre outside Sun/Oracle and he was trying to see where it could fit in with Hadoop. Luster is the other end of the spectrum from HDFS/Hadoop, really quick but assuming fast, highly available storage behind it. I would love to see some integration with Lustre or Ceph in a Hadoop-like system.

I gave a talk on the Flume Firehose Abs and I made at Forward last week, it was OK (though I still think no-one has done a good job of selling ZeroMQ in 10 minutes!). Slides are here (I’ll do another post about it as well, quite an entertaining fallout from it over twitter.)

Share on Facebook

The wiki page HiveCompressedStorage lists the possibilities.

Basically you have 3 decisions:
TextFile or SequenceFile tables
TextFile

• Can be compressed in place.
• Can gzip/bzip before you LOAD DATA into your table
• Only gzip/bzip are supported
• Gzip is not splitable

SequenceFile

• Need to create a SequenceFile table and do a SELECT/INSERT into it
• Can use any supported compression codec
• All compression codecs are splitable. All the cool kids use LZO or Snappy
• Does not work- At least for me (help appreciated!)

Which compression algorithm

• gzip – Quite slow, good compression, not splitable, supported in TextFile table
• bzip – Slowest, best compression, splitable, supported in TextFile table
• LZO – Not in standard distro (licensing issues), fast, splitable
• Snappy – New from google, Not in standard distro (but licence compatable), Very fast

Block or Record compression (for SequenceFile tables)
The docs say

The value for io.seqfile.compression.type determines how the compression is performed. If you set it to RECORD you will get as many output files as the number of map/reduce jobs. If you set it to BLOCK, you will get as many output files as there were input files. There is a tradeoff involved here — large number of output files => more parellel map jobs => lower compression ratio.

But I got the same number of files regardless of what I selected and the total size suggested they were not even compressed so I dont know what is going on.

For simplicity I chose gziped TextFile tables because

• It worked (always criteria zero)
• Most of our files were not huge anyway and the technique described below keeps some of the parallelism
• Can be done on the table in place
• Each partition can be compressed separately
• The space is saved incrementally and realised immediately
• Testing showed for our load it was not much of a performance hit
• We are feeling more pain on space than query performance at the moment, our hourly runs complete in ~20mins)

require 'rubygems'
require 'date'
require 'rbhive'

countrys = %w[at au br de dk es fr in it jp mx nl no pl pt ru se uk us za]
dates = (Date.parse('2011-01-01')..Date.parse('2011-04-30'))

RBHive.connect('hiveserver') do |con|
  dates.each do |date|
    countrys.each do |country|
      query = "insert overwrite table keywords partition (dated='#{date}', country = '#{country}')
              select account,campaign,ad_group,keyword_id,keyword,match_type,status,
              first_page_bid,quality_score,distribution,max_cpc,destination_url,ad_group_status,
              campaign_status,currency_code,impressions,clicks,ctr,cpc,
              cost,avg_position,account_id,campaign_id,adgroup_id 
              from keywords where dated='#{date}' and country='#{country}'"
      begin
        con.set('mapred.output.compression.codec','org.apache.hadoop.io.compress.GzipCodec')
        con.set('hive.exec.compress.output','true')
        con.set('mapred.output.compress','true')
        con.set('mapred.compress.map.output','true')
        con.set('hive.merge.mapredfiles','true')
        con.set('hive.merge.mapfiles','true')
        con.execute(query)
      rescue => e
        puts "#########################"
        puts e.message
        puts "#########################"
      end
    end
  end 
end 

You can actually download and compress the data directly to HDFS as Hive does not know what data is inside the folders on HDFS, just their layout but I thought better to do it via hive and let Hadoop parallelise it. I would have carried on doing it this way but with other tables it was too slow (too many partitions, difficult to parallelise hive server). I stopped using rbhive, dropped to using hive -e to execute the querys and used the lovely autopartitioning in later hive versions. Notice you can SELECT * now and it automatically does what it needs to to insert results into the correct partitions.

require 'rubygems'
require 'date'

countrys = %w[at au br de dk es fr in int it jp kr mx nl no pl pt ru se uk us za]

dates = (Date.parse('2010-12-02')..Date.parse('2011-05-01'))

dates.each do |date|
  query = ""
  query += "SET hive.exec.compress.output=true;"
  query += "SET mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;"
  query += "set mapred.job.priority=VERY_LOW;" 
  query += "set hive.exec.dynamic.partition=true;"
  query += "set mapred.output.compress=true;"
  query += "set mapred.compress.map.output=true;"
  query += "set hive.merge.mapredfiles=true;"
  query += "set hive.merge.mapfiles=true;"
  query += "insert overwrite table hourly_clicks 
            partition (dated='#{date}', country, hour) 
            select * from hourly_clicks where dated='#{date}'"
  query = "hive -e \"#{query}\""
  puts "running #{query}"
  `#{query}`
end

The reason I am not (very) worried about losing parallelism is that some of our partition contained big .csv’s and the output of INSERT OVERWRITE was multiple .gz files (looked to me like as many as there were mappers, for example a 700M text file became ~10 .gz files) so they will still be read in parallel by mappers as the original CSV was.

Open to suggestions about better ways to achieve this, this does not preclude doing something better later.

Share on Facebook

current = ''
file_count = 0
total_size = 0

output = File.open('output.csv','w')

IO.popen('hadoop fs -lsr /user/hive/warehouse').each_line do |line|
  split = line.split(/\s+/)
  #permissions,replication,user,group,size,mod_date,mod_time,path
  next unless split.size == 8
  path = split[7]
  size = split[4]
  permissions = split[0]
  tablename=path.split('/')[4]
  if tablename != current
    average_size = file_count == 0 ? 0 : total_size/file_count
    result = "#{current},#{file_count},#{total_size},#{average_size}"
    unless current==''
      puts result
      output.puts result
    end
    total_size = 0
    current = tablename
    file_count = 0
  end
  file_count += 1 unless permissions[0] == 'd'
  total_size += size.to_i
end

Lots of our files were small so I am going to experiment with different partitioning and compression schemes.

Share on Facebook

old lock file: /var/lib/mongodb/mongod.lock. probably means unclean shutdown
recommend removing file and running –repair
see: http://dochub.mongodb.org/core/repair for more information

As the docs do not seem to have much of an alternative to running –repair I looked for a way to automate it from upstart. Mongo creates a mongod.lock file in the data directory with the processes PID in and on a safe shutdown removes the PID, leaving the file there.

This upstart scripts includes a pre-start script that checks if the lock file exists, reads it, makes sure there is a PID there, makes sure no mongod processes exist with that PID then performs the repair as the mongodb user.

limit nofile 20000 20000

kill timeout 300

env MONGO_DATA=/var/lib/mongodb/
env MONGO_LOGS=/var/log/mongodb/
env MONGO_EXE=/usr/bin/mongod
env MONGO_CONF=/etc/mongodb.conf

pre-start script
  mkdir -p $MONGO_DATA
  mkdir -p $MONGO_LOGS
  if [ -f $MONGO_DATA/mongod.lock ]; then
    mongo_pid=`cat $MONGO_DATA/mongod.lock`
    if [ ! -z $mongo_pid ]; then
      if [ ! `pgrep mongo | grep "$mongo_pid" | wc -l` -gt 0 ]; then
        rm $MONGO_DATA/mongod.lock
        sudo -u mongodb /usr/bin/mongod --config /etc/mongodb.conf --repair
        touch $MONGO_DATA/repaired-`date "+%Y%m%d-%H%M%S"`
      fi
    fi
  fi  
end script

start on runlevel [2345]
stop on runlevel [06]

script
  if [ -f /etc/default/mongodb ]; then . /etc/default/mongodb; fi
  exec start-stop-daemon --start --quiet --chuid mongodb --exec  $MONGO_EXE -- --config $MONGO_CONF
end script

Share on Facebook

I borrowed ideas from John Leach’s excellent Ruby: Reinventing the Wheel talk, this DepOps: The War Is Over presentation and rambled incoherently about a talk I just saw at the UKUUG Spring Conference from the author of cfengine, see here a nice description of the project (you can see how it has influenced Puppet)

Here are the slides (first time I have used Scribd, it is excellent. Much better than slideshare)
DevOps

I like the James White Manifesto , it chimes really strongly with me.

In particular

On Infrastructure
—————–
There is one system, not a collection of systems.
The desired state of the system should be a known quantity.
The “known quantity” must be machine parseable.
The actual state of the system must self-correct to the desired state.
The only authoritative source for the actual state of the system is the system.
The entire system must be deployable using source media and text files.

Soon they will post videos and I will get to see myself give a talk for the first time.

Share on Facebook

Completed – 16
1, Teetotalitarianism for 3 months
2, Cheeseless for 3 months
9, Read GEB
11, Reread all Dennett books
15, Proofread for Project Guttenburg
48, Create a Backblaze storage pod
53, Make Jam
66, Via Feratta in Italy
78, Learn to use Emacs
I suppose you can never fully learn it but I do use it for my development now
82, Visit Egypt
83, Re-visit Louvre
85, Visit Pergamon Museum
86, Give Carrie a British Museum Tour
92, Read “An Ode Less Travelled“, do the exercises (but not share them!)
Read it while in Egypt.
97, Be 1/3 through in 2010
100, Set success criteria / progression metrics for each goal

On Track – 16
5, Lose 2 stone
10, Write book reviews for each book I read
Where I havent yet I have added a task to rememberthemilk to do so
13, Release 303 books on bookcrossing.com
88 available here, let me know if you like any and I will post them to you.
19,Blog on average once a week
50, Move 10 people to FreeAgent
68, Complete Pimsleur German
Changed from Spanish as I now live with a lovely German lady.
72, Read “Winning Ways”
read 1/2 of part 1 (of 4)
74, Read AI: A Modern Approach
75, Watch SICP, do exercises from book
Started a book club in work, seems to have stalled but I’ll start banging the drum again now I’ve settled in my new house.
76, Do on average 1 Project Euler problem per week
77, Complete “Real World Haskell”
88, Go to the theatre on average once a month
Way ahead on this, started a monthly theatre club but we managed to schedule a dozen things for the first few months of 2011
91, Memorise 10 poems
Not quite settled on the 10 but between listening to Jorge Louis Borges, This Craft Of Verse and The Ode Less Travelled I have quite a list to choose from.
95, Pay off all credit cards
96, Let loans run course and dont get any more
101, Do 100 day updates
This is one right

Behind – 4
8, Read all the VSIs
12, Read all PG Wodehouse
81, Watch all TTC Art history DVDs
90, See all world heritage sites in the UK

Changing – 5
Lots of the work related ones dont make sense any more now that I have gone full time and moved into development so I am making the following changes.
43, Visit the rijksmuseum (was Get CCNP)
44, Visit The Uffizi in Florence (was Get CCEE)
45, Give blood every 20 weeks (was Get MCITP – Enterprise Admin)
46, Listen to Radio 4 / British Museum – A History of the World in 100 Objects and view each of them (was Get VCAP)
The above are all taken from a mate who just did his own 101 list.
47, Make a Munro bagging site in Rails (was Say to a recruiter “I dont work ” and turn down work)

Planning – 12
60, Hike on average once a month
61, Do a UK long distance path
67, Do another alpine 4000m peak
62, Do a big hike in Europe
64, Climb a continental highest mountain
33, Safari
20, Organise a big bash for my 30th
The fitness aspect of these goals is where I am behind the most (though I am still a stone lighter than when I started) so I am concentrating the next six months on these goals, ending with summiting kilimanjaro for my 30th then returning to a big party.
35, Visit 5 Michelin 3* restaurants
37, Visit porto
Will go with Petra in the spring
84, Revisit Met Museum
A good mate has just moved to NYC so this should happen as soon as he is settled.
89, Return to the Theatre by the lake
My first trip with Petra was to here and we loved it. Will be going in the spring.
94, Go to Edinburgh festival
Will go at the beginning of August.

Not Started – 48
3, Do a marathon
4, Do a triathlon
6, Attend martial arts classes for 3 months
7, Write an artice for Plus new writers
14, Read a short story for librivox
16, Send Dennett a letter
17, Send Dawkins a letter
18, Read Joyce
21, Read GTD
22, Spend 3 months in another country
23, Organise all my DVDs
24, Swim with sharks
25, Paraglide
26, Learn to play bongos
27, Skydive
28, Drive Offroad
29, Do a banger rally
30, Have a track day
31, Hire the whole of Salvos Salumeria for an evening
32, Bungee Jump
34, Vinyard tour
36, See Northern Lights
38, Take dad to an opera
39, Take Mum, Dad and Carrie to the Welsh Mountain Zoo
40, Do 1000 things in London
41, Do a standup comedy course
42, Visit Japan
49, Work only 100 days in a year
51, Investigate Visa situation for Australia
52, Investigate Visa situation for US
54, Grow mushrooms
55, Paint a water colour
56, Make beer
57, Make wine
58, Cook a 4 course meal for 20 friends
59, Do a photography course
63, Attend NIM
69, Learn to dance
70, Learn to play golf
71, Learn 10 magic tricks
73, Make a Dots and Boxes program
79, Raise £5005 for charity
80, Talk about Free Software at a school
87, Go on wine tasting course
93, Go to Melbourne Comedy Festival
99, Have a completion party
65, Volunteer for the mountain bothys association
98, Have done 2/3 by day 666

I am quite heartened by the progress to be honest, considering that I spent half of last year working outside the UK, now things are settling down I should be able to churn through them faster.

If you want to join in on some, let me know!

Share on Facebook

Original MRI on Linux

  def build
    pid = fork { run_command }
    Process.detach(pid)
  end

  def run_command
    `sudo /opt/script/deployserver/setupnewserver.sh -p #{poolserver} -i #{ip} -s #{@size} -v #{@vlan} -a "#{@owner}" -n #{@name} -e "#{@email}"`
  end

IronRuby
We tried IronRuby and the same bit of the script broke as on win32 MRI (though I was pleased and surprised that Sinatra worked)

  def build
    WindowsProcess.start "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
"-PSConsoleFile \"C:\\Program Files (x86)\\VMware\\Infrastructure\\vSphere PowerCLI\\vim.psc1\" \"& C:\\script\\DataStoreUsage.ps1\""
  end

Using the following DotNet code

class WindowsProcess
  def self.start(file, arguments)
    process = System::Diagnostics::Process.new
    process.StartInfo.FileName = file
    process.StartInfo.CreateNoWindow = true
    process.StartInfo.Arguments = arguments
    process.Start
  end
end

Workaround using Windows “start” command
I had hoped the module at win32utils would let me just use the original script but fork did not work properly still.

def build
  commandstr = "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe -PSConsoleFile \"C:\\Program Files (x86)\\VMware\\Infrastructure\\vSphere PowerCLI\\vim.psc1\" \"& C:\\Sites\\vmdeploy\\PrepNewMachine.ps1 -type #{@type} -machinename #{@name} -size #{@size} -vlan #{@vlan} -creator #{@owner} -creatoremail #{@email} -ipaddress #{ip}"

  system ("start #{commandstr} > ./log/#{@name}.log 2>&1")
end

This uses the windows “start” command and works pretty well.

Share on Facebook

Measuring IOPS
In linux I like the iostat command, though there are a few ways to get at the info. In debian/ubuntu it is in the sysstat package (ie: sudo apt-get install sysstat)

root@MACHINENAME:/home/deploy# iostat 1
Linux 2.6.24-28-server (MACHINENAME.forward.co.uk) 	18/02/11
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
    45.51    0.00    1.85    0.62       0.00       52.03

Device:        tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
cciss/c0d0     4.00       0.00       40.00          0       40
cciss/c0d1     4.00       0.00       64.00          0       64
cciss/c0d2    12.00       0.00      248.00          0       248
cciss/c0d3     0.00       0.00        0.00          0       0
cciss/c0d4    25.00       0.00      320.00          0       320
cciss/c0d5     0.00       0.00        0.00          0       0
cciss/c0d6    30.00       0.00      344.00          0       344
cciss/c0d7    42.00    3144.00        0.00         3144     0

iostat 1 refreshes everysecond, if you do it over a longer period it will average the results. tps is what you are interested in, Transactions Per Second (ie IOPS). -x will give a more detailed output and separate out reads and writes and let you know how much data is going in and out per second.

What is a good or bad number though?
As with most metrics, if the first time you look at it is when you are in trouble then it’s less helpful. You should have an idea of how much IO you typically do, then if you experience issues and are doing 10x that or only getting 1/10 from the disks then you have a good candidate explanation for them.

How much can I expect from my storage?
It depends how fast the disks are spinning, and how many there is.
As a rule of thumb I assume for a single disk:
7.2k RPM -> ~100 IOPS
10k RPM -> ~150 IOPS
15k RPM -> ~200 IOPS
Our hadoop servers were pushing about 70 IOPS to each disk at peak and they are 7.2k ones so that is in line with this estimate.

See here for a breakdown of why these are good estimates for random IOs from a single disk. Interestingly a large amount of it comes from the latency of the platter spinning, which is why SSDs do so well for random IO (Compared to a 15k disk, ~50x for writes, ~200x reads)
See also:
A concrete example of faster CPU causing higher %iowait while actually doing more transactions here

Extreme Linux Performance Monitoring and Tuning: Part 1 (pdf) and Part 2 (pdf) from ufsdump.org/

Share on Facebook

You only need two things – the built-in “sc” command and an executable from Windows Server 2003 Resource Kit Tools called srvany (works with 2008 too). Get just that exe here (if you trust me of course )

Creating the service

Check it exists

Set Parameters In The Registry
Configure it at HKLM/SYSTEM/CurrentControlSet/Services/APPNAME/Parameters

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VMdeploy\Parameters]
"Application"="C:\\Ruby192\\bin\\ruby"
"AppParameters"="C:\\Sites\\vmdeploy\\server.rb -p 80"
"AppDirectory"="C:\\Sites\\vmdeploy"
"AppEnvironment"=hex(7):65,00,78,00,61,00,6d,00,70,00,6c,00,65,00,3d,00,32,00,\
  37,00,00,00,62,00,6c,00,61,00,68,00,3d,00,63,00,3a,00,5c,00,74,00,65,00,6d,\
  00,70,00,66,00,69,00,6c,00,65,00,73,00,00,00,00,00

Note the AppEnvironment is a multiline string, the rest are strings

This lets you run any executable file, change the directory you run it from and pass any arguments or environment variables so should cover most use cases.

I will be sharing the code for both the Sinatra app and the PowerShell deploy script in later posts.

Share on Facebook

Just a brief outline:

Work: Started the year down in Kent working on a XenApp deployment on VMware at the beginning of the year, then over in Dubai building a VMware View solution, went to Libya and built the corporate infrastructure for their biggest telco, spent 2 months working on INGs next gen datacentre in the Hague and ended in London for Forward working on merging USwitch’s infrastructure after the acquisition.
While this was fun, I got fatigued with traveling and living out of a suitcase and was very happy to settle down a bit in London (which I think is the greatest city in the world) and work for a great company which I am very pleased and proud to say I have joined permanently. It is exciting to work somewhere using so much great tech and with so many sound people (with huge brains).

Jollys: I managed to go to Russia, visiting Moscow and St Petersburg (OK, that was Dec 2009 but what the hell), the Czech Republic, Italy, go to Oklahoma for a friends wedding followed by a visit to two ancestral puebloan sites, Berlin, Paris, Vegas and Egypt. I feel privileged to have the opportunity to have done all this and am still a bit amazed at just how much happened.

Life: I have always thought their was an embarrassment of riches when I look at the great people I get to call my friends, I don’t get to see any of them enough and no amount is too much. Thanks to you all, I always say you are what makes the universe great for me, you conspire to make it so even though you don’t all know one another. The biggest change this year was finding someone patient enough to pair up with me full time, thanks Petra – I love you. We are moving in together soon and I look forward to having a permanent home in the greatest city in the world, with a spare room – please come visit!. Thanks to my family too, we had some bad news in 2010 but you all dealt with it with the usual aplomb, you are all ace.

Resolutions There is no new resolutions for 2010, I will be reporting on the 101 goals in 1001 days soon.

2011: Looks set to be the best year ever, thanks in advance for helping make it so

Share on Facebook

I had previously moaned about seeing Jimmy Wales’s face every time I logged in, like this

and laughed my head off at this piss take from The Daily What:

I found today at Information Is Beautiful the following demonstration of just how effective the campaign has been though.

Wikimedia have done some nice analysis of the campaign on the Meta Wiki if you are interested.

Give to Wikipedia here

Share on Facebook

Will #VMware Load-Based Teaming remove the need for #Cisco EtherChannel? Discuss….

I long ago investigated NIC Teaming algorithms and settled on IP Hash with Cisco Etherchannels for most environments, only really using something else if the client happened not have stacked switches. Thanks to Scott Lowe for this superb article on the matter.

When vSphere 4.1 came out with Load Based Teaming, I was pleased that at last we had an algorithm that would have a go at proper load balancing and not just load distribution but had not got round to investigating much more.

At Forward we have just updated to 4.1, Enterprise Plus and have bought some shiny new Extreme Summit X650 Series 10G switches; so Simon’s comment was particularly apropos.

I had decided I wanted to try and use LBT but was unsure if I should port-channel the uplink ports. It turns out you can’t. I thought maybe you should to be honest, it does not mention in the dvSwitch guide as far as I can see but the ESX host requirements for link aggregation KB (updated today) is very clear

• The switch must be set to perform 802.3ad link aggregation in static mode ON and the virtual switch must have its load balancing method set to Route based on IP hash.
• Enabling either Route based on IP hash without 802.3ad aggregation or vice-versa disrupts networking

ie you need both IP Hash and EtherChannel and neither will work without the other.

In answer to Simon’s question, my feeling is you may still get better performance from EtherChannel and IP based hash for some workloads but would guess “usually” LBT wins. I think the case where you may get better utilisation is when certain VMs have very high bandwidth requirements to different IPs. As described here IP Hash is the only way to allow traffic from one vNIC to leave over different pNICs at the same time.

It is interesting that even with LBT bandwidth is still limited to the maximum bandwidth a single pNIC can provide for individual VMs / vmkernels, also IP hash will not get higher than a single pNIC for a vMotion or other point to point connections. So 10G is going to perform better for these operations than 10x1G, however you team them.

Share on Facebook

In the 3par storage I could see that IO was unusually high of late.

Then I looked at the top-n datastores by IOPS and graphed them

A huge jump for sharedstorage8, so I looked at its VMs

and found the culprit VM.

Here it is against our big “Superhero” database and the vCenter server with the DB.

A lot of IO from a machine the owner thought was doing nothing!

Hyper9 is a pretty good tool for reporting, alerting and troubleshooting your VMware infrastructure, the query language is lucene based and this gives you lots of options in creating custom views and alerts.

Share on Facebook

The second is my del.icio.us tags, when is later going to happen?

With delicious closing I am looking at alternatives, send recommendations if you have any.

Share on Facebook

We flew out last Thursday and stayed for 3 nights at the Wynn, which is a great hotel.

Thursday: Arrive and sleep (forgive me I only just got back from Egypt after flying to Manchester instead of London and having to sit all night on a freezing cold coach!).

Friday: Flew in a helicopter into the Grand Canyon which was rather awesome.

Sat: Went to see Zumanity by Cirque du Soleil, it was amazing, I must see them again.

Sunday: A few of us went shooting,
An M16

An H&K MP5

A Mac-10

not firing it gangster style unfortunately.

A Tommygun

A Desert Eagle

And I did a Shotgun

And I’ve got the T-shirt to prove it

Peace Through Superior Firepower indeed

Pics On Flickr

Vids On Youtube

Share on Facebook

You know you are a huge geek when you take as much space for books as clothes.

The trip was ace, a week on the M/S Hamees, Stopping at:
Valley of the Kings
Temple of Queen Hatshepsut
The Colossi of Memnon
Edfu Temple

Kom Ombo, which I had not heard of but has some extraordinarily vibrant original colour remaining

Abu Simbel

Karnak Temple

Luxor Temple (we had the place to ourselves at night, it was amazing)

5 Nights in Movenpick El Gouna on the Red Sea and 2 nights in the Nile Palace

It was incredibly relaxing, nice to be incommunicado for 2 weeks and catch up on some reading and just mince around the beach. I managed to read Godel Escher Bach at last and also ticked off The Ode Less Travelled from my 101 Goals. It was weird not having my phone to distract me in spare moments, making me daydream more and think about people and events I have not thought about it ages, I should do it more often.

Pictures are on flickr

Share on Facebook

Im particularly excited about Abu Simbel,, The Red Chapel of Hatshepsut, Temple of Amenhotep III among tons of others.

Also I am very much looking forward to No Phone and No Computers for the whole trip, the longest I will have been without either for for a good few years. I am taking the chance to catch up on some reading and take a break, I will have my Kindle with loads of books on it but am taking The Dots-and-boxes Game: Sophisticated Child’s Play (one of my 101 goals is to make a dots and boxes program), Gödel, Escher, Bach (another 101, started 3 times and never finished and it has intrigued me since my first year in university) and Winning Ways for your Mathematical Plays. Pen’n'paper geekery for the win!

Share on Facebook

Installing R
Adding the repositry
Create a new file at /etc/sources.list.d/R.list

#R repositry
deb http://rh-mirror.linux.iastate.edu/CRAN/bin/linux/ubuntu hardy/

(we are still using hardy, with the Cloudera packages)

Add the gpg keys for the repository

gpg --keyserver pgp.mit.edu --recv-key E2A11821
gpg -a --export E2A11821 | sudo apt-key add -

Install and update R
Easy:

$ sudo apt-get install r-base r-base-dev pkg-config littler
$ sudo R
> update.packages()

Set environment variables for Rhipe
Add to bottom of /etc/environment

HADOOP=/usr

create it for current session

$ export HADOOP=/usr

install protobuff

# wget http://protobuf.googlecode.com/files/protobuf-2.3.0.tar.bz2
# tar jxf protobuf-2.3.0.tar.bz2
# cd protobuf-2.3.0
# ./configure
# make
# make install
# ldconfig

install Rhipe

# wget http://www.stat.purdue.edu/~sguha/rhipe/dn/Rhipe_0.64.tar.gz
# R CMD INSTALL Rhipe_0.64.tar.gz

So all is well except that the test code here is a bit off.

For me today

> library(Rhipe)

Only works as root

It seems that

> rhwrite(list(1,2,3),"/tmp/x")

should be:

> rhwrite(list(1,2,3),"/tmp/x",1)

then

> rhread("/tmp/x")

works properly.

Also in the longer example

map <- expression({
  lapply(seq_along(map.values),function(r){
    x <- runif(map.values[[r]])
    rhcollect(map.keys[[r]],c(n=map.values[[r]],mean=mean(x),sd=sd(x)))
  })
})

## Create a job object
z <- rhmr(map, ofolder="/tmp/test", inout=c('lapply','sequence'),
          N=10,mapred=list(mapred.reduce.tasks=0),jobname='test')

## Submit the job
rhex(z)

## Read the results
res <- rhread('/tmp/test/p*')
colres  <- do.call('rbind', lapply(res,"[[",2))

colres
       n      mean        sd
 [1,]  1 0.4983786        NA
 [2,]  2 0.7683017 0.2937688
 [3,]  3 0.5936899 0.3425441
 [4,]  4 0.3699087 0.2666379
 [5,]  5 0.5179839 0.4060244
 [6,]  6 0.6278925 0.2952608
 [7,]  7 0.4920088 0.2785893
 [8,]  8 0.4592598 0.2674592
 [9,]  9 0.5734197 0.1928496
[10,] 10 0.4942676 0.2989538

Where line 16 has been changed from the original

res <- rhread('/tmp/test')

Thanks to Saptarshi Guha, the author of Rhipe for so quickly responding to my query in the group and also the authors of this discussion on setting up R in Ubuntu

Share on Facebook

Numbers that fool the Fermat test are called Carmichael numbers, and little is known about them other than that they are extremely rare. There are 255 Carmichael numbers below 100,000,000. The smallest few are 561, 1105, 1729, 2465, 2821, and 6601. In testing primality of very large numbers chosen at random, the chance of stumbling upon a value that fools the Fermat test is less than the chance that cosmic radiation will cause the computer to make an error in carrying out a “correct” algorithm. Considering an algorithm to be inadequate for the first reason but not for the second illustrates the difference between mathematics and engineering.

Share on Facebook

Completed
66 – Via Feratta in Italy
85 – Visit Pergamon Museum
1 – Teetotalitarianism for 3 months
2 – Cheeseless for 3 months
11 -Reread all Dennett books
15 -Proofread for Project Guttenburg
53 – Make Jam
86 – Give Carrie a British Museum Tour
48 – Create a Backblaze storage pod
100 Set success criteria / progression metrics for each goal

Changing
18 – Read epic literature
A bit vague (in spite of meta-goal 100) and I have loads of reading goals so im changing it to “Read Joyce”, in particular:

• Dubliners
• A Portrait of the Artist as a Young Man
• Ulysses
• Finnegans Wake

26 -Scuba
My ears are bad, I worry this may destroy them, not settled on a replacement goal yet.

Sustained Effort – On Track
19 – Blog on average once a week
50 – Move 10 people to FreeAgent
68 – Complete Pimsleur Spanish
88 – Go to the theatre on average once a month
95 – Pay off all credit cards
96 – Let loans run course and dont get any more
101 – Do 100 day updates
5 – Lose 2 stone
13 – Release 303 books on bookcrossing.com
76 -Do on average 1 Project Euler problem per week

Sustained Effort – Behind
8 – Read all the VSIs
12 – Read all PG Wodehouse
60 – Hike on average once a month
79 – Raise £5005 for charity
81 – Watch all TTC Art history DVDs
90 – See all world heritage sites in the UK

In Progress
91 – Memorise 10 poems
92 – Read “An Ode Less Travelled”, do the exercises (but not share them!)
72 – Read “Winning Ways”
75 – Watch SICP, do exercises from book

Share on Facebook