It is quite complicated, section 2 deals with the performance of the naive algorithm and 3 implements the sieve properly. I could not quite grok the Haskell but the following quote helped me know what was going on

Whereas the original algorithm crosses off all multiples of a prime at once, we perform these “crossings off” in a lazier way: crossing off just-in-time. For this purpose, we will store a table in which, for each prime p that we have discovered so far, there is an “iterator” holding the next multiple of p to cross off. Thus, instead of crossing off all the multiples of, say, 17, at once (impossible, since there are infinitely many for our limit-free algorithm), we will store the first one (at 17 × 17; i.e., 289) in our table of upcoming composite numbers. When we come to consider whether 289 is prime, we will check our composites table and discover that it is a known composite with 17 as a factor, remove 289 from the table, and insert 306 (i.e., 289+17). In essence, we are storing “iterators” in a table keyed by the current value of each iterator.

Here it is using a dictionary for each composite number with a list of increments it was reached by:

from itertools import count

def gen_primes():
    yield 2
    def update_composites(number,increment):
        try:
            composites[number] += [increment]
        except:
            composites[number] = [increment]

    composites = {4:[2]}

    c = count(3)
    while 1:
        #print composites
        next = c.next()
        smallest = min(composites.keys())
        incs = composites[smallest]
        del composites[smallest]
        for inc in incs:
            update_composites(smallest+inc,inc)
        if next < smallest:
            yield next
            c.next()
            update_composites(next**2,next)

g=gen_primes()

for i in range(100):
    print g.next()

(On Github)

They mention a speedup moving to a heapqueue as the data structure (as we just take the lowest each lookup) so I implemented that too, here is the code

from itertools import count
from heapq import heappush,heappop

def gen_primes():
    yield 2

    def update_composites(number,increment):
        heappush(composites,(number,increment))
    composites = [(4,2)]

    c = count(3)

    while 1:
        #print composites
        smallest,increment = heappop(composites)
        update_composites(smallest+increment,increment)
        if composites[0][0] == smallest:
            continue
        next = c.next()
        if next < smallest:
            yield next
            c.next()
            update_composites(next**2,next)

g=gen_primes()

for i in range(100):
    print g.next()

(On github)

Share on Facebook

I remembered this while reading the SICP book, the question was about the difference between this program in applicative and normal order evaluation

(define (p) (p))

(define (test x y)
  (if (= x 0)
      0
      y))

It rang a bell as (define (p) p) does not go into an infinite loop if you invoke p. In lisp (p) calls the procedure p with no arguments whereas p is just a reference to the function. In python someinstance.method refers to the method, someinstance.method() calls it, Ruby seems to need Proc objects to get around this (IMHO as a beginner!, see the end for John Leach’s lovely response via email at the time)

I redid all the examples from the book in Python

Eg 1
Ruby

def maybe_do some_proc
  if rand(2) == 0
    some_proc.call
  end
end

def twice_do some_proc
  some_proc.call
  some_proc.call
end

wink = Proc.new do
  puts '<wink>'
end

glance = Proc.new do
  puts '<glance>'
end

Python

import random

def maybe_do(some_proc):
    if random.choice(range(2)) == 0:
        some_proc()

def twice_do(some_proc):
    some_proc()
    some_proc()

def wink():
    print 'wink'

def glance():
    print 'glance'

for i in range(5):
    print 'running for i=',i
    maybe_do(wink)

Eg2
Ruby

def do_until_false first_input, some_proc
  input = first_input
  output = first_input
  while output
    input = output
    output = some_proc.call input
  end
  input
end

build_array_of_squares = Proc.new do |array|
  last_number = array.last
  if last_number <= 0
    false
  else
    # Take off the last number...
    array.pop
    # ...and replace it with its square...
    array.push last_number*last_number
    # ...followed by the next smaller number.
    array.push last_number-1
  end
end

always_false = Proc.new do |just_ignore_me|
  false
end

puts do_until_false([5], build_array_of_squares).inspect

yum = 'lemonade with a hint of orange blossom water'
puts do_until_false(yum, always_false)

Python

def do_untill_false(first_input, some_proc):
    input = first_input
    output = first_input
    while output:
        input = output
        output = some_proc(input)
    return input

def build_array_of_squares(array):
    last_number = array.pop()
    if last_number <= 0:
        return False
    else:
        array.append(last_number * last_number)
        array.append(last_number - 1)
        return array

def always_false(just_ignore_me):
    return False

def just_ignore_me():
    pass

print do_untill_false([5], build_array_of_squares)
yum = 'lemonade with a hint of orange blossom water'
print do_untill_false(yum, always_false)

Eg3
Ruby

def compose proc1, proc2
  Proc.new do |x|
    proc2.call(proc1.call(x))
  end
end

square_it = Proc.new do |x|
  x*x
end

double_it = Proc.new do |x|
  x+x
end

double_then_square = compose double_it, square_it 

square_then_double = compose square_it, double_it

puts double_then_square.call(5) puts square_then_double.call(5)

Python

def compose(proc1,proc2):
    def composed(x):
        return proc2(proc1(x))
    return composed

def square_it(x):
    return x**2

def double_it(x):
    return x*2

double_then_square = compose(double_it,square_it)
square_then_double = compose(square_it,double_it)

print double_then_square(5)
print square_then_double(5)

Eg4

class Array
  def each_even(&was_a_block__now_a_proc)
    # We start with "true" because
    # arrays start with 0, which is even.
    is_even = true
    self.each do |object|
      if is_even
        was_a_block__now_a_proc.call object
      end
      # Toggle from even to odd, or odd to even.
      is_even = !is_even
    end
  end
end

fruits = ['apple', 'bad apple', 'cherry', 'durian']
fruits.each_even do |fruit|
  puts "Yum! I just love #{fruit} pies, don't you?"
end

[1, 2, 3, 4, 5].each_even do |odd_ball|
  puts "#{odd_ball} is NOT an even number!"
end

Python

class MyArray(list):
    def each_even(self):
        for i in range(len(self)):
            if i % 2 == 0:
                yield self[i]

fruits = MyArray(['apple', 'bad apple', 'cherry', 'durian'])

for fruit in fruits.each_even():
    print 'yum! I love %s pies, dont you?' % fruit

for odd_ball in MyArray([1,2,3,4,5]).each_even():
    print '%s is NOT an even number' % odd_ball

Eg5
Ruby

def profile block_description, &block
  start_time = Time.new
  block.call
  duration = Time.new - start_time
  puts "#{block_description}: #{duration} seconds"
end

profile '25000 doublings' do
  number = 1
  25000.times do
    number = number + number
  end

  puts "#{number.to_s.length} digits"
  # That's the number of digits in this HUGE number.
end

profile 'count to a million' do
  number = 0 1000000.times do
    number = number + 1
  end
end

Python

def profile(description, function):
    import time
    start_time = time.time()
    function()
    duration = time.time() - start_time
    print '%s: %s seconds' % (description, duration)
    print function.__name__
    print 'see, "function.__name__" can be used in place of description in python'

def count_to_a_million():
    number = 0
    for i in range(1000000):
        number = number+1

profile('count to a million', count_to_a_million)

def profiled(function):
    def new_function(*args, **kwargs):
        import time
        start_time = time.time()
        result = function(*args, **kwargs)
        print function.__name__, 'took', time.time() - start_time, 'secs'
        return result
    return new_function

@profiled
def count_to_a_million_again():
    number = 0
    for i in range(1000000):
        number = number + 1

count_to_a_million_again()

This uses decorators, a nice Python feature that uses higher order functions (and the fact functions are first class in python).

In Conclusion
IMHO, at this point in my experience of Ruby, with all the disclaimers about my non expert status etc.
Like:

• No restriction on complexity of anonymous functions

Dont Like:

• Methods being different from Procs/Blocs, non-uniform syntax
• Leaving out parenthesis (though I await DSL goodness later!)
• “end” everywhere (I know the indentation thing in python is contentious!)

John Leach’s thought provoking tuppence

Young padawan, you look but you do not see, you will learn

or rather

Yeah, but blocks are closures Tom

Tom goes to google and comes back with http://www.artima.com/intv/closures2.html
Matz

I think it’s not that useful in the daily lives of programmers. It doesn’t matter that much.

Then john came back with

I can think of one example in Rails right away where it's useful, transactions:

r = Record.new params[:record]

Record.transaction do
 r.save
 RecordLog.create(:text => "created a new record")
end


that code takes some input from a browser (in params), instantiates a
new Record object, then writes it and a RecordLog entry to the database
atomically.
All the Record.transaction does is sends a BEGIN to the db server,
executes the block, and sends a COMMIT (or a ROLLBACK if the block
errors for any reason).
The block needs access to the r object. We could have created that
inside the block, but then it'd need access to the params object. So
without real closure support, Record.transaction would have had to
support passing in arbitrary variables.
Remember, that interview with Matz was in 2003 - more people are using
Ruby for more things nowadays, for uses beyond the imagination of it's
creator I'm sure

Final Thoughts
I am waiting to be blown away by Ruby and Rails

Share on Facebook

If p is the perimeter of a right angle triangle with integral length sides, {a,b,c}, there are exactly three solutions for p = 120.
{20,48,52}, {24,45,51}, {30,40,50}
For which value of p < 1000, is the number of solutions maximised?

WARNING: CONTAINS MATHEMATICS

You may remember from school



(Difference of 2 squares)
let

and

so


and substituting these into
gives

Set and to get rid of that root and you have

Primitive triples are ones in lowest terms (if you know (3,4,5) you can get (6,8,10) etc by multiplying each side by some integer)
If p and q are coprime and different parity you just get primitive ones (i did not bother and just used set() to remove dupes from the list)

Here is my code

from __future__ import division
from collections import defaultdict
from time import time
     start = time()
counter = defaultdict(list)
    def triples():
    for q in range(1, 35):
        #35 is greater than sqrt(1000)
        for p in range(q+1, 35):
            a = p**2 - q**2
            b = 2 * p * q
            c = p**2 + q**2
            for m in range(1,int(1000/c)):
                #generate multiples till c is above 1000
                #triples with perimeter > 1000 are filtered later
                yield (m*min(a,b),m*max(a,b), m*c)
    for triple in triples():
        perimeter = sum(triple)
        if perimeter < 1001:
        counter[perimeter] += [triple]
<p>print set((counter[120]))
<p>maxlen = 1<br />
for i in counter:
    length = len(set(counter[i]))<br />
    if length > maxlen:<br />
        maxlen = length<br />
        print i<br />
end = time()</p>
<p>print "took ", end-start</p>
<p>

Which outputs:
set([(24, 45, 51), (20, 48, 52), (30, 40, 50)])
520
528
630
660
720
840
took 0.0140960216522

I tried brute forcing it today and knocked up the following

<br />
from collections import defaultdict<br />
counter = defaultdict(int)<br />
from time import time</p>
<p>def run():<br />
	start = time()<br />
	for perimeter in range(1,1001):<br />
		#print perimeter<br />
		for a in range(1,perimeter):<br />
			for b in range(1,perimeter-a):<br />
				c = perimeter - a - b<br />
				if a**2 + b**2 == c**2:<br />
					counter[a+b+c] += 1<br />
	mosttriples = 0<br />
	for i in counter:<br />
		if counter[i] > mosttriples:<br />
			mosttriples = counter[i]<br />
			print i<br />
	end = time()<br />
	print "TOOK", end-start</p>
<p>print "without psyco"<br />
run()</p>
<p>print "with"<br />
import psyco<br />
psyco.full()<br />
run()<br />

Which gives

without psyco
516
520
528
630
660
720
840
TOOK 134.663383007
with
516
520
528
630
660
720
840
TOOK 21.0988409519

So psyco makes it 6 times faster but a smarter algorithm is 2000 times faster still.

Share on Facebook

I sent Dave the following blurb:

Why I love Python:

A talk on the programming language Python, in 3 parts (feel free to
leave in the interludes if you have had enough)

Part 1: Past, Present, Future.
A bit of history and the design of the language, a look at all the
implementations available today, quick tour of built-in and commonly
used modules and future plans.

Part 2: Language overview
A quick tour of the language: builtin types, control structures, using
modules etc

Part 3: Recent Magic.
Some relatively recent changes that make programming Python even more
pleasurable.
Decorators, Generators, List comprehensions, Iterators, Functools and
anything else I can fit in.
Again a whirlwind tour, but you should be impressed and want to read
up on these some more

I have been revisiting some of the Python talks I have watched over the last few years for ideas and will update my ComSci page with links.

I stumbled across some excellent video from RubyConf, particularly the Rubinius one. Rubinius is a ruby VM partially written in Ruby, taking some lessons from Python and Smalltalk. Some of the stuff he bigs up (compiling to bytecode automatically comes to mind) Python has had for ages, but the self hosting aspect is cool (not as cool as PyPy though). Rubinius seems to be doing what Avi Bryant suggested here, learn from the Smalltalk guys and the papers from the Self team that Sun spun off and later bought back to do the hotspot VM for Java. Interesting times for dynamic languages, target the JVM, CLR, self host and generate code in other languages while always writing in the same fun language. I say Ruby envy only because I think the Ruby community does a better job of looking cool and exciting people than the Python one.

Now Haskell joy. After describing working through Yet Another Haskell Tutorial to the 2 friends doing it with me as “not an obviously pleasurable experience” I had a great moment on the train the other day looking at partial application.
(\y -> y*3)
is Haskell for the anonymous function that takes y and multiplies it by 3 (I wish I had LaTeX here to draw the lamda calculus). What I like is that you can also write that as
(*3)
While this example is trivial, what is happening is interesting. The compiler knows * is an infix operator that takes 2 arguments and that is has been supplied one and “partially applies” the function, making (*3) (a function that takes one argument). One more thing is changing prefix and infix operators around using ( _ ) and ` _ ` , for example:
3 * 5
(*) 3 5

map (*2) [1,2,3]
(*2) `map` [1,2,3]

I hope this second example is clear, map usually is a prefix function that takes a function and a list and returns a list with the result of applying the function to each element (the return value here would be [2,4,6]). This flexibility is neat and is starting to make Haskell a joy to hack in.

Merry Christmas,

Share on Facebook

The slideshow was made using rst2s5 and I am amazed how well it works. Restructuredtext is from Pythons Docutils and is a simple markup language designed to be readable as plain text, see here for details. Below is part of the code for my presentation

==============================================================
Anatomy Of A Python Program - How Much Can You Do In 0.1 KLOC?
==============================================================

:Author: Tom Hall
  ate: 11pm Tues Dec 4th

The Problem - Sudoku
====================

::

   +----------+----------+----------+
   | A1 A2 A3 | A4 A5 A6 | A7 A8 A9 |
   | B1 B2 B3 | B4 B5 B6 | B7 B8 B9 |
   | C1 C2 C3 | C4 C5 C6 | C7 C8 C9 |
   +----------+----------+----------+
   | D1 D2 D3 | D4 D5 D6 | D7 D8 D9 |
   | E1 E2 E3 | E4 E5 E6 | E7 E8 E9 |
   | F1 F2 F3 | F4 F5 F6 | F7 F8 F9 |
   +----------+----------+----------+
   | G1 G2 G3 | G4 G5 G6 | G7 G8 G9 |
   | H1 H2 H3 | H4 H5 H6 | H7 H8 H9 |
   | I1 I2 I3 | I4 I5 I6 | I7 I8 I9 |
   +----------+----------+----------+

Setting The Scene
=================

::

    def cross(A, B):
        return [a+b for a in A for b in B]

    rows = 'ABCDEFGHI'
    cols = '123456789'
    digits   = '123456789'
    squares  = cross(rows, cols)
    unitlist = ([cross(rows, c) for c in cols] +
                [cross(r, cols) for r in rows] +
                [cross(rs, cs) for rs in ('ABC','DEF','GHI') for cs in ('123','456','789')])
    units = dict((s, [u for u in unitlist if s in u])
                 for s in squares)
    peers = dict((s, set(s2 for u in units[s] for s2 in u if s2 != s))
                 for s in squares)

Taking in problems
==================

::

    def parse_grid(grid):
        "Given a string of 81 digits (or . or 0 or -), return a dict of {cell:values}"
        grid = 1
        values = dict((s, digits) for s in squares) ## To start, every square can be any digit
        for s,d in zip(squares, grid):
            if d in digits and not assign(values, s, d):
        	return False
        return values

The second talk went a bit over my head but was interesting, a declarative metaclass pattern. The chat after was great, everyone is interesting and excited about similar things to me.

I used Python the next day in work when our FTP server (a crappy windows app that I have disliked all the time I have been here) decided to recreate its password list as a 0 byte file and left us with no FTP. I took the time to do an upgrade to Filezilla and wrote a little python script to create entries in Filezillas xml config for all the users (though I had to manually create a .csv with all the usernames/passwords in). Simple things like this are another reason I like Python, it’s great tool for a SysAdmin to get those annoying little jobs done.

import md5

users = open("usernames.csv","r")
outputxml = open("usernames.xml","w")

usernamestr = """
<User Name="%s">
<Option Name="Pass">%s</Option>
<Option Name="Group"/>
<Option Name="Bypass server userlimit">0</Option>
<Option Name="User Limit">0</Option>
<Option Name="IP Limit">0</Option>
<Option Name="Enabled">1</Option>
<Option Name="Comments"/>
<Option Name="ForceSsl">0</Option>
<IpFilter>
<Disallowed/>
<Allowed/>
</IpFilter>
<Permissions>
<Permission Dir="F:\FTPFILES\%s">
<Option Name="FileRead">1</Option>
<Option Name="FileWrite">1</Option>
<Option Name="FileDelete">1</Option>
<Option Name="FileAppend">1</Option>
<Option Name="DirCreate">1</Option>
<Option Name="DirDelete">1</Option>
<Option Name="DirList">1</Option>
<Option Name="DirSubdirs">1</Option>
<Option Name="IsHome">1</Option>
<Option Name="AutoCreate">0</Option>
</Permission>
</Permissions>
<SpeedLimits DlType="0" DlLimit="10" ServerDlLimitBypass="0" UlType="0" UlLimit="10" ServerUlLimitBypass="0">
<Download/>
<Upload/>
</SpeedLimits>
</User>
"""

def md5hash(password): return md5.new( password ).hexdigest()

for line in users.readlines():
    user , password = line.split(",")[0], line.split(",")[1][:-1]
    print user, password
    outputxml.write(usernamestr % (user, md5hash(password), user))

outputxml.close()

Passwords md5 hashed in 1 line and most of these 50 lines are the template for Filezilla. Definitely satisfies the mantra “Simple Things Should Be Simple, Complex Things Should Be Possible”. That the same language can be so applicable right across the complexity spectrum is amazing. I love the learning curve for Python, very smooth and it can take you very far.

Share on Facebook

The one from yesterday (#148) is quite interesting, taking postfix notated expressions and returning an infix version.

For example
2 3 5 + * -> 2 * (3 + 5)

I spoilt it a bit by reading Reverse Polish Notation on Wikipedia, which gave away how to evaluate the postfix expressions.

Here is my python code to evaluate postfix expressions

ops = ["+","*","/","-"]

def evaluate(inputpfs):
    numstack =[]
    opstack = []
    for i in inputpfs.split():
        if i not in ops:
            numstack.append(i)
        else:
            right = numstack.pop()
            left = numstack.pop()
            #print (left + i +  right)
            numstack.append(str(eval(left + i +  right)))
    assert len(numstack) == 1
    return eval(numstack[0])

Then I realised all I needed to do was remove eval, add brackets and I would have what I needed.

def bracket(somestring):
    return "(" + somestring + ")"

def post2infix(inputpfs):
    numstack = []
    opstack = []
    resultstr = ""
    for i in inputpfs.split():
        if i not in ops:
            numstack.append(i)
        else:
            right = numstack.pop()
            left = numstack.pop()
            newresult = " ".join([left, i, right])
            newresult = bracket(newresult)
            numstack.append(newresult)
    return numstack[0]

Then only printing brackets when you must. If the operation is – or /; you always need them around the right hand expression, unless is is a plain number as they are not associative. If the operation is +, you never need them. If the operation is * then you need them if you have + or – in the right hand expression. Note that left is always a plain number as only the top element of the stack is a compound expression

def isnum (somestring):
    return all([i not in somestring for i in ops]) 

def post2infixpp(inputpfs):
    numstack = []
    opstack = []
    resultstr = ""
    for i in inputpfs.split():
        if i not in ops:
            numstack.append(i)
        else:
            right = numstack.pop()
            left = numstack.pop()
            if not isnum(right):
            #if (not all([i not in right for i in ops])):
                if i in ["/", "-"]:
                    right = bracket(right)
                elif i=="*" and ("-" in right or "+" in right):
                    right = bracket(right)
            newresult = " ".join([left, i, right])
            numstack.append(newresult)
    return numstack[0]

A subtle bug means I could not do isnum inline (commented out, line 15 here), I had to add an auxiliary function. Though this way is more comprehensible, I cannot figure out why swapping lines 14 and 15 breaks the code.

Testing: I know I should have done a bit more error checking in the above functions, but here is the little test sequence I wrote.

eg1 = "2 3 +"
eg2 = "2 3 5 + *"
eg3 = "56 34 213.7 + * 678 -"
egs = [eg1,eg2,eg3]

for i in egs:
    print "*********"
    print i
    print evaluate(i)
    print post2infix(i)
    print post2infixpp(i)
    assert (#evaluate(i) ==
            eval(post2infix(i)) ==
            eval(post2infixpp(i))
           )

I had to comment out the evaluate(i) part of the assertion as it failed on eg3 due to some sort of floating point rounding error. I will try and track down why tomorrow.

evaluate(eg3) == 13193.200000000001
eval(post2infixpp(eg3)) == eval(post2infix(eg3)) == 13193.199999999999

Get the file Here

Share on Facebook

The one I am excited about most is for SAGE. SAGE is written in python and leverages the hard work of the other projects (its motto is Building the Car Instead of Reinventing the Wheel). See the TeXmacs plugin here and some screenshots of SAGE here.

I will take a look at the code and see if I can contribute anything, it will be nice to get involved in a big project.

Share on Facebook