The difference between code and data

Today I finished watching SICP 2B video and it was what inspired me to write this post.

Recently I started working on SICP (https://github.com/bor0/sicp) and I also implemented tinylisp (similar to Peter Norvig’s lis.py) in PHP (https://github.com/bor0/tinylisp).

So the 2B video ends with Abelson defining cons in terms of procedures that return other procedures. In this way, one can ask the question “What really is data?”

In almost all of the widely used languages today (JavaScript, PHP, Python), you’ll find that functions are all first-class objects, and this is all thanks to Lisp.

The contract for cons, car, and cdr is as follows:

(car (cons x y)) = x
(cdr (cons x y)) = y 

And that’s all really is to data. Any kind of data can be constructed using cons, car, and cdr.
So here is how we can implement cons, car, and cdr in Lisp:

(define (cons a b)
(lambda (pick)
(if (= pick 1) a b)))

(define (car a) (a 1))
(define (cdr a) (a 2))

And then ask the evaluator:

> (cons 1 2)
#
> (car (cons 1 2))
1
> (cdr (cons 1 2))
2
>

One needs to take a moment here and think how these are implemented (and possibly try to reduce the examples using the substitution model to get a sense of what really is going on).

For the funs, I implemented this in PHP:

<?php
function cons($x, $y) {
// context 1
return function ($a) use ($x, $y) {
// context 2
return ($a == 1) ? $x : $y;
};
}

function car($x) {
return $x(1);
}

function cdr($x) {
return $x(2);
}

$x = cons(1, 2);
var_dump($x(1));
var_dump(car($x));
var_dump(cdr($x));
boro@bor0:~$ php test.php
int(1)
int(1)
int(2)
boro@bor0:~$

This can give you an idea of how powerful it is to have multiple contexts, and the things we can do given them.

If you look at the tinylisp code, you’ll get a sense of how cons works in terms of contexts.

Prolog Proof search trees

Somewhere I read that writing programs is much like writing mathematical proofs.
Let’s try to understand what’s meant by that, by playing with Prolog.

Consider the following Prolog facts, that is, given definitions:

bigger(cat, mouse).
bigger(mouse, bug).

Now, we can ask Prolog what’s smaller than a cat?

?- bigger(cat, X).
X = mouse.

It of course responds that a mouse is smaller than a cat.
But what about a bug? Why didn’t it report that a bug is smaller than a cat?
This is so because Prolog doesn’t know much about this relation. Other than what is supplied to it, it assumes everything else is untrue. So, for instance, if we did:

?- bigger(cat, bug).
false.

We would get a negative answer.

To fix this, we need to implement a transitive relation of bigger. What this means is that if we have that A·B and B·C, then A·C holds as well, that is, ∀a, b, c ∈ X : (a·b ∧ b·c) ⇒ a·c. Do you see the mathematical similarity here?

In this case, · would be bigger. Let’s call this relation is_bigger.

This is how it looks:

is_bigger(A, B) :- bigger(A, B).
is_bigger(A, B) :- bigger(A, C), is_bigger(C, B).

Doing a couple of tests:

?- is_bigger(cat, mouse).
true .

?- is_bigger(mouse, bug).
true .

?- is_bigger(cat, bug).
true .

It seems to be working fine!

Now let’s try to explain the definition.

The :- operator here is like mathematical if, that is, if we did a :- b, and then ask for the truth value of a, it would report true if b were true as well. So by is_bigger :- bigger, we say that the former is true if the latter.

When you define multiple definitions in Prolog, it tries to find which one of them is true. In this case, it’s acting kind of an OR, it tries the first definition, then the second, and if all of them fail the result would be false, but even one of them succeeds and the result is true. Note that, we could also rewrite the definition like:

is_bigger(A, B) :- bigger(A, B); bigger(A, C), is_bigger(C, B).

That is, rewrite it using the ; operator as an OR. This is also valid, but much less readable.

In contrast, when you do a, b, c. in Prolog (i.e. comma operator), it does an AND, i.e. it first tries to prove a, then b, then c. If one of them fails, the result is false. Otherwise, it’s true.

One could easily notice that this definition is a recursive one, and the first one should be the base case of the recursion. This is correct.

So, calling is_bigger(cat, bug) would fail for the first definition, and then proceeds to the second (recursive) one to see if it can find a true match. So it tries to prove bigger(A, C), is_bigger(C, B), that is, it tries to find a C such that these 2 terms are logically true. But does there exist such C? Let’s try to further expand.

is_bigger(C, B) would first attempt to prove bigger(C, B), that is, bigger(C, bug). C in this case (according to the given facts) would be mouse. This is a possible match. Let’s return to bigger(A, C) and see if we can also fit mouse to it, i.e. is it true that bigger(cat, mouse)? Of course, this is also in the definitions. So for C = mouse, we can prove that bigger(A, C), bigger(C, B) is true. From this, it follows also that is_bigger(cat, bug) is true.

Would you agree that we’ve just written a mathematical proof? Given a few definitions, using transitivity we were able to deduce some fact, using Prolog.

The gap between Prolog and Mathematics is too small, in contrast to other languages, say Python and Mathematics. This is why notation is important. It makes you think in a different way for tasks you likely already solved using other languages.

So, let’s turn to our everyday languages for a while. What we have to deal with is some trees, some equality checking and we should be able to easily implement this in another language. Let’s try Python.

We’re going to write a program that attempts to prove a given search tree, just what Prolog did. With this, I think we will be able to see how writing programs is actually much like writing mathematical proofs.

I won’t go through the details, but here’s how it could look:


class node(object):
def __init__(self, value, children = []):
self.value = value
self.children = children

def contains(self, value):
for x in self.children:
if x.value == value:
return True

return False

def __repr__(self, level=0):
ret = " "*level+repr(self.value)+"\n"
for child in self.children:
ret += child.__repr__(level+1)
return ret

# The relation tree to work with
tree = node("bigger", [
node("cat", [node("mouse"), node("bug")]),
node("mouse", [node("bug")]),
])

# This function would attempt to find a correct proof for the given task
# It parses the tree
def find_proof(tree, A, B):
if tree.value == A and tree.contains(B):
return True

for node in tree.children:
if find_proof(node, A, B):
return True

return False

# Try to find proof
print(find_proof(tree, 'cat', 'mouse'))
print(find_proof(tree, 'mouse', 'bug'))
print(find_proof(tree, 'cat', 'bug'))

It returns:


True
True
True

What would be more interesting is to write a function that would generate the proof tree instead of us manually supplying it 🙂

Svpion Fifth problem

I attempted myself to solve #5 from Five programming problems every Software Engineer should be able to solve in less than 1 hour in Haskell.

So the task is:
Write a program that outputs all possibilities to put + or – or nothing between the numbers 1, 2, …, 9 (in this order) such that the result is always 100. For example: 1 + 2 + 34 – 5 + 67 – 8 + 9 = 100.

Initially I thought,
“Hey, this should be easy. All we need to do is create an algebra that contains Addition, Subtraction, and Multiplication (being concatenation) and we should be good!”

So I proceeded:

data Expr =
A Expr Expr |
S Expr Expr |
C Expr Expr |
I Int deriving Show

parse :: Expr -> Int
parse (A a b) = parse a + parse b
parse (S a b) = parse a - parse b
parse (C a b) = parse a * 10 + parse b
parse (I a) = a

This should be it! So for example 1 + 2 + 3 would be A (I 1) (A (I 2) (I 3)), and getting the result of it is to call parse like:

*Main> parse $ A (I 1) (A (I 2) (I 3))
6

Now all we need to do is create all possibilities of the form (1 _ 2 _ 3 _ 4 _ 5 _ 6 _ 7 _ 8 _ 9) and replace _ with any of the three operators and then finally do something like filter (== 100) parse x.

But using this grammar, there are some non-valid cases such as C (A (I 1) (I 2)) (I 3).
These are cases that we must exclude, for if we don’t, then what this expression would evaluate to is 33, i.e. (1 + 2) * 10 + 3, but this is not a valid expression given the stated task.
However 1 + 23 or 12 + 3 or 1 + 2 + 3 are.

To take care of this, we will slightly rework our grammar and parsing functions:

data Expr =
A Expr Expr |
S Expr Expr |
CE CExpr deriving Show

data CExpr =
C CExpr CExpr |
I Int deriving Show

parse :: Expr -> Int
parse (A a b) = parse a + parse b
parse (S a b) = parse a - parse b
parse (CE c) = parseCE c

parseCE :: CExpr -> Int
parseCE (C a b) = parseCE a * 10 + parseCE b
parseCE (I a) = a

Another constraint that we need to add is that the concatenations need to be successive. So we somehow need to exclude those cases as well from all of the possibilities. Let’s call this function getCExprs. So what getCExprs should do is, given a list, it should return possible successive concatenations of that list. Successive concatenations is what will allow us to remove the non-valid cases.

E.g. getCExprs [I 1,I 2,I 3] = [I 1,C (I 1) (I 2),C (C (I 1) (I 2)) (I 3)].

Additionally (we’ll see why within the foldingFunction), we want getCExprs to return the remaining part of the list (digits) it’s working on, so:

getCExprs [I 1,I 2,I 3] = [([I 2,I 3],I 1),([I 3],C (I 1) (I 2)),([],C (C (I 1) (I 2)) (I 3))].

To implement this, we’ll need a help function called listToC that given a list [I 1,I 2,I 3], it will turn it into its concatenated algebraic version, C (C (I 1) (I 2)) (I 3).

The definition for this is trivial:

listToC :: [CExpr] -> CExpr
listToC (x:xs) = foldl C x xs
listToC _ = I 0

Now we are ready to go for getCExprs:

getCExprs :: [CExpr] -> [([CExpr], CExpr)]
getCExprs x = go 1 (tail $ inits x)
where
go n xs@(x':xs') = (drop n x, listToC x') : go (n + 1) xs'
go _ [] = []
Some simple tests:

*Main> getCExprs $ map I [1,2,3]
[([I 2,I 3],I 1),([I 3],C (I 1) (I 2)),([],C (C (I 1) (I 2)) (I 3))]
*Main> getCExprs $ map I [1..9]
[([I 2,I 3,I 4,I 5,I 6,I 7,I 8,I 9],I 1),([I 3,I 4,I 5,I 6,I 7,I 8,I 9],C (I 1) (I 2)),([I 4,I 5,I 6,I 7,I 8,I 9],C (C (I 1) (I 2)) (I 3)),([I 5,I 6,I 7,I 8,I 9],C (C (C (I 1) (I 2)) (I 3)) (I 4)),([I 6,I 7,I 8,I 9],C (C (C (C (I 1) (I 2)) (I 3)) (I 4)) (I 5)),([I 7,I 8,I 9],C (C (C (C (C (I 1) (I 2)) (I 3)) (I 4)) (I 5)) (I 6)),([I 8,I 9],C (C (C (C (C (C (I 1) (I 2)) (I 3)) (I 4)) (I 5)) (I 6)) (I 7)),([I 9],C (C (C (C (C (C (C (I 1) (I 2)) (I 3)) (I 4)) (I 5)) (I 6)) (I 7)) (I 8)),([],C (C (C (C (C (C (C (C (I 1) (I 2)) (I 3)) (I 4)) (I 5)) (I 6)) (I 7)) (I 8)) (I 9))]
*Main>

Seems to be working fine!

Next, we want to create a function f that has three parameters:
1. Current expression calculated with add/sub so far, Expr
2. Current operation being done, String
3. Remaining list of expressions (digits and successive concatenated digits) to work on, [CExpr]
and this function should return a list of (Expr, String), i.e. which expression is produced for what operations.

So we have:

f :: Expr -> [String] -> [CExpr] -> [(Expr, [String])]

This should be a fold, so what f should do is basically go through all valid possibilities (which are created by getCExprs), so, what we have so far:

f s ops [] = [(s, ops)]
f s ops xs = foldr foldingFunction [] $ getCExprs xs

In the first definition of f, we pattern match against an empty list, which is basically the base case and it returns the last pair of (sum, operations) done at that point.

So, this is what we currently have:

import Data.List (inits, nub)

data Expr =
A Expr Expr |
S Expr Expr |
CE CExpr deriving Show

data CExpr =
C CExpr CExpr |
I Int deriving Show

parse :: Expr -> Int
parse (A a b) = parse a + parse b
parse (S a b) = parse a - parse b
parse (CE c) = parseCE c

parseCE :: CExpr -> Int
parseCE (C a b) = parseCE a * 10 + parseCE b
parseCE (I a) = a

listToC :: [CExpr] -> CExpr
listToC (x:xs) = foldl C x xs
listToC _ = I 0

getCExprs :: [CExpr] -> [([CExpr], CExpr)]
getCExprs x = go 1 (tail $ inits x)
where
go n xs@(x':xs') = (drop n x, listToC x') : go (n + 1) xs'
go _ [] = []

f :: Expr -> [String] -> [CExpr] -> [(Expr, [String])]
f s ops [] = [(s, ops)]
f s ops xs = foldr foldingFunction [] $ getCExprs xs
where
foldingFunction = undefined

So, all we need to do is implement foldingFunction and we are done.

To be able to implement foldingFunction, we need to look at getCExprs and see what it produces for us. We know that it gives us back a pair, ([CExpr], CExpr). CExpr is the current concatenations done, and [CExpr] is the remaining part of the list.

Therefore,
foldingFunction (a, b) l = undefined

Remember f had three params. The way we defined f makes it easily callable by the foldingFunction.

We need to call f from within the foldingFunction and add the current value we are iterating to the sum. We also need to note which operation we are applying, and to pass the current list of digits we are working on. Note that we have the variable s (expression calculated so far) in the scope since we will define foldingFunction within f itself. We also have the variable b produced by getCExprs, but its type is CExpr. if b :: CExpr, then CE b will be Expr, which is what our function f requires, i.e. to parse a complete expression (Expr), and not just digits or concatenated digits (CExpr).

So:
foldingFunction (a, b) l = f (A s (CE b)) (“+” : ops) a

In this case, we are calling f while adding s and b, i.e., we add b to the current expression so far, and then we pass the remaining list of numbers (a) to f.

This takes care of the addition. To make messages more verbose, we’ll implement a function called “calculated” which in details will explain what’s going on:

foldingFunction (a, b) l = f (A s (CE b)) (calculated “+” b) (drop a xs)
calculated op b = (op ++ show (parseCE b)) : ops

Similarly, we need to do the same for the operation minus. And then we need to append all of the results in a single list:
foldingFunction (a, b) l =
f (A s (CE b)) (calculated “+” b) a
++
f (S s (CE b)) (calculated “-” b) a
++ l

So the full code is:

import Data.List (nub)

data Expr =
A Expr Expr |
S Expr Expr |
CE CExpr deriving Show

data CExpr =
C CExpr CExpr |
I Int deriving Show

parse :: Expr -> Int
parse (A a b) = parse a + parse b
parse (S a b) = parse a - parse b
parse (CE c) = parseCE c

parseCE :: CExpr -> Int
parseCE (C a b) = parseCE a * 10 + parseCE b
parseCE (I a) = a

listToC :: [CExpr] -> CExpr
listToC (x:xs) = foldl C x xs
listToC _ = I 0

getCExprs xs = map (\x -> (drop x xs, listToC (take x xs))) [1..length xs]

f :: Expr -> [String] -> [CExpr] -> [(Expr, [String])]
f s ops [] = [(s, ops)]
f s ops xs = foldr foldingFunction [] $ getCExprs xs
where
foldingFunction (a, b) l = f (A s (CE b)) (calculated "+" b) a
++ f (S s (CE b)) (calculated "-" b) a
++ l
calculated op b = (op ++ show (parseCE b)) : ops

main = do
let l = f (CE (I 0)) [] (map I [1..9])
let l' = filter (\x -> parse (fst x) == 100) l
mapM_ (\(x, y) -> print $ concat $ reverse y) l'

Call main to get:

[1 of 1] Compiling Main             ( test.hs, interpreted )
iOk, modules loaded: Main.
*Main> main
"+1+2+3-4+5+6+78+9"
"+1+2+34-5+67-8+9"
"+1+23-4+5+6+78-9"
"+1+23-4+56+7+8+9"
"-1+2-3+4+5+6+78+9"
"+12+3+4+5-6-7+89"
"+12+3-4+5+67+8+9"
"+12-3-4+5-6+7+89"
"+123+4-5+67-89"
"+123-4-5-6-7+8-9"
"+123+45-67+8-9"
"+123-45-67+89"
*Main>

Solving this problem in a functional language like Haskell reveals its background when it’s represented using an algebraic data type. For instance, if this problem was solved in Python, you would run through all combinations of “1_2_3_4_5_6_7_8_9”, change underscores with plus, minus, or append, and then eval the string expression. But if you solve it this way, you wouldn’t have any deeper insight regarding its algebra, e.g. the “successive concatenations” part might not be immediately visible.

Algebraic data types representation helps us with adding constraints, but to some point. Additional more complex constraints were handled by the functions themselves.

This is why I do not believe this is solvable in under 1 hour when you first meet this problem. There is much more background in this than what the author states.

Playing with the type system

Given these definitions:
(.) :: (b -> c) -> (a -> b) -> a -> c
map :: (a -> b) -> [a] -> [b]
head :: [a] -> a

Find the type of (map . head)

1. (f . g) x = f (g x)
(map . head) x = map (head x)

2. If we set
x :: [a -> b]
then we have
head x :: a -> b

3. \x -> map (head x) :: [a -> b] -> [a] -> [b]
=>
map . head :: [a -> b] -> [a] -> [b]

Let’s now try to find a senseful definition of [a -> b] -> [a] -> [b] by using hole-driven approach (inspired by http://www.youtube.com/watch?v=52VsgyexS8Q):

In this example we are using a silent hole (i.e. undefined). We are also writing the types of each argument (xs and ys)

{-# LANGUAGE ScopedTypeVariables #-}

f :: forall a b. [a -> b] -> [a] -> [b]
f xs ys = undefined
    where
    _ = xs :: [a -> b]
    _ = ys :: [a]

It’s compiling. But it does nothing.

Let’s try to do the same with a noisy hole:

data Hole = Hole

f :: forall a b. [a -> b] -> [a] -> [b]
f xs ys = Hole
    where
    _ = xs :: [a -> b]
    _ = ys :: [a]

We get this error:
Couldn’t match expected type `[b]’ with actual type `Hole’

How do we get from Hole to [b]? We have xs :: [a -> b]. What if instead we tried map?

f :: forall a b. [a -> b] -> [a] -> [b]
f xs ys = map Hole Hole
    where
    _ = xs :: [a -> b]
    _ = ys :: [a]

We now get these errors:
Couldn’t match expected type `a0 -> b’ with actual type `Hole’
Couldn’t match expected type `[a0]’ with actual type `Hole’

The second hole is obviously just ys:

f :: forall a b. [a -> b] -> [a] -> [b]
f xs ys = map Hole ys
    where
    _ = xs :: [a -> b]
    _ = ys :: [a]

Now we get:
Couldn’t match expected type `a -> b’ with actual type `Hole’

What if we try to use just xs instead of Hole?

Couldn’t match expected type `a -> b’ with actual type `[a -> b]’

Now it wants just x, not xs.

One way to get to that is to use (head xs)

f :: forall a b. [a -> b] -> [a] -> [b]
f xs ys = map (head xs) ys
    where
    _ = xs :: [a -> b]
    _ = ys :: [a]

This makes it happy.

Conclusion:
The function of type [a -> b] -> [a] -> [b] is not unique, because we could use something else for map instead of just (head xs).

Bonus:
id’ :: forall a. a -> a
id’ x = Hole
    where
    _ = x :: a

Couldn’t match expected type `a’ with actual type `Hole’

id’ x = x

Folds in imperative languages

Today I was again playing with folds, but this time implementing them in PHP.
So, consider the list [a, b, c]:
foldr f z [a,b,c] = a `f` (b `f` (c `f` z))
foldl f z [a,b,c] = (((a `f` b) `f` c) `f` z) = z `f` (c `f` (a `f` b))

Note the bolded parts of foldr and foldl. They look pretty same, just that the elements are kind of reversed (e.g. (0 – 1 – 2 – 3) against (3 – 2 – 1 – 0)).

Now consider we use array_reduce in PHP like this:


<?php
var_dump(array_reduce(array(1,2,3), function($x, $y) { return $x - $y; } ));

This will print “-6” as a result. And in Haskell:


Prelude> foldl (-) 0 [1,2,3]
-6

So it turns out we’re using a left fold. What can we do to make it a right fold?
What if we try something like


<?php
var_dump(array_reduce(array(1,2,3), function($x, $y) { return $y - $x; } ));

This one will print “2” as a result. Haskell says:


Prelude> foldr (-) 0 [1,2,3]
2

And in Haskell we can also do something like


Prelude> let f x y = y - x
Prelude> foldr f 0 [1,2,3]
-6

So, it’s interesting to play around with associativity of binary operators 🙂
But we don’t want to do it in a “hacky” way by changing the operands in the function. So we want foldr.
Here’s the full PHP code with foldr and foldl implemented:


<?php
interface iBinaryOperator {
public function calc($x, $y);
}

function foldr(iBinaryOperator $f, $z, array $xs) {
try {
$x = array_shift($xs);
if ($x == null) {
return $z;
}
return $f->calc($x, foldr($f, $z, $xs));
} catch (Exception $e) {
return "error";
}
}

// Foldl is tail recursive
function foldl(iBinaryOperator $f, $z, array $xs) {
try {
$x = array_shift($xs);
if ($x == null) {
return $z;
}
return foldl($f, $f->calc($z, $x), $xs);
} catch (Exception $e) {
return "error";
}
}

class Subtraction implements iBinaryOperator {
public function calc($x, $y) {
if (gettype($x) != "integer" || gettype($y) != "integer") {
throw new Exception("MathException");
}
return $x - $y;
}
/*
public function partial_calc($x) {
if (gettype($x) != "integer") {
throw new Exception("MathException");
}
return function ($y) use ($x) {
if (gettype($y) != "integer") {
throw new Exception("MathException");
}
return $x - $y;
};
}
*/
}

$x = new Subtraction();
var_dump(foldr($x, 0, array(1,2,3)));
var_dump(foldl($x, 0, array(1,2,3)));

/*
> php test.php
int(2)
int(-6)
*/