When you have to figure out the difference between two strings your best bet is either Hamming distance or Levenshtein distance.

Because hamming distance only works on strings of equal lengths, you’re usually better off using levenshtein distance. Its name is also harder to spell so you know it’s better!

It is defined as the minimum amount of edits needed to transform one string into another. Like this:

  1. kitten → sitten (substitution of ‘s’ for ‘k’)
  2. sitten → sittin (substitution of ‘i’ for ‘e’)
  3. sittin → sitting (insertion of ‘g’ at the end).

Levenshtein distance also happens to be a great case for learning about dynamic programming, because one of the simplest implementations involves nothing more than keeping track of the already known distances and deciding whether you should increase that number for the next character.

Doing this in a 2D array is trivially simple because it lest you travel through the distance space very efficiently without forgetting that kitten and kiten only have a levenshtein distance of 1.

The meat of the algorithm goes like this:

      if s[i] = t[j] then  
        d[i, j] := d[i-1, j-1]       // no operation required
        d[i, j] := minimum
                     d[i-1, j] + 1,  // a deletion
                     d[i, j-1] + 1,  // an insertion
                     d[i-1, j-1] + 1 // a substitution

Dynamic programming and Haskell

Doing this in Haskell becomes tricky because we don’t have state so we can’t really keep track of what we already know. Translating this into a recursive problem is our only option.

The key insight comes from looking at what those index manipulations are doing – traveling in the space of two strings.

So, if we start from the other end of the “matrix” – bottom right instead of top left – we can express the levenshtein distance recursively:

  • if last two characters are same, ignore them
  • otherwise take the minimum of ignoring either or both the last characters
  • recurse

In Haskell the function ends up looking like this:

-- calculate levenshtein distance between two strings
levenshtein::[Char] -> [Char] -> Int
levenshtein "" "" = 0
levenshtein "" s2 = length s2
levenshtein s1 "" = length s1
levenshtein s1 s2
   | last s1 == last s2 = levenshtein (init s1) (init s2)
   | otherwise = minimum [1 + levenshtein (init s1) s2,
                          1 + levenshtein s1 (init s2),
                          1 + levenshtein (init s1) (init s2)]

Better Haskellers than me can probably write this a bit cleaner – for instance you don’t need the levenshtein “” “” = 0 line, but I think it’s more readable when all the border cases are spelled out explicitly.

However, the problem is that this is incredibly slow when comparing a long string to a short one. So slow in fact, I never waited for the code to stop executing (takes 20s+).

Luckily, speeding up the code is trivial – look at those border cases. We can just handle those first and then go into the meaty algorithm!

The code becomes somewhat uglier to look at, but is usefully quick for all cases:

-- calculate levenshtein distance between two strings
levenshtein::[Char] -> [Char] -> Int
-- this part is mostly a speed optimiziation
levenshtein s1 s2
  | length s1 > length s2 = levenshtein s2 s1
  | length s1 < length s2 =
    let d = length s2 - length s1
    in d + levenshtein s1 (take (length s2 - d) s2)
-- the meat of the algorithm
levenshtein "" "" = 0
levenshtein s1 s2
  | last s1 == last s2 = levenshtein (init s1) (init s2)
  | otherwise = minimum [1 + levenshtein (init s1) s2,
                         1 + levenshtein s1 (init s2),
                         1 + levenshtein (init s1) (init s2)]

If anyone’s got an even better idea for implementing Levenshtein distance in Haskell, I’d love to hear!

edit: A Reddit commenter provides a far better Levenshtein distance function.

Enhanced by Zemanta

You should follow me on twitter, here.

Get 10 of my best articles and talks

Leave your email and over the next few weeks I will send you the best material I've written since 2010.

  • http://activedeveloper.info mhitza

    There are faster version of Levenshtein distance implementations in Haskell (and other languages) on wikibooks: http://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance#Haskell

  • ed
  • configurator

    These are based on your example:
    levenshtein is the same as your example.
    levenshtein2 should be quite a bit faster for longer strings. All I did was replace last and init with head and tail – because lists are implemented as pairs this should be a small boost.
    levenshtein3 is the dynamic programming way. Basically, d is an array with all the results, which would be lazily calculated (d2 actually calculates them) as they’re needed – then you just request the result you actually need by asking for d !! m !! n.

    I don’t have the example on which you ran these, so I’m intrigued to know if these work better for you.

  • WilliamEdwards

    0) really, you can do better than this!
    1) you can use a table…
    2) if you only want the score, you can do it in O(min(n,m)+1) space
    3) you can even do it in better than O(n*m) time using lazy eval and this is where Haskell shines