Image via Wikipedia

The other day @HairyFotr and @zidarsk8 were doing some codegolfing with implementations of nondeterministic finite state machineand asked me to blog their results.

For those of us who often forget what all of this computer science mumbo jumbo means, here’s a quick explanation from wikipedia:

In the theory of computation, a nondeterministic finite state machine or nondeterministic finite automaton (NFA) is a finite state machine where for each pair of state and input symbol there may be several possible next states.

Essentially they were looking for the shortest implementation of an algorithm that can take a bunch of states, go through all of them on each step and then backtrack to find the solution.

@zidarsk8 doesn’t really know python all that well so his way to optimize things was basically “SHORTEN ALL THE CODES!” and as a result he came up with this line nobody understands.

`state = [st for s in state for st in states[(s,letter)]]`

At first it looks just like a double loop. But then you notice the right-most for is taking the list to iterate over from its own body, which is the iterator of the left-most for loop …

What?

Seriously, if you can explain how this works you win a lot of internets, eternal fame and I might just send you a box of cookies.

Here’s the whole implementation in case you were wondering

```from optparse import OptionParser from collections import defaultdict   def run(beseda): a, states = open("machine.txt") ,defaultdict(list) state, final= a.readline().split()[1:],a.readline().split()[1:] [states[(i.split()[0], i.split()[1])].append(i.split()[3]) for i in a] for letter in word: state = [st for s in state for st in states[(s,letter)]] return any(i in state for i in final)   print [(b,"YES") if run(b) else (b,"NO") for b in OptionParser().parse_args()[1]]```

According to our best debugging efforts it works as advertised … even though we can’t actually understand why or how it’s even possible that python knows what to do.

For curiosity’s sake, here’s @HairyFotr’s Scala implementation

```object NKA extends App { import scala.collection.mutable._ val gates = new HashMap[(String,Char), ListBuffer[String]] val lines = io.Source.fromFile("avtomat.txt").getLines.toSeq val (init,finals) = (lines(0).split(" ")(1),lines(1).split(" ").tail) lines.tail.tail.map(_.split(" ")).foreach {s =&gt; gates.getOrElseUpdate((s(0),s(1)(0)), ListBuffer()) += s(3)}   def crawl(state:String, input:String):Boolean = (input!="" &amp;&amp; (false /: gates.getOrElse((state,input(0)), return false)) {_ || crawl(_, input.tail)}) || (input=="" &amp; finals.contains(state))   args.foreach(in =&gt; println(in + (if(crawl(init,in)) ": YES" else ": NO")))```

If you ask me, this looks like a bunch of gibberish and even HairyFotr says it isn’t the prettiest Scala code out there. But hey, this is codegolf, all that matters is minimizing those keystrokes!

To conclude, two challenges:

1. Explain how that line of python works
2. Come up with a shorter solution … I’m guessing golfscript is a good choice

PPS: an example of what the automata decription looks like http://pastebin.com/QLW1BfFj

#### Get 10 of my best articles and talks

Leave your email and over the next few weeks I will send you the best material I've written since 2010.

• anony moose

This is really simple:

state = [st for s in state for st in states[(s,letter)]]You can interpret this as:new_state = []for s in state: for st in states[(s,letter)]: new_state.append(st)state = new_state

• MD

Clue: The ‘state’ on the LHS has nothing to do with the ‘state’ on the RHS.

• dbaupp

The last point of http://www.python.org/dev/peps/pep-0202/ explains why it works.

The line is equivalent to:
state = []for s in state: for st in states[(s,letter)]]: state.append(st)

• Anonymous

(That should have more newlines and indents)

Thank you. I kinda thought that the body of such a for loop was always to the left of the loop. This seems kind of weird, but I guess I learned something new.

• ohhi

The python is actually easy to follow … just takes some time to get used to it.
I have broken it down for you here –> http://pastebin.com/q8HFAczC

• martior

I wouldn’t call it easy, but that is what I got as well when I got my head around it. Also found this explanation: http://rhodesmill.org/brandon/2009/nested-comprehensions/

• http://www.danmorelle.com Dan Morelle

Seems quite elegant.

it would seem all your comments kinda make sense, but I always thought it was something like this

(stuff to do with X) for X in somewhere

and it seemd weird that somewhere had something to do with (stuff to do with X)

• Anonymous

Here’s an implementation in 2 lines of Prolog: https://gist.github.com/1376411

Prolog is awesome, but the python and scala codes are also only 3 lines if you disregard the parsing of data. I’d like to see that in Prolog…

What’s up with the beseda variable? It seems it’s never used.

What’s in machine.txt?

this is at the bottom of the blog: with english words for readability and an automata example

http://pastebin.com/QLW1BfFj

This is a nested list comprehension. I generally avoid these whenever possible as they’re very difficult to read. Here’s the official documentation: http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions

• CipE

new_statevector= [next
for current in statevector  for next in possiblemachinestates[(current,letter)]]

• Arnar Birgisson

Like others have said, it’s not very complex.  state holds the list of states we might be in (since it’s a NFA). Then

[states[(s, letter)] for s in state]

will return a list of lists, giving the possible next states after following letter-edges. The other for only serves to concatenate those lists into one big one with this pattern

[x for xs in listoflists for x in xs]

The body of the rightmost for is only st, not the other for. The translation from the PEP makes it clear as dbaupp pointed out.

• http://zephyrfalcon.org/ Hans Nowak

This is code golf? I’m curious to see the Haskell version.

• Guest

Golfscript is always a good choice!

https://gist.github.com/1376606

• iig

This is very similar to the way you flatten lists:
flat = [item for sublist in list for item in sublist]

• Anonymous

What that line does mainly is confusing future readers. That includes the future self of the original author.

It’s codegolf… it’s meant to be short, fun and confusing

• Sami Hangaslammi

• Greg Turnquist

And I thought the obfuscated C code contests were hard to read!

Python version is definetely written in perl style (programatic and read-only). I’ve rewritten it to make it readable:

https://gist.github.com/1377187

Alternative version without a class:

https://gist.github.com/1377685

• Bonelake

Here’s an analogous, but much simpler example that shows whats going on.
>>> [(s,st) for s in [1,2] for st in [1,2,3]][(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3)]

• http://2leep.com Ruslan

thanks, just few cents for other who will try this code:
In [2]: [(s,st) for s in [1,2] for st in [1,2,3]]
Out[2]: [(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3)]

• Ron

Just rewrite it as a regular for loop:

result = []
for s in state:
for st in states[(s,letter)]:
result.append(st)

• rjsen

Obligatory perl one-liner:

perl -E ‘say”[“,map({open(my\$a,”machine.txt”);my%s;my@l=;my@s=split(” “,shift@l);shift@s;my@f=split(” “,shift@l);shift@f;map{my@i=split(” “,\$_);push@{\$s{\$i[0].\$i[1]}},\$i[3]}@l;map{my\$l=\$_;@s=map{@{\$s{\$_.\$l}}}@s}split(“”,\$_);grep({my\$i=\$_;grep /\$i/,@s}@f)?”(\$_,”YES”)”:”(\$_,”NO”)”}@ARGV),”]”;’

• Christopher Allen-Poole

But where is word defined?

• Derek McLean

Eleven ugly lines without a nested list comprehension: http://pastebin.com/i1tESYuV