Strangest line of python you have ever seen

Nov 18 2011

An example of a NFA state diagram.

Image via Wikipedia

The other day @HairyFotr and @zidarsk8 were doing some codegolfing with implementations of nondeterministic finite state machineand asked me to blog their results.

For those of us who often forget what all of this computer science mumbo jumbo means, here’s a quick explanation from wikipedia:

In the theory of computation, a nondeterministic finite state machine or nondeterministic finite automaton (NFA) is a finite state machine where for each pair of state and input symbol there may be several possible next states.

Essentially they were looking for the shortest implementation of an algorithm that can take a bunch of states, go through all of them on each step and then backtrack to find the solution.

@zidarsk8 doesn’t really know python all that well so his way to optimize things was basically “SHORTEN ALL THE CODES!” and as a result he came up with this line nobody understands.

state = [st for s in state for st in states[(s,letter)]]

At first it looks just like a double loop. But then you notice the right-most for is taking the list to iterate over from its own body, which is the iterator of the left-most for loop …

What?

Seriously, if you can explain how this works you win a lot of internets, eternal fame and I might just send you a box of cookies.

Here’s the whole implementation in case you were wondering

from optparse import OptionParser
from collections import defaultdict
 
def run(beseda):
       a, states = open("machine.txt") ,defaultdict(list)
       state, final= a.readline().split()[1:],a.readline().split()[1:]
       [states[(i.split()[0], i.split()[1])].append(i.split()[3]) for i in a]
       for letter in word:
               state = [st for s in state for st in states[(s,letter)]]
       return any(i in state for i in final)
 
print [(b,"YES") if run(b) else (b,"NO") for b in OptionParser().parse_args()[1]]

According to our best debugging efforts it works as advertised … even though we can’t actually understand why or how it’s even possible that python knows what to do.

For curiosity’s sake, here’s @HairyFotr’s Scala implementation

object NKA extends App {
    import scala.collection.mutable._
    val gates = new HashMap[(String,Char), ListBuffer[String]]
    val lines = io.Source.fromFile("avtomat.txt").getLines.toSeq
    val (init,finals) = (lines(0).split(" ")(1),lines(1).split(" ").tail)
    lines.tail.tail.map(_.split(" ")).foreach 
        {s => gates.getOrElseUpdate((s(0),s(1)(0)), ListBuffer()) += s(3)}
 
    def crawl(state:String, input:String):Boolean = 
        (input!="" && (false /: gates.getOrElse((state,input(0)), return false))
            {_ || crawl(_, input.tail)}) || (input=="" & finals.contains(state))
 
    args.foreach(in => println(in + (if(crawl(init,in)) ": YES" else ": NO")))

If you ask me, this looks like a bunch of gibberish and even HairyFotr says it isn’t the prettiest Scala code out there. But hey, this is codegolf, all that matters is minimizing those keystrokes!

To conclude, two challenges:

  1. Explain how that line of python works
  2. Come up with a shorter solution … I’m guessing golfscript is a good choice

PS: I was serious about those cookies

PPS: an example of what the automata decription looks like http://pastebin.com/QLW1BfFj

Related articles
Enhanced by Zemanta

Related Posts

---
Need a freelance developer? Email me!

You should follow me on twitter
 Subscribe to RSS

33 responses so far

  • anony moose

    This is really simple:

    state = [st for s in state for st in states[(s,letter)]]You can interpret this as:new_state = []for s in state: for st in states[(s,letter)]: new_state.append(st)state = new_state

  • MD

    Clue: The ‘state’ on the LHS has nothing to do with the ‘state’ on the RHS.

  • dbaupp

    The last point of http://www.python.org/dev/peps/pep-0202/ explains why it works.

    The line is equivalent to:
    state = []for s in state: for st in states[(s,letter)]]: state.append(st)

  • Anonymous

    (That should have more newlines and indents)

  • ohhi

    The python is actually easy to follow … just takes some time to get used to it.
    I have broken it down for you here –> http://pastebin.com/q8HFAczC

  • http://www.danmorelle.com Dan Morelle

    Seems quite elegant.

  • http://twitter.com/zidarsk8 Miha Zidar

    it would seem all your comments kinda make sense, but I always thought it was something like this

    (stuff to do with X) for X in somewhere

    and it seemd weird that somewhere had something to do with (stuff to do with X)

    guess I should learn more python

  • http://twitter.com/zidarsk8 Miha Zidar

    Thank you. I kinda thought that the body of such a for loop was always to the left of the loop. This seems kind of weird, but I guess I learned something new.

  • Anonymous

    Here’s an implementation in 2 lines of Prolog: https://gist.github.com/1376411

  • http://profiles.google.com/jonelf Jonas Elfström

    What’s up with the beseda variable? It seems it’s never used.

    What’s in machine.txt?

  • http://twitter.com/btaylordesign Brandon Taylor

    This is a nested list comprehension. I generally avoid these whenever possible as they’re very difficult to read. Here’s the official documentation: http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions

  • CipE

    new_statevector= [next 
    for current in statevector  for next in possiblemachinestates[(current,letter)]]

  • Arnar Birgisson

    Like others have said, it’s not very complex.  state holds the list of states we might be in (since it’s a NFA). Then

    [states[(s, letter)] for s in state]

    will return a list of lists, giving the possible next states after following letter-edges. The other for only serves to concatenate those lists into one big one with this pattern

    [x for xs in listoflists for x in xs]

    The body of the rightmost for is only st, not the other for. The translation from the PEP makes it clear as dbaupp pointed out.

  • http://zephyrfalcon.org/ Hans Nowak

    This is code golf? I’m curious to see the Haskell version. :)

  • Guest

    Golfscript is always a good choice!

    https://gist.github.com/1376606

  • iig

    This is very similar to the way you flatten lists:
    flat = [item for sublist in list for item in sublist]

  • http://twitter.com/HairyFotr HairyFotr

    Prolog is awesome, but the python and scala codes are also only 3 lines if you disregard the parsing of data. I’d like to see that in Prolog… :)

  • Anonymous

    What that line does mainly is confusing future readers. That includes the future self of the original author.

  • Sami Hangaslammi

    A simple Haskell solution: https://gist.github.com/1376795

  • Greg Turnquist

    And I thought the obfuscated C code contests were hard to read!

  • martior

    I wouldn’t call it easy, but that is what I got as well when I got my head around it. Also found this explanation: http://rhodesmill.org/brandon/2009/nested-comprehensions/

  • http://twitter.com/zidarsk8 Miha Zidar

    this is at the bottom of the blog: with english words for readability and an automata example

    http://pastebin.com/QLW1BfFj

  • http://twitter.com/den_sh Den Shabalin

    Python version is definetely written in perl style (programatic and read-only). I’ve rewritten it to make it readable:

    https://gist.github.com/1377187

  • http://twitter.com/HairyFotr HairyFotr

     It’s codegolf… it’s meant to be short, fun and confusing :)  

  • Bonelake

    Here’s an analogous, but much simpler example that shows whats going on.
    >>> [(s,st) for s in [1,2] for st in [1,2,3]][(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3)]

  • http://2leep.com Ruslan

    thanks, just few cents for other who will try this code: 
    In [2]: [(s,st) for s in [1,2] for st in [1,2,3]]
    Out[2]: [(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3)]

  • Ron

    Just rewrite it as a regular for loop:

    result = []
    for s in state:
      for st in states[(s,letter)]:
        result.append(st)

  • http://twitter.com/den_sh Den Shabalin

    Alternative version without a class:

    https://gist.github.com/1377685

  • rjsen

    Obligatory perl one-liner:

    perl -E ‘say”[",map({open(my$a,"machine.txt");my%s;my@l=;my@s=split(" ",shift@l);shift@s;my@f=split(" ",shift@l);shift@f;map{my@i=split(" ",$_);push@{$s{$i[0].$i[1]}},$i[3]}@l;map{my$l=$_;@s=map{@{$s{$_.$l}}}@s}split(“”,$_);grep({my$i=$_;grep /$i/,@s}@f)?”($_,”YES”)”:”($_,”NO”)”}@ARGV),”]”;’

  • Christopher Allen-Poole

    But where is word defined?

  • Derek McLean

    Eleven ugly lines without a nested list comprehension: http://pastebin.com/i1tESYuV

  • Pingback: Links 20/11/2011: GNU/Linux in Tamil Nadu, Flex Donated to Apache | Techrights

  • Pingback: A geek with a hat » Best blogging week I have ever had

« Why I only work with startups Best blogging week I have ever had »