Let’s say you want to launch a service. For coders, by coders.

English: Coder decoder group AN/MSQ-18 (ao in ...

Coder decoder group AN/MSQ-18 (Photo credit: Wikipedia)

Let’s also say you want only coders. Nobody else should be able to get in.

Leaving aside any “Haha, launching is _hard_! Allow everyone and tweak incentives so only coders want to join.” While essentially true, coders act differently around coders than they do around people, and an exclusive environment could be awesome.

How do you make a lock only coders can get through?

The low-tech solution is a manual approval process. When a person signs up, you look at their profile and google around a little bit. Try to find them on Github, see if they have any technical blogs, have ever released a piece of software etc.

Congratulations! You’ve just made yourself a bunch of work and excluded the 80% of the coders who don’t have a strong web presence or aren’t even interested in that sort of thing.

No, we need an automated test … something only coders can solve, but trumps everyone else.

Something that:

  • Every coder can solve.
  • Nobody who isn’t a coder can solve.
  • Is simple to test.
  • Takes less than 15 seconds to solve.

But what’s the common knowledge all coders share?

The for-loop!

Everyone can write a for loop right!? How do you test a for loop? Wikia:code lists 18 different  for loops. Which means there are more … plus there’s languages like Haskell which prefer recursio. I hear there’s also a lot of coders out there using CSS and HTML that don’t even have a concept for looping.

Yes, those people are coders too. They write thousands of lines of code. Maybe not programmers – definitely coders.

Okay that was a stupid idea … but every coder has got a favourite text editor don’t they? If you like a text editor you are definitely a coder. Then again, there are thousands of text editors out there, we can’t test this one either.

Oh hey! Everyone can write a regular expression can’t they? At least a very basic one!

Regular expressions are great, they aren’t language specific since they are a language and pretty much everyone has written a basic regular expression once or twice in some of their projects.

Well, except for the guys writing hardcore C all their lives who are coding rocket launch procedures for SpaceX. Doubt they ever needed a regex. Maybe for grep or sed?

A plausible solution

A friend of mine recently (yesterday) had this problem and it seems the final solution is good enough for now.

To sign up you need to answer one of two questions:

  1. Write a very simple regex. (match New York and New Jersey, but not Newark)
  2. Or find a hidden element on the webpage and click it.

I was too lazy to do either, so I went into the page source and used node.js to run the snippet of javascript that constructs the redirect url. Also proving that I am a coder. I think …

But what about all those coders who are just starting out? Who haven’t encountered any of this stuff before? It took me years to first get into regexes … And what of those who’ve never come into contact with either regexes or front-end web development … someone coding up controllers for your car’s engine, say?

Question is, how would you make a CAPTCHA to distinguish coders from non-coders?

Enhanced by Zemanta
  • Informatimago

    Easy: ask them to implement the fizzbuzz problem. :-)

  • http://blaise.io/ Blaise Kal

    Something like: What is the final value of “mystring”? And then two or three string manipulations in python (or other easy-to-read language), with for loops, substrings, replaces, if statements and other usual suspects that are in most programming languages.

    Good thing about this is that there is only one correct answer and the question can be randomized easily to have multiple versions of a captcha.

    This means someone is at least able to read code. If users know how to run the code and fetch the output, they can be considered coder enough as well :)

    Writing code is more complicated because of differences between programming languages.

  • Scott Brickey

    I would have them sort several implementations of the same function by performance/Big-O… given 5 implementations, they can have maybe 2 incorrect answers (either swapped, or perhaps off-by-one which can cause 3 errors due to shifting) due to being “new to the game”

    examples:
    - sort algorithms… bubble, shell, b-tree, skip lists, etc

    - Fibonacci / factorial… recursion, simple loops, etc

    Granted, non-programmers could look it up… it wouldn’t be quick though

    This could also introduce newbies to new algorithms! :)

  • Anonymous

    How about “Find the missing close paren”?

  • sb

    you are ordered to “go to the supermarket, buy 6 eggs if they have bread take 2″. What do you buy?

  • zidar

    even though this is a nice problem, it’s not something every coder could finish in 15 seconds, or even a minute for that matter.

  • chuck

    Why don’t you put a big sign over the registration form saying “This site is for coders only, there’s nothing here for you otherwise”.

  • Johannes

    remove one syntax-relevant letter from the middle of a random line of code and ask for it, like:

    Whats the missing letter to make the syntax of this line of code correct?

    Answer:
    =

    @import url( http://swizec.com/blog/wp-content/themes/clear-customized/style.css ;)

    /* <!CDATA[ */
    [

    var disqus_domain 'disqus.com';
    =

    more difficult (because it will pass a syntax-checker):

    .website-name ahover,.website-url a:hover {
    :

    However, you'll have to find out what language this is to get a language-dependent list of syntax-relevant letters.

  • https://github.com/download13/ download13

    http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper3.pdf

    If you can detect who was able to form a consistent mental model of variable assignment (even if they got it wrong) you’d have the start of a good test. That would get people who don’t even know any programming languages but have the capability to learn.

  • mpelko

    The above mentioned CAPTCHA would probably exclude me and I’ve been coding for over 10 years (non-web and for some reason I managed to avoid regex). I would probably be able to look the solution up fast enough, though.

    I really thing chuck nails it. All the rest is too much hassle.

  • David Božjak

    This might be a good choice: “Separating Programming Sheep from Non-Programming Goats” (Link: http://goo.gl/qCcA4 ). The article and the academic paper are both quite old but I think that even though they answer your question quite well even though that wasn’t their goal.

    Hope it helps!

  • http://swizec.com Swizec

    I really like this one! It’s a great idea and checks for the “consistent mental model” thing some other commenters mentioned.