Swizec Teller - a geek with a hatswizec.com

    Perl will solve your bug!

    Yesterday upon coming to work I was greeted by an e-mail saying that Detextive is having issues when large files are uploaded. It instantly felt suspicious because it didn't use to have these problems. So off I go and try a 100k word file on for size and woe and behold, the upload hung. Not to be deterred I turned on debugging and tried again, this time the culprit reared its ugly head and said hello.

    The file uploaded well, but where it failed was ... counting the number of words there was. Strange huh? See the word counting "algorithm" kind of took more than 30 seconds to execute on such a monstrous text and so a timeout occured. I could've gone the usual solution, shoved it into an asynchronous process and simply wait it out, but it didn't feel right. I shouldn't have to do something like that just to bloody count how many words there were in a file. And there wasn't anything I could do with the algorithm to speed it up since regex function calls where hanging not the algorithm itself.

    So I turned to python, python is cool right? It's bloody fast, has awesome string support and is overall very easy to program, especially since I already have some experience with it. But what's this? It doesn't support unicode character properties in regexes? WHAT? What the hell ... onward!

    After some shuffling around and trying to figure out what to do I decided, what the hell, let's try perl. And that's exactly what I did, I learned perl from scratch just to solve a bug, crazy. Next thing you know I'm finding perl quite easy to code even though it's just about the same age I am and what's more, it counted the words and such not in a few seconds less, but in a bloody second! In light of this new discovery of perl's quickness I decided to put some other things into perl, for example in a later step I found out php's regexes and such take seven minutes to turn text into an array of words (for a 100k word file) and that's just not acceptable.

    In other news, Tea Cooker has been downloaded over 120 times and yesterday reached a stage where it is in fact very useful.

    Did you enjoy this article?

    Published on July 31st, 2008 in life,

    Learned something new?
    Read more Software Engineering Lessons from Production

    I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.

    Subscribe below 👇

    Software Engineering Lessons from Production

    Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.

    "Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"

    ~ Ashish Kumar

    Join 15,883+ engineers learning lessons from my "raw and honest from the heart" emails.

    ⭐️⭐️⭐️⭐️✨
    4.5 stars average rating

    Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.

    Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.

    Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.

    Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev

    Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization

    Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections

    Want to brush up on modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com

    Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog

    Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com

    By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️

    Created by Swizec with ❤️