For the longest of whiles I've been working on a speed comparison between Node.js and Clojure. Today I had some time in a place with unreliable internets and finally gave it a whirl since there was nothing better to do, I certainly wasn't going to merely enjoy the sun.
I did my best to implement the same algorithm in both. It creates a list of numbers from 3 to N, then filters it of anything that isn't a prime. Through a bit of trial and error this turned out to be the quickest approach, possibly because it means we can run prime-ness tests in parallel on several numbers.
For node.js I also implemented a bit different algorithm that builds a list of known primes and iterates through that instead of everything under the square root of the target. It can't run in parallel, but turns out it's faster ... couldn't figure out how to implement it in clojure to be fast as you can see from my post on prime searching in clojure.
As far as initial observations go there were a few interesting things I discovered:
- Node is super fast for heavy but small calculations and seems to break down for large datasets.
- Clojure really benefits from native implementations of filter and such. I suspect that they are running in parallel because it is consistently burning 15+ threads.
- Jacking up the max N to 100,000,000 caused a memory allocation error in the classic node.js algorithm, but worked fine with the cutesier one.
- Both Node and Clojure were burning around 600 megs of RAM when doing the 1M test
The computer I was testing on is my trusty MBP with 4gigs of RAM and a 2.4gig core2duo. To avoid as much artefacts as possible each test was run five times and the average runtime was calculated. Just in case it's relevant, the computer was running on battery at the time, there's a slight chance this means the CPU was clocked down.
The code
This is the basic implementation in node.js, the more cutesy algorithm just uses a global array of known primes and iterates over those, but otherwise the code is the same so it'd be silly to post it twice.
var async = require('async');
var isPrime = function (n, callback) {
if (n%2 == 0) {
callback(false);
return;
}else{
var root = Math.sqrt(n);
for (var i=3; i<=root; i += 2) {
if (n%i == 0) {
callback(false);
return;
}
}
}
callback(true);
}
var primes = function (n, callback) {
var acc = new Array();
for (var i=2; i<n; acc.push(i++));<p="">
async.filter(acc, isPrime, function(results){
results.unshift(2);
callback(results);
});
}
primes(process.argv[2], function (result) {
});
</n;>
This is practically the same code in clojure:
(defn prime? [n]
(if (even? n) false
(let [root (num (int (Math/sqrt n)))]
(loop [i 3]
(if (> i root) true
(if (zero? (mod n i)) false
(recur (+ i 2))))))))
(defn primes [n]
(let [nums (loop [i 2 acc []]
(if (> i n) acc
(recur (inc i) (cons i acc))))]
(concat (filter prime? nums) [2])))
(primes (Integer/parseInt (first *command-line-args*)))
Immediately we can see that the clojure code is much more concise; which code is more readable is a bit harder to say. Personally I'd lean towards javascript for readability, but there is a certain level of elegance in lisp.
This is the code I used to run the tests. You can see for each run a process is spawned and the cleanest possible time is measured. This helps us avoid any memory leaking issues that could slow down the code, but introduces a small penalty for spawning processes. I'm assuming this penalty is constant.
var spawn = require("child_process").spawn;
var N = 100000000;
var runs = function (n, i, avg) {
var i = i || 0;
var avg = avg || 0;
var before = new Date().getTime();
var child = spawn("node", ["many-primes.js", N], { cwd: __dirname });
//var child = spawn('clj', ['many-primes.clj', N], {cwd: __dirname});
child.on("exit", function (code) {
var time = (new Date().getTime() - before) / 1000;
console.log(time);
if (i < n) {
runs(n, i + 1, avg + time);
} else {
console.log(avg / n);
}
});
};
runs(5);
The results
As you can see, looking for primes is a bit of an exponential problem. However, if you have a linear solution I would absolutely love to see it.
The more interesting part is how differently exponential it is with the same algorithm in different runtime environments. I have no idea what's going on with node.js on those large datasets. Both algorithms seem to be running on a nice exponential curve and then BAM, shoots through the roof and even dies completely. Whereas Clojure's biggest problem with the small datasets is apparently the run-up time itself and then it continues growing on a predictable exponential curve.
Conclusion
My conclusion from all this is that despite everything, despite all the awesome optimiziations the V8 engine does, clojure is simply more appropriate when you're doing serious calculation on serious datasets. Go ahead and use node.js for everything nice and small, it's absolutely magnificent there.
The next relevant test would probably input/output since that is supposedly node's strongpoint, but something tells me the story will be similar. Node better for small bursts of activity and clojure better for more sustained hardcore work.
Let me know what you think, where did I fuck up this test?
PS: I know clojure has type hinting, they proved to slow down the code.
Continue reading about Comparing clojure and node.js for speed
Semantically similar articles hand-picked by GPT-4
- Checking for primes? Dumber algorithm is faster algorithm
- Benchmarking node, tornado and django for concurrency
- Project euler is a fun way to become a better geek
- Chrome's console.log is the slowest
- Hard work is a total waste of time
Learned something new?
Read more Software Engineering Lessons from Production
I write articles with real insight into the career and skills of a modern software engineer. "Raw and honest from the heart!" as one reader described them. Fueled by lessons learned over 20 years of building production code for side-projects, small businesses, and hyper growth startups. Both successful and not.
Subscribe below 👇
Software Engineering Lessons from Production
Join Swizec's Newsletter and get insightful emails 💌 on mindsets, tactics, and technical skills for your career. Real lessons from building production software. No bullshit.
"Man, love your simple writing! Yours is the only newsletter I open and only blog that I give a fuck to read & scroll till the end. And wow always take away lessons with me. Inspiring! And very relatable. 👌"
Have a burning question that you think I can answer? Hit me up on twitter and I'll do my best.
Who am I and who do I help? I'm Swizec Teller and I turn coders into engineers with "Raw and honest from the heart!" writing. No bullshit. Real insights into the career and skills of a modern software engineer.
Want to become a true senior engineer? Take ownership, have autonomy, and be a force multiplier on your team. The Senior Engineer Mindset ebook can help 👉 swizec.com/senior-mindset. These are the shifts in mindset that unlocked my career.
Curious about Serverless and the modern backend? Check out Serverless Handbook, for frontend engineers 👉 ServerlessHandbook.dev
Want to Stop copy pasting D3 examples and create data visualizations of your own? Learn how to build scalable dataviz React components your whole team can understand with React for Data Visualization
Want to get my best emails on JavaScript, React, Serverless, Fullstack Web, or Indie Hacking? Check out swizec.com/collections
Did someone amazing share this letter with you? Wonderful! You can sign up for my weekly letters for software engineers on their path to greatness, here: swizec.com/blog
Want to brush up on your modern JavaScript syntax? Check out my interactive cheatsheet: es6cheatsheet.com
By the way, just in case no one has told you it yet today: I love and appreciate you for who you are ❤️