Javascript performance: callback (async) vs Q ..

Promises/A+ Performance Hits You Should Be Aware Of

The Promises/A+ specification is a fresh and very interesting way of dealing with the asynchronous nature of Javascript. It also provides a sensible way to deal with error handling and exceptions. In this article we will go through the performance hits you should be aware of and as a side-effect do a comparison between the two most popular Promises/A+ implementations, When and Q and how they compare to Async, the lowest abstraction you can get on asynchronicity.


Basic nodejs single thread architecture:


The Case

My motivation for looking deeper into the performance of Promises/A+ was a Job Queuing system i’ve been working on named Kickq. It is expected that the system will get hammered when used on production so stress testing was warranted. After stubbing all the database interactions, essentially making the operation of job creation synchronous, I was getting odd performance results.

The test was simple, create 500 jobs in a loop and measure how long it takes for all the jobs to finish.

The measurements were in the ~550ms range and my eyeballs started to roll. “That’s a synchronous operation, it should finish in less than 3ms, WHAT THE????!?!”. After taking a few moments to let it sip in the suspect was found, it was Promises. I used them as the only pattern to handle asynchronous ops and callbacks throughout the whole project. Brian Cavalier, one of the authors of When.js, helped me pinpoint the real culprit, it was the tick:

Promises/A+ Specification, Note 4.1 In practical terms, an implementation must use a mechanism such as setTimeout, setImmediate, or process.nextTick to ensure that onFulfilled and onRejected are not invoked in the same turn of the event loop as the call to then to which they are passed.

In other words, Promises, per the specification, must be resolved Asynchronously! That comes with a cost, a heavy one apparently.

In the process of studying performance I had to create a performance library, poor mans profiling. And a benchmark test for Promises/A+ implementations that’s already used to optimize the future versions of When.


Creating The Promises/A+ Benchmark

I tried to broaden the definition of the test case. If an application uses the Promises pattern as the only way to manage how the internal parts interact, we can make a few assumptions:

  • There will be a series of promises chained together, representing the various operations that will be performed by your application.
  • The Deferred Object is used on each link of the chain to control resolution and how the promise object is exposed.
  • Throughout the whole chain of promises there can be operations that are actually synchronous, we will measure all cases.

Promises, Total Time to Resolve, 500 Loops

Promises, Memory Consumption

Difference to First Resolved Promise, 500 Loops

Perf Type Async When 2.1.0 Q 0.9.5 Promise 3.0.1
Sync Diff 0.01ms 36.62ms 186.43ms 63.96ms
Mixed Diff 5.37ms 41.78ms 226.34ms 83.83ms
Async Diff 22.42ms 58.18ms 241.80ms 93.68ms
Sync Diff vs AsyncLib 1x 3,662x 18,643x 6,396x
Mixed Diff vs AsyncLib 1x 7.78x 42.15x 15.61x
Async Diff vs AsyncLib 1x 2.60x 10.79x 4.18x

Libraries When.js v1.8.1 and Deferred are not included in this table because they resolve promises synchronously. This difference makes the Diff metric inapplicable.

Total Time of execution, 500 Loops

Perf Type Async When 1.8.1 When 2.1.0 Q 0.9.5 Deferred 0.6.3 Promise 3.0.1
Sync Total 5.15ms 12.35ms 72.35ms 301.47ms 71.25ms 80.50ms
Mixed Total 18.94ms 40.57ms 80.21ms 325.49ms 94.58ms 95.67ms
Async Total 35.70ms 50.63ms 90.52ms 337.82ms 105.87ms 107.01ms
Sync Total vs AsyncLib 1x 2.40x 14.05x 58.54x 13.83x 15.63x
Mixed Total vs AsyncLib 1x 2.14x 4.23x 17.19x 4.99x 5.05x
Async Total vs AsyncLib 1x 1.42x 2.54x 9.46x 2.97x 3.00x

Average Memory Difference – Single 500 Loop Runs

Pert Type Async When 1.8.1 When 2.0.1 When 2.1.x Q Q longStack=0 Deferred
Sync 113.29% 160.98% 840.21% 866.88% 1106.67% 684.56% 354.07%
Async 159.29% 458.44% 811.32% 834.63% 1110.21% 691.41% 429.18%