Why You Should Consider Node.JS as a Backend Option for Your Project

I. HISTORY OVERVIEW

In the early 2000s, when the web was small, servers were slow and client machines were even slower, developers faced the C10K problem. The problem was about concurrently handling 10,000 client connections on a single server machine. As a solution, multi-process and multi-threaded architectures (a new process or thread per each request) became very popular in mainstream software platforms for web development.

But the web continued to grow (and it still does), C10K goal became implementable on most software platforms and frameworks and the community stated the next goal – C10M problem. As you might have guessed, it’s about dealing with 10,000,000 concurrent client connections, which is a tremendous load.

The C10M problem is still relevant now and developers need new solutions to reach much higher load requirements. Most modern web apps include RESTful backend along with web and mobile apps that consume backend API. This increases load that the backend server should be able to hold.

Async I/O based platforms were created to help developers reach the goal. In this post we’ll talk specifically about Node.js as our preferred platform for high load systems development.

II. NODE.JS OVERVIEW

Node.js was initially created in 2009 by Ryan Dahl and other developers working at Joyent company (main contributor of Node.js). The idea was to use extremely fast Google V8 JavaScript engine and implement system libraries that provide common APIs which are absent in browser environments (as browsers provide sandboxes for JS code actually), like file manipulation, HTTP client/server libraries, and so on.

Because of async nature of JS (which is executed in a single thread and does not support multithreading at all), all of Node.js system libraries provide evented and asynchronous API for I/O operations, like reading a file or sending an HTTP request.

So Node.js itself might be described as an asynchronous event-driven platform. Code written by developers is executed in a single thread and, speaking of web backend development, at each moment of time only single client request is being processed, while all others are idle. However, due to the async nature of Node.js, all calls that are potentially blocking, like execution of SQL queries, have event-based API and lead to switching from current request processing to next awaiting operation (all of this is handled by the event loop).

test

III. COMPARISON WITH “CLASSICAL” SOLUTIONS

Let’s look at “classical,” thread-per-request based solutions, like Java servlet containers (Tomcat, Jetty, etc.) or Apache web server. By default, the whole thread is blocked on I/O operations here. Thus, maximum load that can be handled by a single web server instance is bounded by the maximum amount of threads that the server can handle.

Apart of the fact that creation of a new thread is relatively expensive, threads have quite a heavy footprint themselves. For example, each Java thread requires 1024KB of memory for its stack in 64-bit JVM, so 10K threads require 10Gb of RAM just for the thread stack.

So building highroad systems with such software platforms is still doable, but it’ll require many more server instances to handle the same load.

IV. OUR EXPERIENCE WITH NODE.JS

At DSR we have implemented several projects with Node.js-based backends and have been very pleased with the scalability and performance of the platform.

The strength of Node.js is the async, non-blocking I/O nature. Modern RESTful backends do a lot of I/O operations mostly, without involving heavy computations, and Node.js shines here. While reaching C10K goal requires certain effort from the developers on multi-threaded platforms, it’s just the worst performance level in case of Node.js.

Our experience confirms that Node.js is a stable and developer-friendly platform. Since Node.js 4.0, there are LTS releases now, which are stable and supported for a long period of time. As for developers, we find built-in platform tools, like npm, very handy, and, as for the code, promises cooked with generators are just simply awesome.

Of course, there are some negative points, like small amount of production-ready 3rd party libraries/frameworks. But the platform is rather young and Node.js ecosystem is rapidly growing and maturing. Growth of npm package count also indicates this well enough (see the chart below).

file

We’re actively participating in the Node.js platform community and are looking forward to building more great, high load systems using it.

ZigBee 3.0 Evolution of Things

Another CES has come and gone. The wheels have touched down, and you are likely back home. You and your team have refueled with a few well deserved, solid nights of sleep and it’s now time to reflect on what made CES 2016’ special. Let’s highlight one of the exciting moments; the ZigBee Alliance announcement of ZigBee 3.0.

With ZigBee 3.0, there is no reinvention of standard, sudden updates, or unpredictable changes – we are looking at the refinement of a proven technology. Using natural selection as an example, we are watching the substantial evolution and adaption of ZigBee with the IoT market confirming the technology maturity. Observing changes in the wireless technologies and connected IT business environments, we are tracking the reactions of the ZigBee standard in response to this.

The key features of ZigBee 3.0 include dramatically improved interoperability and strengthened security. DSR has been continuously involved in the implementation of ZigBee Pro since the 2006’ standard and we can confirm that the new features in ZigBee 3.0 are a real game changer, especially the convergence of the application profiles to a unified base device implementation. At first glance, this change is the kind of revolution in ZigBee that casts doubt on the previous specification. We do not view it this way.

Earlier, when the profiles were developed, the market was a union of isolated areas. Which areas, you might ask? Well, let me challenge you to quickly recite the ZigBee profile names. If you’re like us, you don’t like separate smart home, light control, or energy measurement functions. We want the Internet of Things and we now have extremely inexpensive, more powerful microcontrollers to build it with. We don’t need profiles anymore. We need the unified implementation enabled by ZigBee 3.0.

Structural consequence of the profile evolution into a base device approach is strengthening the role of a cluster as a unified application building block (clusters were developed for this, of course). ZigBee Alliance goes further and standardizes device types. For us, this approach becomes quickly rudimentary because all the tools are ready for dynamic discovering of devices. We’re talking about EZ-mode commissioning that is now able to discover all the features of the added device right at the commissioning step. After finding and binding, the application has full details about the joined device and bound clusters, so the device type information could be used only for predictions. What we would like to see instead of standardized devices is the strict, “survival recommendation” list for different groups of devices. For example, recommendations for implementing optional attributes/commands or, more specially, having poll control cluster for sleepy end devices, etc. (see our previous blog post).

Overall, a transformation of the profiles multiplies many times the core and indisputable advantage of ZigBee – mesh networks. Devices that previously joined the different networks will truly co-exist now. The new standard allows ZigBee to keep their status as one of the most energy saving choices. Moreover, with the Green Power feature in ZigBee 3.0, devices without batteries can operate in the network.

In conclusion, to all the benefits of ZigBee 3.0 painless backward compatibility and OTA Upgrade feature guarantee, that neither user nor developer will have trouble with switching to the new standard or supporting old devices. What is the best, now only a ZigBee sign on the device’s box makes sense: not profile, even not ZigBee PRO or ZigBee 3.0. For example, how often do you care about 1.1, 2.0 or 3.0 USB device you buy? That is the same.

What do we have as a result? The mesh self-healing network of green, low-power devices with the unified easy installation mechanism, growing community, and continuous evolution. Isn’t that a synonym of IoT?