NodeJS, when outsourcing Node.js development tasks, has a specific way of managing its operations. According to the official documentation, it is an asynchronous event-driven JavaScript runtime. But what does that mean in the most straightforward terms? Here, I will address that question and explore the best practices to follow when writing asynchronous code.
What is Asynchronous Programming?
Asynchronous programming means that part of the code can run while other parts of the code are waiting for a response. Let’s look at a concrete example: As a web developer, you will deal with API calls extensively: you will write code that sends a request to an external server and waits for a response. The thing is, first of all, you don’t even know if you’ll get the response you are expecting, you don’t also know how much time it will take to execute that function, the situation of the server you are calling, and so on.
So, when you make a request to an API call, you’re completely at the mercy of the server’s state, and your application should stop whatever it’s doing and wait for the response, as with your user patiently waiting for the page to load. Right? Well, obviously not. In modern web development, even milliseconds are counted. We need a way to make our application run smoothly. That is, when we send a request to an API, we should be able to do something else while we wait for the response. That’s where async programming comes in.
There are various benefits of asynchronous programming. The moment you explore the world of async programming in NodeJS, you’ll hear terms like event loop, callback functions, promises, scalability, non-blocking, and so on. All of these terms center around the idea of asynchronous programming, the ability to run code while other parts of the code are waiting for a response.
In the example above, we were making a hypothetical API call to an external server. Since NodeJS is non-blocking, which means that our code will be able to continue with other operations while that former operation is waiting for a response. This makes NodeJS very scalable because it can handle lots of concurrent requests. The most important concept to understand is that in asynchronous programming, in layman’s terms, you can run some operations while some other operations are running already. Let’s see this in action.
Event Loop
In NodeJs, operations are handled in a single thread. This means that only one operation can be executed at a time. But, as we’ve seen above, we can still do something while another thing is being processed.
How is that possible? NodeJS has a mechanism called an event loop that allows NodeJS to perform non-blocking I/O (input/output) operations, Like a loop that checks the event queue and executes operations in order. If you’d visit the official NodeJS docs in https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick you’ll learn that the event loop consists of the following phases: timers, pending callbacks, idle/prepare, poll, check, and close callback functions.
In the same link, a brief overview of those phases are given as such:
timers: this phase executes callbacks scheduled by setTimeout() and setInterval().
pending callbacks: executes I/O callbacks deferred to the next loop iteration.
idle, prepare: only used internally.
poll: retrieve new I/O events; execute I/O related callbacks (almost all with the exception of close callbacks, the ones scheduled by timers, and setImmediate()); node will block here when appropriate.
check: setImmediate() callbacks are invoked here.
close callbacks: some close callbacks, e.g. socket.on('close', ...).
While the event loop is quite a complex topic that would require a whole post on its own, the most important thing to understand is that the event loop is a mechanism that allows NodeJS to perform non-blocking I/O operations. This means that NodeJS can handle lots of concurrent requests, and that’s why it is so scalable. Let’s see an example of how the event loop works.
Since all modern browsers have a Javascript engine, we can use the browser console to see how the event loop works. Now, what if we opened the dev tools of our browser of choice and wrote the following?
console.log("one");
console.log("two");
console.log("three");
The browser will log them in order, as we’d expect: one, two, and then three. But let’s consider another example:
console.log("one");
setTimeout(() => {
console.log("two");
}, 1000);
console.log("three");
Something different will happen here. Due to the setTimeout method, the browser will log one, three, and only after a second later, two. You see, setTimeout function is asynchronous here. While the code is following the queue from top to bottom, when it encounters the setTimeout function, it will not just stand there and wait for one second so that it can execute the code. No, it will continue its execution, and only after the execution is complete, it will check the event queue and see if there are any callbacks to be executed.
In this case, the setTimeout function will be executed after one second. This is the most basic example of how the event loop works. This is the very same mechanism that allows NodeJS to handle lots of concurrent requests, making it very scalable and fast.
Now that we’ve discussed the event loop, touching upon scalability and non-blocking nature of NodeJS, let’s cover the other common terms we’ve described above, like callbacks, promises, and more.
Callback Function
Callback functions in NodeJS are functions that are passed as arguments to other functions, and they are called once the main function is done. One of the most basic examples you can find is timeouts. Let’s see an example:
// Example function that performs an asynchronous operation
function fetchData(callback) {
// Simulate a delay
setTimeout(() => {
const data = { name: "John", age: 30 };
// Call the callback function with the fetched data
callback(data);
}, 3000);
}
// Call the fetchData function and pass a callback function
fetchData((data) => {
console.log(data); // { name: "John", age: 30 }
});
Since NodeJS is just Javascript, we can use this same concept in our browser console to easily understand how callbacks work. So, if you paste this code to the browser console, the data will be fetched after 3 seconds. While the data is being fetched, the rest of the code will be executed. Once the data is fetched, the callback function will be called and the data will be printed to the console.
The reason why we’re giving a setTimeout function is to mock an API call for instance, and since we don’t know how much time it will take for the API to return, we need to simulate that delay. But callbacks can get out of hand. Maybe you’ve heard of the term callback hell. Let’s see what that means.
Callback Hell
Callback hell is a term used to describe a situation where you have a lot of nested callbacks. There is even a website named callbackhell.com just for explaining this concept. If you visit that page, you’ll be prompted with the following example:
fs.readdir(source, function (err, files) {
if (err) {
console.log("Error finding files: " + err);
} else {
files.forEach(function (filename, fileIndex) {
console.log(filename);
gm(source + filename).size(function (err, values) {
if (err) {
console.log("Error identifying file size: " + err);
} else {
console.log(filename + " : " + values);
aspect = values.width / values.height;
widths.forEach(
function (width, widthIndex) {
height = Math.round(width / aspect);
console.log(
"resizing " + filename + "to " + height + "x" + height
);
this.resize(width, height).write(
dest + "w" + width + "_" + filename,
function (err) {
if (err) console.log("Error writing file: " + err);
}
);
}.bind(this)
);
}
});
});
}
});
That’s a lot of nested callbacks. As you can see, it’s not even that readable. We’d want to avoid such things in our code for our own sake.
Promises
Now, since asynchronicity is such an important concept in NodeJS, there are a couple of ways to handle them. While one of them is callbacks, another option is to use promises. Let’s rewrite the same setTimeout example from above but with promises:
// Example function that returns a Promise
function fetchData() {
return new Promise((resolve, reject) => {
// Simulate a delay
setTimeout(() => {
const data = { name: "John", age: 30 };
// Resolve the Promise with the fetched data
resolve(data);
}, 1000);
});
}
// Call the fetchData function and handle the Promise result
fetchData()
.then((data) => {
console.log(data); // { name: "John", age: 30 }
})
.catch((error) => {
console.error(error);
});
This way seems a bit cleaner. At least we get to deal with possible errors. Remember that we have no idea about the state of the server we’re making a request to. There is no guarantee that our request will return with the response we are expecting. So, we need a way to handle possible errors. The catch statement here is a good way to do that. But there’s also a caveat here: Maybe we’re not in callback hell anymore, but we might easily fall into the piths of another hell that’s called a promise hell, or a “then hell”. Here, “then hell” takes its name from having lots of chained then statements in the code. Let’s see what that means.
Promise Hell
Promise hell is a term used to describe a situation where you have a lot of nested promises. Let us look at an example:
getData()
.then(function (data) {
return processData(data)
.then(function (processedData) {
return saveData(processedData)
.then(function (savedData) {
return displayData(savedData);
})
.catch(function (err) {
console.log("Error while saving data:", err);
});
})
.catch(function (err) {
console.log("Error while processing data:", err);
});
})
.catch(function (err) {
console.log("Error while getting data:", err);
});
That’s no better than callbacks. So, what’s the solution? One way is to use async/await. Here’s how that works.
Asynchronous Functions [await]
Async/await is a way to handle asynchronous code in a synchronous way. It is by far my personal favorite and helps avoiding the pitfalls of a callback function and promises. Let’s see how it works:
async function fetchData() {
try {
const res = await fetch("https://jsonplaceholder.typicode.com/users");
const data = await res.json();
console.log(data);
return data;
} catch (error) {
console.error(error);
}
}
Here, instead of the setTimeout, we’re actually making a real API request. For reference, jsonplaceholder is a dummy API that we can call to test API requests. The above request should return a list of users. Now, this method of doing asynchronous programming works in the following way: The function has to be called async in order for us to be able to use the await keyword.
The await keyword is the moment where we wait for the asynchronous operation to finish. Once it’s done, we can continue with the rest of our code. You’ll also see that we use try/catch statements, which are extremely useful in this specific case: We can handle errors in a much easier way thanks to this method.
Working With Asynchronous Code in NodeJS
Embracing the power of asynchronous programming in NodeJS not only streamlines our code, but it also significantly enhances the performance and responsiveness of our applications.
Creating a Basic Express Server
Now, here’s how we can use asynchronous programming in NodeJS with an example. We’ll be creating a basic express server and send a get request to the jsonplaceholder API. We’ll be using the async/await method for this as it’s the most convenient way. To follow through, you’ll need node and npm installed in your system.
First, we create a new folder called ‘async-node’ and cd into it. Note that we’re using Linux terminal, so we use mkdir command to create the folder. Once I’m in the folder, I run npm init -y to initialize a new npm project.
The -y flag is to skip the questions that npm asks us. Once that’s done, I’ll install express and axios packages via npm. I’ll also use nodemon in this project, but will not install is as I have it globally installed. For ease of use, I recommend you do the same. If you don’t want to, you can install it locally as a dev dependency as well.
To install express and axios, we run the following command:
npm i express axios
For nodemon, if you wanted to install it globally, you’d write npm i -g nodemon. Also note that here you might need to provide root access via specifying sudo. If I wanted to install nodemon locally as a dev dependency, I’d run npm i –save-dev nodemon
Now I’ve installed the packages, I can create an index.js file with the command touch index.js and start coding. I’ll open the project with VsCode via code . command. Once I’m in there, I’ll go to package.json file and make some changes there. I’ll add the following line to the scripts section:
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node index.js",
"dev": "nodemon index.js"
},
Now, whenever we run npm run start or npm run dev, it’ll run the index.js file with the specified packages. Now that we’re done with the package.json file, we go to index.js file and populate it with a basic express server:
const express = require("express");
const app = express();
app.get("/", (req, res) => {
res.send("Hello, world!");
});
app.listen(3000, () => {
console.log("Server listening on port 3000");
});
Here a couple of things are happening. First and foremost, I’m importing the express package and initializing it via const app = express();. Then, we’re creating a get request to the root route, and sending a response with the text “Hello, world!”. Finally, we’re starting the server on port 3000. Now, if I run npm run dev and go to localhost:3000, if everything is correct, we should see the text “Hello, world!”.
Send Request With Axios and Write the Response to a File
But we want to do more than just creating a basic server. We want to send a get request to an external API, and write it to a file. So let’s do that. Check the following code snippet out:
const axios = require("axios");
const fs = require("fs");
const express = require("express");
const app = express();
app.get("/", async (req, res) => {
try {
// Make API call using Axios
const response = await axios.get(
"https://jsonplaceholder.typicode.com/todos/-1"
);
debugger;
// Write data to file using asynchronous file system method
await fs.promises.writeFile("response.txt", JSON.stringify(response.data));
res.send("Data written to file successfully!");
} catch (error) {
console.error(error);
res.status(500).send("An error occurred");
}
});
app.listen(3000, () => console.log("Listening on port 3000"));
In this code, we import Axios and fs along with express. Axios allows us to make API requests, while fs gives access to the filesystem to read and write files.
After initializing the Express app, we add a GET route handler for the root route. Notice the async keyword – this signifies the route contains asynchronous code.
Inside the handler, we use a try/catch block to gracefully handle any errors. In the try portion, we make a GET request to the dummy API using axios and await a call to fs.promises to create a new file called response.txt. This file will contain the API response data.
When we run the server with npm run dev and visit localhost:3000, we expect to see the “Data written to file successfully!” message. But instead, we get an error.
Clearly, there is a bug preventing the file from being written properly. Now it’s time to debug our asynchronous code to uncover and fix the issue. We’ll use Node’s built-in debugger along with other techniques to trace the execution flow and identify problems.
Debugging asynchronous code like this takes some specialized skills, but learning these skills will allow us to build complex Node apps with confidence.
Debugging Nodejs With Chrome Dev Tools
Debugging is a critical skill for developers. When bugs appear, we need tools to identify the root cause. In this example, our code encounters an error instead of writing data as expected. Thankfully, Node.js has a built-in debugger we can leverage.
To use it, first close the server and run nodemon --inspect index.js. This restarts the server in inspect mode. Next, open Chrome and find the Node debugger panel using the green Node.js icon.
In the Sources tab, we see line 10 is highlighted where our Axios request is made. This indicates the error occurs here. We can pause execution on line 10 and inspect variables to get more context.
Checking the API endpoint, we spot the issue – there is an invalid “/todos/-1” path. I fix this to “/todos/1” and resume execution. Now in the debugger, we see a 200 status code on the request – success!
The debugger was invaluable for tracing the execution flow and pinpointing the root cause. Now with the bug fixed, we restart normally with npm run dev. Visiting localhost, we correctly get the “Data written” message and a response.txt file containing the API data.
Being able to debug asynchronous Node code is crucial for any developer. Mastering the built-in debugger and other debugging workflows will give you the skills to efficiently squash bugs in your apps.
Conclusion
Here, we’ve discussed asynchronous programming in NodeJs. We covered some ways to handle asynchronicity like promises, callback functions, and the relatively new async/await block. We’ve also examined how to debug our code using nodeJS’s built-in debugger. So now it’s up to you to incorporate and write asynchronous code into your future projects. However, if you need further assistance or wish to scale your project, consider hiring a reputable Node.js development company.
FAQ
What is the difference between asynchronous and synchronous programming?
Synchronous programming is a linear approach where tasks are executed one after the other, meaning a task must complete before the next one can begin. It can make your program easier to understand, but it also means that your program can hang or become unresponsive if a task takes a long time to complete.
An asynchronous function, on the other hand, allows tasks to be executed concurrently. If a task is going to take a long time to complete (like reading a file from a disk or fetching data from the network), the program can continue with other tasks. Once the long task is complete, a callback function is typically invoked to handle the result. The non-blocking nature of asynchronous operations makes them particularly well-suited for executing tasks within JavaScript code that require waiting for external resources or need to run in the background. This approach can lead to the creation of potentially more efficient and responsive programs.
How does the event loop work in Node.js to handle asynchronous operations?
The event loop in Node.js is the engine that handles asynchronous operations. When you hire Node.js developers, they leverage the event loop by initiating asynchronous tasks like reading files or making database requests. These tasks run outside the event loop, allowing it to continue processing other JavaScript code. When an asynchronous task completes, its callback function is added to a queue, or ‘callback queue’. The event loop checks this queue and processes the callbacks one by one, thus handling the results of asynchronous operations efficiently.
