When you run a Node.js program, it creates a process.

Process is the representation of a program that consumes memory. This memory is exclusively available for the process. Apart from memory, a process also receives computing time from the processor.

A process can be divided into threads. Think of a thread as a strand of execution. Since the thread is a part of the process, it consumes the resources available to a process. Multi-threaded applications handle workload by creating multiple threads within the process.

However Node.js is single-threaded. This means that the process running a Node.js program creates a single thread. Operations taking a long time to complete can block this single thread and delay the execution of other code.

Check out the below illustration that compares multi-threaded processes with single-threaded process.

single threaded vs multi-threaded process
Single-threaded vs multi-threaded process

So how can you handle long-running tasks in Node.js?

The easiest way is to create a child process in Node.js. Think of a child process as a process created by another process. When a child process is launched, the operating system can utilize multi-processing techniques to execute the main process and child process concurrently.

Node.js provides a special module called the child_process to create a child process. Though this Node.js module provides various ways of creating a child process, this post will focus on exec and execFile methods.

1 – Node.js Child Process using exec

It is best to start with an example of creating a child process. See the below code snippet:

import { exec } from 'child_process';

const cmd = 'cat package.json';

exec(cmd, (err, stdout, stderr) => {
    if (err) {
        console.error(`error: ${err.message}`);
        return;
    }

    if (stderr) {
        console.log(`stderr: ${stderr}`);
        return;
    }

    console.log(`Command Output: ${stdout}`);
})
  • In the first step, you need to import the exec method from the child_process module.
  • Next, declare the command you would like to execute. Let’s assume you want to run cat package.json. This command displays the contents of the file in the console.
  • Next, call the exec method. It takes two arguments as input. One is the command that you want to run. Second is a callback function that will be executed as soon as the result of the command is available.
  • The callback function has a few important arguments. First is an error object that contains a value if an error occurs during the command’s execution. Second and third arguments are buffer objects. stdout contains the standard output of the command. stderr contains the value of the standard error output.

You can run the program using the node index.js command where index.js is the name of the file.

So what happens under the hood?

When you run the program, the specified command is executed by calling exec in a sub-shell or a separate command line. The output of the command is buffered and the result is made available in the buffer object of the callback function. You can read more about stdout and stderr objects in this detailed post.

The execution of the command is asynchronous. In other words, the application is not blocked by the processing of the external command.

2 – Running a Script using Node.js execFile

The exec method lets us to execute typical shell commands. However, if you want to execute a file without using the shell, you can use execFile. It behaves like the exec function but does not use a shell.

For example, you can use it to run a full-fledged Node.js application that runs in a child process of the parent application.

See the below example:

import { execFile } from 'child_process';

execFile('./input.js', (err, stdout, stderr) => {

    if (err) {
        console.error(`error: ${err.message}`)
        return;
    }
    console.log(stdout);
})

Here, you import execFile instead of exec from the child_process module.

Then, call execFile that takes the file path (./input.js) as the input and the callback function.

The contents of the input.js file are as follows:

#!/usr/local/bin/node
console.log("Hello from the Child Process");

The first line of this file specifies the application that should be used to run the file. In this case, the application is /usr/local/bin/node. You can find out this path using which node command on a UNIX system.

The second line is a simple console.log statement.

Just like a Node.js script, you can also execute Python scripts or even normal bash commands using the execFile approach.

3 – Node.js exec child process options

The exec function also provides several options that can affect the execution. You can provide these options by specifying an object as the second argument of the exec function.

See the below code snippet:

exec(cmd, { encoding: 'utf-8' }, (err, stdout, stderr) => {
    if (err) {
        console.error(`error: ${err.message}`);
        return;
    }

    if (stderr) {
        console.log(`stderr: ${stderr}`);
        return;
    }

    console.log(`Command Output: ${stdout}`);
})

Some of the important options are as follows:

  • cwd – This is to specify the current working directory for the child process. Default is null
  • encoding – Character encoding for the output
  • env – Environment variables for the child process as key-value pairs
  • maxBuffer – Maximum size for the standard output and standard error. If this size is exceeded, the child process gets terminated. Default is 1024 * 1024
  • killSignal – This is the signal used to terminate the child process if it runs longer than the specified time-out value
  • timeout – This is to specify the time-out value in milliseconds.
  • shell – Specifies the shell on which the command should be run.

Important Points about Node.js Child Process using exec

Here are a few things to keep in mind:

  • The shell syntax has a security risk if you are executing any dynamic input received from an external source. A user can carry out a command injection attack by passing dangerous commands such as rm -rf.
  • The exec command buffers the whole data in memory before passing it to the callback function. This makes them more suitable for cases where size of the data output from the command is small.
  • In the case of execFile, you have to specify the extension of the file as .js when using the ECMAScript Module System. Not doing do will cause Node.js to throw an error.

Conclusion

The exec method is the simplest way to create a Node.js child process using the child_process module with exec and execFile. You can use it for a wide variety of cases when there are no dangers of an injection attack or large size of output.

There are other approaches to create a Node.js child process such as spawn and fork. We will be seeing them in the upcoming posts.

If you found this post as useful, consider sharing it with friends and colleagues. Also, do write your queries in the comments section.

You can also connect with me on other platforms:

ProgressiveCoder Newsletter

Twitter

LinkedIn

Youtube

Categories: BlogNodeJS

Saurabh Dashora

Saurabh is a Software Architect with over 12 years of experience. He has worked on large-scale distributed systems across various domains and organizations. He is also a passionate Technical Writer and loves sharing knowledge in the community.

0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *