✏ Asynchronous JS with MongoDB/Mongoose
Over the years, the way to write asynchronous code in JavaScript has gone through various stages. Applications with a connection to a database naturally require a lot of async operations, so in this post, I'd like to go over a few snippets to show how to write the same code with different methods/syntax, using:
callbacks
Promise.then
async/await
I'm still working on my Recipe DB App, so I'll take that code as example. The database has two collections (recipes and ingredients), but for the sake of simplicity, I'll concentrate on the ingredients collection. It's just a list of names, so the Schema is:
const IngredientSchema = new Schema(
{
name: {
type: String,
required: true,
unique: true
}
}
);
const Ingredient = mongoose.model('ingredient', IngredientSchema);
The name is required (it's basically the only piece of data for each record), and it should also be unique.
With the unique field set, Mongoose will check before saving if a record with the same name already exists, and throw an error (and not save) if it finds one. The way I'm handling this in my code for a POST request to the backend is to
first find out if the ingredient is already in the collection
only call the
.save
method on the ingredient if the above is falsereturn the (potentially) updated ingredients collection
Each step is an asynchronous operation, where the next step depends on the result of the previous, so I'll have to make sure to account for that in my code.
There might be better ways to make sure that only unique/new records are saved, instead of manually checking before saving like I'm doing here, but I couldn't find one right away, and this example works well for illustrating the point of this post
✏ Asynchronous code with Callbacks
Mongoose's .find
and .findOne
methods take two arguments: an object to specify the condition (if you have some experience with SQL databases, think of it as a WHERE
clause), and a callback function. .save
also takes a callback. The code would look like this (for readability, I'll leave out any error logging/catching):
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
// STEP 1
Ingredient.findOne({ name: newIngName }, function (err, result) {
if (!result) {
const newIng = new Ingredient({ name: newIngName });
// STEP 2
newIng.save(function (err, result) {
// STEP 3
Ingredient.find({}, function (err, result) {
res.json(result);
})
})
}
})
})
The nesting of callbacks is only three levels deep, but you can already see how this would soon become a triangle of doom when the code gets more complex. Because Mongoose's methods return a Promise, I can rewrite this now, using .then
syntax.
For a long time, that was my preferred way of writing async code, because it's kind of descriptive and easy to read ("doStuff.then(doMoreStuff)
"), but this example will demonstrate that you'll still have to be very careful about what gets chained where/when:
✏ Asynchronous code with Promise.then()
This is the new code without callbacks:
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
// STEP 1
Ingredient.findOne({ name: newIngName }).then(result => {
if (!result) {
const newIng = new Ingredient({ name: newIngName });
// STEP 2
newIng.save().then(() => {
// STEP 3
Ingredient.find({}).then(result => res.json(result));
});
}
})
})
The Promise syntax allows to write asynchronous code in a more synchronous way, but does the above look any less like a beginning callback hell? In order to "fix" that, one could be tempted to write the code like this instead:
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
// STEP 1
Ingredient.findOne({ name: newIngName })
.then(result => {
if (!result) {
const newIng = new Ingredient({ name: newIngName });
// STEP 2
newIng.save();
}
})
.then(() => {
// STEP 3
Ingredient.find({}).then(result => res.json(result));
})
})
That's less nesting, but the code is erroneous, because STEP 3 will happen before STEP 2 is finished. Saving to the database takes longer than reading from it, so while the .save
method is still busy doing its job, the .find
method will be performed on the not-yet-updated collection. This shows that even with the simpler .then
syntax, you'll still have to be very mindful about how the code gets processed.
✏ Asynchronous code with Async/Await
The third and most recent version of async JS syntax, introduced with ES2017, finally allows to write code in a "truly" synchronous manner. Under the hood, it's all still Promises, but async/await gives an interface that simplifies writing the code.
The structure changes a bit, you'll have to wrap your database operations in a function, declared using the async
keyword. Then, whenever you want your code to wait until the previous operation has finished, use the await
keyword:
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
postIngredient(newIngName);
async function postIngredient(newName) {
// STEP 1
const result = await Ingredient.findOne({ name: newName });
if (!result) {
const newIng = new Ingredient({ name: newName });
// STEP 2
await newIng.save();
// STEP 3
res.json(await Ingredient.find({}));
}
}
});
This minimises nesting, and maximises readability. STEP 2 won't happen before the assignment to result
in the previous step has finished. STEP 3 won't happen before the .save
method in the previous step has finished.
It takes a little while to get used to the syntax, but once you're comfortable with it, it really simplifies the code and makes it easy to reason with.
✏ Why do I need the function wrapper?
A common mistake when first using the async/await syntax is to try to await
outside an async
function like this:
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
const result = await Ingredient.findOne({ name: newName });
// Syntax Error: await is only valid in async function
});
This behaviour is likely to change soon, though. The proposition to include top level await as a language feature is currently in Stage 3 (which is one step before finished).
There's a lot of discussion whether this would be a good idea (see top level await is a footgun), but the details are beyond the scope of this article.
✏ Does Await block the Thread?
I'd say the answer to this question is yes and no. The execution of the current code branch is indeed blocked while await
ing the result of an asynchronous operation like a network request, but other code can still run. To demonstrate, add a little test function and console.log
different time stamps:
app.post('/api/ingredients', (req, res) => {
const newIngName = req.body.name.toLowerCase();
const now = new Date();
postIngredient(newIngName);
testFun();
async function postIngredient(newName) {
const result = await Ingredient.findOne({ name: newName });
console.log('awaited findOne()', new Date() - now);
if (!result) {
const newIng = new Ingredient({ name: newName });
await newIng.save();
console.log('awaited save()', new Date() - now);
res.json(await Ingredient.find({}));
}
}
function testFun() {
console.log('testFun', new Date() - now);
setTimeout(() => console.log('testFun timeout 0', new Date() - now), 0);
setTimeout(() => console.log('testFun timeout 20', new Date() - now), 20);
}
});
The above code (with my machine and a local database) logs:
testFun starts: 1
testFun timeout 0: 5
awaited findOne(): 8
testFun timeout 20: 23
awaited save(): 27
So, even though the postIngredient function is busy with its database requests, the other function can still run its code.
✏ Recap
I've learned/revisited
all three ways of writing asynchronous code
Promise.then
, although simpler than callbacks, still has some pitfallsasync/await
is a wrapper for Promise-based code, which allows to write asynchronous code as if it was synchronous
✏ Thanks for reading!
I do my best to thoroughly research the things I learn, but if you find any errors or have additions, please leave a comment below, or @ me on Twitter. If you liked this post, I invite you to subscribe to my newsletter. Until next time 👋
✏ Previous Posts
You can find an overview of all previous posts with tags and tag search here: