#100DaysOfMERN - Day 37

#100DaysOfMERN - Day 37

·

7 min read

✏ Asynchronous JS with MongoDB/Mongoose

Over the years, the way to write asynchronous code in JavaScript has gone through various stages. Applications with a connection to a database naturally require a lot of async operations, so in this post, I'd like to go over a few snippets to show how to write the same code with different methods/syntax, using:

  • callbacks

  • Promise.then

  • async/await

I'm still working on my Recipe DB App, so I'll take that code as example. The database has two collections (recipes and ingredients), but for the sake of simplicity, I'll concentrate on the ingredients collection. It's just a list of names, so the Schema is:

const IngredientSchema = new Schema(
    {
        name: {
            type: String,
            required: true,
            unique: true
        }
    }
);

const Ingredient = mongoose.model('ingredient', IngredientSchema);

The name is required (it's basically the only piece of data for each record), and it should also be unique.

With the unique field set, Mongoose will check before saving if a record with the same name already exists, and throw an error (and not save) if it finds one. The way I'm handling this in my code for a POST request to the backend is to

  • first find out if the ingredient is already in the collection

  • only call the .save method on the ingredient if the above is false

  • return the (potentially) updated ingredients collection

Each step is an asynchronous operation, where the next step depends on the result of the previous, so I'll have to make sure to account for that in my code.

There might be better ways to make sure that only unique/new records are saved, instead of manually checking before saving like I'm doing here, but I couldn't find one right away, and this example works well for illustrating the point of this post


✏ Asynchronous code with Callbacks

Mongoose's .find and .findOne methods take two arguments: an object to specify the condition (if you have some experience with SQL databases, think of it as a WHERE clause), and a callback function. .savealso takes a callback. The code would look like this (for readability, I'll leave out any error logging/catching):

app.post('/api/ingredients', (req, res) => {

    const newIngName = req.body.name.toLowerCase();

    // STEP 1
    Ingredient.findOne({ name: newIngName }, function (err, result) {

        if (!result) {
            const newIng = new Ingredient({ name: newIngName });

            // STEP 2
            newIng.save(function (err, result) {

                // STEP 3
                Ingredient.find({}, function (err, result) {
                    res.json(result);
                })
            })
        }
    })
})

The nesting of callbacks is only three levels deep, but you can already see how this would soon become a triangle of doom when the code gets more complex. Because Mongoose's methods return a Promise, I can rewrite this now, using .then syntax.

For a long time, that was my preferred way of writing async code, because it's kind of descriptive and easy to read ("doStuff.then(doMoreStuff)"), but this example will demonstrate that you'll still have to be very careful about what gets chained where/when:


✏ Asynchronous code with Promise.then()

This is the new code without callbacks:

app.post('/api/ingredients', (req, res) => {

    const newIngName = req.body.name.toLowerCase();

    // STEP 1
    Ingredient.findOne({ name: newIngName }).then(result => {

        if (!result) {
            const newIng = new Ingredient({ name: newIngName });

            // STEP 2
            newIng.save().then(() => {

                // STEP 3
                Ingredient.find({}).then(result => res.json(result));
            });
        }
    })
})

The Promise syntax allows to write asynchronous code in a more synchronous way, but does the above look any less like a beginning callback hell? In order to "fix" that, one could be tempted to write the code like this instead:

app.post('/api/ingredients', (req, res) => {

    const newIngName = req.body.name.toLowerCase();

    // STEP 1
    Ingredient.findOne({ name: newIngName })
        .then(result => {

            if (!result) {
                const newIng = new Ingredient({ name: newIngName });

                // STEP 2
                newIng.save();
            }
        })

        .then(() => {
            // STEP 3
            Ingredient.find({}).then(result => res.json(result));
        })
})

That's less nesting, but the code is erroneous, because STEP 3 will happen before STEP 2 is finished. Saving to the database takes longer than reading from it, so while the .save method is still busy doing its job, the .find method will be performed on the not-yet-updated collection. This shows that even with the simpler .then syntax, you'll still have to be very mindful about how the code gets processed.


✏ Asynchronous code with Async/Await

The third and most recent version of async JS syntax, introduced with ES2017, finally allows to write code in a "truly" synchronous manner. Under the hood, it's all still Promises, but async/await gives an interface that simplifies writing the code.

The structure changes a bit, you'll have to wrap your database operations in a function, declared using the async keyword. Then, whenever you want your code to wait until the previous operation has finished, use the await keyword:

app.post('/api/ingredients', (req, res) => {

    const newIngName = req.body.name.toLowerCase();

    postIngredient(newIngName);


    async function postIngredient(newName) {
        // STEP 1
        const result = await Ingredient.findOne({ name: newName });

        if (!result) {

            const newIng = new Ingredient({ name: newName });
            // STEP 2
            await newIng.save();

            // STEP 3
            res.json(await Ingredient.find({}));
        }
    }
});

This minimises nesting, and maximises readability. STEP 2 won't happen before the assignment to result in the previous step has finished. STEP 3 won't happen before the .save method in the previous step has finished.

It takes a little while to get used to the syntax, but once you're comfortable with it, it really simplifies the code and makes it easy to reason with.


✏ Why do I need the function wrapper?

A common mistake when first using the async/await syntax is to try to await outside an async function like this:

app.post('/api/ingredients', (req, res) => {

    const newIngName = req.body.name.toLowerCase();

    const result = await Ingredient.findOne({ name: newName });
    // Syntax Error: await is only valid in async function

});

This behaviour is likely to change soon, though. The proposition to include top level await as a language feature is currently in Stage 3 (which is one step before finished).

There's a lot of discussion whether this would be a good idea (see top level await is a footgun), but the details are beyond the scope of this article.


✏ Does Await block the Thread?

I'd say the answer to this question is yes and no. The execution of the current code branch is indeed blocked while awaiting the result of an asynchronous operation like a network request, but other code can still run. To demonstrate, add a little test function and console.log different time stamps:

app.post('/api/ingredients', (req, res) => {
    const newIngName = req.body.name.toLowerCase();

    const now = new Date();

    postIngredient(newIngName);
    testFun();

    async function postIngredient(newName) {
        const result = await Ingredient.findOne({ name: newName });
        console.log('awaited findOne()', new Date() - now);

        if (!result) {
            const newIng = new Ingredient({ name: newName });

            await newIng.save();
            console.log('awaited save()', new Date() - now);

            res.json(await Ingredient.find({}));
        }
    }

    function testFun() {
        console.log('testFun', new Date() - now);
        setTimeout(() => console.log('testFun timeout 0', new Date() - now), 0);
        setTimeout(() => console.log('testFun timeout 20', new Date() - now), 20);
    }
});

The above code (with my machine and a local database) logs:

testFun starts:  1
testFun timeout 0:  5
awaited findOne():  8
testFun timeout 20:  23
awaited save():  27

So, even though the postIngredient function is busy with its database requests, the other function can still run its code.


✏ Recap

I've learned/revisited

  • all three ways of writing asynchronous code

  • Promise.then, although simpler than callbacks, still has some pitfalls

  • async/await is a wrapper for Promise-based code, which allows to write asynchronous code as if it was synchronous


✏ Thanks for reading!

I do my best to thoroughly research the things I learn, but if you find any errors or have additions, please leave a comment below, or @ me on Twitter. If you liked this post, I invite you to subscribe to my newsletter. Until next time 👋


✏ Previous Posts

You can find an overview of all previous posts with tags and tag search here:

#100DaysOfMERN - The App