Fixing Heroku H18 server request interrupted errors with Node.js

Since launch, one of my Node.js APIs has occasionally thrown errors on Heroku. Randomly. Any time of day. Not always on the same route, but usually when someone was sending a lot of data (like a file upload or a form) and it was unsuccessful.

It only seemed to happen once or twice a year, but it would keep me awake at night. Literally - I would get an alert when there were too many errors on my server. Luckily, these H18 errors never brought my server down and only affected a few users, but still, I need my sleep!

So how could I figure out what was causing these errors?

Honestly, I found it hard to reproduce the errors on my development server. And locally you won't get an H18 error, as this is a Heroku error code. In Node.js, you might see an EPIPE or ECONNRESET error.

But I struggled to get these errors. I would carry out the same actions, and not get the error. The user could carry out the same actions and not get the error. It was infuriating!

After many hours of research, and learning from people on the internet far cleverer than me who also had this problem, I think I finally figured it out.

It's been a few months with no H18 server errors, so here's what helped!

Server request interrupted

The H18 error on Heroku means "Server Request Interrupted".

So a request is being interrupted. But why?

What I have found this means (in my case) was that I was trying to respond to a request before the full body of the request had been read.

Let's imagine that a user sends a large payload, like a form, to the server. They have written a lot of content. Or uploaded a large file. But it turns out, that their session has expired, or their API key is invalid. They are no longer authenticated.

I can't process that form without valid authentication, so I must return an error instead.

When a request arrives at my server, I have a middleware that checks it. If the auth token in the header isn't there (or isn't valid), the server responds with an error.

Pretty standard stuff.

Unfortunately, if the error is sent before the rest of the request stream (like the body) has arrived, you get the H18 error. Because the request has been interrupted!

It doesn't matter that I don't care about the rest of the request, and I want to discard it, or that I don't need the server to read it, because authentication or some other requirement has failed. The request has been interrupted.

I found a closed issue in the Node.js repository and a few similar reports on Github:

Server Request Interrupted Error on Heroku after a while Fix Heroku H18 error when responding with 4xx errors on file uploads requests

It's hard to reproduce because it will only happen if the request hasn't finished (so not all the time).

So how did I fix it? 🤞

The completeRequest function

I took a lot of inspiration from a comment by domharrington on the original issue I had found since his description of what was happening seemed to pretty much match up with my situation:

He suggested fixing this issue by waiting for the incoming request to end before sending a response. Clever.

I took the example code and tweaked it into a function that returns a promise I could await before sending a response.

Here's the completeRequest function:

const completeRequest = function(req) {
  return new Promise(function (resolve, reject) {
    try {
      if (req.complete) {
        resolve(); 
      }
      req.on('data', () => {});
      req.once('end', resolve);
    } catch {
      reject();
    }
  });
};

I've returned a promise so that I can await the response. If the request is already complete (req.complete) the promise will resolve immediately. Otherwise, it will receive the data (req.on('data', () => {})) and then once the request ends, resolve (req.once('end', resolve)).

Now we have a function we can use to make sure the server request is complete before responding.

I tend to handle all server errors in an error middleware, so this just meant calling and awaiting my new 'completeRequest` function before sending an error response from the server.

📄 errors.middleware.js

// check the request is complete before responding
await completeRequest(req);
// send error response
res.status(err.statusCode).send(err.message);

I pray to the server gods that all my requests will be received and the H18 error won't return!

If you're interested, here's how I structure my Node.js projects with express.