Recently, a project required NodeJS to use a proxy. I initially thought it would be a simple configuration, but it turned out to be quite challenging.

NodeJS doesn't read system proxies by default, and since NodeJS has multiple methods for sending network requests, the ways to set up proxies also vary. Here's a summary.

Prerequisites

Let's assume the proxy server addresses come from environment variables, which is the standard practice for many web apps. For convenience, let's call them HTTP_PROXY and HTTPS_PROXY.

Fetch

Recent versions of NodeJS support the Fetch interface, which tries to maintain consistency with browser's Fetch, but if it were completely consistent, there would be no way to set up proxies. Fortunately, NodeJS opened a backdoor by adding a unique parameter dispatcher to modify network request parameters, including proxy servers.

We can use the undici package to set up the dispatcher:

npm install undici

Setting up fetch proxy

Use ProxyAgent to set it up:

import { ProxyAgent } from "undici";

const proxyUrl = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
if (proxyUrl) {
  await fetch(url, { dispatcher: new ProxyAgent(proxyUrl) });
}

Global fetch proxy setup

setGlobalDispatcher is used to set up proxies for all fetch calls in the NodeJS program:

import { setGlobalDispatcher, ProxyAgent } from "undici";

const proxyUrl = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
if (proxyUrl) {
  setGlobalDispatcher(new ProxyAgent(proxyUrl));
}

To remove the proxy, you can set an empty Agent:

import { Agent, setGlobalDispatcher } from "undici";
setGlobalDispatcher(new Agent());

How it works

setGlobalDispatcher actually uses a hidden entry point in NodeJS. The global object in NodeJS has a Symbol for setting the global dispatcher:

global[Symbol.for("undici.globalDispatcher.1")] = yourDispatcher;

This doesn't seem to be documented in NodeJS (at least I couldn't find it). For more details, see: https://github.com/nodejs/node/issues/43187

What else can dispatcher do besides setting up proxies

If you import Agent instead of ProxyAgent, you can also set network parameters like keepAlive and timeout:

import { Agent } from "undici";

const res = await fetch("https://example.com", {
  dispatcher: new Agent({
    keepAliveTimeout: 10,
    keepAliveMaxTimeout: 10,
  }),
});
const json = await res.json();
console.log(json);

For more details, check out the undici documentation.

NodeJS Native Modules http and https

As mentioned above, NodeJS doesn't have built-in methods for setting up proxies, so we need to develop them using http.Agent and https.Agent. Currently, the popular packages are http-proxy-agent and https-proxy-agent.

Here's an example using https-proxy-agent:

import https from "https";
import { HttpsProxyAgent } from "https-proxy-agent";

const proxyUrl = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
if (proxyUrl) {
  const agent = new HttpsProxyAgent(proxyUrl);

  https.get(
    {
      host: "example.com",
      path: "/",
      agent,
    },
    (res) => {
      // Handle response
    }
  );
}

Axios

Axios is almost the industry standard for JavaScript. It has multiple ways to set up proxies (in order of priority from low to high):

Environment variables: Axios itself reads the environment variables HTTP_PROXY and HTTPS_PROXY

proxy parameter:

import axios from "axios";

axios.get("https://example.com", {
  proxy: {
    host: "127.0.0.1",
    port: 7890,
  },
});

httpAgent and httpsAgent parameters: You can pass NodeJS's built-in http.Agent and https.Agent to control network parameters.

Here we use the http-proxy-agent and https-proxy-agent mentioned in the previous section:

import axios from "axios";
import { HttpsProxyAgent } from "https-proxy-agent";
import { HttpProxyAgent } from "http-proxy-agent";

const proxyUrl = process.env.HTTPS_PROXY || process.env.HTTP_PROXY;
if (proxyUrl) {
  const httpsAgent = new HttpsProxyAgent(proxyUrl);
  const httpAgent = new HttpProxyAgent(proxyUrl);

  const instance = axios.create({
    httpAgent,
    httpsAgent,
  });
}

For more details, refer to the Axios documentation.

HTTPS Connection Limitations

I found that if you set up an HTTP proxy using the first two methods, axios cannot establish HTTPS connections. The reason is that axios doesn't send the CONNECT method, so it can't establish TLS connections (if you don't understand the CONNECT method, you can read this article).

Setting Up Proxy in NodeJS