Skip to content

Proxy errors

502 Bad Gateway through a proxy

Something upstream returned an invalid response. Test the same URL without the proxy to find out whether the proxy or the destination is at fault.

A 502 through a proxy is ambiguous: it can come from the proxy failing to reach the target, or from the target's own infrastructure. Telling them apart is the whole task.

Error
502 Bad Gateway
Comes from
Varies, see below

Two different 502s

A 502 means a server acting as a gateway received an invalid response from the server behind it. When a proxy is in the path, there are two candidates for which gateway is complaining.

The proxy may be unable to reach the target, and reports that to you as a 502. Or the target site may sit behind its own load balancer, which is failing to reach the application behind it, in which case the 502 is the site's own problem and has nothing to do with your proxy.

These need opposite responses, so identifying which one you have is the first step rather than an afterthought.

The one test that separates them

Request the same URL directly, without the proxy, at roughly the same moment. The pair of results is conclusive.

If both fail with 502, the site is broken for everyone and there is nothing to fix on your side. If only the proxied request fails, the proxy or the exit address is involved.

# direct
curl -sS -o /dev/null -w 'direct: %{http_code}\n' https://example.com

# through the proxy
curl -sS -o /dev/null -w 'proxied: %{http_code}\n' \
  --proxy proxy.example.com:8000 --proxy-user 'USER:PASS' \
  https://example.com

When only the proxied request fails

Try a second, unrelated site through the same proxy. If that works, the proxy is healthy and the problem is specific to the combination of that exit address and that destination, which usually means the site is rejecting the address in a way that surfaces as a gateway error rather than a clean 403.

If every site returns 502 through the proxy, the proxy itself is in trouble. Rotating to a different address, or raising it with the provider, is the right move. A 502 on all destinations is not something you can configure around.

Transient versus persistent

A 502 that appears once and clears is usually a brief upstream hiccup and does not deserve investigation. Build a single retry with a short backoff and move on.

A 502 that repeats on the same URL for more than a few minutes is structural. At that point run the direct comparison above rather than retrying, because retrying a persistent 502 just burns bandwidth.

If you are scraping, treat 502 differently from 429. A 429 means slow down, and retrying sooner makes it worse. A 502 is usually not about your rate at all, so the same backoff policy is not appropriate for both.

Retrying without making it worse

Because a 502 is frequently transient, retrying is reasonable, which makes it easy to retry badly. The failure mode is a tight loop that hammers an already struggling upstream and turns a brief outage into a sustained one.

A sound policy is at most three attempts, with the wait doubling each time and a random offset added so parallel workers do not retry in unison. If the third attempt fails, record the URL and move on rather than blocking the queue. Coming back to a small list of failures later is far cheaper than stalling a whole job.

import random, time

def fetch(session, url, proxies, attempts=3):
    for attempt in range(attempts):
        r = session.get(url, proxies=proxies, timeout=30)
        if r.status_code != 502:
            return r
        if attempt < attempts - 1:
            time.sleep((2 ** attempt) + random.uniform(0, 1))
    return None  # record and revisit later

Telling a 502 apart from the errors it resembles

Three status codes all mean something went wrong upstream, and they call for different responses. A 502 means an invalid response came back from behind the gateway. A 503 means the service is deliberately unavailable, often during deployment or overload, and frequently carries a Retry-After header worth honouring. A 504 means the upstream did not answer in time at all.

Of the three, 503 is the one that most often resolves itself on a stated schedule, and 504 most often points at something genuinely slow rather than broken. Logging the exact code rather than grouping them as server errors makes a scraping job much easier to reason about after the fact.

Frequently asked questions

Is a 502 my fault?

Usually not. It means a gateway got an invalid response from something behind it. The direct comparison shows whether that gateway is the proxy or the site.

Should I retry a 502?

Once, with a short backoff. If it repeats for several minutes, diagnose it instead of retrying.

Does changing proxy fix a 502?

Only when the proxy or the exit address is the cause. If the site returns 502 directly too, no proxy will help.

Is 502 the same as 504?

They are related. A 502 is an invalid response from upstream, a 504 is no response in time. Both point upstream rather than at your request.

Ready to try it?

Create an account, add funds from US$1 and pick your country and proxy type.