Debugging HTTP with curl: headers, DNS overrides, timing and TLS

When a page is slow, a webhook fails or a redirect loops, the browser hides a lot: cached connections, HSTS, extensions, service workers, automatic retries. A colleague recently spent an afternoon on a "random" 502 that turned out to be one of three backends behind a load balancer. One curl command pinned to each backend's IP found it in two minutes.

This guide is for developers and on-call engineers who use curl to fetch things but not yet to interrogate them. We will go from reading verbose output to timing every phase of a request, testing servers before DNS points at them, forcing protocol versions and checking certificates. All examples were run with curl 8.16, but nearly everything works on any curl 8.x.

Read the conversation with -v

-v prints what curl does and every header it sends and receives. Lines starting with * are curl's notes, > is what it sent, < is what it got back.

curl -v https://www.example.com/ -o /dev/null

* Host www.example.com:443 was resolved.
* IPv4: 203.0.113.20
*   Trying 203.0.113.20:443...
* Connected to www.example.com (203.0.113.20) port 443
* ALPN: curl offers h2,http/1.1
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* Server certificate:
*  subject: CN=www.example.com
*  expire date: Dec  1 23:59:59 2025 GMT
* using HTTP/2
> GET / HTTP/2
> Host: www.example.com
> User-Agent: curl/8.16.0
> Accept: */*
>
< HTTP/2 200
< content-type: text/html; charset=utf-8
< cache-control: max-age=60

In one screen you know which IP answered, the negotiated protocol, the TLS version and certificate expiry, and the exact response headers. For more detail, --trace-ascii trace.txt --trace-time writes every byte with timestamps, which is useful for intermittent failures.

Look at headers only, and follow redirects

-I sends a HEAD request. Some servers handle HEAD differently from GET, so for faithful results send a GET and dump only the headers:

curl -sS -D - -o /dev/null https://www.example.com/

-D - dumps response headers to stdout, -o /dev/null discards the body, and -sS hides the progress bar but keeps errors. To trace a redirect chain, add -L and print each hop:

curl -sSL -D - -o /dev/null http://example.com/login | grep -iE '^(HTTP|location)'

HTTP/1.1 301 Moved Permanently
Location: https://example.com/login
HTTP/2 301
location: https://www.example.com/login
HTTP/2 200

If the same location appears twice, you have a loop. --max-redirs 5 makes curl stop early instead of following up to 50 hops.

Test a server before DNS points at it

Two options let you send a request to a different IP while keeping the correct hostname for SNI, certificate validation and the Host header. This is the right way to test a new server, a specific backend or a blue/green environment. Overriding only -H 'Host: ...' is not enough for HTTPS, because SNI and certificate checks still use the URL host.

--resolve pins a host and port to an address:

# Ask backend 2 directly, as if DNS returned its IP
curl -sS -o /dev/null -w '%{http_code} %{remote_ip}\n' \
  --resolve www.example.com:443:198.51.100.12 https://www.example.com/health

--connect-to maps a host and port to another host and port, which is handy when the target is itself a name, such as a load balancer hostname or a different port:

# Send traffic for www.example.com:443 to the staging load balancer
curl -sS https://www.example.com/ --connect-to www.example.com:443:staging-lb.example.net:443

Loop over every backend IP to find the bad one:

for ip in 198.51.100.11 198.51.100.12 198.51.100.13; do
  printf '%s ' "$ip"
  curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' --resolve "www.example.com:443:$ip" \
    https://www.example.com/api/status
done

Measure where the time goes with -w

-w (--write-out) prints variables after the transfer. The timing variables are cumulative from the start of the request:

Put the format in a file, curl-timing.txt, so it is reusable (everything in the file is printed, so no comments):

     dns: %{time_namelookup}s\n
 connect: %{time_connect}s\n
     tls: %{time_appconnect}s\n
    ttfb: %{time_starttransfer}s\n
   total: %{time_total}s\n
    code: %{http_code}  proto: %{http_version}  ip: %{remote_ip}\n

curl -sS -o /dev/null -w @curl-timing.txt https://www.example.com/

     dns: 0.002871s
 connect: 0.034512s
     tls: 0.081944s
    ttfb: 0.912306s
   total: 0.915877s
    code: 200  proto: 2  ip: 203.0.113.20

Subtract adjacent values to get each phase. Here DNS took 3 ms, TCP 32 ms, TLS 47 ms and the server spent about 830 ms before the first byte, so the problem is in the application, not the network. A large DNS value points at resolver trouble; a large gap between connect and TLS often means certificate revocation checks or a slow TLS terminator. For scripting, -w '%{json}' prints all variables as one JSON object.

Reproduce what the browser sent

Browser DevTools can copy any request from the Network panel as a curl command (right-click, Copy, Copy as cURL). That gives you the same headers, cookies and body. Then remove headers one at a time until the behaviour changes; that usually identifies the culprit, such as a stale cookie or a missing Origin header that a CORS check depends on.

Two other options make hand-written requests easier:

# JSON body with the right Content-Type and Accept headers in one flag (curl 7.82+)
curl -sS --json '{"email":"user@example.com"}' https://api.example.com/v1/signup

# Simulate a CORS preflight exactly like a browser would
curl -sS -X OPTIONS -D - -o /dev/null https://api.example.com/v1/orders \
  -H 'Origin: https://app.example.com' \
  -H 'Access-Control-Request-Method: POST' \
  -H 'Access-Control-Request-Headers: authorization,content-type'

Add --compressed when comparing response sizes with a browser, since browsers always ask for gzip or Brotli.

Force protocol versions

Some bugs only appear on one protocol: an HTTP/2-only proxy bug, a header that is valid in HTTP/1.1 but illegal in HTTP/2, an HTTP/3 deployment behind a firewall that drops UDP.

curl -sS -o /dev/null -w '%{http_version}\n' --http1.1 https://www.example.com/
curl -sS -o /dev/null -w '%{http_version}\n' --http2 https://www.example.com/
curl -sS -o /dev/null -w '%{http_version}\n' --http3-only https://www.example.com/

--http3 tries HTTP/3 and falls back, while --http3-only fails if HTTP/3 does not work, which is what you want when testing it. HTTP/3 needs a curl build with QUIC support; check with curl -V and look for HTTP3 in the features line. Many distribution packages do not include it yet.

Inspect TLS and certificates

curl validates certificates by default. When verification fails, the error and the ssl_verify_result variable tell you why:

curl -sS -o /dev/null -w 'verify=%{ssl_verify_result} certs=%{num_certs}\n' https://api.example.com/

A num_certs of 1 on a publicly trusted site usually means the server sends only the leaf and forgot the intermediate, which breaks curl, Java and Android clients even if browsers cope. To force a TLS version or test a private CA:

curl -sS --tlsv1.3 --tls-max 1.3 https://www.example.com/ -o /dev/null
curl -sS --cacert internal-ca.pem https://intranet.example.com/

Never fix verification failures with -k (--insecure) outside a quick local experiment. It turns off exactly the check that protects you from a man-in-the-middle, and it has a way of ending up in scripts that run in production.

Make scripts robust

For health checks and CI, a few options turn curl from interactive tool into reliable probe:

curl --fail-with-body -sS \
  --connect-timeout 3 --max-time 10 \
  --retry 3 --retry-all-errors --retry-delay 2 \
  https://www.example.com/health

--fail-with-body returns a non-zero exit code on HTTP 4xx and 5xx but still prints the error body. --connect-timeout bounds only the connect phase; --max-time bounds the whole transfer. Without them a hung server can block a pipeline for minutes.

Common mistakes

Key takeaways

• -v shows the IP, protocol, TLS details and headers in one view; start there.
• --resolve and --connect-to test specific servers without touching DNS.
• The -w timing variables separate DNS, TCP, TLS and server time.
• Copy as cURL from DevTools turns a browser problem into a reproducible command.
• Force protocol versions explicitly when a bug might be protocol specific.
• Keep certificate verification on and add timeouts to every scripted call.

References

• curl man page — https://curl.se/docs/manpage.html
• Everything curl: the book — https://everything.curl.dev/
• curl: SSL certificate verification — https://curl.se/docs/sslcerts.html
• curl: HTTP/3 support — https://curl.se/docs/http3.html
• Chrome DevTools: Network features reference — https://developer.chrome.com/docs/devtools/network/reference