API Response Debugging Guide: Read Headers, Status Codes, JSON Errors, and Pagination

Overview

Debugging an API is easier when you separate the request into observable parts: the URL and method, request headers, request body, response status, response headers, and response body. A failure in any one of these areas can produce a similar-looking symptom in your application, such as an empty result, an unexpected exception, or a generic “request failed” message.

Start by reproducing the request outside your application. A command-line request with curl, an API client, or your browser’s network tools can show the raw response before application code transforms it. Preserve the exact method, endpoint, query parameters, headers, and body used by the failing request. Small differences—such as sending JSON as form data or omitting an Accept header—can change the result.

A useful first-pass checklist is:

• Confirm the HTTP method and complete URL.
• Record the status code and response time.
• Inspect response headers before reading only the body.
• Check whether the body is JSON, text, HTML, or empty.
• Compare authentication, content type, and query parameters with the API documentation.
• Look for request identifiers that can connect your test to server-side logs.

For JSON-heavy workflows, a JSON formatter and validator can make nested error objects easier to inspect. Avoid pasting production tokens, personal data, or confidential payloads into third-party tools.

Checklist by scenario

1. The response is a 2xx status, but the data is wrong

A successful status does not guarantee that your application received the expected records. First check the response shape. An endpoint may return an object containing an array rather than an array at the top level, or it may return a valid empty collection because your filters match nothing.

• Compare the actual JSON keys with the documented response shape.
• Check query parameters, filters, sorting, and date or status values.
• Confirm that you are using the intended environment, account, tenant, or resource ID.
• Check whether the response includes a warning, partial-result flag, or pagination metadata.
• Log the raw response before mapping it into application objects.

2. The response is 4xx: request or authentication problem

Client-error responses generally indicate that the server received the request but could not process it as submitted. The exact meaning depends on the provider, so use the response body and documentation together.

Do not assume that 401 and 403 mean the same thing. One commonly points to missing or invalid authentication, while the other often indicates that the authenticated identity lacks permission. Providers may use these statuses differently, so treat the response details as authoritative.

3. The response is 5xx or the connection fails

A 5xx response usually means the server could not complete a valid request, but the request can still contribute to the failure. Reproduce the issue with a minimal request and capture the response body, timestamp, endpoint, and request ID. If the same request works intermittently, compare timing, payload size, rate limits, and upstream dependencies.

A connection failure is different from an HTTP error: your client may not have received an HTTP response at all. Check DNS resolution, TLS or certificate errors, proxy settings, firewall rules, timeouts, and whether the hostname is correct. In application code, distinguish connection errors from non-2xx responses so the right retry behavior can be applied.

4. The body is not valid JSON

Before parsing the body as JSON, inspect the Content-Type response header. An HTML error page, plain-text message, redirect, or empty response can cause a JSON parser exception even when your request code is correct. Log a bounded preview of the body for diagnosis, but avoid exposing secrets or sending unlimited response data to logs.

On the request side, send JSON only when the endpoint expects it. A typical curl template is:

curl --request POST \
  --url https://api.example.test/v1/items \
  --header 'Authorization: Bearer YOUR_TOKEN' \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{"name":"Example item"}'

Replace the placeholder host, token, fields, and method with values from the API documentation. Never commit a real token to source control or paste it into a shared debugging transcript.

5. Authentication appears correct, but access still fails

Compare the working and failing requests character by character where practical. Check the authentication scheme, header spelling, whitespace, token audience or scope, and the account or environment associated with the credential. Also verify that the token is being sent to the intended host and not exposed through a redirect or an incorrectly configured proxy.

For signed requests, inspect the canonical URL, timestamp, nonce, body hash, and header order required by the provider. A valid credential can still produce an authorization error if the signature was generated from a different request than the one sent.

6. Results are missing because of pagination

Many APIs return only part of a collection. Look for fields such as next, nextCursor, offset, limit, page, or a pagination link in the response. Do not assume that increasing a page-size parameter removes the need to follow a cursor; some APIs enforce a maximum.

• Record the initial page size and the number of items returned.
• Follow the provider’s cursor or next-page value exactly.
• Stop when the API indicates there is no next page.
• Protect against repeated cursors to avoid an infinite loop.
• Check for duplicates or gaps if records can change while you paginate.

What to double-check

Headers

Request and response headers often explain what the body cannot. Check Content-Type, Accept, caching directives, compression, request IDs, retry hints, and rate-limit fields. Header names are case-insensitive, but header values and formatting are not always interchangeable. A response may also include a redirect or a different content type than expected.

Request construction

Confirm that URL encoding is correct for query parameters and path values. Reserved characters, spaces, ampersands, and Unicode text can alter the request if they are concatenated manually. Use your HTTP client’s parameter and JSON-body features where available instead of building strings by hand.

Retries and timeouts

Set a finite connection and response timeout. Retry only errors that are plausibly temporary, and use a bounded strategy with increasing delays. Be especially careful with non-idempotent operations such as creating a payment or submitting an order: a timeout does not prove that the server did not process the request. Prefer provider-supported idempotency keys or a status-check endpoint when available.

Application logging

Useful logs include the method, route pattern, status, duration, response size, retry count, and correlation or request ID. Redact authorization headers, cookies, API keys, and sensitive fields. If the response is transformed before storage, retain enough diagnostic context to compare the raw shape with the transformed result. For longer-running extraction or integration jobs, the guidance in How to Monitor a Web Scraper can also help structure error tracking and data-quality checks.

Common mistakes

• Reading only the status code: The body may contain a field name, validation rule, or request ID that identifies the problem.
• Parsing every response as JSON: Check the content type and handle empty or non-JSON bodies safely.
• Retrying all failures: Repeatedly retrying a malformed request or a permission error adds noise and can worsen rate limiting.
• Assuming an empty array means success: Verify filters, permissions, tenant context, and pagination before treating it as a complete result.
• Logging secrets while debugging: Redact credentials before sharing logs or curl commands.
• Testing only inside the application: A minimal reproducible request helps separate API behavior from client-side serialization, parsing, or state-management bugs.
• Ignoring version and environment details: An endpoint can behave differently across development, staging, and production configurations.

When to revisit

Keep this API response debugging checklist close to any integration that matters. Revisit it when an API provider changes versions, authentication requirements, schemas, pagination rules, rate limits, or error formats. It is also worth reviewing before a seasonal planning cycle, a major release, a migration between environments, or a change in the HTTP client used by your application.

When an incident occurs, save a sanitized reproduction containing the method, endpoint pattern, relevant headers, status code, response shape, and pagination state. That record turns a one-time fix into a reusable runbook. After the issue is resolved, add a test for the failure mode: malformed input, expired authentication, an empty page, a non-JSON error, a timeout, or a repeated cursor.

For your next failed request, work in this order: reproduce it with a minimal command, capture status and headers, validate the body format, compare authentication and content type, inspect pagination or rate-limit signals, then apply a narrowly targeted fix. This sequence keeps API debugging evidence-driven and makes it easier to tell whether the problem is in the request, the provider, or the code handling the response.