Webhooks & logs
Inspect webhook delivery across apps, re-send parked events, and review the audit trail and gateway-callback evidence from the admin panel.
Three operational logs help an operator debug deliveries and trace what happened: the webhook delivery log, the append-only audit log, and the gateway-callback evidence log. All three are session-authenticated admin surfaces gated by RBAC authorities — they are not part of the app-facing API.
These are read-and-act surfaces for operators. App developers integrating webhook consumers should start with the Webhooks guide.
All responses use the standard envelope:
{
"data": <payload | null>,
"meta": { "success": true, "message": null, "errorCode": null, "timestamp": "2026-06-30T12:00:00Z" },
"pagination": { "page": 0, "size": 20, "totalElements": 42, "totalPages": 3 }
}The examples below show just the data payload after the first one. Admin
endpoints authenticate with a session bearer token, not an API key:
-H "Authorization: Bearer SESSION_TOKEN"Webhook delivery log
GET /api/v1/admin/webhooks lists outbox events with their delivery state across
every app the caller is scoped to. The coarse authority audit:read is checked
on the endpoint, then the result set is narrowed to the apps the operator can see
(RBAC-4) — one app never appears in another operator's view. See
app isolation.
Query parameters
Every filter is optional; a missing filter leaves that dimension unconstrained.
| Field | Type | Required | Notes |
|---|---|---|---|
page | int | No | Zero-based page index. Default 0. |
size | int | No | Page size. Default 20. |
sortBy | string | No | Sort property. Default createdAt. |
order | ASC | DESC | No | Sort direction. Default DESC. |
paginate | boolean | No | When false, returns the full result set unpaged. Default true. |
applicationId | Long | No | Restrict to one app. |
endpointId | Long | No | Restrict to one webhook endpoint. Since an app may have several per mode, filtering by app and mode alone returns every sibling's rows interleaved — this is how you answer "is this consumer receiving anything?". |
mode | TEST | LIVE | No | Enum name (see below). |
type | WebhookEventType | No | Either form — the enum name PAYMENT_SUCCEEDED or the dotted wire name payment.succeeded. |
deliveryStatus | PENDING | DELIVERED | FAILED | No | Enum name. |
See pagination & filtering for the
shared page / size / sortBy / order / paginate conventions.
type accepts either form; mode and deliveryStatus want the enum name.
Filtering by type=SUBSCRIPTION_RENEWED and by type=subscription.renewed (the
dotted form your consumer receives) both work. mode and deliveryStatus have no
wire alias, so they take the constant name only — mode=TEST, not mode=test. An
unrecognised value on any of the three is a 400 naming the offending parameter.
The response field type is always rendered as the dotted wire name.
The full set of catalogue names you can pass as type:
PAYMENT_SUCCEEDED PAYMENT_FAILED PAYMENT_CANCELLED PAYMENT_EXPIRED
REFUND_PENDING REFUND_SUCCEEDED REFUND_FAILED
INVOICE_ISSUED INVOICE_PARTIALLY_PAID INVOICE_PAID INVOICE_VOIDED
SUBSCRIPTION_CREATED SUBSCRIPTION_ACTIVATED SUBSCRIPTION_TRIAL_WILL_END
SUBSCRIPTION_RENEWED SUBSCRIPTION_PAYMENT_SUCCEEDED SUBSCRIPTION_PAYMENT_FAILED
SUBSCRIPTION_PAST_DUE SUBSCRIPTION_CANCELED SUBSCRIPTION_PAUSED
SUBSCRIPTION_RESUMED SUBSCRIPTION_ENTITLEMENTS_UPDATEDWebhookEventDto (response row)
Each delivery-log row is a WebhookEventDto. Null fields are omitted from the JSON.
| Field | Type | Required | Notes |
|---|---|---|---|
id | Long | Yes | Outbox event id. This is also the X-Webhook-Id header your endpoint received — use it to correlate. |
applicationId | Long | Yes | The owning app. |
applicationName | string | No | Resolved app display name. Omitted when the app can no longer be resolved (fall back to applicationId). |
endpointId | Long | No | Which webhook endpoint this row was delivered to. Omitted for rows captured before endpoints existed, for the sentinel written when an app has no endpoint at all, and for rows whose endpoint has since been deleted — delivery history deliberately outlives its target. |
endpointUrl | string | No | Resolved URL of endpointId, for display. Omitted on the same three cases; the deletion itself is recorded in the audit log with the URL. Never a signing secret. |
eventGroupUid | UUID | No | Groups the sibling rows one logical event fanned out into, so a dashboard can show "this event, to each of its destinations". Not part of the delivered envelope — consumers still dedupe on id. Omitted for pre-migration rows, which are groups of one. |
mode | string | Yes | TEST or LIVE. |
type | string | Yes | The dotted wire name carried in the delivered envelope (e.g. subscription.renewed). |
schemaVersion | int | Yes | Payload schema version. |
payload | string | Yes | The event's typed data object as a JSON string (the app's own data). No signing secret ever appears here. |
deliveryStatus | string | Yes | PENDING, DELIVERED, or FAILED. |
attempts | int | Yes | Delivery attempts made so far. |
lastResponseStatus | Integer | No | HTTP status of the most recent attempt. null (omitted) when the endpoint was unreachable. |
lastError | string | No | Short diagnostic for the most recent failed attempt. Omitted once delivered. |
nextAttemptAt | datetime | No | When the row is next eligible for an attempt. Omitted when not scheduled (delivered, parked, or due now). |
lastAttemptAt | datetime | No | When the most recent attempt ran. Omitted until the first attempt. |
deliveredAt | datetime | No | When a 2xx acknowledgement was received. Omitted until delivered. |
createdAt | datetime | Yes | Event-creation time — authoritative for ordering and consumer dedupe. |
updatedAt | datetime | Yes | Last update to the row. |
The mode filter is an enum name, but unlike the dashboard analytics endpoints
it is optional here — there is no required mode query parameter on the
delivery log. The mode shown on each row is the mode of the producing record; it
is never set by a caller. See modes.
Example — failed subscription renewals
curl "http://localhost:8080/api/v1/admin/webhooks?type=SUBSCRIPTION_RENEWED&deliveryStatus=FAILED&size=20" \
-H "Authorization: Bearer SESSION_TOKEN"{
"data": [
{
"id": 90817,
"applicationId": 42,
"applicationName": "Acme Storefront",
"mode": "LIVE",
"type": "subscription.renewed",
"schemaVersion": 1,
"payload": "{\"subscription_id\":\"sub_8f3a\",\"amount\":150000,\"currency\":\"BDT\"}",
"deliveryStatus": "FAILED",
"attempts": 6,
"lastResponseStatus": 503,
"lastError": "503 Service Unavailable",
"lastAttemptAt": "2026-06-30T09:41:12",
"createdAt": "2026-06-30T08:55:00",
"updatedAt": "2026-06-30T09:41:12"
}
],
"meta": { "success": true, "message": null, "errorCode": null, "timestamp": "2026-06-30T12:00:00Z" },
"pagination": { "page": 0, "size": 20, "totalElements": 1, "totalPages": 1 }
}Note the row above has deliveryStatus: FAILED and attempts: 6, and no
nextAttemptAt or deliveredAt — it is parked. The amount of 150000 in
the payload is paisa: 1,500.00 BDT. See money.
Delivery status & the retry lifecycle
A captured event starts PENDING and is driven by the background delivery loop.
The status field has exactly three values:
| Status | Meaning |
|---|---|
PENDING | Captured and awaiting a delivery attempt — either never attempted yet, or failed-with-budget and scheduled to retry at nextAttemptAt. |
DELIVERED | The consumer endpoint returned 2xx. Terminal; deliveredAt is set and nextAttemptAt/lastError are cleared. |
FAILED | The retry budget was exhausted without a 2xx. The event is parked — it is never retried automatically again, only via a manual re-send. |
Two conditions park on the first attempt instead of consuming the retry budget. Both are things only an operator or the merchant can fix, so re-deriving the same answer thirteen more times over six hours would waste claim capacity and — worse — delay the moment anyone notices, because the row stays non-terminal until the budget drains:
lastError | Meaning | Fix |
|---|---|---|
No enabled webhook endpoint is configured for app N in M mode | The app has no enabled endpoint for that mode. Written terminal at capture, so it never enters the claim pool — one insert, no delivery work. It names the mode because the app may well have healthy endpoints in the other one. | Add an endpoint for that mode on the app's Webhooks screen. |
no enabled webhook endpoint for app N in M mode | The endpoint this row was addressed to was deleted or disabled between capture and delivery. | Re-enable it, or re-send the row once another endpoint exists. |
Webhook endpoint deleted before this event could be delivered | The endpoint was deleted while this row was still queued. Parking is deliberate: the alternative would be redelivering it to a different endpoint, signed with a different secret. | Nothing to fix — re-send only if you want it delivered elsewhere. |
LIVE_REQUIRES_HTTPS | A LIVE event whose endpoint is not https. Only reachable for endpoints migrated from the legacy shared column; new ones cannot be saved that way. | Change that endpoint's URL to https. See Webhook endpoints and URL requirements. |
LIVE_REQUIRES_HTTPS is deliberately not SCHEME_BLOCKED. SCHEME_BLOCKED means
the deployment's scheme allowlist refused the URL and is changed by configuration.
LIVE_REQUIRES_HTTPS is unconditional and no configuration overrides it. They used to
share one code, which put a message in front of operators that never mentioned TLS and
pointed them at an allowlist that was not the problem.
The delivery loop is configured in app.webhook.delivery.*. The values that govern
when a row parks:
| Tunable | Default | Meaning |
|---|---|---|
maxAttempts | 14 | Total attempts before the event is parked FAILED. |
initialBackoff | 10s | Wait before the first retry; doubles each subsequent retry. |
maxBackoff | 1h | Upper bound on the exponential backoff. |
jitterRatio | 0.2 | ±20% randomisation applied to each computed backoff. |
batchSize | 250 | How many due events one sweep claims and delivers. |
connectTimeout | 3s | Connect timeout for the delivery POST. |
responseTimeout | 8s | Response timeout for the delivery POST. |
requestTimeout | 15s | Absolute ceiling on one delivery attempt. |
Backoff before the n-th retry (0-based) is min(maxBackoff, initialBackoff · 2ⁿ),
then jittered by ±jitterRatio, giving a retry horizon of roughly 6 hours across
all 14 attempts. After maxAttempts failed attempts the event parks. The two
immediate-park conditions above bypass this schedule entirely.
Webhook endpoints and URL requirements
An application configures up to five webhook endpoints per mode — five TEST and five
LIVE — at /api/v1/admin/apps/{id}/webhook-endpoints. Each is an independent row with
its own URL, its own signing secret, its own enabled state and its own event subscription.
A captured event fans out to every enabled endpoint of its own mode, producing one delivery row per endpoint. Those rows have independent statuses, attempt counts and retry schedules, so one merchant consumer failing never holds up another.
This replaced a single shared webhook_url. Under the old model a merchant could not
point a test consumer anywhere without repointing the destination their production traffic
used — so in practice production apps could not verify TEST webhooks at all. V38
migrated each app's existing URL into one TEST and one LIVE endpoint carrying the app's
existing signing secret, so the change is invisible to a merchant who does nothing.
LIVE events are never transmitted over an unencrypted connection. The delivery
signature proves the payload was not altered; it does not keep it private, and a
plaintext LIVE payload exposes amounts, customer references and event types to
anyone on the network path.
Enforcement
Because endpoints now carry their own mode, the TLS rule is stated per mode. That was not
possible before: with one column serving both modes there was no "the LIVE URL" to hold to
a stricter standard, so the write-time rule had to be "https, always" and the TEST
plaintext allowance was theoretical.
| Where | Behaviour |
|---|---|
Create endpoint (POST /api/v1/admin/apps/{id}/webhook-endpoints), LIVE | A non-https URL is rejected with 400 and error code WEBHOOK_URL_REQUIRES_TLS, regardless of the scheme allowlist. |
Create endpoint, TEST | Governed by the scheme allowlist (https alone by default). |
| Update endpoint | Same rules, evaluated against the endpoint's own mode — a LIVE endpoint cannot be edited down to plaintext. |
| Sixth endpoint in one mode | 409 / WEBHOOK_ENDPOINT_LIMIT_REACHED. The cap is per mode, so the other mode is unaffected. |
| Duplicate URL in the same mode | 409 / WEBHOOK_ENDPOINT_DUPLICATE_URL. The same URL in both modes is fine — that is what the migration produces. |
Delivery (LIVE) | Still refused before any connection is opened; the row parks with lastError: "LIVE_REQUIRES_HTTPS". Now only reachable for endpoints migrated from the legacy column, since new ones cannot be saved that way. |
Endpoints backfilled by V38 whose inherited URL was not https are created DISABLED
in LIVE. Those events already parked, so nothing is lost — but the merchant now sees a
disabled endpoint in the dashboard rather than a quiet trail of parked rows.
Signing secrets
Each endpoint has its own, so one can be rotated or revoked without invalidating every
other consumer, and a test consumer's key can never verify live traffic. A secret is
returned exactly twice — by create and by rotate — and by no read endpoint. Both are gated
on app:manage plus recent re-auth, and both are audited.
Rotation has no grace window, unlike api-credential rotation. A key we verify can accept two values for a while; a key we sign with cannot, because each delivery carries exactly one signature. The old secret stops working the instant the rotation commits, so the merchant must deploy the new one before the next event fires.
Relaxing it (self-hosted / sandbox)
Plaintext endpoints are permitted by widening the allowlist the transport already
uses — app.webhook.delivery.http.allowed-schemes, env WEBHOOK_DELIVERY_SCHEMES:
WEBHOOK_DELIVERY_SCHEMES=https,httpThe shipped default is https alone. This is deliberately the same key the
delivery transport reads rather than a second switch, so the two can never be set to
disagree.
Widening this permits storing an http:// URL and delivering TEST events to
it. It does not make LIVE delivery over plaintext work — nothing does. On such a
deployment the split brain is re-enabled rather than removed: TEST delivers, LIVE
parks immediately with LIVE_REQUIRES_HTTPS. Use it for sandbox and CI stacks where no
app is live, never in production.
A URL stored before this rule existed, or written directly into the column, is not retroactively rejected — it is caught at delivery time by the runtime check, which remains in place as the last line rather than the only one.
Re-send a parked event
POST /api/v1/admin/webhooks/{id}/resend re-queues one parked, failed, or still-pending
event for re-delivery and returns its re-queued state.
This action is gated by app:manage. The authority is checked coarsely on the
endpoint, then re-checked app-scoped against the event's owning app inside the
use case (RBAC-4). It is a sensitive action and additionally requires a recent
re-authentication:
- Unknown event id →
404(RESOURCE_NOT_FOUND). - Operator lacks
app:managefor that event's app →403(FORBIDDEN). - No recent re-authentication →
403(FORBIDDEN). - The event is being delivered right now →
409. Retry a few seconds later.
Re-queue
The row is set back to PENDING, due now, with attempts reset to 0 and
deliveredAt cleared. A re-send buys a full fresh retry budget — an operator asking
to re-send a parked event means "try properly again", not "try once more".
The diagnostic fields — lastError, lastResponseStatus, lastAttemptAt — are
deliberately not cleared. They still describe the attempt that failed before the
re-send, which is what lets you look at a re-queued row and still see what you were
re-sending it for. The next attempt overwrites them.
Audit
A WEBHOOK_RESENT entry is written to the append-only audit log (targetType: WEBHOOK_EVENT, targetId = the event id), recording the operator, the event type,
the app, and the attempt count.
Deliver on the next sweep
Delivery is enqueued, not inline — no HTTP call is made inside the admin request,
so the endpoint returns immediately and a slow consumer can never hold the request
open. The returned WebhookEventDto therefore shows the re-queued state
(PENDING, attempts: 0), not a delivery outcome. Refresh the log to see the result.
A re-send keeps the same X-Webhook-Id (the event id never changes), so a
well-behaved consumer that
dedupes on the id treats the
replay as the same event and does not double-process it.
Example
curl -X POST "http://localhost:8080/api/v1/admin/webhooks/90817/resend" \
-H "Authorization: Bearer SESSION_TOKEN"A successful re-send returns the re-queued row — deliveryStatus back to
PENDING, attempts reset to 0, deliveredAt cleared. This is the state before the
next sweep picks it up, not a delivery outcome:
{
"id": 90817,
"applicationId": 42,
"applicationName": "Acme Storefront",
"mode": "LIVE",
"type": "subscription.renewed",
"schemaVersion": 1,
"payload": "{\"subscription_id\":\"sub_8f3a\",\"amount\":150000,\"currency\":\"BDT\"}",
"deliveryStatus": "PENDING",
"attempts": 0,
"lastResponseStatus": 503,
"lastError": "503 Service Unavailable",
"lastAttemptAt": "2026-06-30T09:41:12",
"createdAt": "2026-06-30T08:55:00",
"updatedAt": "2026-06-30T12:01:30"
}Two things about that response are easy to misread.
lastError, lastResponseStatus and lastAttemptAt are carried over, not cleared.
They describe the attempt that failed before the re-send. A re-queued row that shows
503 has not just failed again — it has not been tried yet. Only attempts,
deliveryStatus and deliveredAt are reset.
Null fields are omitted, never emitted as null. The response uses
NON_NULL serialization, so deliveredAt is absent above rather than present-and-null.
Do not write a client that waits for an explicit null.
Refresh the delivery log to see the outcome. If the consumer is still down the row
stays PENDING with a new nextAttemptAt and works through its fresh 14-attempt
budget, then parks FAILED again with lastError and lastResponseStatus
describing the latest failure.
Audit log
GET /api/v1/admin/audit-logs (requires audit:read) is the append-only trail
of sensitive actions — credential rotation/revocation, gateway overrides, user and
role management, refund approvals, and webhook re-sends (WEBHOOK_RESENT). Filter by
actorType, actorId, action, targetType, and targetId, with the same
page / size / sortBy / order / paginate conventions.
actorType is the enum name — USER (a human admin), APP (a client
application acting via its API key), or SYSTEM (an automated action).
Each row is an AuditLogDto:
| Field | Type | Required | Notes |
|---|---|---|---|
id | Long | Yes | Entry id. |
actorType | string | Yes | USER, APP, or SYSTEM. |
actorId | Long | No | Admin-user id or application id, per actorType. |
actorLabel | string | No | Resolved human-readable actor label. |
action | string | Yes | The audited action, e.g. WEBHOOK_RESENT. |
targetType | string | No | The kind of entity acted on, e.g. WEBHOOK_EVENT. |
targetId | string | No | The target entity id. |
metadata | string | No | Free-form context string captured at the time of the action. |
createdAt | datetime | Yes | When the action occurred. |
The audit log can be read but never edited or deleted. It is the record of who did what, when — treat a missing entry as significant.
Gateway-callback evidence
GET /api/v1/admin/gateway-callbacks (requires audit:read) records every inbound
gateway callback (IPN) the service received, with the decision it applied — useful
when a payment's state and the gateway appear to disagree. Filter by mode,
outcome, reference, matchedPaymentId, gateway, and signatureValid.
Callback evidence is forensic data with no single owning app (an unmatched
callback matched no payment), so — like the audit trail — it is not
app-scope-narrowed; the audit:read authority is the only gate.
Each row is a GatewayCallbackLogDto:
| Field | Type | Required | Notes |
|---|---|---|---|
id | Long | Yes | Evidence row id. |
gateway | string | Yes | The gateway that sent the callback (e.g. sslcommerz). |
mode | string | Yes | TEST or LIVE. |
reference | string | No | The gateway's transaction reference. |
matchedPaymentId | Long | No | The payment the callback was matched to, if any. |
signatureValid | boolean | Yes | Whether the callback's signature verified. |
outcome | string | Yes | The decision applied (see below). |
note | string | No | Short handler note. |
rawPayload | string | No | The callback's fields exactly as received, for forensic inspection (gateway POST data, no card/PAN material). |
createdAt | datetime | Yes | When the callback was processed. |
The outcome enum name explains what the handler did:
| Outcome | Meaning |
|---|---|
SETTLED | Server-confirmed success matching amount/currency → payment marked succeeded. |
SETTLED_AFTER_EXPIRY | The same, but against a payment that had already been expired — the gateway session outlived our deadline. The payment is revived to succeeded and payment.succeeded is emitted after the earlier payment.expired. |
FAILED | Gateway reported the attempt failed → payment marked failed. |
CANCELLED | Customer abandoned the hosted checkout → payment marked cancelled. |
DUPLICATE | Verified, but the payment was already terminal — acknowledged, no change. |
MISMATCH | Confirmed success whose amount/currency did not match — flagged for review, payment left unpaid. |
SIGNATURE_INVALID | Signature did not verify — rejected, no state change. |
UNVERIFIED | Could not be authoritatively confirmed server-side — payment left unchanged. |
UNMATCHED | No payment matched the callback's reference — recorded and ignored. |
An outcome of SIGNATURE_INVALID or DUPLICATE here explains why a callback did
not move a payment, without you having to dig into the gateway's own dashboard. A run of
SETTLED_AFTER_EXPIRY rows is worth acting on rather than just noting: it means the checkout
window is shorter than customers are actually taking, and every one of those is a merchant who
was told "expired" and then "succeeded". Raise the app's Checkout window in its settings. See
reconciliation for matching settled gateway records
against the ledger.
Related
Webhooks guide
Build and verify a webhook consumer: signatures, dedupe, and the event catalogue.
Transactions
The cross-app payment list these callbacks settle against.
Reconciliation
Match gateway settlement against the double-entry ledger.
RBAC
The audit:read and app:manage authorities these surfaces gate on.