Event Delivery Troubleshooting
When the Event Pipeline cannot deliver an event to a critical sink after all retry attempts, the event is moved to the dead-letter queue. This page covers how to inspect delivery failures, understand error patterns, and resolve or requeue dead-lettered events.
Dashboard Pipeline Tile
The Dashboard shows a Pipeline status tile with a quick overview: pending event count, dead-letter count, and sink status. The tile color reflects the pipeline health: green (all delivered), yellow (events pending), or red (dead-letters present). Click Details to open the Pipeline Detail window.
Common Error Patterns
| Error | Classification | Resolution |
|---|---|---|
| HTTP 401 / 403 | Permanent | Check API key, bearer token, or credentials in siem.xml / webhook.xml. If using CredRef, verify the Credential Manager entry exists. |
| HTTP 404 | Permanent | Verify the endpoint URL. For SIEM, check <EndpointUrl>. For webhooks, check each <Url> element. |
| HTTP 429 | Transient | Rate-limited by the target. Events will be retried with backoff. Consider reducing event volume via SIEM filtering. |
| HTTP 5xx | Transient | Server-side error. Events will be retried. If persistent, check the target server's health. |
| Connection refused | Transient | Target is unreachable. Verify host, port, and firewall rules. |
| Timeout / Timed out | Transient | Increase <SendTimeoutSeconds> in eventpipeline.xml or <TimeoutSeconds> per endpoint. |
| DNS / Name resolution | Transient | Verify hostname resolves. Check DNS settings and network connectivity. |
| Certificate / SSL / TLS | Permanent | Certificate validation failed. Install the correct CA certificate, or set <ValidateCertificate>false</ValidateCertificate> for testing only. |
Dead-Letter Management
Dead-lettered events are stored in deadletter.jsonl within the EventPipeline folder. The following operations are available:
| Operation | Description |
|---|---|
| Requeue | Resets failed sinks back to Pending and clears timing fields. The event re-enters the retry cycle. Use after fixing the root cause (e.g. corrected credentials). |
| Resolve | Marks the event as resolved with an optional comment. The event remains in the dead-letter file for auditing but is excluded from active processing. |
| Purge | Permanently deletes dead-letter entries older than a specified date. |
| Export | Returns structured diagnostics JSON for a specific dead-letter entry, including the full event payload and all sink delivery states. |
Queue Files
The queue files are located in the folder configured by <QueueFolder> in eventpipeline.xml. Default: %ProgramData%\IT-Consulting Kinner\AppProfileSafe\EventPipeline.
| File | Format | Contents |
|---|---|---|
queue.jsonl |
NDJSON (one JSON object per line) | Active events with per-sink delivery state |
deadletter.jsonl |
NDJSON | Permanently failed events, optionally marked as resolved |
Both files are included in the diagnostics bundle (queue summary and last 100 dead-letter entries).
Troubleshooting Checklist
- Check the Pipeline tile on the Dashboard for current status.
- Open the application log (
app.csv) and search forEventPipelineServiceordead-letteredentries. See Where to Find Logs. - Review the
LastErrorandLastHttpStatusin the dead-letter entry to classify the failure (see table above). - Fix the root cause (credentials, URL, network, certificate).
- Requeue the affected events to retry delivery.
- If the events are no longer needed, resolve or purge them.
- Create a diagnostics bundle if further support analysis is needed.