Network health
Track timing phases, redirects, status, bytes, validators, throttling, and grouped host failures.
Build source health →Feed monitoring
Feed monitoring that follows the whole pipeline.
Distinguish network reachability, document validity, source freshness, entry identity, queue processing, and destination delivery.
ReachabilityDNS, TLS, status, latency
ValidityFormat, parser, warnings
FreshnessChange and discovery lag
DeliveryQueue, retry, acceptance
Health dimensions
One green light hides too much.
Model each stage so an outage points to the publisher, ingestion pipeline, scheduler, or destination—not an undifferentiated “feed failed” state.
Document health
Detect HTML substitutions, malformed XML, unsupported formats, invalid dates, and parser regressions.
Validate feeds →Content health
Measure document change, new entries, updated entries, identity collisions, and source-specific quiet periods.
Model entries →Scheduler health
Observe due-source lag, host fairness, adaptive intervals, backoff state, and backlog recovery.
Scale polling →Delivery health
Track queue delay, attempts, response class, signature configuration, dead letters, and replay.
Monitor delivery →Incident evidence
Keep bounded response samples, normalized diffs, parser versions, event IDs, and timeline context.
Read the runbook →Source timeline
Put fetches, changes, and deliveries on one clock.
Operators should be able to trace a release from publisher observation to accepted destination delivery.
1
Last attemptWhen the scheduler started the most recent network request.
2
Last successWhen a valid response or not-modified result was received.
3
Last new itemWhen a previously unseen entry was committed.
4
Last deliveryWhen a configured destination accepted the event.
Alert design
Notify on transitions with evidence.
Degraded source
Several attempts fail or latency rises beyond the source baseline, while the last snapshot remains usable.
Define thresholds →Invalid replacement
A previously valid feed begins serving HTML, malformed content, or an unsupported document.
Debug the response →Destination failure
Entries are committed, but a webhook or downstream API rejects delivery beyond the retry policy.
Plan replay →FAQ
Monitoring decisions.
Is HTTP 200 enough to call a feed healthy?
No. The response may be HTML, stale, malformed, or valid but no longer include expected entries. Monitor reachability, document validity, content behavior, and downstream delivery separately.
How should staleness be measured?
Use the source's own historical publication intervals when enough data exists, and keep publisher dates separate from first-seen observation times.
What should an alert contain?
Include the affected source, health dimension, transition time, recent attempts, redirect or response evidence, parser signature, and a runbook link.
Monitor the contract, not only the URL.
Reachability, validity, freshness, identity, and delivery are different promises. Keep them visible.