Silent Job Failures: Monitoring Cron Jobs and Imports

Most shops monitor three layers, and each answers a different question. Uptime monitoring asks: is the server responding? Checkout monitoring asks: can an order still be completed? Log analysis asks: is the application writing errors to its log files? All three layers are worthwhile, and all three share the same blind spot. They observe what happens. A failed background job, however, is defined precisely by the fact that nothing happens -- no request, no error code, frequently not even a line in the log. What is missing is the fourth question: did the thing that should run regularly actually run?

Economically this gap is no small matter. The damage to the German economy in 2025 from data theft, espionage and sabotage came to 289.2 billion euros (Bitkom), of which 202.4 billion euros (Bitkom) related to cyberattacks; 87 percent (Bitkom) of the companies surveyed reported incidents. Silent job failures do not appear in these figures, because they are usually not an attack but a plain operational fault. That is exactly what makes them stubborn: an attack has reporting chains, templates and insurance policies. For a cron job that has not run for eleven days, by contrast, nobody has often claimed responsibility -- all the attention sits on external threats while internal processes keep running unobserved, or do not.

A job that aborts with a clear error is the pleasant case: it leaves a trace. The expensive cases are those where the system behaved correctly as far as it was concerned. The scheduler started what it was told to start; the fact that the result came back empty was not an event for it. Six patterns explain the bulk of the silent failures we encounter in practice -- and none of them reliably produces an entry that error monitoring could build on.

What all six patterns share is that classic error monitoring cannot see them. Anyone waiting solely for log entries at ERROR level waits in vain during a silent failure. The database behaves similarly: a job that runs into a timeout because of a blocking query sometimes leaves nothing but an entry in the slow query log -- a side effect that tidy database maintenance noticeably defuses.

A sign of life alone is not enough. The second most common silent failure is the run that takes place, reports in and is still wrong: the import processes nine hundred rows instead of forty thousand because the source file was truncated. The heartbeat arrives on schedule, no alert is raised, and the catalogue is unusable anyway. That is why every sign of life carries two additional metrics: the duration of the run and the volume it moved. Both are compared against a rolling average for the same hour of the week rather than a fixed value -- a Monday morning simply looks different from a Sunday night.

The thresholds are allowed to be rough. This is not statistical fine work but a line between normal and obviously wrong. For a catalogue import a downward deviation of twenty percent is usually reason enough to take a look; eighty percent is an alert. The opposite direction matters just as much: an import that suddenly writes three times as many rows as usual points to a file read in twice, and produces duplicates that later have to be untangled by hand. How tightly the thresholds can sit before alert fatigue sets in is the same trade-off as in log monitoring and alerting.

An alert that nobody reads at 3:40 a.m. is a log line. The question of who responds when is therefore not an organisational afterthought but part of the design. Background jobs typically run at night because the load is low then -- which means they also fail at night. A nightly import failure handled only the following morning has by then produced half a selling day with wrong stock levels. A staged chain makes sense here, in which urgency decides the route and not every alert receives the same treatment.

Which window is realistic depends on the business model. A shop with strong evening and night trade may well need a response time of thirty minutes; a trade-focused shop with orders during office hours is frequently fine with the next working day. What matters is that the window is committed and measurable -- which metrics need clean definitions for that is described in the article on response times in the SLA. For cases outside the agreed service hours, emergency support takes over with a call chain of its own.

Heartbeat monitoring is not a project that is set up once and then forgotten. Every new import, every new interface and every major update brings additional processes with it or changes existing cadences. Job monitoring therefore belongs in the same rhythm as updates, backups and continuous monitoring: an inventory that is kept current, a regular review of the mute and the overly talkative alerts, and a responsibility that holds during holidays too. In the SLA maintenance contract this becomes a committed service with a defined response time rather than a good intention. Trade keeps growing meanwhile -- nominal goods revenue growth of 3.8 percent (bevh and EHI Retail Institute) is expected for 2026, and with every percentage point the number of transactions affected by a silent hour rises.

Two transitions deserve particular attention. When the service provider changes, background jobs are typically the first thing lost, because they appear on no handover list -- what to watch out for is described in the article on taking over maintenance without shop downtime. And an agency looking after client projects needs an inventory that works across all tenants; how to organise that is shown in the article on white-label maintenance for agencies. In both cases the list of background jobs is precisely the document that is most often missing.

It starts with an inventory: which cron jobs, scheduled tasks and workers run in your shop, at what cadence should they run, and when did each of them last deliver a valid business result? That list becomes the monitoring plan, with time windows, runtime and volume thresholds and an escalation chain that reaches someone at night as well. You can request this inventory without obligation -- as a first step towards processes that report in themselves before anyone else does.