Back where the schedule is: a restart that cuts in at the next keyframe, and the case our test missed

What a restart used to do

Before 14 October 2026 the engine kept each channel’s running schedule in memory and nowhere else. A restart, whether a crash, a power cut or an engineer’s deliberate stop, opened the schedule file again and played its first item, whatever the time. A restart at ten in the morning on a day that began at six aired the six o’clock item again, every item after it ran late until the next item fixed to a time, and every playlist loaded into the channel during the day was gone.

What we wanted instead, in our own words at the time: write the playlist every two seconds, so a restart loads the latest edited version; find where the schedule says we should be; find the next keyframe at least 0.3 seconds from now, to give the decoder its cue time; and cut in there. We built it on 14 October 2026.

How it works

The copy. Every two seconds each channel’s live timeline, with every playlist loaded into it and every length the engine has measured, is written to disk if it changed. The write goes to a temporary file, is synced, renamed into place and the directory synced, so a power cut leaves the last complete copy and never half of one. A clean stop writes it once more on the way down, so it loses nothing; a crash can lose up to the last two seconds of edits.

Which copy a start takes. Only one that grew from the same schedule file, by its checksum, for the same channel. Anything else starts from the file, and the old copy is kept to one side under a new name. A schedule an operator has deliberately replaced is never overridden by yesterday’s edits.

Where it cuts in. From the master clock’s first tick, the engine looks 0.3 seconds ahead, finds the item the timeline has on air then, and finds that file’s first keyframe at least that far in, looking at most 30 seconds ahead. A decoder can only start cleanly at a keyframe, and the 0.3 seconds is the time to open the file and decode that first picture before it is due. If there is no keyframe in the window, or the item ends first, the channel goes to the next item, at its own time. Until the cut-in the channel is black.

On the channel’s frame grid. The keyframe becomes the airing’s first frame on the tick its scheduled time falls to, so every frame after it counts from a tick. Our first version placed it at the keyframe’s own time instead, and a review of our own work found what that did: a 50p file whose keyframe fell on an odd frame, played on a 25p channel, aired every frame half a frame late and the item after it a tick late. That was fixed before the feature went anywhere.

How we tested it

Until 9 November 2026 its proofs ran on one source, an H.264 file with a keyframe every two seconds, and every restart was a clean stop. So on 9 November 2026 we built a matrix. Each case is a day schedule: the source under test for 30 seconds, six seconds of bars after it, and an item fixed to a time 44 seconds in. The engine is stopped at one of three places in the first item, 2 seconds in, 13.7 seconds in, or 29.2 seconds in (inside its last second, so the next keyframe after the restart falls past its end), either cleanly or by a kill, which skips the last write and leaves the copy up to two seconds old. A second engine starts on the same schedule file at once.

Each case is judged from the engine’s own records against a reading written before it ran: one resumed line per channel; the cut-in at one of the source’s own keyframes, read from the file with ffprobe, at least 0.3 seconds after the second engine started; that airing’s start in the compliance log at the cut-in’s time to half a frame; every item after it on its time, the fixed one on its exact tick; the first item never aired again from its start; and the day’s compliance log verifying across the restart. It runs without the SDI card, on our development laptop, with the release build of the engine. The sources are made for the test from a generated picture; none is anyone’s programme.

As the test first judged them, all 36 cases passed. When the engine was stopped in the first item’s last second, every channel came back at a later item at its own time. The ten-second GOP is the slow row because the cut-in waits for the next keyframe, up to ten seconds of black; that is the trade the design makes, and the alternative, starting at the keyframe before and decoding forward to the right frame, costs more decoding at the moment the engine is starting.

Not every case passed first time, and the reasons are worth saying. Nineteen passed on the first run. Three came back as cannot-determine, because the laptop was busy with other builds and the master clock woke late; our test tooling refuses to call a failure on a loaded machine either way. The other fourteen failed because of two mistakes in the test, not in the engine. It judged an item’s end in the channel’s frames, when a 25p source on a 50p channel ends a source frame, two ticks, before the item’s scheduled end. And it compared the cut-in with the keyframe’s exact time, when the cut-in airs on the tick that time falls to. With both fixed, the 17 were run again on a quiet machine and passed.

What the test missed

Writing this entry, we read the twelve cases stopped in the first item’s last second again. Each said the channel came back at the item fixed to 44 seconds, not at the six seconds of bars that follow the first item. The bars clip has one keyframe, its first frame. The second engine’s first tick came just after the bars began (in the case we read closely, within 13 milliseconds of it), so the bars could not be joined, and the engine went to the next item that had not begun: the fixed one, about fourteen seconds later. The bars aired as black.

Two things were wrong, and the test saw neither. The engine skips an item that began just before its restart if the item’s file has no keyframe left in it: a long programme loses at most its last GOP, but a short item encoded as one GOP, an ident, a promo or a slate, loses its whole length. And nothing in the compliance log names the skipped item: not aired, not skipped. The log verified as valid, with no skips in it. Our own rule is that an item that never airs is recorded as passed over, and this path broke it. The test missed it because it gave the second engine a few seconds' allowance to start, and judged no item due inside it.

We corrected the test first. It now requires every scheduled item to be accounted for, aired or recorded skipped, and run again, all twelve name exactly this: ten fail, and our tooling calls the other two cannot-determine, the laptop having woken the master clock late during them. The engine is unchanged: whether it should start at the keyframe before and decode forward, or air an item begun a moment before the restart from its start, is a choice we are making with care, because both change what a restart puts on air. Recording the skipped item is not in doubt.