Unify the start/end time extraction functions

This isn't limited to the calendar handler, but we use this in post-
processing and the timeline model.