Fix DepartureReply::departures to avoid copies
Also, make it less departure-centric, now that arrivals are handled here too.