Fix parking engine, and make observatory startup job-centric

Authored by TallFurryMan on Aug 28 2018, 8:14 AM.

Description

Fix parking engine, and make observatory startup job-centric

Summary:

  • Consider a mount that cannot park is always unparked. This fixes the state of the mount as seen by the Scheduler when the mount cannot park.
  • Ensure getting parking state returns a valid state, even if last action was failed. This fixes an issue arising when parking operation is aborted, that caused the Mount interface to always return PARKING_ERROR.
  • Make the Scheduler use isMountParked when starting in case the state machine is not up-to-date. This fixes the issue where the Scheduler is started with a mount parking state that was manipulated from the INDI panel in-between.
  • Fix remaining issues in transitions of the parking engine states. This fixes incorrect transitions in the state machine, unrelated to the actual state they were attempted in.

The next changes prepare the rework of the D-Bus connection of the Scheduler towards an event-driven implementation.
These improve Scheduler robustness by managing device events that are not reflected in the state machine.
They consolidate on the parking state changes, and are pushed now because they fix remaining issues with job startup.

  • Refactor INDI connection status test, to avoid depending on the machine state.
  • Move unpark check down to the slew procedure.
  • Refactor state checks into a call to checkStatus, managing most external events.
  • Move job startup from evaluateJobs to executeJob.

The big change in this differential is that the first job is executed AFTER the observatory starts up.
This makes Ekos take control of the observatory at the moment the Scheduler is started, with device connections and setup unparks.
The advantage of this method is that any device issue will fire when the end-user presses the start button, not a few hours later when the Scheduler wakes up.

Test Plan:
As this differental relates to parking states, use a basic scheduler job with only tracking set.
Uncheck "Stop Ekos After Shutdown" in the Ekos options to avoid losing driver states when the job finishes.

  • Start Scheduler, check job/Ekos/INDI are started, executed and terminated.
  • Test with "UnPark Mount" and "Park Mount" checked and unchecked.
  • Test with a Scheduler stop in the middle of slewing or capturing, and a restart.

Observe that if "UnPark Mount" is unchecked, the mount is still unparked ; this is by design.
Option "UnPark Mount" is here to make sure the order of unparking is honored.
This will be replaced later by one single checkbox with that specific meaning.

BUG - Observe that if "Park Mount" is unchecked, the mount is NOT unparked at the end of the procedure.
This is by design, and is clearly not symmetric with the "UnPark Mount" option, thus needs to be reworked.

  • Remove the capture storage.
  • Start Scheduler, check job/Ekos/INDI are started, and kill/disconnect indiserver.
  • Check that Scheduler cancels the running action, and attempts to restore connection with devices.
  • If indiserver is still accessible, Scheduler will reconnect and restart the job.
  • If indiserver is not accessible anymore, Scheduler will attempt to reconnect multiple times then stop.
  • If indiserver is accessible but not responsive, Scheduler will attempt to reconnect devices, fail then stop.

Note that using a local Simulator makes it difficult to test because the system is really quick to recover.
During that test, a remote indiserver is easier to test with, and allows for various test cases with the network.

BUG - Observe the issue that Scheduler is unable to handle the case of a single device being disconnected.

  • Remove the capture storage.
  • Start Scheduler, check job/Ekos/INDI are started, wait for mount slew to start.
  • Abort the slew while it is running.
  • Check that Scheduler does notice the problem and restarts the slew.
  • Park the mount while it is slewing.
  • Check that Scheduler does NOT notice the problem until the mount is parked, but then unparks and restarts the slew.
  • Park the mount then abort the park while it is running by unparking the mount, this causes IPS_ALERT on the parking property.
  • Check that Scheduler does notice the problem and restarts the slew.

Originally a parking property in IPS_ALERT would cause the Mount interface to return PARKING_ERROR, causing confusion and hanging Scheduler.

Next test requires two runs, one with guiding NOT set, the other with guiding set. This is obviously an edge case.

  • Remove the capture storage.
  • Start Scheduler, check jobs/Ekos/INDI are started, wait for the mount to capture.
  • Park the mount.
  • Check that Scheduler does NOT notice the problem if guiding is NOT set, and continues to capture until the end of the job.
  • Check that Guider notices the problem though, but nonetheless resumes guiding before the mount is fully parked.
  • Check that situation cannot be recovered.

BUG - Observe the issue that while Guider notices mount is parking, guiding is resumed by stubborn Scheduler, and successfully guides when close to the pole because of the lower slew speed.
This is obviously an edge case, and probably cannot be considered a valid test case as a mount that is parking may not answer to guide commands. Simulator does.

Next test requires the Dome Simulator, and two distant jobs - one that executes now, one that executes later on (repeated duplicate is ok).

  • Remove the capture storage.
  • Check Unpark Dome, Unpark Mount.
  • Start Scheduler, check jobs/Ekos/INDI are started, wait for the end of the first job.
  • Check that Scheduler shuts observatory down if job is farther than preemptive shutdown delay and that option is set.
  • Check that Scheduler parks the setup then sleeps if job is farther than lead time.
  • Check that Scheduler waits if job is sooner than lead time.
  • Abort Scheduler, remove capture storage.
  • Uncheck Unpark Dome, Unpark Mount, and pay attention to current state of those.
  • Start Scheduler, check jobs/Ekos/INDI are started, wait for the end of the first job.
  • Check that Scheduler shuts observatory down if job is farther than preemptive shutdown delay and that option is set.
  • Check that Scheduler does not park the setup then sleeps if job is farther than lead time.
  • Check that Scheduler waits if job is sooner than lead time.

BUG - Observe the issue that if Unpark Dome is not checked, Scheduler is unable to unpark the dome device.
The issue was fixed for the mount device, but is probably the same for the dome and needs to be revisited.

BUG - Observe the issue that if Unpark options are checked, and the first job is farther than the lead time, Scheduler will open the dome, then either shut the dome
immediately if preemptive shutdown is enabled, or leave the dome open while sleeping.

Reviewers: mutlaqja, wreissenberger

Reviewed By: mutlaqja, wreissenberger

Subscribers: kde-edu

Tags: KDE Edu

Differential Revision: https://phabricator.kde.org/D15073

Details

Committed
mutlaqjaAug 28 2018, 8:21 AM
Reviewer
mutlaqja
Differential Revision
D15073: Fix parking engine, and make observatory startup job-centric
Parents
R321:85bf46276f5c: Bugfix for #397650 flat creation failed
Branches
Unknown
Tags
Unknown