Differential D18335 Diff 49826 doc/ekos-scheduler.docbook

Changeset View

Standalone View

View Options

doc/ekos-scheduler.docbook

  • This file was added.
1​<sect2 id="ekos-scheduler">
2​ <title>Scheduler</title>
3​ <indexterm>
4​ <primary>Tools</primary>
5​ <secondary>Ekos</secondary>
6​ <tertiary>Scheduler</tertiary>
7​ </indexterm>
8​ <screenshot>
9​ <screeninfo>
10​ Ekos Scheduler Module
11​ </screeninfo>
12​ <mediaobject>
13​ <imageobject>
14​ <imagedata fileref="ekos_scheduler.png" format="PNG"/>
15​ </imageobject>
16​ <textobject>
17​ <phrase>Ekos Scheduler Module</phrase>
18​ </textobject>
19​ </mediaobject>
20​ </screenshot>
21​ <sect3 id="ekos-scheduler-introduction">
22​ <title>Introduction</title>
23​ <para>
24​ Ekos Scheduler is an indispensable arsenal in building your robotic observatory. A Robotic observatory is an observatory composed of several subsystems that are orchestrated together to achieve a set of scientific objectives without human intervention. It is the only Ekos module that does not require Ekos to be started as it is utilized to <emphasis>start</emphasis> and <emphasis>stop</emphasis> Ekos. It is designed to be straightforward and intuitive. However, the scheduler should only be used after you mastered Ekos and knows all the quirks of your equipment. Since the complete process is automated, including focus, guiding, and meridian flip. All equipment should be thoroughly used with Ekos and all their parameters and settings adjusted to achieve the best result.
25​ </para>
26​ <para>
27​ With Ekos, the user can utilize the powerful sequence queue to image batches of images for a particular target. In simple setups, the user is expected to focus the CCD, align the mount, frame the target, and start guiding before initiating the capture process. For more complex observatory environments, there are usually predefined custom procedures to be executed to prepare the observatory for imaging, and another set of procedures on shutdown. The user may plan to image one or more targets during the night and expects data to be ready by morning. In &kstars;, tools such as the <link linkend="tool-obsplanner">Observation Planner</link> and <link linkend="tool-whatsup">What's up Tonight</link> help the user in selecting candidates for imaging. After selecting the desired candidates, the user can add them to the Ekos Scheduler list for evaluation. The user may also add the targets directly in Ekos scheduler or select a FITS file of a previous image.
28​ </para>
29​ </sect3>
30​
31​ <sect3 id="ekos-scheduler-settings">
32​ <title>Settings</title>
33​ <para>
34​ Ekos Scheduler provides a simple interface to aid the user in setting the conditions and constraints required for an observation job. Each observation job is composed of the following:
35​ </para>
36​ <itemizedlist>
37​ <listitem>
38​ <para>
39​ <guilabel>Target name and coordinates</guilabel>: Select target from the Find Dialog or Add it from Observation Planner. You can also enter a custom name.
40​ </para>
41​ </listitem>
42​ <listitem>
43​ <para>
44​ <guilabel>Optional FITS file</guilabel>: If a FITS file is specified, the astrometry solver shall solve the file and use the central RA/DEC as the target coordinates.
45​ </para>
46​ </listitem>
47​ <listitem>
48​ <para>
49​ <guilabel>Sequence File</guilabel>: The sequence file is constructed in the Ekos <link linkend="ekos-capture">Capture Module</link>. It contains the number of images to capture, filters, temperature settings, prefixes, download directory, &etc;
50​ </para>
51​ </listitem>
52​ <listitem>
53​ <para>
54​ <guilabel>Priority</guilabel>: Set job priority in the range of 1 to 20 where 1 designates the highest priority and 20 the lowest priority. Priority is applied in calculating the weight used to select the next target to image.
55​ </para>
56​ </listitem>
57​ <listitem>
58​ <para>
59​ <guilabel>Profile</guilabel>: Select which equipment profile to utilize when starting Ekos. If Ekos &amp; INDI are already started and online, this selection is ignored.
60​ </para>
61​ </listitem>
62​ <listitem>
63​ <para>
64​ <guilabel>Steps</guilabel>: The user selects which Ekos modules should be utilized in the observation job execution workflow.
65​ </para>
66​ </listitem>
67​ <listitem>
68​ <para>
69​ <guilabel>Startup Conditions</guilabel>: Conditions that must be met <emphasis role="bold">before</emphasis> the observation job is started. Currently, the user may select to start as soon as possible <guimenuitem>Now</guimenuitem>, or when the target is near or past culmination, or at a specific time.
70​ </para>
71​ </listitem>
72​ <listitem>
73​ <para>
74​ <guilabel>Constraints</guilabel>: Constraints are conditions that must be met <emphasis role="bold">at all times</emphasis> during the observation job execution process. These include minimum target altitude, minimum moon separation, twilight observation, and weather monitoring.
75​ </para>
76​ </listitem>
77​ <listitem>
78​ <para>
79​ <guilabel>Completion Conditions</guilabel>: Conditions that trigger completion of the observation job. The default selection is to simply mark the observation job as complete once the sequence process is complete. Additional conditions enable the user to repeat the sequence process indefinitely or up until a specific time.
80​ </para>
81​ </listitem>
82​ </itemizedlist>
83​ <para>
84​ You must select the <guilabel>Target</guilabel> and <guilabel>Sequence</guilabel> before you can add a job to the Scheduler. When the scheduler starts, it evaluates all jobs in accord to the conditions and constraints specified and attempts to select the best job to execute. Selection of the job depends on a simple heuristic algorithm that scores each job given the conditions and constraints, each of which is weighted accordingly. If two targets have identical conditions and constraints, usually the higher priority target followed by higher altitude target is selected for execution. If no candidates are available at the current time, the scheduler goes into sleep mode and wakes up when the next job is ready for execution.
85​ </para>
86​ <screenshot>
87​ <screeninfo>
88​ Scheduler + Planner
89​ </screeninfo>
90​ <mediaobject>
91​ <imageobject>
92​ <imagedata fileref="scheduler_planner.png" format="PNG"/>
93​ </imageobject>
94​ <textobject>
95​ <phrase>Scheduler + Planner</phrase>
96​ </textobject>
97​ </mediaobject>
98​ </screenshot>
99​ <para>
100​ The description above only tackles the <emphasis role="bold">Data Acquisition</emphasis> stage of the observatory workflow. The overall procedure typically utilized in an observatory can be summarized in three primary stages:
101​ </para>
102​ <orderedlist>
103​ <listitem>
104​ <para>
105​ Startup
106​ </para>
107​ </listitem>
108​ <listitem>
109​ <para>
110​ Data Acquisition (including preprocessing and storage)
111​ </para>
112​ </listitem>
113​ <listitem>
114​ <para>
115​ Shutdown
116​ </para>
117​ </listitem>
118​ </orderedlist>
119​ </sect3>
120​
121​ <sect3 id="ekos-scheduler-startup-procedure">
122​ <title>Startup Procedure</title>
123​ <para>
124​ Startup procedure is unique to each observatory but may include:
125​ </para>
126​ <itemizedlist>
127​ <listitem>
128​ <para>
129​ Turning on power to equipment
130​ </para>
131​ </listitem>
132​ <listitem>
133​ <para>
134​ Running safety/sanity checks
135​ </para>
136​ </listitem>
137​ <listitem>
138​ <para>
139​ Checking weather conditions
140​ </para>
141​ </listitem>
142​ <listitem>
143​ <para>
144​ Turning off light
145​ </para>
146​ </listitem>
147​ <listitem>
148​ <para>
149​ Fan/Light control
150​ </para>
151​ </listitem>
152​ <listitem>
153​ <para>
154​ Unparking dome
155​ </para>
156​ </listitem>
157​ <listitem>
158​ <para>
159​ Unparking mount
160​ </para>
161​ </listitem>
162​ <listitem>
163​ <para>
164​ &etc;
165​ </para>
166​ </listitem>
167​ </itemizedlist>
168​ <para>
169​ Ekos Scheduler only initiates the startup procedure once the startup time for the first observation job is close (default <emphasis>lead</emphasis> time is 5 minutes before <emphasis>startup</emphasis> time). Once the startup procedure is completed successfully, the scheduler picks the observation job target and starts the sequence process. If a startup script is specified, it shall be executed first.
170​ </para>
171​ </sect3>
172​
173​ <sect3 id="ekos-scheduler-data-acquisition">
174​ <title>Data Acquisition</title>
175​ <para>
176​ Depending the on the user selection, the typical workflow proceeds as follows:
177​ </para>
178​ <itemizedlist>
179​ <listitem>
180​ <para>
181​ Slew mount to target. If a FITS file was specified, it first solves the files and slew to the file coordinates.
182​ </para>
183​ </listitem>
184​ <listitem>
185​ <para>
186​ Auto-focus target. The autofocus process automatically selects the best star in the frame and runs the autofocus algorithm against it.
187​ </para>
188​ </listitem>
189​ <listitem>
190​ <para>
191​ Perform plate solving, sync mount, and slew to target coordinates.
192​ </para>
193​ </listitem>
194​ <listitem>
195​ <para>
196​ Perform post-alignment focusing since the frame might have moved during the plate solving process.
197​ </para>
198​ </listitem>
199​ <listitem>
200​ <para>
201​ Perform calibration and start auto-guiding: The calibration process automatically selects the best guide star, performs calibration, and starts the autoguide process.
202​ </para>
203​ </listitem>
204​ <listitem>
205​ <para>
206​ Load the sequence file in the <link linkend="ekos-capture">Capture module</link> and start the imaging process.
207​ </para>
208​ </listitem>
209​ </itemizedlist>
210​ </sect3>
211​
212​ <sect3 id="ekos-scheduler-shutdown">
213​ <title>Shutdown</title>
214​ <para>
215​ Once the observation job is completed successfully, the scheduler selects the next target. If the next target scheduled time is not due yet, the mount is parked until the target is ready. Furthermore, if the next scheduled target is not due for a user-configurable time limit, the scheduler performs a <emphasis>preemptive</emphasis> shutdown to preserve resources and performs the startup procedure again when the target is due.
216​ </para>
217​ <para>
218​ If an unrecoverable error occurs, the observatory initiates shutdown procedure. If there is a shutdown script, it will be executed last.
219​ </para>
220​ <para>
221​ The following video demonstrates an earlier version of the scheduler, but the basic principles still apply today:
222​ </para>
223​ <mediaobject>
224​ <videoobject>
225​ <videodata contentdepth="315" contentwidth="560" fileref="https://www.youtube.com/embed/v8vIXD1kois"/>
226​ </videoobject>
227​ <caption>
228​ <para>
229​ <phrase>Ekos Scheduler</phrase>
230​ </para>
231​ </caption>
232​ </mediaobject>
233​ </sect3>
234​
235​ <sect3 id="ekos-scheduler-weather-monitoring">
236​ <title>Weather Monitoring</title>
237​ <para>
238​ Another critical feature of any remotely operated robotic observatory is weather monitoring. For weather updates, Ekos relies on the selected INDI weather driver to continuously monitor the weather conditions. For simplicity sake, the weather conditions can be summed in three states:
239​ </para>
240​ <orderedlist>
241​ <listitem>
242​ <para>
243​ <emphasis role="bold">Ok</emphasis>: Weather conditions are clear and optimal for imaging.
244​ </para>
245​ </listitem>
246​ <listitem>
247​ <para>
248​ <emphasis role="bold">Warning</emphasis>: Weather conditions are not clear, seeing is subpar, or partially obstructed and not suitable for imaging. Any further imaging process is suspended until the weather improves. Warning weather status does not pose any danger to the observatory equipment so the observatory is kept operational. The exact behavior to take under Warning status can be configured.
249​ </para>
250​ </listitem>
251​ <listitem>
252​ <para>
253​ <emphasis role="bold">Alert</emphasis>: Weather conditions are detrimental to the observatory safety and shutdown must be initiated as soon as possible.
254​ </para>
255​ </listitem>
256​ </orderedlist>
257​ </sect3>
258​
259​ <sect3 id="ekos-scheduler-startup-and-shutdown-scripts">
260​ <title>Startup &amp; Shutdown Scripts</title>
261​ <para>
262​ Due to the uniqueness of each observatory, Ekos enables the user to select startup and shutdown scripts. The scripts take care of any necessary procedures that must take place on startup and shutdown stages. On startup, Ekos executes the startup scripts and only proceeds to the remainder of the startup procedure (unpark dome/unpark mount) if the script completes successfully. Conversely, the shutdown procedure begins with parking the mount &amp; dome before executing the shutdown script as the final procedure.
263​ </para>
264​ <para>
265​ Startup and shutdown scripts can be written any language that can be executed on the local machine. It must return 0 to report success, any other exist value is considered an error indicator. The script's standard output is also directed to Ekos logger window. The following is an sample demo startup script in Python:
266​ </para>
267​ <programlisting language="python">
268​#!/usr/bin/env python
269​# -*- coding: utf-8 -*-
270​
271​import os
272​import time
273​import sys
274​
275​print "Turning on observatory equipment..."
276​sys.stdout.flush()
277​
278​time.sleep(5)
279​
280​print "Checking safety switches..."
281​sys.stdout.flush()
282​
283​time.sleep(5)
284​
285​print "All systems are GO"
286​sys.stdout.flush()
287​
288​exit(0)
289​ </programlisting>
290​ <para>
291​ The startup and shutdown scripts must be <emphasis>executable</emphasis> in order for Ekos to invoke them (&eg; use <userinput>chmod +x startup_script.py</userinput> to mark the script as executable). Ekos Scheduler enables truly simple robotic operation without the need of any human intervention in any step of the process. Without human presence, it becomes increasingly critical to gracefully recover from failures in any stage of the observation run. Using &plasma; notifications, the user can configure audible alarms and email notifications for the various events in the scheduler.
292​ </para>
293​ </sect3>
294​
295​ <sect3 id="ekos-scheduler-mosaic-wizard">
296​ <title>Mosaic Wizard</title>
297​ <screenshot>
298​ <screeninfo>
299​ Mosaic Wizard
300​ </screeninfo>
301​ <mediaobject>
302​ <imageobject>
303​ <imagedata fileref="mosaic_wizard.png" format="PNG"/>
304​ </imageobject>
305​ <textobject>
306​ <phrase>Mosaic Wizard</phrase>
307​ </textobject>
308​ </mediaobject>
309​ </screenshot>
310​ <para>
311​ Hubble-like super wide field images of <ulink url="http://billsnyderastrophotography.com/?page_id=3275">galaxies</ulink> and nebulae are truly awe-inspiring, and while it takes great skills to obtain such images and process them; many notable names in the field of astrophotography employ gear that is not <emphasis>vastly</emphasis> different from yours or mine. I emphasize <emphasis>vastly</emphasis> because some do indeed have impressive equipment and dedicated observatories worth tens of the thousands of dollars. Nevertheless, many amateurs can obtain stellar wide-field images by combining smaller images into a single grand mosaic.
312​ </para>
313​ <para>
314​ We are often limited by our camera+telescope Field of View (FOV). By increasing FOV by means of a focal reducer or a shorter tube, we gain a larger sky coverage at the expense of spatial resolution. At the same time, many attractive wide-field targets span multiple FOVs across the sky. Without any changes to your astrophotography gear, it is possible to create a super mosaic image <emphasis>stitched</emphasis> together from several smaller images. There are two major steps to accomplish a super mosaic image:
315​ </para>
316​ <orderedlist>
317​ <listitem>
318​ <para>
319​ Capture multiple images spanning the target with some overlap between images. The overlap is necessary to enable the processing software from aligning and joining the sub-images.
320​ </para>
321​ </listitem>
322​ <listitem>
323​ <para>
324​ Process the images and <emphasis>stitch</emphasis> them into a super mosaic image.
325​ </para>
326​ </listitem>
327​ </orderedlist>
328​ <para>
329​ The 2nd step is handled by image processing applications such as <ulink url="https://pixinsight.com">PixInsight</ulink>, among others, and will not be the topic of discussion here. The first step can be accomplished in Ekos Scheduler where it creates a mosaic suitable for your equipment and in accordance with the desired field of view. Not only Ekos creates the mosaic panels for your target, but it also constructs the corresponding observatory jobs required to capture all the images. This greatly facilitates the logistics of capturing many images with different filters and calibration frames across a wide area of the sky.
330​ </para>
331​ <para>
332​ Before starting the <guilabel>Mosaic Job Creator</guilabel> in Ekos Scheduler, you need to select a target and a sequence file. The Sequence File contains all the information necessary to capture an image including exposure time, filters, temperature setting, &etc; Start the Mosaic Job Creator by clicking on the icon next to the <guibutton>Find</guibutton> button in Ekos Module.
333​ </para>
334​ <para>
335​ On first use, you need to enter your equipment settings including your telescope focal length in addition to camera's width, height, and pixel dimensions. Finally, you need to enter the rotation of the camera with respect to north or the position angle. If you don't know this value, start Ekos and slew to your desired target then use the Align module to solve the image and obtain the position angle.
336​ </para>
337​ <para>
338​ Next, enter the desired number of horizontal and vertical panels (&eg; 2x2, 3x3, &etc;) and then click <guibutton>Update</guibutton>. The target FOV shall be calculated given the number of panels and your camera's FOV and the mosaic overlap shall be displayed. By default, the percentage of the overlap among images is 5%, but you can change this value to your desired value. You can also move the complete mosaic structure around to fine tune the position of the mosaic panels. When satisfied, click <guibutton>Create Jobs</guibutton> and Ekos shall create an observation job and a corresponding customized sequence file for each panel. All the jobs shall be saved to an Ekos Scheduler List (<filename role="extension">.esl</filename>) file that you can load on any suitable observing night and it will pick off where you left. Before starting the <guilabel>Mosaic Job Creator</guilabel>, check that all the observation job conditions, constraints, and startup/shutdown procedures are as per your requirements since these settings shall be copied to all the jobs generated by the Mosaic tool.
339​ </para>
340​ <para>
341​ With Ekos Scheduler, multi-night imaging is greatly facilitated and creating super mosaics has never been so easy.
342​ </para>
343​ </sect3>
344​</sect2>