Oh I totally forgot about this one. Actually, it's interesting in the sense it can be used to ensure a mean ADU over the frame. That is, calculate the ideal exposure duration. But there are things to fix, as seen in the comments. I keep this in my list and I'll have a look just because it's cool.

Good spot, I had missed this. But wouldn't it have been clearer to std::erase the result of each std::remove_if? This would have avoided the num_remaining gymnastics.

• Fix log crash when scaling SimClock without current job.
• Assert prerequisite of valid current job when testing whether Scheduler should sleep or not.
• Update altitude each time Scheduler timer elapses while running.
• Fix preemptive shutdown warning when preemptive shutdown is enabled.
• Adjust altitude-related tooltip in queue table.
• Fix jobs being reset when Scheduler wakes up.

I think I got the thing solved. I'm running further tests and I will update this differential.

OK so as far as I have investigated this issue doesn't seem to be a regression from D22446: when Scheduler wakes up, Scheduler::wakeUpScheduler() calls Scheduler::start(), which calls Scheduler::startJobEvaluation(), which resets all jobs to JOB_IDLE.
Scheduler::startJobEvaluation() is supposed to be called from the UI when the end-user really presses the play button.
Here state is SCHEDULER_IDLE, and the wake up procedure doesn't seem to care whether Scheduler is actually more or less already running.
That needs a bit more time to check.

Proceeded with a test with two schedule jobs, one that is able to finish inside one night, and one that cannot.
First job runs after dusk, and manages to finish before dawn, and is marked Completed.
Second job starts to run, is preempted by dawn, and is marked Scheduled for the next night while the first remains Completed.
So far so good.
But when Scheduler wakes up on next dusk, the first job is reset to Scheduled and is run again.

Regression crash found when SimClock is scaled while no job is current.

Still in progress, so far so good.
This morning I tested a very simple Simulator observation schedule using Deneb with default constrains, and a SimClock interval of 5 minutes.
Deneb starts after dusk, proceeds to focus and align, is cancelled when dawn is there and is rescheduled to the next night.
That situation repeats indefinitely as if a multi-night schedule.
It would be quite important to use pre-emptive shutdown here, as the Mount Simulator probably wrapped cables up multiple times during my test... (I spotted a wrong warning there, I'll fix that)

Not tested yet due to weather. But changes seem OK to me from code review.

I have a small request : when phd2 is not started and Ekos starts, Ekos will keep the guider module disconnected as expected. But when Ekos requests the guider module to start guiding, I think no attempt to connect again is made. Could this be added in? This would improve robustness. Well that can be another diff of course.

Thanks a million for this Robert. I'm immediately feeding this diff to the observatory build chain for the next clear night.

Actually the use case I stumbled upon was:

• launch Ekos in the afternoon
• change the simclock to see how the sky is at 22:00
• configure schedule with objects available at that time
• start scheduler, expecting it to fall asleep until 22:00

After playing with this diff, I think I will add a warning pop-up when the simulation clock is too far from the system clock. That would be a timed pop-up defaulting to start the schedule. But this pop-up could also reset the simulation time instead. If we go for full-dbus, there should be a way for the remote user to intercept such requests. Opinions?

Changed all Ekos logs to use the simulation time as timestamp.

D24151 should help us debugging those situations: it allows to change the simulation step in KStars and have Scheduler update the delay it waits sleeping when that happens.
Changing the simulation scale also helps bringing restriction events in faster.

I can update that differential to use the SimClock time in all Ekos logs if you want.

Rebased.

About restriction, I remember I used a hack to use the simulation time instead of the system time back in 2017. That might be an interesting feature to put back in.

I didn't note the log was strange, I'll double check. It may be related to the change I made on dawn and dusk offsets, which was merged this week.
About restriction, I remember I used a hack to use the simulation time instead of the system time back in 2017. That might be an interesting feature to put back in.

Additional issue found by @jpaana. I'll check how to fix this one, it's a bit more tricky:

So we try to set jobs to COMPLETED based on frame count, we make sure we do not touch the frame count except when receiving a frame or recounting storage frames, and we do that state change when starting evaluation only? In other words, we never complete jobs during execution? The problem with states is that we must not jitter between values needlessly. Do you have a state diagram in head?

Hm, quite complicated. Has this ever worked?

Illustration for the first regression.

Oh no. You mean I've been tweaking the UI to death just because I thought you meant 800x600 and not 800x800? That's embarrassing for the hairs I've been pulling out for that diff...
Could you simply tweak as an additional differential the minimum size block at the very beginning of manager.ui to relax my restriction to 768 instead of 568?

Yes, given my conditions, make scheduler complete a job, then abort a second one because of a guide failure. Have the second job re-scheduled far away enough that scheduler falls asleep. Let scheduler wake up to process jobs again. When scheduler wakes up, it re-evaluates jobs, and the completed job is incorrectly reset for execution. Using preemptive shutdown and dawn and dusk offsets could ease that test by reducing the amount of time to wait.

I didn't touch the manager, except for adding the dimension you mentioned and that was not there in the first place. As you can see, the manager is unable to achieve 800x600 without overlapping gadgets as it is now. I made sure Scheduler was not preventing the dimension from being reduced to the required size. The manager issue must be tackled by another diff.

The labels are simple drawText calls. There could be a first loop consolidating a list of rectangles bounding labels, then a second pushing labels away from each other but keeping them close to their star marker. But the added value is not high enough for me to implement this now. It'd more important to be able to tweak SEP parameters to detect stars more efficiently I think.

Updated tabbed UI with clearer groups.

Makes sense?

Awesome job, Eric!
I've tested it with simulators and everything looks good. Having the HFR-values displayed makes it much more transparent what the focusing algorithm does.

Thanks. Along with the area filter, it's really more stable than the single-star focus procedure in my opinion (with my setup in fact).

1. Advanced tab .. maybe we should rename this to "Parameters"?

That would make "Settings" and "Parameters". Maybe I should replace "Settings" with "Basic" if I move some stuff in from "Advanced"?
I'll try to move stuff around. I would expect for instance SEP parameters to get a new tab.

Oh, and I removed years from dates to compact the view even more.

Dawn and dusk offsets tested over two nights, work as expected (this change is actually far simpler than the UI stuff...).

Also slightly changed the Option dialog creation so that the dialog is effectively prepared before modules connect signals to it.

Manager dimensions restricted to 796x568, to leave room for window manager decoration and remain under 800x600.
Rehauled the Scheduler UI completely, removed ALL size specificities, and had the layout manager do its job on its own for the entire UI.
Qt layouts are nice, but one need to trust logic and hierarchy, not Qt Designer views.

OK, understood. But why did it happen that no frame has been captured? Was it due to the case that as soon as it would have been there turn to be started, they already hit their altitude or twilight limit, right?

Yes, the first job started, proceeded to focus, but in the meantime altitude got under 15°, at 14.6°, and the job aborted (completed).
The second job started at that moment, but immediately saw that its altitude was lower than 15° (I think), and aborted too (completed).

Yeah, the issue I haven't mentioned is that those jobs are marked completed without any frame captured, so the behavior is not correct. I believe that setting those as ABORTED would have been OK : in the case of my mosaic, there's no problem retrying the job at the end of the queue, or "immediately", here rescheduled at a valid time.

This night I had a 25-job mosaic run. All jobs were constrained with altitude 15° and twilight.
In the morning, job #11 failed to find a guide star after 2 runs and was aborted (OK).
When job #12 crossed the altitude restriction at 15° while focusing, it was incorrectly set to completed instead of aborted (KO).
When job #13 crossed the altitude restriction at 14.3° before starting, it was incorrectly set to completed instead of aborted (KO).
Further jobs were kept as scheduled, and observatory fell asleep as expected (OK).

Focus UI is now smaller than 800x600 when I restrict the manager to that size. I'll tackle the rest of the modules in another diff.

Rendered HFR only if it can be fully displayed (fixes clipped text).
Used an autovar in the enumeration of star centers.

Adjust focus UI again.

I quickly checked on an old version on my lubuntu, Ekos is roughly 835x750 when minimal, because of Capture I think. Thanks for these insights.
By the way, wasn't the relative profile curve on a tab in the past?

I'll share a screenshot in my environment too. In any case, I stand my point about layout constraints, I will make sure the minimal size fits the required dimensions. What is that tool you used on the left?

Procedure tested this night, works smoothly again. I'm fixing the UI dimensions, but I'm not sure it currently fits in 800x600 with the icon tab and the bottom log?

Ah you mean there is a missing restriction in the manager? I'll double check ok. Thanks for the connection example, I'll try to use that!

We need applyConfig to trigger updates in modules following a change of settings. For instance, altering lead time in the options requires a re-evaluation of the schedule. Granted, there are not so many cases for now because most procedures use options when they execute, without preview. Scheduler prepares execution and present it to the end-user for verification as a job list. Thus while the evaluation would be done again when starting the observation, and options taken into account at that moment, scheduler needs to update its presentation when options are applied to retain coherence.

My opinion on this is that observatory setups are too diverse to use a set predefined durations. Those durations need to be at least customizable via options. We thus get a whole new set of options, incidentally one or two per Ekos module.
Based on this observation, it means the estimation of procedure duration needs to rely on each module instead of being computed by scheduler. By extension, each module has to be responsible for the calculation of those duration, based on the actual work it has to do.
Therefore, I suggest we review that estimation activity when each Ekos module is assigned a blob of settings, as I presented on the forum.
"Hi Ekos module, here's what you need to do!" "- Yeah, that will probably take me <insert double> hours!"
Makes sense?

Excellent, thanks for this change!

I'm marking this OK, but we'll have to return to it. I'm nearly done with my other activity, temporarily, and hope to return soon to Scheduler.

See my comments. This is defintely a better behavior, but I'm still not happy with JOB_ERROR :)

Owww I totally forgot about this one. Not that I would have been able to test much that said. Let me try to review this tomorrow.

I'd suggest using std::fmod.

I agree but did not test. I was wondering what the calibration stage was referring to? Is it the same as the calibration for flats?

Sorry I need more time to come back on this. The end of week is hectic.

Please see my comments. I'm on with the idea :)
Have you seen my post on the forum about the future of scheduler? We need to talk about this at some point.

I didn't test yet, but this seems just great. You're addressing real problems there. There are probably a few Bugzilla issues that could get resolved with this.

That's a real bug actually: we are in the period where dusk and dawn are on the same day! So observation time is later than both dawn and dusk...

Testing with a random target with only tracking got me an assertion abort on the dark sky score in the first 30 seconds I manipulated... What regression is this... Twilight isn't functional anymore?

Oh don't get me wrong, I completely agree the current state is bugged, and I could reproduce the issue with the pause button, no issue there. So this diff is obviously important! I should have made this clearer in my post, sorry.

First I need to say I haven't built your differential, so my comment is from code and explanations only. So I'm not sure I understand the diff properly.

Sorry for the delay.

Would it be possible to state a sentence describing the use case for that change? Like "I want to...", eventually multiple sentences. I don't disagree obviously, but this would help progressively gathering a list of specifications.

I was minutes from saying OK :)

So please rebase your changes and eventually issue a preliminary differential to make observatory_work live.

Jasem created the branch "observatory_work" for this activity.

This is neat. Feature branch for the time being indeed.

Do we create a feature branch for this, or do we use the trunk?

No more problem with reordering of jobs when the scheduler is running. The behavior is now coherent with the Capture module.
However, the end-user now has the same feedback when clicking (job characteristics are displayed) and double-clicking (job characteristics are edited).
But that's a better behavior I think (and it matches with the draft UI I'm preparing).

Oh sorry this requires my review. I'll have a look this morning.

I think we should merge this. I'll then separate scheduler.cpp into several file to try to clarify what functions are for. I'll also separate the interface from the implementation, so we don't have too many private stuff cluttering.

Rebased to 20190430 state.

I'll double check this as soon as I can. Sorry for the delay.

I'm sorry my availability is really low these days. Hopefully "better" in the coming w16 as I'll be in holidays.
When a scheduler job aborts, it does not change the completed frame count. So probably an in-sequence focus failing might do the job?

It's unclear, but maybe, yes. Could I offer my own implementation on the two fixes that are in this differential? I'd like to first fix the FindNextJob issue, then in another diff the frame counting via messages from the capture module.

That's a good idea, but weeell I have two disagreements : first this is integer calculation and you probably need to reorder your operators, and second, if I understand correctly, you are considering the amount of captures to get equiprobable scattering over all sequence jobs.
I'll nonetheless test this asap.

In D20029#439409, @wreissenberger wrote:

Well, filling the map in any cases will not help, because the intention of not ticking "Remember progress" is exactly, that we start counting from zero - no matter how many are in the file system.

Situation is clear, thanks, and sorry for missing the point at first.

In D20029#438502, @wreissenberger wrote:

By the way: I found out that the null pointer exception occurs also when there is only one scheduler entry. So I think it's quite urgent that we at least fix this issue...

I propose we then divide SchedulerJob into SchedulerJobModel and SchedulerJob, deriving from SchedulerJobModel.
We can then move all the accessor stuff away into SchedulerJobModel, and focus on making SchedulerJob more a controller.
We should be able to tweak SchedulerJobModel to comply with the QT model interface perhaps.