Restructure tests, add simulator tests, spot expected failures.
ClosedPublic

Authored by TallFurryMan on Feb 26 2020, 1:17 PM.

Details

Summary

This differential restructures test execution to allow use of regular test classes for UI tests of KStars.

A few tests are reworked with additional features:

  • Geographical location is set to Greenwich and verified against city database
  • Date and time of simulation is set to 2020/01/01 11:00pm and verified
  • Clock is set to not run and verified
  • Basic test and warn test are removed
  • Fixture data is created for Messier objects higher than the horizon at current simulation time
  • Slew test is created to point at each fixture in sequence using the Simulators profile
  • Various dialog appearing for questions are managed by tests

Tests reveal that:

  • There is no definitive UI feedback to end the connection procedure of devices
  • There is limited homogeneity on dialogs presented to the end-user (QDialog, KMessageBox, KNotification)
  • Ekos unconditionally resumes the simulation clock on startup, and resets its date and time to current system time
Test Plan

Execute kstars_ui_test, either from the command line or from the QtCreator project run or from the QtCreator test run (check timeout in options)
The test can be executed in a virtual framebuffer X server using "xfvbrun".

Diff Detail

Repository
R321 KStars
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
TallFurryMan created this revision.Feb 26 2020, 1:17 PM
Restricted Application added a project: KDE Edu. · View Herald TranscriptFeb 26 2020, 1:17 PM
Restricted Application added a subscriber: kde-edu. · View Herald Transcript
TallFurryMan requested review of this revision.Feb 26 2020, 1:17 PM

Please don't test this on your own setup, because it may change your userdb (it doesn't right now, but will certainly).
I need a way to use another database, so that options are all default.

Tests/kstars_ui/kstars_ui_tests.cpp
76

This was cheating: I used the city database, while I should have used arbitrary latitude and longitude, eventually verifying this was Greenwhich.

96

There's apparently no way to *not* use the system properties when initializing KStars, due to KStarsDateTime management of UTC/localtime. This should be tested in a unitary test elsewhere.

99

I always end up with a comparison between xxx.999ms and xxx.000ms. Rounding issues somewhere on the QDateTime/KStarsDataTime chain.

127

This is currently useless.

157

I had to use this method in order to use multiple test classes that execute in parallel with the GUI.

165

This value hard-codes a limit to the duration of the test. I'll probably remove it and rely on another method to detect the end of the asynchronous tests. Normally QTest::qExec calls should not break out of the singleShot() call above, so this is more a safeguard against lock-up. But that safeguard would be better done with a timeout shell wrapper.

Tests/kstars_ui/test_ekos.cpp
67

This one might seem unrelevant, but it actually reveals that Ekos is resetting and resuming the Simulation Clock of KStars. It results in a relatively long update of all sky objects of KStars. As I have downloaded many Minor Planet elements, this can take around 30 seconds on my Core i7, which is far from negligible.

This has the side effect of breaking all sky object coordinates fixtures, which are computed from initial conditions below in TestEkosSimulator.

Tests/kstars_ui/test_ekos_simulator.cpp
64

I would expect this to use the initial conditions in KStarsUiTest, but actually Ekos resets the Simulation Clock.
What is weird is that this test fixture generation is executed *after* the first Ekos startup, so lt() should provide the updated local time.
Still, many sky objects are placed incorrectly in the sky and end below the horizon, even though altitude is tested.
So, this must be bugged. But the test manages that with an expected failure.

112

First, I tried to use the UI to slew to specific RA/DEC.
It appears the Mount Control box is the only one to provide a way to enter numeric values.
However, QML just doesn't provide you a gadget if that gadget miss the "objectName" property.

Also that reveals that apart from the Mount Control box, only the Scheduler is able to slew with numerical values.
The other way is to right-click on the sky map, which is not very numerical.

129

This block is a contrived way to bypass the notification box produced by the telescope handler...

138

I had a certain number of rounding issues with the Simulator telescope, and ended up testing closeness instead of equality.
I'm not sure this is expected from the Simulator Telescope, but I know this happens with real mounts, so...

Thanks for writing all these tests... can we have a *template* KStars UI test so that prospective users/developers can use in the future as a starting point?

I'm writing that for kstars classes, with empty test plans. I'll see what I can do for UI tests, although that's less easy as the source of a UI test is a use case.

  • Fix formatting.

Thanks in advance for fixing minor issues.

Tests/kstars_ui/test_ekos.h
2

Is it possible to add copyright statement?

53

Missing newline at the end.

Tests/kstars_ui/test_ekos_simulator.cpp
3

2018?

154

Missing newline at the end.

Tests/kstars_ui/test_ekos_simulator.h
2

Is it possible to add copyright statement?

56

Missing newline at the end.

TallFurryMan planned changes to this revision.Feb 27 2020, 9:12 AM
  • Fix copyrights.
TallFurryMan marked 3 inline comments as done.Feb 27 2020, 8:17 PM

Updated. @yurchor there were apparently no issues with newlines at the end of files.

Is this ready to land?

Yes, as it now, it's ok to me.

mutlaqja accepted this revision.Mar 1 2020, 4:04 AM
This revision is now accepted and ready to land.Mar 1 2020, 4:04 AM
This revision was automatically updated to reflect the committed changes.