Rework of UI tests, KStars window and Ekos profile.
ClosedPublic

Authored by TallFurryMan on Jan 29 2020, 8:48 PM.

Details

Summary

This diff picks where the UI tests were left a few years ago.
The implementation is reworked to pass the tests that were then implemented.

The idea is to execute the KStars application, and run all tests asynchronously
from a QTimer::singleShot call. From there, some actions trigger modal dialogs,
which block the click handler used by the test, and require a second layer of
QTimer::singleShot. Some other dialogs are KSMessageBox widgets, which don't
block the clieck handler, so test methods may vary uncomfortably.

Because QTest uses macros that simply "return;" with no code when they assess
failure, it is difficult to call functions to run sub-tests used by other tests
that may fail. Instead, macros are used to factor common functionalities in (we
don't care for code size here).

Test Plan

Build and run "kstars_ui_tests".
There are 7 passing tests, one of which produces an expected warning.
Ekos profile is tested by creating a simple telescope/CCD simulator profile,
saving it, editing it and verifying its contents, and deleting and verifying
it is not present anymore. Nothing new, but works on current trunk.

Diff Detail

Repository
R321 KStars
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
TallFurryMan created this revision.Jan 29 2020, 8:48 PM
Restricted Application added a project: KDE Edu. · View Herald TranscriptJan 29 2020, 8:48 PM
Restricted Application added a subscriber: kde-edu. · View Herald Transcript
TallFurryMan requested review of this revision.Jan 29 2020, 8:48 PM

The way the project was implemented originally is to test the end-user side of manipulations, using UI as much as possible.
Effectively, this is final integration test which could probably be difficult to maintain and automate.
Running tests with a full stack, from star map down to drivers, could also have a long duration, making it less suitable to automation (imagine scheduler tests over three days).

However, with efficient refactoring and adequate preparation (maybe read "a few thousands of code test lines"?), we should be able to implement and test use cases very precisely.
This could be helpful to re-generate user manual videos automatically for each version. Interesting?

This is great! Just this morning I was thinking we should add a project for this years' Google Summer Of Code just to help in streamlining all the testing, especially GUI testing which is dearly needed.

In this diff, quite a few blocks could be refactored: finding gadgets, clicking things, managing dialogs...
Currently, the test runs as fast as it can by using timeouts on states instead of sleeping (except for QTimer::singleShot calls), so when tests all pass, the duration is ~5 seconds on my laptop (most of which is KStars loading).
Fixtures should be integrated in order to test many more situations with a single action flow.
Also, I did not investigate the lack of menu and toolbar in KStars.

mutlaqja accepted this revision.Feb 11 2020, 12:13 PM
This revision is now accepted and ready to land.Feb 11 2020, 12:13 PM

Can you please rebase so I can merge it?

  • Update for rebase.

Getting this:

[ 97%] Building CXX object Tests/kstars_ui/CMakeFiles/kstars_ui_tests.dir/test_ekos.cpp.o
[ 97%] Building CXX object kstars/CMakeFiles/kstars.dir/main.cpp.o
In file included from /home/jasem/Projects/kstars/Tests/kstars_ui/test_ekos.cpp:17:0:
/home/jasem/Projects/kstars/Tests/kstars_ui/kstars_ui_tests.h:21:10: fatal error: test_ekos.h: No such file or directory
#include "test_ekos.h"

^~~~~~~~~~~~~

compilation terminated.
Tests/kstars_ui/CMakeFiles/kstars_ui_tests.dir/build.make:146: recipe for target 'Tests/kstars_ui/CMakeFiles/kstars_ui_tests.dir/test_ekos.cpp.o' failed
make[2]: * [Tests/kstars_ui/CMakeFiles/kstars_ui_tests.dir/test_ekos.cpp.o] Error 1
make[2]:
* Waiting for unfinished jobs....

  • Fix rogue include.
This revision was automatically updated to reflect the committed changes.