Differential D14288 Diff 39668 plugins/clazy/tests/test_clazyjob.cpp

Changeset View

Standalone View

View Options

plugins/clazy/tests/test_clazyjob.cpp

  • This file was added.
1​/* This file is part of KDevelop
2​
3​ Copyright 2018 Anton Anikin <anton@anikin.xyz>
4​
5​ This program is free software; you can redistribute it and/or
6​ modify it under the terms of the GNU General Public
7​ License as published by the Free Software Foundation; either
8​ version 2 of the License, or (at your option) any later version.
9​
10​ This program is distributed in the hope that it will be useful,
11​ but WITHOUT ANY WARRANTY; without even the implied warranty of
12​ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
13​ General Public License for more details.
14​
15​ You should have received a copy of the GNU General Public License
16​ along with this program; see the file COPYING. If not, write to
17​ the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
18​ Boston, MA 02110-1301, USA.
19​*/
20​
21​#include "test_clazyjob.h"
22​
23​#include "job.h"
24​
25​#include <tests/autotestshell.h>
26​#include <tests/testcore.h>
27​#include <language/editor/documentrange.h>
28​
29​#include <QtTest/QTest>
kossebauUnsubmitted
Done

Please remove module prefix QtTest

kossebau: Please remove module prefix QtTest
30​
31​using namespace KDevelop;
32​using namespace Clazy;
33​
34​class JobTester : public Job
35​{
36​ Q_OBJECT
37​
38​public:
39​ JobTester()
40​ {
41​ connect(this, &JobTester::problemsDetected,
42​ this, [this](const QVector<KDevelop::IProblem::Ptr>& problems) {
43​ m_problems += problems;
44​ });
45​
46​ connect(this, &JobTester::infoMessage,
47​ this, [this](KJob*, const QString& name) {
48​ m_started += name;
49​ });
50​ }
51​
52​ ~JobTester() override = default;
53​
54​ using Job::postProcessStdout;
55​ using Job::postProcessStderr;
56​
57​ const QVector<KDevelop::IProblem::Ptr>& problems() const
58​ {
59​ return m_problems;
60​ }
61​
62​ const QList<QString>& started() const
63​ {
64​ return m_started;
65​ }
66​
67​ void setTotalCount(int totalCount)
68​ {
69​ m_totalCount = totalCount;
70​ }
71​
72​ int finishedCount() const
73​ {
74​ return m_finishedCount;
75​ }
76​
77​private:
78​ QVector<KDevelop::IProblem::Ptr> m_problems;
79​ QList<QString> m_started;
80​};
81​
82​void TestClazyJob::initTestCase()
83​{
pinoUnsubmitted
Done

Here, make sure to QSKIP the whole test if clazy is not installed.

pino: Here, make sure to `QSKIP` the whole test if clazy is not installed.
antonanikinAuthorUnsubmitted
Done

Clazy is runtime dependency, so I think we should always run the test.

antonanikin: Clazy is runtime dependency, so I think we should always run the test.
kossebauUnsubmitted
Done

A failing test though means something is broken in the code/logic. FAIL sends a wrong message here IMHO.
+1 for QSKIP if runtime dep is not found.

kossebau: A failing test though means something is broken in the code/logic. FAIL sends a wrong message…
pinoUnsubmitted
Done

Exactly, it is a runtime dependency: if I build kdevelop and run the test suite with no clazy installed, this test fails. As Friedrich said, a failure means something went wrong, not that an optional tool was not found.

pino: Exactly, it is a runtime dependency: if I build kdevelop and run the test suite with no clazy…
antonanikinAuthorUnsubmitted
Not Done

Sorry for not the exact formulation - clazy-standalone is runtime dependency for the plugin (real code analysis). This test will works without this executable - here we test only output and error parsing without executable calls. So should we skip the test when clazy-standalone is missing?

antonanikin: Sorry for not the exact formulation - clazy-standalone is runtime dependency for the plugin…
pinoUnsubmitted
Not Done

Clazy::Job creates a makefile that itself invokes clazy-standalone. Also, the outputs that are checked for (e.g. stderrOutput1 & friends) are printed by clazy-standalone, aren't they?

pino: Clazy::Job creates a makefile that itself invokes clazy-standalone. Also, the outputs that are…
antonanikinAuthorUnsubmitted
Not Done

Clazy::Job creates a makefile only when it created with "right" constructor. JobTester uses "fake" empty one and can't be run. In our test we call only postProcessStdout() and postProcessStderr() methods - so this test is only "parser test". Such approach also used by Cppcheck plugin - we tests only parser without execution of any external tools/libs.

stderrOutput1 & friends are taken form clazy-standalone output for some real code and then adapted for our tests (filename, line and column numbers are changed).

antonanikin: `Clazy::Job` creates a makefile only when it created with "right" constructor. `JobTester`…
84​ AutoTestShell::init({"kdevclazy"});
85​ TestCore::initialize(Core::NoUi);
86​}
87​
88​void TestClazyJob::cleanupTestCase()
89​{
90​ TestCore::shutdown();
91​}
92​
93​void TestClazyJob::testJob()
94​{
95​ JobTester jobTester;
96​
97​ // test progress parsing =======================================================================
98​
99​ static const QStringList stdoutOutput1 = {
100​ QStringLiteral("Clazy check started for source2.cpp"),
101​ QStringLiteral("Clazy check started for source1.cpp")
102​ };
103​
104​ static const QStringList stdoutOutput2 = {
105​ QStringLiteral("Clazy check finished for source2.cpp"),
106​ QStringLiteral("Clazy check started for source3.cpp"),
107​ QStringLiteral("Clazy check started for source4.cpp")
108​ };
109​
110​ static const QStringList stdoutOutput3 = {
111​ QStringLiteral("Clazy check finished for source1.cpp"),
112​ QStringLiteral("Clazy check finished for source4.cpp")
113​ };
114​
115​ static const QStringList stdoutOutput4 = {
116​ QStringLiteral("Clazy check finished for source3.cpp"),
117​ };
118​
119​ jobTester.setTotalCount(4);
120​
121​ jobTester.postProcessStdout(stdoutOutput1);
122​ QCOMPARE(jobTester.started().size(), 2);
123​ QCOMPARE(jobTester.finishedCount(), 0);
124​ QCOMPARE(jobTester.percent(), 0);
125​
126​ jobTester.postProcessStdout(stdoutOutput2);
127​ QCOMPARE(jobTester.started().size(), 4);
128​ QCOMPARE(jobTester.finishedCount(), 1);
129​ QCOMPARE(jobTester.percent(), 25);
130​
131​ jobTester.postProcessStdout(stdoutOutput3);
132​ QCOMPARE(jobTester.started().size(), 4);
133​ QCOMPARE(jobTester.finishedCount(), 3);
134​ QCOMPARE(jobTester.percent(), 75);
135​
136​ jobTester.postProcessStdout(stdoutOutput4);
137​ QCOMPARE(jobTester.started().size(), 4);
138​ QCOMPARE(jobTester.finishedCount(), 4);
139​ QCOMPARE(jobTester.percent(), 100);
140​
141​ QCOMPARE(jobTester.started().at(0), QStringLiteral("source2.cpp"));
142​ QCOMPARE(jobTester.started().at(1), QStringLiteral("source1.cpp"));
143​ QCOMPARE(jobTester.started().at(2), QStringLiteral("source3.cpp"));
144​ QCOMPARE(jobTester.started().at(3), QStringLiteral("source4.cpp"));
145​
146​ // test errors parsing =========================================================================
147​
148​ static const QStringList stderrOutput1 = {
149​ QStringLiteral("source2.cpp:13:10: warning: unused variable 'check' [-Wunused-variable]"),
150​ QStringLiteral(" auto check = db.checks()[\"returning-void-expression\"];")
151​ };
152​
153​ static const QStringList stderrOutput2 = {
154​ QStringLiteral("source3.cpp:248:21: warning: Don't call QList::first() on temporary [-Wclazy-detaching-temporary]"),
155​ QStringLiteral(" auto item = pContext->items().first();"),
156​ QStringLiteral(" ^"),
157​ QStringLiteral("1 warning generated.")
158​ };
159​
160​ static const QStringList stderrOutput3 = {
161​ QStringLiteral("source4.cpp:47:9: warning: unused QString [-Wclazy-unused-non-trivial-variable]"),
162​ QStringLiteral(" auto test = QString(\"%1 : %2\").arg(\"a\").arg(\"b\");"),
163​ QStringLiteral(" ^"),
164​ QStringLiteral("source4.cpp:47:47: warning: Use multi-arg instead [-Wclazy-qstring-arg]"),
165​ QStringLiteral(" auto test = QString(\"%1 : %2\").arg(\"a\").arg(\"b\");"),
166​ QStringLiteral(" ^"),
167​ QStringLiteral("2 warnings generated.")
168​ };
169​
170​ jobTester.postProcessStderr(stderrOutput1);
171​ QCOMPARE(jobTester.problems().size(), 0);
172​
173​ jobTester.postProcessStderr(stderrOutput2);
174​ QCOMPARE(jobTester.problems().size(), 1);
175​
176​ jobTester.postProcessStderr(stderrOutput3);
177​ QCOMPARE(jobTester.problems().size(), 3);
178​
179​
180​ // test common values
181​ auto problems = jobTester.problems();
182​ foreach (auto problem, problems) {
183​ QCOMPARE(problem->severity(), KDevelop::IProblem::Warning);
184​ QCOMPARE(problem->source(), KDevelop::IProblem::Plugin);
185​ }
186​
187​ // test problem description
188​ QCOMPARE(problems[0]->description(), QStringLiteral("Don't call QList::first() on temporary"));
189​ QCOMPARE(problems[1]->description(), QStringLiteral("unused QString"));
190​ QCOMPARE(problems[2]->description(), QStringLiteral("Use multi-arg instead"));
191​
192​ // test problem location (file)
193​ QCOMPARE(problems[0]->finalLocation().document.str(), QStringLiteral("source3.cpp"));
194​ QCOMPARE(problems[1]->finalLocation().document.str(), QStringLiteral("source4.cpp"));
195​ QCOMPARE(problems[2]->finalLocation().document.str(), QStringLiteral("source4.cpp"));
196​
197​ // test problem location (line)
198​ QCOMPARE(problems[0]->finalLocation().start().line(), 247);
199​ QCOMPARE(problems[1]->finalLocation().start().line(), 46);
200​ QCOMPARE(problems[2]->finalLocation().start().line(), 46);
201​
202​ // test problem location (column)
203​ QCOMPARE(problems[0]->finalLocation().start().column(), 20);
204​ QCOMPARE(problems[1]->finalLocation().start().column(), 8);
205​ QCOMPARE(problems[2]->finalLocation().start().column(), 46);
206​}
207​
208​QTEST_GUILESS_MAIN(TestClazyJob)
209​
210​#include "test_clazyjob.moc"