Differential D14288 Diff 40180 plugins/clazy/tests/test_clazyjob.cpp

Changeset View

Standalone View

View Options

plugins/clazy/tests/test_clazyjob.cpp

  • This file was added.
1​/* This file is part of KDevelop
2​
3​ Copyright 2018 Anton Anikin <anton@anikin.xyz>
4​
5​ This program is free software; you can redistribute it and/or
6​ modify it under the terms of the GNU General Public
7​ License as published by the Free Software Foundation; either
8​ version 2 of the License, or (at your option) any later version.
9​
10​ This program is distributed in the hope that it will be useful,
11​ but WITHOUT ANY WARRANTY; without even the implied warranty of
12​ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
13​ General Public License for more details.
14​
15​ You should have received a copy of the GNU General Public License
16​ along with this program; see the file COPYING. If not, write to
17​ the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
18​ Boston, MA 02110-1301, USA.
19​*/
20​
21​#include "test_clazyjob.h"
22​
23​#include "job.h"
24​
25​#include <tests/autotestshell.h>
26​#include <tests/testcore.h>
27​#include <language/editor/documentrange.h>
28​
29​#include <QTest>
30​
kossebauUnsubmitted
Done

Please remove module prefix QtTest

kossebau: Please remove module prefix QtTest
31​using namespace KDevelop;
32​using namespace Clazy;
33​
34​class JobTester : public Job
35​{
36​ Q_OBJECT
37​
38​public:
39​ JobTester()
40​ {
41​ connect(this, &JobTester::problemsDetected,
42​ this, [this](const QVector<KDevelop::IProblem::Ptr>& problems) {
43​ m_problems += problems;
44​ });
45​
46​ connect(this, &JobTester::infoMessage,
47​ this, [this](KJob*, const QString& name) {
48​ m_started += name;
49​ });
50​ }
51​
52​ ~JobTester() override = default;
53​
54​ using Job::postProcessStdout;
55​ using Job::postProcessStderr;
56​
57​ const QVector<KDevelop::IProblem::Ptr>& problems() const
58​ {
59​ return m_problems;
60​ }
61​
62​ const QList<QString>& started() const
63​ {
64​ return m_started;
65​ }
66​
67​ void setTotalCount(int totalCount)
68​ {
69​ m_totalCount = totalCount;
70​ }
71​
72​ int finishedCount() const
73​ {
74​ return m_finishedCount;
75​ }
76​
77​private:
78​ QVector<KDevelop::IProblem::Ptr> m_problems;
79​ QList<QString> m_started;
80​};
81​
82​void TestClazyJob::initTestCase()
83​{
pinoUnsubmitted
Done

Here, make sure to QSKIP the whole test if clazy is not installed.

pino: Here, make sure to `QSKIP` the whole test if clazy is not installed.
antonanikinAuthorUnsubmitted
Done

Clazy is runtime dependency, so I think we should always run the test.

antonanikin: Clazy is runtime dependency, so I think we should always run the test.
kossebauUnsubmitted
Done

A failing test though means something is broken in the code/logic. FAIL sends a wrong message here IMHO.
+1 for QSKIP if runtime dep is not found.

kossebau: A failing test though means something is broken in the code/logic. FAIL sends a wrong message…
pinoUnsubmitted
Done

Exactly, it is a runtime dependency: if I build kdevelop and run the test suite with no clazy installed, this test fails. As Friedrich said, a failure means something went wrong, not that an optional tool was not found.

pino: Exactly, it is a runtime dependency: if I build kdevelop and run the test suite with no clazy…
antonanikinAuthorUnsubmitted
Not Done

Sorry for not the exact formulation - clazy-standalone is runtime dependency for the plugin (real code analysis). This test will works without this executable - here we test only output and error parsing without executable calls. So should we skip the test when clazy-standalone is missing?

antonanikin: Sorry for not the exact formulation - clazy-standalone is runtime dependency for the plugin…
pinoUnsubmitted
Not Done

Clazy::Job creates a makefile that itself invokes clazy-standalone. Also, the outputs that are checked for (e.g. stderrOutput1 & friends) are printed by clazy-standalone, aren't they?

pino: Clazy::Job creates a makefile that itself invokes clazy-standalone. Also, the outputs that are…
antonanikinAuthorUnsubmitted
Not Done

Clazy::Job creates a makefile only when it created with "right" constructor. JobTester uses "fake" empty one and can't be run. In our test we call only postProcessStdout() and postProcessStderr() methods - so this test is only "parser test". Such approach also used by Cppcheck plugin - we tests only parser without execution of any external tools/libs.

stderrOutput1 & friends are taken form clazy-standalone output for some real code and then adapted for our tests (filename, line and column numbers are changed).

antonanikin: `Clazy::Job` creates a makefile only when it created with "right" constructor. `JobTester`…
84​ AutoTestShell::init({"kdevclazy"});
85​ TestCore::initialize(Core::NoUi);
86​}
87​
88​void TestClazyJob::cleanupTestCase()
89​{
90​ TestCore::shutdown();
91​}
92​
93​void TestClazyJob::testJob()
94​{
95​ JobTester jobTester;
96​
97​ // test progress parsing =======================================================================
98​
99​ static const QStringList stdoutOutput1 = {
100​ QStringLiteral("Clazy check started for source2.cpp"),
101​ QStringLiteral("Clazy check started for source1.cpp")
102​ };
103​
104​ static const QStringList stdoutOutput2 = {
105​ QStringLiteral("Clazy check finished for source2.cpp"),
106​ QStringLiteral("Clazy check started for source3.cpp"),
107​ QStringLiteral("Clazy check started for source4.cpp")
108​ };
109​
110​ static const QStringList stdoutOutput3 = {
111​ QStringLiteral("Clazy check finished for source1.cpp"),
112​ QStringLiteral("Clazy check finished for source4.cpp")
113​ };
114​
115​ static const QStringList stdoutOutput4 = {
116​ QStringLiteral("Clazy check finished for source3.cpp"),
117​ };
118​
119​ jobTester.setTotalCount(4);
120​
121​ jobTester.postProcessStdout(stdoutOutput1);
122​ QCOMPARE(jobTester.started().size(), 2);
123​ QCOMPARE(jobTester.started().at(0), QStringLiteral("source2.cpp"));
124​ QCOMPARE(jobTester.started().at(1), QStringLiteral("source1.cpp"));
125​ QCOMPARE(jobTester.finishedCount(), 0);
126​ QCOMPARE(jobTester.percent(), 0);
127​
128​ jobTester.postProcessStdout(stdoutOutput2);
129​ QCOMPARE(jobTester.started().size(), 4);
130​ QCOMPARE(jobTester.started().at(2), QStringLiteral("source3.cpp"));
131​ QCOMPARE(jobTester.started().at(3), QStringLiteral("source4.cpp"));
132​ QCOMPARE(jobTester.finishedCount(), 1);
133​ QCOMPARE(jobTester.percent(), 25);
134​
135​ jobTester.postProcessStdout(stdoutOutput3);
136​ QCOMPARE(jobTester.started().size(), 4);
137​ QCOMPARE(jobTester.finishedCount(), 3);
138​ QCOMPARE(jobTester.percent(), 75);
139​
140​ jobTester.postProcessStdout(stdoutOutput4);
141​ QCOMPARE(jobTester.started().size(), 4);
142​ QCOMPARE(jobTester.finishedCount(), 4);
143​ QCOMPARE(jobTester.percent(), 100);
144​
145​ QCOMPARE(jobTester.started().at(0), QStringLiteral("source2.cpp"));
146​ QCOMPARE(jobTester.started().at(1), QStringLiteral("source1.cpp"));
147​ QCOMPARE(jobTester.started().at(2), QStringLiteral("source3.cpp"));
148​ QCOMPARE(jobTester.started().at(3), QStringLiteral("source4.cpp"));
149​
150​ // test errors parsing =========================================================================
151​
152​ static const QStringList stderrOutput1 = {
153​ QStringLiteral("source2.cpp:13:10: warning: unused variable 'check' [-Wunused-variable]"),
154​ QStringLiteral(" auto check = db.checks()[\"returning-void-expression\"];")
155​ };
156​
157​ static const QStringList stderrOutput2 = {
158​ QStringLiteral("source3.cpp:248:21: warning: Don't call QList::first() on temporary [-Wclazy-detaching-temporary]"),
159​ QStringLiteral(" auto item = pContext->items().first();"),
160​ QStringLiteral(" ^"),
161​ QStringLiteral("1 warning generated.")
162​ };
163​
164​ static const QStringList stderrOutput3 = {
165​ QStringLiteral("source4.cpp:47:9: warning: unused QString [-Wclazy-unused-non-trivial-variable]"),
166​ QStringLiteral(" auto test = QString(\"%1 : %2\").arg(\"a\").arg(\"b\");"),
167​ QStringLiteral(" ^"),
168​ QStringLiteral("source4.cpp:47:47: warning: Use multi-arg instead [-Wclazy-qstring-arg]"),
169​ QStringLiteral(" auto test = QString(\"%1 : %2\").arg(\"a\").arg(\"b\");"),
170​ QStringLiteral(" ^"),
171​ QStringLiteral("2 warnings generated.")
172​ };
173​
174​ jobTester.postProcessStderr(stderrOutput1);
175​ QCOMPARE(jobTester.problems().size(), 0);
176​
177​ jobTester.postProcessStderr(stderrOutput2);
178​ QCOMPARE(jobTester.problems().size(), 1);
179​
180​ jobTester.postProcessStderr(stderrOutput3);
181​ QCOMPARE(jobTester.problems().size(), 3);
182​
183​ // test common values
184​ auto problems = jobTester.problems();
185​ foreach (auto problem, problems) {
186​ QCOMPARE(problem->severity(), KDevelop::IProblem::Warning);
187​ QCOMPARE(problem->source(), KDevelop::IProblem::Plugin);
188​ }
189​
190​ // test problem description
191​ QCOMPARE(problems[0]->description(), QStringLiteral("Don't call QList::first() on temporary"));
192​ QCOMPARE(problems[1]->description(), QStringLiteral("unused QString"));
193​ QCOMPARE(problems[2]->description(), QStringLiteral("Use multi-arg instead"));
194​
195​ // test problem location (file)
196​ QCOMPARE(problems[0]->finalLocation().document.str(), QStringLiteral("source3.cpp"));
197​ QCOMPARE(problems[1]->finalLocation().document.str(), QStringLiteral("source4.cpp"));
198​ QCOMPARE(problems[2]->finalLocation().document.str(), QStringLiteral("source4.cpp"));
199​
200​ // test problem location (line)
201​ QCOMPARE(problems[0]->finalLocation().start().line(), 247);
202​ QCOMPARE(problems[1]->finalLocation().start().line(), 46);
203​ QCOMPARE(problems[2]->finalLocation().start().line(), 46);
204​
205​ // test problem location (column)
206​ QCOMPARE(problems[0]->finalLocation().start().column(), 20);
207​ QCOMPARE(problems[1]->finalLocation().start().column(), 8);
208​ QCOMPARE(problems[2]->finalLocation().start().column(), 46);
209​}
210​
211​QTEST_GUILESS_MAIN(TestClazyJob)
212​
213​#include "test_clazyjob.moc"