WIP slave command behavior assertion system
Needs RevisionPublic

Authored by sitter on Oct 23 2019, 12:44 PM.

Details

Reviewers
dfaure
Summary

This implements a simple expectation system covering slave commands.
At its heart sits the CommandExpectation, which contains the expectations
of a given command. Currently the sole expectation is about signal emission
notably a whitelist of signals that any given command may emit during its
emission. To assist this there's a new Signal enum which is used to
refer to the signals. Signals by default are considered exclusive and
repeatedly emitting the same signal will result in assertion failures.
Multi-emission signals such as data/written/processedsize are of course
excluded from this.
All signal implementation in SlaveBase should call VERIFY_CALL to verify
the validity of the signal call at call-time (notably: can this signal
be called during this Command and may it be called 1 or N times).
After the command finished the pre-existing verifyState function has
been refitted for the new expectation tech.

  • slavebase.h now tries to document expectations WRT signals
  • makes KIO to a Q_NAMESPACE which kinda exposes unintended symbols I guess e.g. staticMetaObject. this does enable us to convert the enum values to strings via QMetaMethod but has no additional purpose beyond that.
  • m_state/States are no more. m_state was solely used to track whether error/finished was called and then assert expectations. this is replaced by m_receivedSignals which tracks any number of signals
  • verifyErrorFinishedNotCalled and verfifyFinishedNotCalled have been merged into verifyCall which is now the one-place-stop to get in-call verification (i.e signals call verifyCall and verifyCall makes sure that signal should even have been called)
  • verifyState has been reworked to be applicable to all expectations

todo:

  • I am not 100% on most of the expectations
  • do we really want Q_NAMESPACE on KIO? we could replace this with enumToString() functions switching over the values manually. seeing as I am lazy I'd rather use the moc generated data though
  • conceivably an empty expectation should mean everything is allowed, currently it's meant to mean nothing is allowed as that is rather more handy to make sure the expectations themselves are in fact correct and complete
  • warning todos
  • stray debugs
Test Plan

casually use all the slaves

Diff Detail

Repository
R241 KIO
Branch
assert-file
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18070
Build 18088: arc lint + arc unit
sitter created this revision.Oct 23 2019, 12:44 PM
Restricted Application added a project: Frameworks. · View Herald TranscriptOct 23 2019, 12:44 PM
Restricted Application added a subscriber: kde-frameworks-devel. · View Herald Transcript
sitter requested review of this revision.Oct 23 2019, 12:44 PM

About the list of unexpected signals in forCommand():

It looks that at least http, ftp, and sftp do call connected() as part of various commands. e.g. all do it during get(). Does that make sense?
connected() refers to openConnection() in its documentation, and openConnection() says the slave is operating in connection-oriented mode when called, so if openConnection() puts it into connection-oriented mode then not having had a call to openConnection() means it shouldn't be operating in connection-oriented mode. I have no additional understanding besides what the documentation tells me and so it would seem to me that connected() shouldn't be called anywhere but openConnection(). At the same time the documentation for openConnection() does have the neat qualifier forced * Opens the connection (forced)

So I guess my questions are :

  • what does connection-oriented mean exactly?
  • what exactly is a forced connection open?
  • how is a forced open different from a casual open?
  • should connected() really be called all over the place?
  • if so, does the client software actually need to do something based on the signal or is it simply a case of "it does no harm, so emitting it unconditionally is easier than not"?

Based on our specific requirements here the expectation class needs some rejiggering to differentiate state violations (e.g. openConnection() neither calling connected() nor error()) from unexpected updates (connected() during get()) from useless updates (listEntry() during stat()).

dfaure requested changes to this revision.Apr 14 2020, 9:56 PM

After much delay.... some replies.... but there are yet more questions in here....

src/core/commands_p.h
27

parse error

src/core/slavebase.cpp
96

why? does another global ctor use it?

It's safe. It's just costly -- happens in each and every process that links to KIO.

Maybe this should be all in NDEBUG anyhow?

128

In order to get progress information when listing very large directories, the idea used to be "emit totalSize(number of files) from listDir()".

Then kdelibs4 commit 23ba25b6505dc7 changed the logic for the batching in KIO and this information is no longer necessary.

I just removed outdated docu from slavebase.h in commit 2261f2045c0a (the kdelibs4 commit removed most of it).

I'm wondering how ListJob can emit percent though, as comments over there say, without knowing about the totalsize. Something seems amiss here.
I can see how batching (based on time) doesn't need to know the total number of items, but progress information surely does...

And in fact AFAICS this information does end up in ListJob, and is used there to calculate progress information.
So I was wrong in today's commit, and that kdelibs4 commit was wrong in removing that docu too.
This is useful, and slaves should emit it.

The porting to KJob turned a simple int into something explicitly called "Bytes" though, so it looks even more like a hack than it did before (see SimpleJobPrivate::slotTotalSize, also called from ListJob).
ListJob should have its own slot, and use Files instead of Bytes.

kio_file should also emit totalSize, 1) for slow network-mounted directories, 2) so that all this can be unittested.

One question, 3 TODOs, this is going to be a long patch to review ;-)

131

Sounds wrong (i.e. but g in kio_desktop), listjob takes care of that.

137

Right. This allows to pick the app based on mimetype, and it will get all the data.

139

right

140

right, on error it won't call statEntry

185

That's internal, doesn't get to SlaveBase.

186

same

187

That's a reply (slave->app). No expectations, since it's the other way around.

188

unused, should be removed in KF6

189

depends on the subcommand, as you documented in slavebase.h

190

fire-and-forget command app->slave, no expectations.

191

handled internally in slavebase.cpp, doesn't get to the SlaveBase subclass

192

Obsolete. There's much cleanup to be done around that stuff in KF6. Unless someone feels like reviving the support for chaining kioslaves using suburls... but IMHO this has all been dead for long.

193

app->slave reply to the slave calling messageBox(). No expectations.

194

also an app->slave reply to the slave asking

195

handled internally, no expectations from kioslaves

196

app->slave reply

844

there's nothing here.... This method should be removed IMHO

src/core/slavebase.h
399

All the documentation changes to slavebase.h are good to go, they could land independently from the rest of this commit...

This revision now requires changes to proceed.Apr 14 2020, 9:56 PM