[SMS App] Make requestMoreMessages asynchronous, blocking, and caching
ClosedPublic

Authored by sredman on Oct 28 2018, 7:01 AM.

Details

Summary

The most serious change from this patch is to move the asynchronous replying to a request from the app for more messages to a newly-spawned, self-destructing thread. Within that thread, we block until the remote device replies with the requested messages.

All gotten messages are cached in the ConversationDbusInterface, so all future requests are fast and don't hit the remote device.

Test Plan

After applying this diff, the messaging app should show 10 messages every time it is opened

Diff Detail

Repository
R224 KDE Connect
Branch
asynchonous-message-fetch
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 5976
Build 5994: arc lint + arc unit
sredman created this revision.Oct 28 2018, 7:01 AM
Restricted Application added a project: KDE Connect. · View Herald TranscriptOct 28 2018, 7:01 AM
Restricted Application added a subscriber: kdeconnect. · View Herald Transcript
sredman requested review of this revision.Oct 28 2018, 7:01 AM
sredman updated this revision to Diff 44340.Oct 28 2018, 7:13 AM
  • Ignore duplicate messages from being added to model

Looks ok to me

show all messages in the conversation when the conversation is first loaded

What's your future plan wrt this?

Looks ok to me

show all messages in the conversation when the conversation is first loaded

What's your future plan wrt this?

I don't have a good plan yet... This is the main problem I was trying to fix by making the interface return an array

We could make requestConversation block on a condition variable which is unblocked by some call in addMessages, then only send conversationUpdated for new messages and the actually-requested messages. I just barely thought of this, but it sounds like it's worth trying.

Another thing to try is to add request window size to the Android interface like I have always been planning to do. I want to have this anyway so it would not be a waste of effort. Then, we request only and exactly the messages we want, and conversationUpdated only gets called with the messages we asked for.
The reason this plan is less good is because I was hoping to use the window size to pre-fill the cache in ConversationDbusInterface, thus reduce visible delays for the user

If none of that works, we could make ConvresationModel keep track of which messages it has requested and only display those. That is a bit of a hack...

sredman updated this revision to Diff 44359.Oct 28 2018, 4:26 PM
  • Only send conversationUpdated if the conversation was truly updated
  • Add requestConversationWorker
  • Add blocking to wait for reply from remote device

I also got carried away and changed all QString conversationIDs to qint32. It was too much of a mess to un-tangle by the time I was finished... Sorry!

sredman edited the summary of this revision. (Show Details)Oct 28 2018, 4:30 PM
sredman edited the test plan for this revision. (Show Details)
sredman retitled this revision from [SMS App] Make requestMoreMessages asynchronous and caching to [SMS App] Make requestMoreMessages asynchronous, blocking, and caching.
apol added a subscriber: apol.Oct 28 2018, 11:38 PM
apol added inline comments.
plugins/sms/requestconversationworker.h
35

Two things:
Do we still need threads now that we don't have the QEventLoop? I don't think so.
If we did, maybe it would be useful if this class inherited QThread.

plugins/sms/smsplugin.cpp
106

messagesList.reserve(messages.count());

sredman updated this revision to Diff 44393.Oct 28 2018, 11:47 PM
  • Reserve space for messagesList
sredman marked an inline comment as done.Oct 28 2018, 11:48 PM
sredman added inline comments.
plugins/sms/requestconversationworker.h
35

We still need something to solve the same problem as the QEventLoop was trying to solve. That is, we need some way to wait until the remote (Android) device replies with the messages we requested, otherwise we don't have anything to return to the user request and the GUI remains annoyingly empty until they make a second request. If we wait on the main thread, the event loop can't process the incoming packet, so we can never stop waiting

This could be done by inheriting from QThread, but the Qt people are very firm that inheriting from QThread to do a simple task is the wrong way to do it. The way I have done it is the recommended way

sredman updated this revision to Diff 44394.Oct 29 2018, 12:02 AM

Address comments from D15979

  • Simplify value-getting
  • Simplify condition
sredman planned changes to this revision.Nov 7 2018, 3:48 AM

With some time to think about this, I think I can combine some of Aleix's last comment. Rather than having the worker inherit from QThread, and rather than implicitly expecting the caller to know a QThread is required, the worker could have a QThread which it is in charge of creating, using, and destroying. I think this would make things much nicer.

sredman updated this revision to Diff 45339.Nov 12 2018, 3:08 AM
  • Move messy details of handling RequestConversationWorker into constructor
apol added a comment.Nov 12 2018, 3:52 PM

I would be okay with accepting this patch. I wonder why it's better than the previous code though. It seems to be just doing the same but from another thread?

In D16475#358422, @apol wrote:

I would be okay with accepting this patch. I wonder why it's better than the previous code though. It seems to be just doing the same but from another thread?

By previous code, do you mean the earlier version of this patch or the stuff currently on the master branch?

In case of comparing to the earlier version, it is basically the same, but no new threads have been created. The work in the constructor (even after this->moveToThread(..)) is all done on the main thread. Future signals delivered to the Worker object are handled on the new thread.

In case of comparing to the master branch, the new thread is doing the same work, but it also blocks (in ConversationsDbusInterface::updateConversation) while it waits for messages to be delivered from the remote device. The alternatives to blocking are:

  • Return immediately, only using the messages currently cached. On the first try, this will be no messages, but the request will be delivered to the phone and handled. If the user of the SMS app waits until that request is handled and tries to load messages again, there will be cached messages the second try. This is not a very nice user experience.
  • Return immediately, only using the messages currently cached. On the first try, this will be no messages. As the phone replies with its messages, immediately announce them on dbus. The first time the user opens a conversation in the messaging app, every message in the conversation will be shown (after a quick delay). When opening future times, the cache will be populated, so only the requested messages will be shown. This is also kind of ugly.
nicolasfella requested changes to this revision.Nov 28 2018, 10:19 PM

I get some compile errors, apparently you need to explicitly covert int to QString

/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp: In member function 'void ConversationModel::sendReplyToConversation(const QString&)':
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:79:51: error: invalid user-defined conversion from 'qint32' {aka 'int'} to 'const QString&' [-fpermissive]
     m_conversationsInterface->replyToConversation(m_threadId, message);
                                                   ^~~~~~~~~~
In file included from /usr/include/qt/QtCore/qhashfunctions.h:44,
                 from /usr/include/qt/QtCore/qlist.h:47,
                 from /usr/include/qt/QtCore/qvariant.h:45,
                 from /usr/include/qt/QtCore/qabstractitemmodel.h:43,
                 from /usr/include/qt/QtGui/qstandarditemmodel.h:44,
                 from /usr/include/qt/QtGui/QStandardItemModel:1,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:25,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/usr/include/qt/QtCore/qstring.h:693:31: note: candidate is: 'QString::QString(const char*)' <near match>
     inline QT_ASCII_CAST_WARN QString(const char *ch)
                               ^~~~~~~
/usr/include/qt/QtCore/qstring.h:693:31: note:   conversion of argument 1 would be ill-formed:
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:79:51: error: invalid conversion from 'qint32' {aka 'int'} to 'const char*' [-fpermissive]
     m_conversationsInterface->replyToConversation(m_threadId, message);
                                                   ^~~~~~~~~~
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:79:51: error: invalid conversion from 'qint32' {aka 'int'} to 'const char*' [-fpermissive]
In file included from /usr/include/qt/QtCore/qhashfunctions.h:44,
                 from /usr/include/qt/QtCore/qlist.h:47,
                 from /usr/include/qt/QtCore/qvariant.h:45,
                 from /usr/include/qt/QtCore/qabstractitemmodel.h:43,
                 from /usr/include/qt/QtGui/qstandarditemmodel.h:44,
                 from /usr/include/qt/QtGui/QStandardItemModel:1,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:25,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/usr/include/qt/QtCore/qstring.h:693:51: note:   initializing argument 1 of 'QString::QString(const char*)'
     inline QT_ASCII_CAST_WARN QString(const char *ch)
                                       ~~~~~~~~~~~~^~
In file included from /home/nico/workspace/kdeconnect-kde/interfaces/dbusinterfaces.h:39,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:29,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/home/nico/workspace/kdeconnect-kde/interfaces/conversationsinterface.h:45:32: note:   initializing argument 1 of 'QDBusPendingReply<> OrgKdeKdeconnectDeviceConversationsInterface::replyToConversation(const QString&, const QString&)'
     inline QDBusPendingReply<> replyToConversation(const QString &conversationID, const QString &message)
                                ^~~~~~~~~~~~~~~~~~~
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp: In member function 'void ConversationModel::requestMoreMessages(const quint32&)':
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:88:51: error: invalid user-defined conversion from 'qint32' {aka 'int'} to 'const QString&' [-fpermissive]
     m_conversationsInterface->requestConversation(m_threadId, numMessages, numMessages + howMany);
                                                   ^~~~~~~~~~
In file included from /usr/include/qt/QtCore/qhashfunctions.h:44,
                 from /usr/include/qt/QtCore/qlist.h:47,
                 from /usr/include/qt/QtCore/qvariant.h:45,
                 from /usr/include/qt/QtCore/qabstractitemmodel.h:43,
                 from /usr/include/qt/QtGui/qstandarditemmodel.h:44,
                 from /usr/include/qt/QtGui/QStandardItemModel:1,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:25,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/usr/include/qt/QtCore/qstring.h:693:31: note: candidate is: 'QString::QString(const char*)' <near match>
     inline QT_ASCII_CAST_WARN QString(const char *ch)
                               ^~~~~~~
/usr/include/qt/QtCore/qstring.h:693:31: note:   conversion of argument 1 would be ill-formed:
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:88:51: error: invalid conversion from 'qint32' {aka 'int'} to 'const char*' [-fpermissive]
     m_conversationsInterface->requestConversation(m_threadId, numMessages, numMessages + howMany);
                                                   ^~~~~~~~~~
/home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:88:51: error: invalid conversion from 'qint32' {aka 'int'} to 'const char*' [-fpermissive]
In file included from /usr/include/qt/QtCore/qhashfunctions.h:44,
                 from /usr/include/qt/QtCore/qlist.h:47,
                 from /usr/include/qt/QtCore/qvariant.h:45,
                 from /usr/include/qt/QtCore/qabstractitemmodel.h:43,
                 from /usr/include/qt/QtGui/qstandarditemmodel.h:44,
                 from /usr/include/qt/QtGui/QStandardItemModel:1,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:25,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/usr/include/qt/QtCore/qstring.h:693:51: note:   initializing argument 1 of 'QString::QString(const char*)'
     inline QT_ASCII_CAST_WARN QString(const char *ch)
                                       ~~~~~~~~~~~~^~
In file included from /home/nico/workspace/kdeconnect-kde/interfaces/dbusinterfaces.h:39,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.h:29,
                 from /home/nico/workspace/kdeconnect-kde/smsapp/conversationmodel.cpp:22:
/home/nico/workspace/kdeconnect-kde/interfaces/conversationsinterface.h:58:32: note:   initializing argument 1 of 'QDBusPendingReply<> OrgKdeKdeconnectDeviceConversationsInterface::requestConversation(const QString&, int, int)'
     inline QDBusPendingReply<> requestConversation(const QString &conversationID, int start, int end)
This revision now requires changes to proceed.Nov 28 2018, 10:19 PM

I get some compile errors, apparently you need to explicitly covert int to QString

Thanks for looking in to this! It looks like you might need to do a make clean -- The particular error you're getting looks like the dbus interface might not have been re-built compared to the current version on master

The patch is kind of a mess right now because there have been so many changes to master. I will rebase it when I have time

sredman updated this revision to Diff 47401.Dec 12 2018, 12:46 AM

Rebase onto master

sredman updated this revision to Diff 47406.Dec 12 2018, 1:10 AM
  • Rebase on master (again)
albertvaka accepted this revision.Dec 12 2018, 2:06 PM
albertvaka added a subscriber: albertvaka.

I don't like the fact that we block waiting on a packet, but it's better to move this forward anyway. If it becomes a problem we can revise it later.

smsapp/conversationmodel.cpp
54 ↗(On Diff #47406)

!=

95 ↗(On Diff #47406)

!=

124 ↗(On Diff #47406)

!=

sredman updated this revision to Diff 47486.Dec 13 2018, 5:40 AM
  • Clean inequalities
This revision was not accepted when it landed; it landed in state Needs Review.Dec 13 2018, 5:43 AM
This revision was automatically updated to reflect the committed changes.