Refactor objecttreeparser
Closed, ResolvedPublic

Description

OTP actutally is written in good old php3 stlye and mixing logic into creating hmtl. Let's create a OTP that creates a tree of objects, that than can be self create html or anything else.

Details

Differential Revisions
D702: Move root logic to MessagePart.
D701: move html creation at the end of the chain.
D700: Move attachment logic into MessagePart
D699: move icon logic into messagepart.
D582: bit hack to get multiple formateres for one type working.
D581: allow multiple Formatters for one type
D580: move processMailman to process/render
D560: Move writeBodyString to TextMessagePart
D559: move descicion of process/render futher up
D558: move processAlternativeSubtype to process/render
D557: move standardHandling to process/render
D556: move processTextHtmlSubtype to process/render
D555: move processMultiPartMixedSubtype to process/render
D553: start with smime refactor
D552: add & use CertMessagePart
D500: update processTextPlainSubtype to process/render
D499: add TextMessagePart
D474: delete unnessarey includes
D473: move processMultiPartEncryptedSubtype to process/format
D472: OTP: Get rid of keepEncryptions
D436: Remove MessageViewer::BodyPartFormatter (only MessageViewer::Interface::BodyPartFormatter is left now)
D435: Move subOtp headling into MessagePart
D443: Remove includeSignatures
D442: Add more signature tests and cleanup SignedPart
D441: Move processMultiPartSignedSubtype to process/render
D440: Move partMetaData to MessagePart, to make MessagePart be a return value
D439: Add MimeMessagePart
D438: split MessageRfc822BodyPartFormatter into process/format
D437: Move link to attachment to AttachmentMarkBlock.
D407: replace processMessageRfc822Subtype with a propper Formatter&renderer
D406: use Inerface::BodyPartFormatter to process nodes
D405: Extend Interface::BodyPart with otp and processResults
D404: create&use own AttachmentMarkBlock
D395: Split parsing and rendering of tempNode of encrypted/signed parts.
D394: Move Deferred and ProgressBlock to CryptoBlock
D393: move sign to messagepart
D372: All parts, that do decyption use CryptoMessagePart now.
D368: move html creation to MessagePart.
D367: merge sign codepath together
knauss created this task.Sep 9 2015, 11:02 AM
knauss updated the task description. (Show Details)
knauss raised the priority of this task from to Normal.
knauss claimed this task.
knauss added a project: KDE PIM.
knauss added subscribers: knauss, vkrause, mbohlender.

The current implementation is mind, is that we change otp so, that all rendering parts are get outsourced into plugins and return a tree of objects that can rendernd aterwards

knauss moved this task from Backlog to In Progress on the KDE PIM board.Sep 11 2015, 7:42 AM
knauss added a comment.Oct 7 2015, 5:19 PM

Let's start think about the way we want to reach.

Currently it is used like that (this is a synced version of a async process, the idea is that the second rendering is done when it is ready - this makes the example more compact than creating a comple class):

KMime::Message::Ptr msg = <the mail>;
NodeHelper nodeHelper;
EmptySource emptySource;
emptySource.setAllowEncryption(true);
{
    OEventLoop loop;
    ObjectTreeParser otp(&emptySource, &nodeHelper);
    connect(&nodeHelper, SIGNAL(update(MessageViewer::Viewer::UpdateMode)), &loop, quit());
    otp.setAllowAsync(true);
    otp.parseObjectTree(msg.data());
    loop.exec(); //Wait for encryption/signing job to be finished
}

{
    ObjectTreeParser otp(&emptySource, &nodeHelper);
    otp.setAllowAsync(true);
    otp.parseObjectTree(msg.data());
}
  • NodeHelper holds the persistant data, if we have async part in it
  • OTP is only a throw-away parser
  • Source are settings for the rendering part not for the parsing

The new interface should look like following:

KMime::Message::Ptr msg = <the mail>;
NodeHelper nodeHelper;
EmptySource emptySource;
emptySource.setAllowEncryption(true);
{
    OEventLoop loop;
    ObjectTreeParser otp(&nodeHelper);
    connect(&nodeHelper, SIGNAL(update(MessageViewer::Viewer::UpdateMode)), &loop, quit());
    otp.setAllowAsync(true);
    otp.parseObjectTree(msg.data());
    otp.render(&emptySource);
    loop.exec(); // Wait for encryption /signing job to be finished
}

{
    ObjectTreeParser otp(&emptySource);
    otp.setAllowAsync(true);
    otp.parseObjectTree(msg.data());
    otp.render(&emptySource);
}

This works for text and html output, but for qtquick we return a tree of QObjects with QPROPERTIES. This tree is interally anyhow, to refelct the contrete visual represantation of the mail. The render step can than also be moved easily to grantlee.

Parsing step

Internally otp interate over each node and ask BodyPartFormatterFactory to get a BodyPartFormatter for the matching mediaType/subType of the node. BodyPartFormatterFactory is a plugin interface where you can register BodyPartFormatter for mediaType.

  • What about multiple BodyFormatter for the same mediaType? Does that makes senśe?

and runs a BodyFormater accoring to the nodes mediaType (BodyFormatter.process) as result we get a tree of MessageParts, that represent the mail visualy and has nothing to do with the represantation in the mail structure. For example for an encrypted+signed mail with attachments:

->enc block
    -> signed block
         -> phabricator block
         -> signed att1
              -> diff block
        -> signed att2
   -> non signed att1
-> non enc att1

But we also need a second level of plugins, that do not have own mediaType/subType for example Inline PGP, Diff View or a Phabrictor/Mailman visual block. All visual string output is performed via own method (writeBodyStr/writeBodyString). So we need there another Pugin infratrcture that process on the simple text. and splits the single string into multiple Blocks.

foreach(auto formatter, formatters) {
      if(formatter.handles(text)) {
          auto listBlocks = formatter.process(text);
          break;
      }
}

or use the Factory pattern here, too?

For Phabricator/Mailman own view we must act on special headers of the main part. So we would either need a own BodyPartFormatter for that or add a third plugin infrastructor on the text/plain process. Like it is done now for Mailman

!isMailmanMessage(node) || !processMailmanMessage(node)

General approach looks good to me.

My idea for the multiple plugins for the same mimetype problem (most probably text/plain) was to enable that by having a sorted list of plugins per mimetype, by giving them some form of numeric priority. The generic text/plain plugin would be the last in the chain.

The plugins would then be tried one after the other, as you suggested as well. One approach when the first one accepts a node is to stop there, but one could also consider continuing, if their output is not a special part object, but something that can be fed back into a plugin. Conceptually as if the output would be an expansion in the KMime::Content* tree, same as the crypto containers do it. Then we can chain the plugins arbitrarily until we hit the default one in the end. I know it's technically not as easy as that, I just want to outline the concept here :)

For rendering it's slightly different, you need to abort after a line has been rendered, you definitely don't want to end up in the default formatter and render again. We therefore probably want to expose the default rendering methods for other plugins to use (render a text line with emoticons, render an attachment box, etc).

knauss moved this task from In Progress to Done on the KDE PIM board.Jul 8 2016, 12:15 PM
knauss closed this task as Resolved.Sep 5 2016, 6:41 PM

Resolved with release KDE/Applications 16.08