Note: in order to fully work https://phabricator.kde.org/D8844 is need in addtion to this.

This commit breaks caldav on kolab servers.
I have reverted locally and my caldav calendars work again.

In D8843#173006, @winterz wrote:

This commit breaks caldav on kolab servers.
I have reverted locally and my caldav calendars work again.

Same here.

looks like there should be more tests to make sure, that a change don't break things. This sounds like there is only a small fine line that a caldav is working...

In D8843#173006, @winterz wrote:

This commit breaks caldav on kolab servers.

Would you mind giving some more details here? Unfortunately I don't have access to a kolab instance. I.e. does it only break for existing kolab resources or does it also break if you recreate the kolab resource from scratch? If it's just the former that might give a hint what the bug is.

I have reverted locally and my caldav calendars work again.

It also failed for a newly created instance for me. As a result I see all my calendars, but they are all empty.
Sorry, I don't have much more details yet, it failed at work here, where I have limited time to debug this.

In D8843#173242, @vkrause wrote:

It also failed for a newly created instance for me. As a result I see all my calendars, but they are all empty.
Sorry, I don't have much more details yet, it failed at work here, where I have limited time to debug this.

Hm, I am not able to reproduce the problem. I just tried this patch against a fresh Kolab 16 deployment and it worked fine (I also tested it against nextcloud and groupwise). Any more hints on how to reproduce it are appreciated.

Same bug here (but that's not surprising, we're all using the same Kolab server).

If there's no time to debug this for 17.12 (RC today!), I would like to request that this is reverted from the branch Applications/17.12 for now, and debugged in master.

Indeed we need to revert it for 17.12
And we keep some week in master for debugging it.

I reverted it before 17.12 RC tag.

I hope that it will debug in master soon.

In D8843#173783, @dfaure wrote:

Same bug here (but that's not surprising, we're all using the same Kolab server).

What Version of Kolab is that? I tried multiple times and it's working just fine for me with Kolab 16.

If there's no time to debug this for 17.12 (RC today!), I would like to request that this is reverted from the branch Applications/17.12 for now, and debugged in master.

Fine with me. Though I am pretty certain that the original code is broken. Especially if your caldav server is behind a reverse proxy or something like that. Because KDAV would insert the hostname/URL of the reverse proxy in that case into the mulitget request. Which might confuse the caldav server.

I don't know much about kolab server versioning and how to find out, but telnet mail.kdab.com imap says:
Cyrus IMAP v2.4.17-Kolab-2.4.17-30.6.el6.kolab_3.1 server ready
Does that help?

I would like to re-start the discussion. I tried rhaferkamps patches and they are working. The original code still fails to sync with mailbox.org.
The difference is that the original code includes the hostname in the request while the patch omits the hostname.
I would like to help to get mailbox.org working with KDAV again, but I don't know how. I can't think of a solution which might work for both cases and I wouldn't like to add a server specific code to to change the URL.

Can any one give me some directions, what might be a working approach?

I've done some Akonadi DAV resource bug triaging and think this, or a fixed patch, would resolve an issue affecting a large number of users.

After reading through the comments here, I was a bit surprised this patch was not taken up again after reverting it before 17.12.
From the comments, especially the following:

@dfaure wrote: Same bug here (but that's not surprising, we're all using the same Kolab server).

I gather that you decided to revert based on the fact that a single server (version?) failed (or were there more instances?), while the patch fixed an issue arising for a multiplicity of user servers.

So to those that reverted it and have access to the problematic server: please revisit this patch soon and gift those users a working DAV Groupware resource for Christmas!

Here's some debug output to understand what's going on:

KDAV::DavItemsListJob::davJobFinished: D8843 difference: href="/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics"

old code QUrl("https://caldav.kdab.com/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics")
    DavUrl: [ CalDav QUrl("https://dfaure%40kdab.com@caldav.kdab.com/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dc3eb14d-fb62-4ddc-b7c5-9d8b3eb5fc02.ics") ]

new code QUrl("/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics")
    DavUrl: [ CalDav QUrl("//dfaure%40kdab.com@/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/901926d0-3732-4e0b-87d1-800f6db33c29.ics") ]

And when enabling the new code (i.e. this patch), here's what I get later on:

KDAV::DavItemsFetchJob::start <calendar-multiget xmlns="urn:ietf:params:xml:ns:caldav">
 <prop xmlns="DAV:">
      <getetag xmlns="DAV:"/>
  <calendar-data xmlns="urn:ietf:params:xml:ns:caldav"/>
 </prop>
  <href xmlns="DAV:">/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics</href>
 [...]

DavGroupwareResource::onMultigetFinished: Empty item returned.  "/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics"

The QUrl inside the DavUrl in the "new code" case looks very broken to me, with a username and no host and no scheme.

In D8843#360672, @dfaure wrote:

Here's some debug output to understand what's going on

Thanks for having a look at it again. I guess we now need similar output for the servers people are having issues with. Either they can provide this or they can provide test accounts.

Could you add a comment to https://bugs.kde.org/show_bug.cgi?id=386985 with instructions for getting similar debug output and a request for test accounts?

What's wrong with current patch and when will we see caldav working with any non-kolab server? Now calendar is unusable.

It would be interessting to know the official DAV-specification. As other clients seem to work without the hostname-stuff, maybe kolab is doing something out-of-spec and it would be better to implement this patch here and open a bugreport at kolab?

Download and apply http://www.davidfaure.fr/2019/debug_output.diff to your kdav library to get debug output similar to mine.

Note that the comment "QUrl would treat this as a file URL be default" seems to be incorrect, there's no file: anywhere in my debug output.

I have a problem with akonadi since the current update (19.08.1 -> 19.08.2). I asked in the kdepim-users mailinglist about it and I was directed to this discussion. I am unsure if the issue I have is related to this discussion.

Can you tell me if this is a newly introduced bug or somehow related to the current one? If related, I was willing to help getting things running smoothly again. So please tell me what you need then.

That's unrelated to this patch. What you're experiencing is a regression in KF 5.63, which is already fixed in KF5 git master, the fix will be in KF 5.64
See https://bugs.kde.org/show_bug.cgi?id=413316 and https://phabricator.kde.org/D24880

@rhaferkamp ping?

In D8843#580407, @cgiboudeaux wrote:

@rhaferkamp ping?

Sorry, I had to drop the ball on this as I am no longer using kontact/korganizer for calendaring and I currently don't have time left to follow up on this.

In D8843#360672, @dfaure wrote:

Here's some debug output to understand what's going on:

[..]

KDAV::DavItemsFetchJob::start <calendar-multiget xmlns="urn:ietf:params:xml:ns:caldav">
 <prop xmlns="DAV:">
      <getetag xmlns="DAV:"/>
  <calendar-data xmlns="urn:ietf:params:xml:ns:caldav"/>
 </prop>
  <href xmlns="DAV:">/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics</href>
 [...]
 
DavGroupwareResource::onMultigetFinished: Empty item returned.  "/calendars/dfaure@kdab.com/02af0002-0181-4d55-88a7-db543be67d83/dd261076-60eb-40c2-b6fe-a245f69eac66.ics"

The QUrl inside the DavUrl in the "new code" case looks very broken to me, with a username and no host and no scheme.

The "username" in there is not really a username (in terms of an URI). It's just a part of the path. And the URL itself IIRC is not something that the client is sending any request to directly. It's just included as a relative href in the XML document that is to be processed on the server side. Most of the examples of CALDAV multiget request really only contain relative URLs there. It's even in the RFC. See: https://tools.ietf.org/html/rfc4791#page-64 (7.9.1. Example: Successful CALDAV:calendar-multiget REPORT)

possibly fixed in https://invent.kde.org/frameworks/kdav/-/commit/bf6628ff1da1661a6485a2199d2222611e1b2ae5