@bcooksley for akonadi-search we need some special settings for the abi-create step. Where we should store such settings?

okay with merging D17534 CI now successfully builds the abis for the mentioned packages. (I modified the last comment)

I use the ci-docker images to test the create-abi, but I can't reproduce why those are failing:

I now looked at every repository that failed to build successfully a ABI dump
for Pim. Most of them build successful, but a few of them have issues:

I now looked at every repository that failed to build successfully a ABI dump for Frameworks.
As I'm not really deep into the Framworks code, so I'm unsure, if those are real issues or if those Frameworks need special handling.
Please give me any response you have to those issues. (@aacid , @dfaure, @kde-frameworks-devel)

In D17431#373556, @bcooksley wrote:

We're only intending on monitoring the ABI on Linux correct?

YEAH - the artifacts are now generated. Thaks a lot!

mail to find a date is sent now.

Find a date at: https://bitpoll.mafiasi.de/poll/8BYKJA5B/

okay still the artifacts are not saved:

Yes most parts are resolved. But still packages have build errors in the create-abi step. Like:
https://build.kde.org/job/Applications/job/kdav/job/kf5-qt5%20SUSEQt5.9/50/console

@bcooksley: Thanks, the script now runs successfully *yeah* But the artifacts are not saved, because of a typo
compat_reports/*_compat_reports.html -> compat_reports/*_compat_report.html

Okay I see R857:a2c2c8fc3f9f in the ci-tooling repo, but jenkins jobs were not updated. When will the jobs been updated?

Okay now the last step: enabling check-abi for our builds: D17099

fix missing parameter platform:

diff --git a/helpers/create-abi-dump.py b/helpers/create-abi-dump.py
index ef29138..cc9830b 100755
--- a/helpers/create-abi-dump.py
+++ b/helpers/create-abi-dump.py
@@ -312,7 +312,7 @@ for library in foundLibraries:
                 "branchGroup": arguments.branchGroup,
                 "platform": arguments.platform,
         }
-        packageName = "{name}_{scmRevision}_{platform}".format(name=library.name, scmRevision=scmRevision)
+        packageName = "{name}_{scmRevision}_{platform}".format(name=library.name, scmRevision=scmRevision, platform=arguments.platform)
         ourArchive.storePackage(packageName, fileName, scmRevision, extraMetadata)
     except subprocess.CalledProcessError as e:
         retval = e.returncode

Here my first email draft for the doodle. Please proofread and change like you think:

In D16719#358284, @bcooksley wrote:

In D16719#357496, @knauss wrote:

The metadata not being uptodate is getting only an issue, if the metadata is important. With the normal tar files the only field used from the metadata is the timestamp check, but this even used as internal stuff. Even the checksum is not used to check, that the download was correctly. As the metadata is not used for anything outside the class, you won't even notice if the metadata does not match the data.

On the other side I can fully understand your point of view . As we won't change the contentsSuffix, when there would be a update of the package.

Sorry, i'm not sure what you're concerned about happening here. While I appreciate your code is leaning on the metadata more, the window of time during which you could potentially have a newer individual metadata file vs. the server manifest is perhaps a couple of seconds at most (because we move the file over first, then update the individual metadata file, and finally rebuild the master manifest file).

I also note that a job would be comparing against prior runs of itself, so you are guaranteed that nobody else will publish a new result (Plus given you put the commit hash in the archive name, once published files should never change correct?).

To use KCoreAddons as an example, the only job that can add results to that job are the jobs for KCoreAddons itself, and because jobs can only run one at a time you'll never have a collision (because you're contained within the branch group and platform constraints as well).

In regards to publishing the updated manifest file, the publishPackage invocation doesn't return until the publishing process completes in full - so the capture command in the build job can't finish and exit until the server manifest is up to date.

In D16717#358235, @bcooksley wrote:

Given that what is really interesting is tagged releases, perhaps instead of collecting ABI Dumps from normal builds, there should be special collection jobs run around release time (you could use a modified version of build-product-dependencies.py for this)?

rebase.

In T3689#166764, @bcooksley wrote:

Yes, you will need to split by that as well. This is handled normally by having one repository per platform but for ABI dumps you will have to handle that in the metadata.

Only work on correct platform.

rename stableBuild -> keepBranchGroup.

update environment.

Fix issues.

In D16719#356622, @bcooksley wrote:

We can rely on the serverManifest being the latest information here, as the window of time between the master manifest file being downloaded and parsed and resources fetched should be less than a minute at the maximum.

In the case of the ABI Reference information, you don't know about build results until they're published in the master manifest in any case as each result is stored by the latest Git commit hash. This also means that results won't change after they've been published, so for your usecase you can depend on the server manifest containing the latest information.

For the dependencies used by the rest of the system, at worst a build uses a slightly out of date result (and to date this hasn't been an issue, and is highly unlikely to occur in any case). The window of time where this would potentially happen is perhaps a maximum of 30-45 seconds.

If we were to fetch the individual metadata files that would defeat part of the reason for caching the files on the nodes locally (reducing master server load). In the case of some projects, they depend on upwards of 20-30 projects, and at least one of our build nodes is located in Canada (which has much greater latency to the master server), so this would cause builds for those projects to take much longer.

In D16717#356608, @bcooksley wrote:

The Jenkins master does not hold copies of any of the repositories, however from my understanding you are primarily interested in the oldest result set so that shouldn't need a list of tags (it can infer a tag exists from the metadata)?

Oh frameworks is built against different Qt versions (Qt5.9 and Qt5.10). We should to save the abidumps in separately for different Qt versions, as different Qt version may trigger different ABIs. Okay than we need to use platform in the abidumps, too.

In D16717#355429, @bcooksley wrote:

This utility will have to run on the server, rather than on the individual build nodes (they have very limited access to the master repository)
We'd therefore look to run this over all projects at once, rather than on a project by project basis - thoughts on how to accomplish this?

For retrievePackage we have the issue about race-condition:

• serverManifest is only downloaded once when initalizing the Archive -> may have outdated information
• localMetadata and package are more tight together IMO that's why it makes sense to always use localMetadata as reference.

fixes issues raised by ben.

update indent.

add documentation to Version and replace branch with branchGroup.

use branchGroup instead of git branch.

just a small update.

After thinking more about it I implement the whole logic different.

add logic to find the last released version of a library.

Fixed bcooksley issues.

than we have no blockers anymore.

cleanup if we store .abidump files in build-artifacts.

D16589 now saves the the reports into $WORKSPACE/compat_report/LIB_NAME_compat_report.html.
Before activating the check-abi.py script we need to cleanup the build-artifacts.kde.org ABIReference directory and I need to cleanup check-abi.py to not create the symlinks.

• run for all libraries, do not stop at first failure.
• safe comapt_reports to a place where it is accessible by jenkins for artifacts.
• cleanup script.

Keep in mind:

• Automatically checking DKIM signature for every mail can have a privacy breach, as you trigger automatically access to the internet - what happens if the server creates one cert per mail, than the server can tell when you access/view the mail.
• When viewing old mails, the cert may be expired /replaced already.

so I need to specify the exact path to html files, for the archiveArtifacts? The default is compat_reports/LIB_NAME/V1_to_V2/compat_report.html, but I can change this to \$WORKSPACE/LIB_NAME_compat_report.html than it would be '*.html'.

Do I just need to copy them to \$WORKSPACE directory and than tell Jenkins to store *.html. Like this:

In T3689#166093, @bcooksley wrote:

The file(s) could be captured using something like this:
https://cgit.kde.org/sysadmin/binary-factory-tooling.git/tree/craft/pipeline-templates/win64.pipeline#n105

In T3689#165996, @bcooksley wrote:

Alas there is no native Jenkins publishing plugin - https://issues.jenkins-ci.org/browse/JENKINS-48221

This means we'd have to publish the actual compatibility report.

The easiest way in this case would probably be using Jenkins artifact system, which would attach the report as an artifact to the job.

I now created a first script to check the ABI in D16589. How we get the compat_report.html available in jenkins? (D16581 should make it possiible to get rid of the creation/removal symlinks.)

Fixed Ben's issues.

Also I need the project name in the metadata, otherwise I would need to reparse the buildlog again.

I need a way to differ the stable/master branch builds afterwards. branchGroup is suitable for that, as it also gives the information about the Qt version used. But any other other field is also fine, like branch. Maybe we can also modify the Archive.name and Archive.platform so we have different directories for stable/master builds.

yes I was referring to exactly this.

Can you somehow get an overview, where we get an error for the ABI checker? I was wondering, why only some parts of messagelib are inside the ABIReference repo and found, that the ABI checker was failing for KF5MessageViewer.

Cool, now I'm happy about the metadata and the data itself.
Can you keep enable this build of ABI References for some packages (stable+master) enabled for several build, so I have test data for the next step, where I need to compare the ABI against each other. I want some packages, so I cover more cases for the begining.

fixed the issues from @bcooksley

use git log to extract current hash.

vkrause: ping