Sorry for the delay in responding to this - it's indeed quite a tricky issue with no easy solution.
I can see a couple of ways around this though.

I've just had a quick look through the code and have noticed the following:

# Read in our parameters
#export BUILD_PREFIX=$1
#export KDENLIVE_SOURCES=$2
export BUILD_PREFIX=/build
export KDENLIVE_SOURCES=/kdenlive
export DEPS_INSTALL_PREFIX=/external

I've now applied that fix.

Ok, I have now prepared the scripts and they produce a working AppImage using the kdeorg/appimage-base docker image (I still need to include a few additionnal dependencies).
The scripts are in kdenlive's git master, under packaging/appimage:
https://cgit.kde.org/kdenlive.git/tree/packaging/appimage

fix missing parameter platform:

diff --git a/helpers/create-abi-dump.py b/helpers/create-abi-dump.py
index ef29138..cc9830b 100755
--- a/helpers/create-abi-dump.py
+++ b/helpers/create-abi-dump.py
@@ -312,7 +312,7 @@ for library in foundLibraries:
                 "branchGroup": arguments.branchGroup,
                 "platform": arguments.platform,
         }
-        packageName = "{name}_{scmRevision}_{platform}".format(name=library.name, scmRevision=scmRevision)
+        packageName = "{name}_{scmRevision}_{platform}".format(name=library.name, scmRevision=scmRevision, platform=arguments.platform)
         ourArchive.storePackage(packageName, fileName, scmRevision, extraMetadata)
     except subprocess.CalledProcessError as e:
         retval = e.returncode

This was caused by Java 8, as shipped by Ubuntu, having the following piece of configuration:

root@charlotte ~ # cat /etc/java-8-openjdk/accessibility.properties
#
# The following line specifies the assistive technology classes 
# that should be loaded into the Java VM when the AWT is initailized.
# Specify multiple classes by separating them with commas.
# Note: the line below cannot end the file (there must be at
# a minimum a blank line following it).
#
assistive_technologies=org.GNOME.Accessibility.AtkWrapper

Changing to Resolved as tests are adjusted to support running prior to installation.

Thanks for the answers. I am working on adapting my bash scripts to work on the Ubuntu 14.04 image mentionned and will ping you when I have it working locally.

Having Kdenlive also use the same "simple" bash system as KMyMoney (and Krita) would be non-ideal mainly as it's fairly resource inefficient if a dependency needs to be rebuilt (because it requires a full rebuild of everything).

No problem, thank you!

Sorry, this was an internal woopsie on the part of the CI system, and has now been repaired.
Replacement builds have been initiated.

I completely understand your frustration. It has been a long while since we had working appimages for you. Create those bash scripts and I will either integrate them into my system or the current kmymoney way.
Scarlett

Back to my original question, KMyMoney has AppImages built on the binary-factory right now, using what seems a very simple bash script system:
https://cgit.kde.org/kmymoney.git/commit/packaging/linux/appimage

My current code is at https://anongit.kde.org/sysadmin/appimage-tooling

@scarlettclark is your (old?) code available somewhere so I could look a bit into it? I'd like to see how much work it'd be to maintain.

Sorry, grumpy warning...

@scarlettclark since I'm a passionate Pythoneer, but have no clue about Ruby (I can read most of it, but like with PHP, I try to avoid it as good as I can), I guess we could team up to port your stuff to Python?

@TheAssassin yes, I was using them for non KDE applications as well.
@ everyone - I guess what I am saying here is I am more than happy to use my existing tools ( ruby ) and help with appimages. I just simply don't have the time to learn python right now. Not to say this is never going to happen, just not now.
Cheers,
Scarlett

@scarlettclark I can provide assistance of course, just let us know what we can help you with. I'd offer to collaborate on developing the scripts for craft or whatever system you use, they might come in handy for non-KDE projects as well. Also, it's always good to see how people use our tools, as we then get an impression how we can improve them.

Sorry last year was madness for me,
With that said, I have more time now.
I failed miserably with craft even getting anything to build, much less integration with the appimage stuff. Python hates me.
I have resurrected my work from BS appimages. I am more than happy to use it on binary factory to produce appimages relatively easily , and will even help with the scripts. I am very close to ready should it be decided this is an acceptable solution.
Cheers, Scarlett

I just saw that KMyMoney seems to have an AppImage built on binary.kde.org. How is it done ? from what I saw in their git repository, it seems like this is simply based on bash scripts ?
We really could benefit from some help on that side. Would it be possible to use a simple pipeline to build the AppImage on the CI ?
I had a quick look at Craft but honestly don't have time to learn how the system works, and since I already have some scripts to build my AppImage locally it would be much much simpler on my side if I could simply put some scripts on our git repository that are then executed on a docker container by the CI or something similar...

In T3689#166764, @bcooksley wrote:

Yes, you will need to split by that as well. This is handled normally by having one repository per platform but for ABI dumps you will have to handle that in the metadata.

Yes, you will need to split by that as well. This is handled normally by having one repository per platform but for ABI dumps you will have to handle that in the metadata.

Oh frameworks is built against different Qt versions (Qt5.9 and Qt5.10). We should to save the abidumps in separately for different Qt versions, as different Qt version may trigger different ABIs. Okay than we need to use platform in the abidumps, too.

I can confirm that all builds for Qt 5.11 on Linux run the tests prior to installation.

I've removed the old *.tar files from the ABIReference/ repository now.

Possible approach: D16682

all right, that would mean we need to "install" the scripts to bin directory, so that we can find them again.

D16589 now saves the the reports into $WORKSPACE/compat_report/LIB_NAME_compat_report.html.
Before activating the check-abi.py script we need to cleanup the build-artifacts.kde.org ABIReference directory and I need to cleanup check-abi.py to not create the symlinks.

I asked similar question some time ago in #plasma. IIRC, the tests fail because they are run uninstalled.

The $WORKSPACE part will be added for you (Jenkins won't archive anything outside that directory actually - it has to be within that folder or a subfolder of it)
If changing the name of the reports is easier then that also works (just be wary that projects might ship a *.html file in their repository that would then be caught by this)

so I need to specify the exact path to html files, for the archiveArtifacts? The default is compat_reports/LIB_NAME/V1_to_V2/compat_report.html, but I can change this to \$WORKSPACE/LIB_NAME_compat_report.html than it would be '*.html'.

Because the compat_reports are in $WORKSPACE you can specify this instead:
archiveArtifacts artifacts: 'path/to/files/*.html', onlyIfSuccessful: false

Do I just need to copy them to \$WORKSPACE directory and than tell Jenkins to store *.html. Like this:

Capturing them as artifacts in Jenkins means there will be an easy way for people to access the report for that particular job run from the job results page.
There isn't a easy way of adding a link to the main job results (basically impossible without a custom plugin) hence my suggestion of artifacts (because those do get linked)

In T3689#166093, @bcooksley wrote:

The file(s) could be captured using something like this:
https://cgit.kde.org/sysadmin/binary-factory-tooling.git/tree/craft/pipeline-templates/win64.pipeline#n105

The file(s) could be captured using something like this:
https://cgit.kde.org/sysadmin/binary-factory-tooling.git/tree/craft/pipeline-templates/win64.pipeline#n105

In T3689#165996, @bcooksley wrote:

Alas there is no native Jenkins publishing plugin - https://issues.jenkins-ci.org/browse/JENKINS-48221

This means we'd have to publish the actual compatibility report.

The easiest way in this case would probably be using Jenkins artifact system, which would attach the report as an artifact to the job.

Alas there is no native Jenkins publishing plugin - https://issues.jenkins-ci.org/browse/JENKINS-48221

I now created a first script to check the ABI in D16589. How we get the compat_report.html available in jenkins? (D16581 should make it possiible to get rid of the creation/removal symlinks.)

Also I need the project name in the metadata, otherwise I would need to reparse the buildlog again.

I need a way to differ the stable/master branch builds afterwards. branchGroup is suitable for that, as it also gives the information about the Qt version used. But any other other field is also fine, like branch. Maybe we can also modify the Archive.name and Archive.platform so we have different directories for stable/master builds.

yes I was referring to exactly this.

Are you referring to checking all of the jobs across the whole CI system to see where ABI Checker runs are failing?

Can you somehow get an overview, where we get an error for the ABI checker? I was wondering, why only some parts of messagelib are inside the ABIReference repo and found, that the ABI checker was failing for KF5MessageViewer.

Thanks Harald.

Some FTRs:

This should now be setup.

Cool, now I'm happy about the metadata and the data itself.
Can you keep enable this build of ABI References for some packages (stable+master) enabled for several build, so I have test data for the next step, where I need to compare the ABI against each other. I want some packages, so I cover more cases for the begining.

Test run done https://build-artifacts.kde.org/production/ABIReference/

The suffix .tar is added by the Archive.storePackage method line 163 but we need to rewrite the whole Archive logic a bit as it expects the tar on other places too. For the moment I would ignore this, as the data is correct. IMO: it more a cosmetic issue. I keep this in mind and fix this when implementing the next step.

I've tested this and can confirm that KTextEditor is happy now.
Could you please check the output and verify it is what you expected?

I've now finished tweaking the regex accordingly and it is now live for the Linux builds.

These files are generated in the build tree.

The problem is that there are other possibly uses for files which match those regexes.

Why not something like this to apply it only on Skrooge even if I believe that all projects using XML gui and settings should have the issue?

From my understanding it uses/supports fairly standard regular expressions. The Windows one for instance is:
excludePattern: '(C|c):/(Program Files \\(x86\\)|program files \\(x86\\)|Craft|craft)/.*'

Hi Ben,

In T9811#163369, @bcooksley wrote:

This has now been deployed.

This has now been deployed.

Thanks. If you could please land that i'll try to schedule some time to test it.

okay made KTextEditor built successfully with the patch D15706.

In the case of Windows we already do this, by excluding the Compiler and Windows SDKs, along with the "system libs" provided by Craft (like Qt and the like) so this would mostly be a case of someone writing an appropriate regex (to give to the excludePattern parameter of the Jenkins warnings plugin) for Linux and FreeBSD.

This change has now been implemented as we've discussed.

This has now been deployed.

Thank you. It was worth it. The failing test case has now been fixed.

You are correct that the CI system is separated from regular Blueprint updates. It is updated separately, on a manually triggered process.
This is done to ensure the system remains broadly consistent and to keep it stable (in terms of Qt and underlying library versions).

In T6308#156575, @bcooksley wrote:

The rollout is now complete.

The rollout is now complete.

sorry, it will properly take some time till I'll find time to look into it. I'm still traveling around and will be at home with the beginning of september...