new CI system-windows build for Skrooge
Closed, ResolvedPublic

Description

Hi Sysadmin team,

First, thank you for your job on new CI system.
Could you activate the windows build for Skrooge?

Of course, I will do my best to improve the build.
Regards.

Restricted Application added a subscriber: sysadmin. · View Herald TranscriptJun 10 2017, 9:10 AM
bcooksley closed this task as Resolved.Jun 10 2017, 9:30 AM
bcooksley claimed this task.
bcooksley added a subscriber: bcooksley.

Once the Dependency Build job has been completed the first build of Skrooge can be attempted.
https://build-sandbox.kde.org/view/CI%20Management/job/Dependency%20Build%20WindowsApplications%20kf5-qt5%20WindowsQt5.7/1/console

bcooksley closed this task as Resolved.Jun 10 2017, 8:48 PM

The Dependency Build has now completed - https://build-sandbox.kde.org/view/CI%20Management/job/Dependency%20Build%20WindowsApplications%20kf5-qt5%20WindowsQt5.7/2/

Unfortunately Skrooge itself cannot be built as it depends on QtWebkit which isn't available on Windows after Qt 5.6:
https://build-sandbox.kde.org/job/WindowsApplications%20skrooge%20kf5-qt5%20WindowsQt5.7/1/console

WebEngine is fully available for 5.7 onwards.

smankowski reopened this task as Open.Sep 17 2017, 8:19 AM

Hi Ben,

I did a first version of the port to WebEngine.
But I still have build error due to missing sqlcipher.

Could you help?
Regards.

I've checked and unfortunately SQLCipher doesn't seem to be available from Craft.
I'd suggest talking to @kfunk / @vonreth

It looks like at a minimum it'll require adding ActiveTcl to Craft before SQLCipher can be looked into.

bcooksley removed bcooksley as the assignee of this task.Sep 17 2017, 10:19 AM
bcooksley changed the visibility from "Custom Policy" to "Public (No Login Required)".
bcooksley changed the edit policy from "Custom Policy" to "All Users".
bcooksley edited projects, added build.kde.org; removed Sysadmin.
bcooksley moved this task from Backlog to Awaiting Response on the build.kde.org board.

Transitioning this task from the private Sysadmin queue to the public CI queue as this doesn't need to be restricted.

Hi All,

What can I do to progress on this compilation error on windows due to missing dependencies (sqlcypher)?

Regards.

You'd need to port https://github.com/sqlcipher/sqlcipher to CMake and add msvc support.
Or we add it to the autotools category like gpgme and use prebuild binaries (build with craft) with msvc....

Hi Hannah,

Thank you for this quick answer.
I don't know how to do the option 1 (port sqlcipher to CMake + msvc).
Is it possible to launch the option 2?
Regards.

Hm I had a look.
While it is C only, which would allow to use a mingw build with msvc. But It depends on openssl...
So for now I don't think we will get sqlcipher on Windows anytime soon, I'm sorry....

According to https://github.com/sqlitebrowser/sqlitebrowser/wiki/Win64-setup-%E2%80%94-Compiling-SQLCipher it is already setup for Windows builds as such, but does need a few extra things like Tcl.

Ping on my above comment. Have I misunderstood the issue?

Hi Ben,

Is this ping for me? If yes, what can I do?
Regards.

More for Hannah, I don't have any expertise in regards to what is needed here.

meh this looks pretty ugly... but I'll have a look in about a week, at least I'll try to have a look

Thanks Hannah

Hi,
Do you have some news?

I had a look on it.

Either support plain SQLite or write a CMake build system or sqlcipher

@smankowski Can you respond to Hannah please?

With the now available mingw cache we might also build it with mingw only?

@bcooksley @vonreth For security reason, using SQLite instead of SQLCipher is not an option.
I am still trying to include the build of SQLCipher with CMake in Skrooge (at least for windows) but this is not so easy.

Good point @vonreth. Have you looked into whether it can be built on Windows in a MingW environment @smankowski?

Hi @bcooksley, Hi @vonreth,

MingW is certainly a good idea.
My major constraint is that I do not have a Windows machine and it is difficult for me to do tests.
I had hoped that build.kde.org could provide me this build capability under windows and that step by step I would have converged towards a build that works.
I had not imagined that SQLCipher is more difficult to have under Windows than under Linux.
And since Skrooge was already running (not with SQLCipher) under windows thanks to the "KDE on Windows" project, I had imagined to make these builds step by step.

This seems more complex than expected.

I can understand that this is a great constraint for you and that you would prefer to close this incident.
Otherwise, if you have an idea to build Skrooge under Windows despite my constraints, I'm a taker.

Thanks again for everything you do for KDE.

Okay. There are currently some inbound changes happening to Craft which will change how the MingW builds for MSVC usage are done, so let's wait for those to land and settle then revisit this. As SQLCipher is purely a C library there shouldn't be too much trouble with getting this to work (in theory at least).

Have you had a chance to look into setting up a SQLCipher blueprint in Craft?

Hi Ben,

I'm starting but:

  • I don't know Craft
  • I have an issue with the installation of Craft because my system doesn't have python 3.6

I will try with "anaconda".

@smankowski Have you been looking into this?

While i'm not sure if it's related, it does seem that someone has been working on SQLCipher on Windows, as it shows up in the KMyMoney builds (and fails unfortunately).
https://binary-factory.kde.org/job/KMyMoney_Nightly_mingw64/88/consoleText

If you've been working on this, you might want to collaborate with @wojnilowicz who has been looking into the KMyMoney side of things.

bcooksley closed this task as Resolved.Aug 14 2018, 4:44 AM
bcooksley claimed this task.

This has now been provisioned.

Unfortunately the code doesn't successfully compile at the moment due to issues with QSQLite and SQLCipher, but those are problems with the codebase you'll need to fix i'm afraid!
https://binary-factory.kde.org/job/Skrooge_Nightly_mingw64/3/console

Please let us know if anything further is required.

(Please note that this is on the Binary Factory, and only runs once a day, as the mainline CI system has no support for SQLCipher as building that requires MingW)

This has now been provisioned.

Unfortunately the code doesn't successfully compile at the moment due to issues with QSQLite and SQLCipher, but those are problems with the codebase you'll need to fix i'm afraid!
https://binary-factory.kde.org/job/Skrooge_Nightly_mingw64/3/console

Please let us know if anything further is required.

It can be fixed by something like this

target_compile_definitions(qsqlcipher
  PRIVATE
  QT_PLUGIN # fails to build on MS Windows without this
  SQLITE_HAS_CODEC
  SQLITE_TEMP_STORE=2)

It's from our CMakeList.txt

(Please note that this is on the Binary Factory, and only runs once a day, as the mainline CI system has no support for SQLCipher as building that requires MingW)

Only MSVC from mainline CI system hasn't got sqlcipher. FreeBSD and Linux already have it. Could you install it for MSVC?

As SQLCipher has too many dependencies(ssl, zlib, icu tcl...), we can't provide a mingw prebuild, as we do for gpg.

The only fix in this case would be to have a build system we can run natively on Windows (ie. CMake) for SQLCipher for MSVC builds I gather?

This has been confirmed. The only things that can be done in this case are:

  1. Develop/Port a CMake build system for SQLCipher
  2. Introduce CI support for MingW builds.

Any comments on this @smankowski and @wojnilowicz ?

Note that introducing MingW support won't be trivial and will require modification of a few parts of the CI Tooling so some assistance with that will be needed.

This has been confirmed. The only things that can be done in this case are:

  1. Develop/Port a CMake build system for SQLCipher
  2. Introduce CI support for MingW builds.

    Any comments on this @smankowski and @wojnilowicz ?

    Note that introducing MingW support won't be trivial and will require modification of a few parts of the CI Tooling so some assistance with that will be needed.

I'm not sure what are you talking about. SQLCipher is already available on MinGW and MSVC, but in Build Factory only.

I've checked and it seems this won't be too problematic to rollout.
Currently doing testing / staging to make sure nothing is doing to break, then we'll move to live production.

bcooksley reopened this task as Open.Aug 21 2018, 6:08 PM
bcooksley closed this task as Resolved.Aug 24 2018, 4:20 AM

The rollout is now complete.

Skrooge still fails to build due to the issues noted above. KMyMoney finds SQLCipher successfully during it's build now.

The rollout is now complete.

I see that that it's not updating together with craft-blueprints-kde. That's fine.
However I try to fix segfaulting SQLCipher test on MSVC and would like to see how new SQLCipher would behave. I see that CI on MSVC reports
Found sqlcipher, version 3.4.2-5
while it should report
Found sqlcipher, version 3.4.2-6
as this is the latest patch level in its build script. Could you please update SQLCipher for MSVC?

You are correct that the CI system is separated from regular Blueprint updates. It is updated separately, on a manually triggered process.
This is done to ensure the system remains broadly consistent and to keep it stable (in terms of Qt and underlying library versions).

I've now run the deploy process, which has pulled in the newer SQLCipher.

Thank you. It was worth it. The failing test case has now been fixed.