[OS X] build in a more "traditional" way (for distribution systems like MacPorts, Fink etc) (and a bit of code clean-up)
Needs ReviewPublic

Authored by rjvbb on Apr 15 2016, 1:37 PM.

Details

Reviewers
None
Group Reviewers
Marble
Summary

Marble currently aims to build 1 or 2 more or less standalone app bundles when being built on Mac OS X. There is nothing wrong with that, but it presents a problem for distribution systems like MacPorts or Fink or HomeBrew that aim to share libraries among packages and to remain as close as possible to the runtime context for which packages were designed (i.e. XDG/Freedesktop). Just like AppImage builds are possible but optional on other Unix variants, standalone app bundle builds should be optional on Mac too.

The patch achieves this by separating all APPLE-specific things in the CMake files into for required things like frameworks and build-type related things that is controlled by the APPLE_STANDALONE_BUNDLE option (ON by default).
The result when the option is turned off is that there is still an app bundle that is created, for the KF5-based marble (but not for marble-qt though that ought to be possible too). Everything else is installed in its usual location under the prefix.
The patch also creates and installs an application icon for marble.app with the official ECM function, and ensures that the .ini format is used for settings files, for maximum compatibility with other Unix desktops.

See also T10112 .

Test Plan

On OS X 10.9.5, with Qt 5.6 - 5.9.7 and FWs 5.20.0 - 5.52.0, installed under /opt/local. Also confirmed on Linux.

Diff Detail

Repository
R34 Marble
Lint
Lint Skipped
Unit
Unit Tests Skipped
rjvbb updated this revision to Diff 3361.Apr 15 2016, 1:37 PM
rjvbb retitled this revision from to [OS X] build in a more "traditional" way (for distribution systems like MacPorts, Fink etc).
rjvbb updated this object.
rjvbb edited the test plan for this revision. (Show Details)
rjvbb added a reviewer: Marble.
rjvbb set the repository for this revision to R34 Marble.
rjvbb added a project: Marble.

We had at least two maintainers for the Mac OS version, but unfortunately none of them is active anymore. I'm happy to accept anything that improves the Mac OS part of the build system. So please go ahead and clean up the patch. There's no need to keep old behavior if it makes no sense, feel free to clean things up where needed.

nienhueser, are you focusing on Marble as a standalone application, or more in the context of the surrounding KDE4/KF5 environments?

rahn added a subscriber: rahn.Apr 16 2016, 11:04 AM

We regard the stand-alone version essential and we'd really like to see an updated app bundle for the stand-alone version.
For us it's important to be able to pass a simple lean Marble version on to people who are interested in Marble. We don't want to force them to install the KDE environment. So KDE integration can be considered optional.

So whatever changes you make it should still be possible to compile the stand-alone version.

rahn added a comment.Apr 16 2016, 11:17 AM

Ideally we'd like to distribute a Mac OS bundle that

  • can be installed on recent plain vanilla Mac OS X systems out of the box
  • comes as a standalone package without further dependencies
  • has a package size similar to our Windows version (=38 MB)
  • is thoroughly tested on a few systems
  • optionally exposes headers and the libmarblewidget library for development

Unfortunately the standalone version we distribute right now is extremely outdated (version 1.6). It would be great if somebody could build such a standalone package of the 1.13.90 / 1.14 release.

rjvbb updated this revision to Diff 3381.Apr 17 2016, 10:08 AM

This version introduces an option/switch that defaults to building as a bundle but in the more traditional way when turned off on the cmake command-line.

As I said, building as standalone bundles isn't my priority, I came to using KDE on OS X because KDE4 is readily available through MacPorts (and I already used MacPorts for most everything not provided by Apple). By extension, my KF5 frameworks are built for use with traditional shared resources, so aren't fit for embedding.

Note that once all packages ("ports") are included in the official MacPorts repository users will be able to install Marble from binary packages, a priori with a single port install kf5-marble command.

My Qt5 build does support both kinds of build, so I did rebuild Marble yesterday (from unpatched sources) to create the standalone bundle around marble-qt. It runs but doesn't find any of its resources for some reason. A full build takes over 30 min on my system, which is too long for me ATM to start extensive experimenting. I suspect though that something is not as it should be in your use of QStandardPaths.

  • has a package size similar to our Windows version (=38 MB)

That can of course not be guaranteed.

  • optionally exposes headers and the libmarblewidget library for development

This is not possible. Embedded frameworks (libraries even more so) cannot be exposed. If you want this you'll have to create a MarbleWidget.framework bundle and figure out how to set things up that it can be embedded in dependent applications (that wish to embed the framework).

From a MacPorts viewpoint this would probably mean that libmarble should be a potentially independent target (i.e. it should be possible to build only that). That's probably not a hard requirement, but being able to use libmarblewidget or MarbleWidget.framework as a shared (= not embedded) resource will be a hard requirement.

Regarding resource finding: I suspect the issue is between setting data_dir as .../Contents/MacOS/resources/data whereas in MarbleDirs there is this:

//do some magick so that we can still find data dir if
//marble was not built as a bundle
if (myPath.contains(".app"))  //its a bundle!
{
  systempath = myPath + "/Contents/Resources/data";
}

Similar for plugin_dir in cmake vs. MarbleDirs.

rjvbb added a comment.Apr 17 2016, 2:16 PM

Regarding resource finding: I suspect the issue is between setting data_dir as .../Contents/MacOS/resources/data whereas in MarbleDirs there is this:

Which is conditional on Q_OS_MACX, a macro that's been deprecated since Qt 5.2.0 . I'll see if replacing it with Q_OS_OSX makes any difference.

rjvbb added a comment.Apr 17 2016, 2:46 PM
In D1420#26562, @rjvbb wrote:

Regarding resource finding: I suspect the issue is between setting data_dir as .../Contents/MacOS/resources/data whereas in MarbleDirs there is this:

Which is conditional on Q_OS_MACX, a macro that's been deprecated since Qt 5.2.0 . I'll see if replacing it with Q_OS_OSX makes any difference.

Anyway, good eye. Using Q_OS_OSX probably didn't change squat, but evidently code has to look where cmake puts stuff. After changing the code in MarbleDirs to correspond the marble-qt app bundle seems to work fine.

Built with -Os -g it's 44Mb but that is of course without any Qt libraries. Embedding those will probably require a step involving macdeployqt . I'd suggest looking at the buildsystem of the Charm time tracker, IIRC it has the necessary additional steps to accomplish this.

It's not uncommon to embed shared libraries (.dylibs) in Contents/MacOS/lib . The plugins should preferably go into Contents/Resources/Plug-Ins and the other data can a priori go into Contents/Resources directly. Am I right that that would require only changing the 2 lines in the toplevel CMake file and in MarbleDirs.cpp, or are there other locations or things to adapt?

It's not uncommon to embed shared libraries (.dylibs) in Contents/MacOS/lib . The plugins should preferably go into Contents/Resources/Plug-Ins and the other data can a priori go into Contents/Resources directly. Am I right that that would require only changing the 2 lines in the toplevel CMake file and in MarbleDirs.cpp, or are there other locations or things to adapt?

You're right about that, no more changes are needed.

rjvbb retitled this revision from [OS X] build in a more "traditional" way (for distribution systems like MacPorts, Fink etc) to [OS X] build in a more "traditional" way (for distribution systems like MacPorts, Fink etc) (and a bit of code clean-up).Apr 17 2016, 9:41 PM
rjvbb updated this revision to Diff 3391.Apr 17 2016, 9:44 PM

This gives me a working standalone-except-for-Qt5 Marble.app bundle.

I've put the plugins in Contents/Resources/plugins (forgot to rename it to Plug-Ins but that's maybe not really necessary) and the other data in Contents/Resources.

I did have to correct the plugin path set in qtmain.cpp!

Question: shouldn't MarbleDirs.cpp use QStandardPaths (QSP) everywhere? The current code seems almost counter-intuitive in that it only calls QStandardPaths on MSWin, where it's maybe the most problematic...

Not that I have anything against the current solution, which happens to be compatible with what my QSP patch does when its XDG-compliant mode is activated ...

rahn added a comment.Apr 18 2016, 11:26 AM

Yes, XDG compliance and backwards compability is certainly a concern there.
Dennis: What do you think about this? :)
Also looking at the code I wonder whether we shouldn't get rid of the MarbleDirs::oldLocalPath and DataMigration stuff (I guess it was only relevant 6 years ago ..) ..

rjvbb updated this revision to Diff 3629.May 3 2016, 8:34 PM

Updated for git/master, and corrects a (stupid!) typo/error in the toplevel CMake file

rjvbb added a comment.May 3 2016, 9:21 PM
This comment was removed by rjvbb.
rjvbb updated this revision to Diff 3643.May 4 2016, 5:33 PM

with this patch the traditional build works again

rjvbb updated this revision to Diff 9377.Dec 27 2016, 9:46 AM

For completeness sake, this is the patch updated for the Marble version included in KF5 Applications 16.08.2 .

Everything works perfectly with -DAPPLE_STANDALONE_BUILD=OFF in this version (I cannot recall having tested to what extent the standalone build failed in expectable ways).

rjvbb updated this revision to Diff 9378.EditedDec 27 2016, 10:07 AM

This updates the patch for git master/head. Aside from the standalone vs. "traditional" build it also moves from using the now deprecated Q_OS_OSX platform token to the new Q_OS_MACOS token.

Tested with the MacPorts "port" available at
https://github.com/RJVB/macstrop/tree/master/kf5/kf5-marble

A few observations:

  • without the patch or with and built with -DAPPLE_STANDALONE_BUILD=ON I get a Marble.app application required but lacking the embedded libraries, and a marble-qt.app application which does contain them
  • neither of those builds finds the other resources it requires, which is undoubtedly to be expected given how I build and install (= for MacPorts)
  • with the patch and -DAPPLE_STANDALONE_BUILD=OFF both marble and marble-qt hang during initialisation, before the main window is rendered or drawn completely.

That last point is very annoying and suggests I missed something that changed since 16.08.2 despite the fact that refactoring the patch was trivial.
This hang also happens on Linux where my patch shouldn't do anything so something fishy is going on that I'm currently trying to pinpoint. I'm uploading the patch anyway in case somebody else already encountered something similar. (On Linux I just get the KWin frame containing a snapshot of whatever screen content it covered).

On Linux I also see this on the calling terminal:

No connection: Connection refused
wlan2     Scan completed :
                    Signal: 51

lo        Interface doesn't support scanning.

eth4      Interface doesn't support scanning.

wlan2     Scan completed :
                    Signal: 51

lo        Interface doesn't support scanning.

eth4      Interface doesn't support scanning.

No connection: Connection refused

which I cannot recall seeing before.

A part of the backtrace on Linux which looks very much like what I saw on Mac:

#0  _int_free (av=0x7ffff3819760 <main_arena>, p=0x1a1b680, have_lock=0) at malloc.c:3897
#1  0x00007ffff773025f in deallocate (data=<optimised out>) at /opt/local/include/qt5/QtCore/qarraydata.h:222
#2  ~QString (this=<optimised out>, __in_chrg=<optimised out>) at /opt/local/include/qt5/QtCore/qstring.h:1065
#3  Marble::GeoDataCoordinates::lonToString (lon=lon@entry=-nan(0x8000000000000), notation=<optimised out>, 
    notation@entry=Marble::GeoDataCoordinates::DMS, unit=unit@entry=Marble::GeoDataCoordinates::Degree, 
    precision=<optimised out>, precision@entry=-1, format=format@entry=103 'g')
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/lib/marble/geodata/data/GeoDataCoordinates.cpp:519
#4  0x00007fffc86e3d1f in Marble::GraticulePlugin::renderLongitudeLines (this=<optimised out>, 
    painter=0x7fffffffb620, viewLatLonAltBox=..., step=inf, skipStep=0, northPolarGap=30, southPolarGap=30, 
    labelPositionFlags=...)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/plugins/render/graticule/GraticulePlugin.cpp:621
#5  0x00007fffc86e4d63 in Marble::GraticulePlugin::renderGrid (this=this@entry=0xbc5dc0, 
    painter=painter@entry=0x7fffffffb620, viewport=viewport@entry=0x80c650, equatorCirclePen=..., 
    tropicsCirclePen=..., gridCirclePen=...)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/plugins/render/graticule/GraticulePlugin.cpp:387
#6  0x00007fffc86e55be in Marble::GraticulePlugin::render (this=0xbc5dc0, painter=0x7fffffffb620, 
    viewport=0x80c650, renderPos=..., layer=<optimised out>)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/plugins/render/graticule/GraticulePlugin.cpp:286
#7  0x00007ffff78992cd in Marble::LayerManager::renderLayers (this=0x80c670, painter=painter@entry=0x7fffffffb620, 
    viewport=0x80c650)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/lib/marble/LayerManager.cpp:181
#8  0x00007ffff77fcd92 in Marble::MarbleMap::paint (this=0x803480, painter=..., dirtyRect=...)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/lib/marble/MarbleMap.cpp:842
#9  0x00007ffff77ed335 in Marble::MarbleWidget::paintEvent (this=0x7df2b0, evt=0x7fffffffb910)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_kf5_kf5-marble/kf5-marble/work/marble-16.12.0/src/lib/marble/MarbleWidget.cpp:709
#10 0x00007ffff5063b68 in QWidget::event (this=0x7df2b0, event=0x7fffffffb910)
    at /opt/local/var/lnxports/build/_opt_local_site-ports_aqua_qt5-kde-devel/qt5-kde-devel/work/qt-everywhere-opensource-src-5.6.2/qtbase/src/widgets/kernel/qwidget.cpp:9117

Edit: Note to self: floating point operations may be involved; try with -O3 instead of -Ofast .

rjvbb updated this revision to Diff 9379.Dec 27 2016, 10:44 AM

check APPLE AND APPLE_STANDALONE in CMake files, in case someone gives -DAPPLE_STANDALONE_BUILD on a different platform.

rjvbb added a subscriber: kde-mac.Dec 27 2016, 12:41 PM
In D1420#71454, @rjvbb wrote:

Edit: Note to self: floating point operations may be involved; try with -O3 instead of -Ofast .

That was it. Built with -O3 things work flawlessly again on Mac and Linux. Good to know.

rjvbb updated this revision to Diff 47061.Dec 7 2018, 6:45 PM

Refactored for current git/master.

rjvbb edited the summary of this revision. (Show Details)Dec 7 2018, 7:10 PM
rjvbb edited the test plan for this revision. (Show Details)
rjvbb set the repository for this revision to R34 Marble.
rjvbb removed a subscriber: kaning.
Restricted Application added a project: KDE Edu. · View Herald TranscriptDec 7 2018, 7:10 PM
Restricted Application added subscribers: kde-edu, marble-devel. · View Herald Transcript