diff --git a/README.md b/README.md index 2b3fb93..e1fb9c4 100644 --- a/README.md +++ b/README.md @@ -1,270 +1,272 @@ # kdesrc-build This script streamlines the process of setting up and maintaining a development environment for KDE software. It does this by automating the process of downloading source code from the KDE source code repositories, building that source code, and installing it to your local system. ## Note the Alternatives NOTICE! If you are a power user just trying to test the latest KDE releases like [KDE Plasma 5](https://www.kde.org/plasma-desktop) or the [KDE Applications](https://www.kde.org/applications/) then there are potentially easier options you may wish to consider first. KDE provides a quick-starter distribution, [KDE neon Developer Edition](https://neon.kde.org/download), and your favorite distribution may also have bleeding edge packages that may be easier to try out. However if you're testing out the latest KDE Frameworks or are involved in development yourself, you'll probably find it easiest to use kdesrc-build. Continue on, to learn how to set it up. ## QUICK HOWTO 1. Set up minimum dependencies - You must have at least [Perl](https://www.perl.org/get.html) version 5.14 installed. Most distributions include at least a minimal Perl set up, and Perl is also required for some portions of Qt and KDE software builds so it is good to have regardless. - You must have [Git](https://git-scm.com/) installed to download KDE sources and kdesrc-build itself. We recommend at least version 2.10. +- You must have [dialog](https://invisible-island.net/dialog/) installed to run kdesrc-build-setup. + 2. Install kdesrc-build: - Clone kdesrc-build from git, by running from a terminal: ```shell $ mkdir -p ~/kde/src $ cd ~/kde/src $ git clone https://anongit.kde.org/kdesrc-build $ cd kdesrc-build # kdesrc-build is in this directory ``` - Make sure it works by running: ```shell $ cd ~/kde/src/kdesrc-build $ ./kdesrc-build --version ``` You should see output similar to `kdesrc-build 18.10 (v18.10-20-g1c39943)`. Later we will set up kdesrc-build to keep itself updated automatically. 2. Set up kdesrc-build: - Now that kdesrc-build is installed and works, you need to set up kdesrc-build to work appropriately on your particular system. Do this by running the provided set up script to generate the **configuration file** (~/.kdesrc-buildrc): ```shell $ cd ~/kde/src/kdesrc-build $ ./kdesrc-build-setup ``` - Answer the questions given, but do not fret if you don't know what exactly you want to build, it is easy to edit the configuration later or just to re-run `kdesrc-build-setup` again. - This script will reference a standard configuration provided as part of the kdesrc-build repository that you downloaded earlier. As kdesrc-build self-updates, these changes will reflect for your configuration as well. 3. Download the KDE project and dependency data: ```shell $ cd ~/kde/src/kdesrc-build $ ./kdesrc-build --metadata-only ``` This will download information describing the KDE source repositories and their dependencies, which will help kdesrc-build figure out what to build. kdesrc-build will maintain this automatically, but running this step separately helps to verify that kdesrc-build can properly reach the KDE source repository and allows the `--pretend` option in the next step to provide more accurate output. 4. Verify kdesrc-build has a good build plan: ```shell $ cd ~/kde/src/kdesrc-build $ ./kdesrc-build --pretend ``` This will have kdesrc-build go through the steps that it would perform, but without actually doing them. kdesrc-build will do some basic pre-checks in this stage to ensure that required command-line commands are available, including `cmake`, `git`, `qmake`, and others. This is the last good chance to make sure that kdesrc-build is set the way you want it. If this command gives you a message that all modules were successfully built, you can move onto the next step. 5. Perform your first build: ```shell $ cd ~/kde/src/kdesrc-build $ ./kdesrc-build --stop-on-failure dolphin ``` This will build [Dolphin](https://www.kde.org/applications/system/dolphin/), the Plasma 5 file manager and its KDE-based dependencies. We choose Dolphin since it is a good test case to exercise the whole build process. For each module built, kdesrc-build will complete these steps: - Update source code (initial download or later update) - Set up the build system and configure source code with your options, if needed - Perform the build, if needed - Install the module The `--stop-on-failure` flag causes kdesrc-build to stop the first time a module fails to build, in case there are missing dependencies. If not set, kdesrc-build would otherwise try to build all the modules even if some fail. Hopefully everything will go well the first time, and kdesrc-build will be able to download and build all of the modules that you ask for. :) ## UPGRADING KDESRC-BUILD Upgrading is simple. You can delete your old kdesrc-build directory (make sure you don't have any local changes in there first, or your kdesrc-buildrc file!) and then install the new version where the old kdesrc-build directory used to be. In fact, it is recommended to use git to update kdesrc-build itself, so that kdesrc-build updates itself automatically when run. This is set up already in the sample configuration for KF5, where kdesrc-build is configured to update itself. One thing to keep in mind when using kdesrc-build to manage keeping itself up to date is that updates won't take effect until the *next* time you run kdesrc-build. You may want to edit your ~/.kdesrc-buildrc to make sure any new options are included. You should always read the changes for the new version however, as sometimes there are slight changes in behavior necessary to adapt to updates in the source repository. If you are running kdesrc-build from its git repository, you can use the "git log" command from inside the kdesrc-build source directory to see the latest changes. You can use the `./kdesrc-build --version` command to ensure that you have successfully upgraded kdesrc-build. ## SAMPLE CONFIGURATION A sample configuration file is included for demonstration purposes. You could copy it to your `~/.kdesrc-buildrc` and edit, but you should use the provided `kdesrc-build-setup` script instead. ## HELP!!! This is only a very cursory guide. For more information please see the KDE Community [Get Involved for Development](https://community.kde.org/Get_Involved/development) page. ## REFERENCE kdesrc-build includes a limited command-line description with the --help option. You can read the [kdesrc-build handbook](https://docs.kde.org/index.php?application=kdesrc-build) online. Once you've set up a KDE development environment, kdesrc-build itself can generate and build documentation (a handbook and a man page). The handbook would be available in KHelpCenter (help:/kdesrc-build), while the man page would be available in the KDE man pages or in the kdesrc-build build directory: ```shell $ cd ~/kde/build/kdesrc-build/doc $ man ./kdesrc-build.1 ``` You can also ask for help online on the #kde-devel channel of IRC (irc.kde.org). Additionally you can ask for help on the KDE support mailing lists, such as kde-devel@kde.org Finally you can drop me an email at mpyne@kde.org (although I have a job/family and therefore don't always have time to respond) ### Behind the Curtain For each build, kdesrc-build does several things: - Finds the configuration file (based on the --rc-file option or by looking for `kdesrc-buildrc` in the current directory and falling back to `~/.kdesrc-buildrc`) - Reads the configuration file to generate: - Default options to apply for each module - A list of modules to build. Modules can be grouped in "module-sets", but kdesrc-build converts each set to a list of modules. - Reduces the module list to modules chosen on the command line (either by name or through options like `--resume-from`). - For modules known to be KDE repositories (derived from a module-set using the special `kde-projects` repository): - If `--include-dependencies` is enabled, adds needed KDE modules into the build, then - Reorders KDE modules with respect to each other to ensure they are built in dependency order. - Builds each module in the resulting list of modules. This is broken into "phases", and each phase's output is logged to a specific directory for inspection later (by default, ~/kde/src/log). kdesrc-build takes some pains to do perform source code updates and builds in the way that a developer really would at the command line, using the same `git`, `cmake`, `make` commands a user would. This means that users are free to explore the source directory and build directory for a module without trampling on additional data maintained by kdesrc-build: kdesrc-build does nothing special in either the source or build directories. ### Important Command Line Options These options are the most useful. Others are documented at [the kdesrc-build online handbook](https://go.kde.org/u/ksbcmdline). | option | Description | | ------ | ----------------- | | `--include-dependencies` | Adds any missing modules that are needed for the modules being built. Only works for KDE modules. | | `--pretend` | Lists the actions kdesrc-build would take but doesn't actually do them. Useful for a double-check before a long build. | | `--resume-from` | Starts the build from the given module instead of building all modules. Can combine with `--stop-after` or `--stop-before`. | | `--resume-after` | Starts the build from *after* the given module, otherwise same as `--resume-from`. | | `--stop-before` | Stops the build just before the given module instead of building all modules. Can combine with `--resume-from` or `--resume-after`. | | `--stop-after` | Stops the build just *after* the given module, otherwise the same as `--stop-before`. | | `--no-src` | Perform module builds as normal but don't try to update source directories. Use this when you've updated source codes yourself. | | `--refresh-build` | Completely cleans under the module build directories before building. Takes more time but can help recover from a broken build directory set up. | ### Cleaning the build and install directories kdesrc-build will if possible avoid regenerating the build system and avoid complete rebuilds of existing modules. This avoids wasting significant amounts of time rebuilding source codes that have not changed, as all supported build systems are smart enough to rebuild when necessary. However it can sometimes happen that a rebuild is needed but wasn't detected. If this happens you can force a build directory to be fully rebuilt using the `--refresh-build` option to kdesrc-build. If all else fails and your development environment which was working fine now can't seem to upgrade modules anymore, another option is to consider deleting the install directory (~/kde/usr by default) completely and rebuilding everything (using `--refresh-build`), but this can take a significant amount of time! ## CONTACT INFO If you find a bug, please report it at the [KDE Bugzilla](https://bugs.kde.org/) If you have any questions, please let me know: Michael Pyne diff --git a/custom-qt5-libs-build-include b/custom-qt5-libs-build-include index 665afa5..63ad1c7 100644 --- a/custom-qt5-libs-build-include +++ b/custom-qt5-libs-build-include @@ -1,51 +1,57 @@ # These libs are likely part of your distro # So you only need to include this file if you compile your own Qt5, # to avoid mixing your own Qt with the distro's Qt. # For okular module poppler repository git://git.freedesktop.org/git/poppler/poppler branch master cmake-options -DWITH_GLIB=OFF -DLIB_SUFFIX=64 -DENABLE_UNSTABLE_API_ABI_HEADERS=1 end module # For kaccounts-integration module libaccounts-glib # doesn't depend on qt, but this way it's the right version for libaccounts-qt below repository https://gitlab.com/accounts-sso/libaccounts-glib.git end module module libaccounts-qt repository https://gitlab.com/accounts-sso/libaccounts-qt.git qmake-options PREFIX=${kdedir} end module module signond repository https://gitlab.com/accounts-sso/signond.git qmake-options PREFIX=${kdedir} end module +# For apper + +module packagekit-qt + repository https://github.com/hughsie/PackageKit-Qt.git +end module + # For telepathy-accounts-signon module libsignon-glib # this one does depend on qt, it seems repository https://gitlab.com/accounts-sso/libsignon-glib.git qmake-options PREFIX=${kdedir} end module # Mandatory for kmouth, optional for kpimtextedit module qtspeech repository git://code.qt.io/qt/qtspeech.git branch dev end module # Mandatory for krita module quazip repository https://github.com/stachenov/quazip.git branch master end module diff --git a/doc/index.docbook b/doc/index.docbook index 098d305..d1e0377 100644 --- a/doc/index.docbook +++ b/doc/index.docbook @@ -1,4809 +1,4840 @@ + Copyright (c) 2005-2008, 2010-2020 Michael Pyne Copyright (c) 2005 Carlos Leonhard Woelz Copyright (c) 2009 Burkhard Lück Copyright (c) 2007, 2011 Federico Zenith Copyright (c) 2009-2011 Yuri Chornoivan ... and possibly others. Check the git source repository for specifics. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in COPYING.DOC. The license will be included in the generated documentation as well. --> kdesrc-build"> BSD'> Git'> CMake'> Make'> Ninja'> SSH'> Cron'> Subversion'> Sudo'> URL'> configure-flags'> kdedir'> qtdir'> build-dir'> module-base-path'> override-url'> source-dir'> email-address'> email-on-compile-error'> colorful-output'> tag'> branch'> do-not-compile'> checkout-only'> svn-server'> make-install-prefix'> niceness'> set-env'> libpath'> binpath'> --nice'> --ignore-modules'> --resume-from'> --resume-after'> --reconfigure'> --refresh-build'> ]> &kdesrc-build; Script Manual &Michael.Pyne; &Michael.Pyne.mail; CarlosWoelz carloswoelz@imap-mail.com 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Michael Pyne 2005 Carlos Woelz &FDLNotice; 2018-01-20 &kdesrc-build; 17.12 &kdesrc-build; is a script which builds and installs &kde; software directly from the &kde; project's source code repositories. KDE kdesdk SVN Subversion git KDE development kdesvn-build kdecvs-build Introduction A brief introduction to &kdesrc-build; What is &kdesrc-build;? &kdesrc-build; is a script to help the &kde; community install &kde; software from its &git; and &subversion; source repositories, and continue to update that software afterwards. It is particularly intended to support those who need to supporting testing and development of &kde; software, including users testing bugfixes and developers working on new features. The &kdesrc-build; script can be configured to maintain a single individual module, a full &plasma; desktop with &kde; application set, or somewhere in between. To get started, see , or continue reading for more detail on how &kdesrc-build; works and what is covered in this documentation. &kdesrc-build; operation <quote>in a nutshell</quote> &kdesrc-build; works by using the tools available to the user at the command-line, using the same interfaces available to the user. When &kdesrc-build; is run, the following sequence is followed: &kdesrc-build; reads in the command line and configuration file, to determine what to build, compile options to use, where to install, &etc; &kdesrc-build; performs a source update for each module. The update continues until all modules have been updated. Modules that fail to update normally do not stop the build – you will be notified at the end which modules did not update. Modules that were successfully updated are built, have their test suite run, and are then installed. To reduce the overall time spent, &kdesrc-build; will by default start building the code as soon as the first module has completed updating, and allow the remaining updates to continue behind the scenes. A very good overview of how &kde; modules are built, including informative diagrams, is provided on an online article discussing &kde;'s &krita; application. This workflow is what &kdesrc-build; automates for all &kde; modules. Documentation Overview This guide is an overview to describe the following aspects of &kdesrc-build; operation: An overview of the steps required to get started. Notable features. The configuration file syntax and options. The command line options. Also documented are the steps which you should perform using other tools (&ie; steps that are not automatically performed by &kdesrc-build;). Getting Started In this chapter, we show how to use the &kdesrc-build; to checkout modules from the &kde; repository and build them. We also provide a basic explanation of the &kde; source code structure and the steps you have to perform before running the script. All topics present in this chapter are covered with even more detail in the Build from Source article, at the &kde; Community Wiki. If you are compiling &kde; for the first time, it is a good idea to read it, or consult it as a reference source. You will find detailed information about packaging tools and requirements, common compilation pitfalls and strategies and information about running your new &kde; installation. Preparing the System to Build &kde; Setup a new user account It is recommended that you use a different user account to build, install, and run your &kde; software from, since less permissions are required, and to avoid interfering with your distribution's packages. If you already have &kde; packages installed, the best choice would be to create a different (dedicated) user to build and run the new &kde;. Leaving your system &kde; untouched also allows you to have an emergency fallback in case a coding mistake causes your latest software build to be unusable. You can do also setup to install to a system-wide directory (⪚ /usr/src/local) if you wish. This document does not cover this installation type, since we assume you know what you are doing. Ensure your system is ready to build &kde; software Before using the &kdesrc-build; script (or any other building strategy) you must install the development tools and libraries needed for &kde;. The nearly complete list of required tools can be found from the &kde; Community Wiki Build Requirements page. Here is a list of some of the things you will need: You will need &cmake;, this software is what &kde; uses to handle build-time configuration of the source code and generation of the specific build commands for your system. The required version will vary depending on what versions of &kde; software you are building (see TechBase for specifics), but with modern distributions the &cmake; included with your distribution should be quite sufficient. You must also install the source control clients needed to checkout the &kde; source code. This means you need at least the following: The Git source control manager, which is used for all &kde; source code Although it is not required, the Bazaar source control manager is used for a single module (libdbusmenu-qt) that is required for the &kde; libraries. Most users can install this library through their distribution packages but &kdesrc-build; supports building it as well if you desire. But to build libdbusmenu-qt, you must have Bazaar installed. The Perl scripting language is required for &kdesrc-build;, some &kde; repositories, and &Qt; (if you build that from source). The Perl that comes with your distribution should be suitable (it needs to be at least Perl 5.14), but you will also need some additional modules (&kdesrc-build; will warn if they are not present): IO::Socket::SSL YAML::PP, YAML::XS, or YAML::Syck The Mojolicious web framework. It is small, self-contained, and allows &kdesrc-build; to support interfaces beyond the simple command-line interface while continuing to permit high performance. You will need a full C++ development environment (compiler, standard library, runtime, and any required development packages). The minimum required versions vary based on the &kde; module: the &kde; Frameworks 5 collection supports the oldest compilers, while &kde; Plasma 5 and &kde; Applications tend to require more recent compilers. The GCC 4.8 or Clang 4 compilers are the minimum recommended. Many distributions support easily installing these tools using a build-essentials package, an option to install "build dependencies" with &Qt;, or similar features. The KDE Community Wiki has a page tracking recommended packages for major distributions. You will need a build tool that actually performs the compilation steps (as generated by &cmake;). GNU Make is recommended and should be available through your package manager. &cmake; does support others options, such as the &ninja; build tool, which can be used by &kdesrc-build; using the custom-build-command configuration file option. Finally, you will need the appropriate &Qt; libraries (including development packages) for the version of &kde; software you are building. &kdesrc-build; does not officially support building &Qt; 5 (the current major version), so it is recommended to use your distribution's development packages or to see the KDE Community wiki page on self-building Qt 5. Most operating system distributions include a method of easily installing required development tools. Consult the Community Wiki page Required devel packages to see if these instructions are already available. Some of these packages are divided into libraries (or programs or utilities), and development packages. You will need at least the program or library and its development package. Setup &kdesrc-build; Install &kdesrc-build; The &kde; developers make frequent changes to &kdesrc-build; to keep it in sync with advances in &kde; development, including improvements to the recommended &kdesrc-build; configuration, added modules, improving &cmake; flags, &etc; Because of this, we recommend obtaining &kdesrc-build; directly from its source repository and then periodically updating it. You can obtain &kdesrc-build; from its source repository by running: $ git Replace with the directory you would like to install to. You can update &kdesrc-build; later by running: $ cd $ git We recommend adding the &kdesrc-build; installation directory to your PATH environment variable, so that you can run &kdesrc-build; without having to fully specify its path every time. Prepare the configuration file &kdesrc-build; uses a configuration file (located at ~/.kdesrc-buildrc) to control which modules are built, where they are installed to, etc. You can use a program included with &kdesrc-build;, called kdesrc-build-setup in order to prepare a simple kdesrc-build configuration. You can then edit the ~/.kdesrc-buildrc from there to make any changes you see fit. kdesrc-build-setup itself runs from a terminal (instead of using a graphical interface), just like &kdesrc-build;, so you can use it even if you have no graphical interface available yet. Manual setup of configuration file You can also setup your configuration file manually, by copying the included sample configuration file kdesrc-buildrc-kf5-sample to ~/.kdesrc-buildrc and then editing the file. will be a useful reference for this, especially its table of configuration options. &kdesrc-build; contains many recommended configuration files to support &kde; Frameworks 5, &plasma; 5, and other &kde; applications. The kdesrc-build-setup refers to these files in the configuration file it generates, but you can also use them yourself. See for information on how to use other configuration files from your own ~/.kdesrc-buildrc. You can find more information about the syntax of the configuration file in and in . Setting the Configuration Data To use &kdesrc-build;, you should have a file in your home directory called .kdesrc-buildrc, which sets the general options and sets the modules you would like to download and build. It is possible to use different configuration files for &kdesrc-build;, which is described in . If you need to use multiple configurations, please see that section. Here, we will assume the configuration is stored in ~/.kdesrc-buildrc. The easiest way to proceed is to use the kdesrc-buildrc-kf5-sample file as a template, changing global options to match your wants, and also change the list of modules you want to build. The default settings should be appropriate to perform a &kde; build. Some settings that you may wish to alter include: kdedir, which changes the destination directory that your &kde; software is installed to. This defaults to ~/kde, which is a single-user installation. branch-group, which can be used to choose the appropriate branch of development for the &kde; modules as a whole. There are many supported build configurations but you will likely want to choose so that &kdesrc-build; downloads the latest code based on &Qt; 5 and &kde; Frameworks 5. &kdesrc-build; will use a default branch group if you do not choose one, but this default will change over time, so it's better to choose one so that the branch group does not change unexpectedly. source-dir, to control the directory &kdesrc-build; uses for downloading the source code, running the build process, and saving logs. This defaults to ~/kdesrc. cmake-options, which sets the options to pass to the &cmake; command when building each module. Typically this is used to set between debug or release builds, to enable (or disable) optional features, or to pass information to the build process about the location of required libraries. make-options, which sets the options used when actually running the make command to build each module (once &cmake; has established the build system). The most typical option is , where N should be replaced with the maximum number of compile jobs you wish to allow. A higher number (up to the number of logical CPUs your system has available) leads to quicker builds, but requires more system resources. Configuring Make for 4 compiles at once, with exceptions global make-options -j4 … end global … module-set big-module-set repository kde-projects use-modules calligra make-options -j2 # Reduced number of build jobs for just these modules end module-set Some very large Git repositories may swamp your system if you try to compile with a too many build jobs at one time, especially repositories like the &Qt; WebKit and &Qt; WebEngine repositories. To maintain system interactivity you may have to reduce the number of build jobs for specific modules. gives an example of how to do this. You may want to select different modules to build, which is described in . Using the &kdesrc-build; script With the configuration data established, now you are ready to run the script. Even if you still have some tweaking or other reading you wish to do, it is a good idea to at least load the &kde; project metadata. Loading project metadata From a terminal window, log in to the user you are using to compile &kde; software and execute the script: % kdesrc-build This command will setup the source directory and connect to the KDE &git; repositories to download the database of &kde; git repositories, and the database of dependency metadata, without making any further changes. It is useful to run this separately as this metadata is useful for other &kdesrc-build; commands. Previewing what will happen when kdesrc-build runs With the project metadata installed, it is possible to preview what &kdesrc-build; will do when launched. This can be done with the command line option. % ./kdesrc-build You should see a message saying that some packages were successfully built (although nothing was actually built). If there were no significant problems shown, you can proceed to actually running the script. % kdesrc-build This command will download the appropriate source code, and build and install each module in order, but will stop if a module fails to build (due to the option). Afterwards, you should see output similar to that in : Example output of a kdesrc-build run % kdesrc-build Updating kde-build-metadata (to branch master) Updating sysadmin-repo-metadata (to branch master) Building libdbusmenu-qt (1/200) No changes to libdbusmenu-qt source, proceeding to build. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) Building taglib (2/200) Updating taglib (to branch master) Source update complete for taglib: 68 files affected. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) Building extra-cmake-modules from <module-set at line 32> (3/200) Updating extra-cmake-modules (to branch master) Source update complete for extra-cmake-modules: 2 files affected. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) ... Building kdevelop from kdev (200/200) Updating kdevelop (to branch master) Source update complete for kdevelop: 29 files affected. Compiling... succeeded (after 1 minute, and 34 seconds) Installing.. succeeded (after 2 seconds) <<< PACKAGES SUCCESSFULLY BUILT >>> Built 200 modules Your logs are saved in /home/kde-src/kdesrc/log/2018-01-20-07 Resolving build failures Depending on how many modules you are downloading, it is possible that &kdesrc-build; will not succeed the first time you compile &kde; software. Do not despair! &kdesrc-build; logs the output of every command it runs. By default, the log files are kept in ~/kdesrc/log. To see what the caused an error for a module in the last &kdesrc-build; command, usually it is sufficient to look at ~/kdesrc/log/latest/module-name/error.log. Perhaps the easiest way to find out what error caused a module to fail to build is to search backward with a case-insensitive search, starting from the end of the file looking for the word error. Once that is found, scroll up to make sure there are no other error messages nearby. The first error message in a group is usually the underlying problem. In that file, you will see the error that caused the build to fail for that module. If the file says (at the bottom) that you are missing some packages, try installing the package (including any appropriate -dev packages) before trying to build that module again. Make sure that when you run &kdesrc-build; again to pass the --reconfigure option so that &kdesrc-build; forces the module to check for the missing packages again. Or, if the error appears to be a build error (such as a syntax error, incorrect prototype, unknown type, or similar) then it is probably an error with the &kde; source, which will hopefully be resolved within a few days. If it is not resolved within that time, feel free to mail the kde-devel@kde.org mailing list (subscription may be required first) in order to report the build failure. You can find more common examples of things that can go wrong and their solutions, as well as general tips and strategies to build &kde; software in the Build from Source. On the other hand, assuming everything went well, you should have a new &kde; install on your computer, and now it is simply a matter of running it, described next in . For more information about &kdesrc-build;'s logging features, please see . Building specific modules Rather than building every module all the time, you may only want to build a single module, or other small subset. Rather than editing your configuration file, you can simply pass the names of modules or module sets to build to the command line. Example output of a kdesrc-build specific module build % kdesrc-build dolphin Updating kde-build-metadata (to branch master) Updating sysadmin-repo-metadata (to branch master) Building extra-cmake-modules from frameworks-set (1/79) Updating extra-cmake-modules (to branch master) No changes to extra-cmake-modules source, proceeding to build. Running cmake... Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) Building phonon from phonon (2/79) Updating phonon (to branch master) No changes to phonon source, proceeding to build. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) Building attica from frameworks-set (3/79) Updating attica (to branch master) No changes to attica source, proceeding to build. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) ... Building dolphin from base-apps (79/79) Updating dolphin (to branch master) No changes to dolphin source, proceeding to build. Compiling... succeeded (after 0 seconds) Installing.. succeeded (after 0 seconds) <<< PACKAGES SUCCESSFULLY BUILT >>> Built 79 modules Your logs are saved in /home/kde-src/kdesrc/log/2018-01-20-07 In this case, although only the dolphin application was specified, the flag caused &kdesrc-build; to include the dependencies listed for dolphin (by setting the include-dependencies option). The dependency resolution worked in this case only because dolphin happened to be specified in a kde-projects-based module set (in this example, named base-apps). See . Setting the Environment to Run Your &kde; &plasma; Desktop Assuming you are using a dedicated user to build &kde; &plasma;, and you already have an installed &plasma; version, running your new &plasma; may be a bit tricky, as the new &plasma; has to take precedence over the old. You must change the environment variables of your login scripts to make sure the newly-built desktop is used. Automatically installing a login driver Starting from version 1.16, &kdesrc-build; will try to install an appropriate login driver, that will allow you to login to your &kdesrc-build;-built &kde; desktop from your login manager. This can be disabled by using the configuration file option. Session setup does not occur while &kdesrc-build; is running in pretend mode. This driver works by setting up a custom xsession session type. This type of session should work by default with the &kdm; login manager (where it appears as a Custom session), but other login managers (such as LightDM and gdm) may require additional files installed to enable xsession support. Adding xsession support for distributions The default login managers for some distributions may require additional packages to be installed in order to support xsession logins. The Fedora &Linux; distribution requires the xorg-x11-xinit-session package to be installed for custom xsession login support. Debian and Debian-derived &Linux; distributions should support custom xsession logins, but require the option to be set in /etc/X11/Xsession.options. See also the Debian documentation on customizing the X session. For other distributions, go to . Manually adding support for xsession If there were no distribution-specific directions for your distribution in , you can manually add a Custom xsession login entry to your distribution's list of session types as follows: Adding an .xsession login session type. This procedure will likely require administrative privileges to complete. Create the file /usr/share/xsessions/kdesrc-build.desktop. Ensure the file just created has the following text: Type=XSession Exec=$HOME/.xsession Name=KDE Plasma Desktop (unstable; kdesrc-build) The $HOME entry must be replaced by the full path to your home directory (example, /home/user). The desktop entry specification does not allow for user-generic files. When the login manager is restarted, it should show a new session type, KDE Plasma Desktop (unstable; kdesrc-build) in its list of sessions, which should try to run the .xsession file installed by &kdesrc-build; if it is selected when you login. It may be easiest to restart the computer to restart the login manager, if the login manager does not track updates to the /usr/share/xsessions directory. Setting up the environment manually This documentation used to include instruction on which environment variables to set in order to load up the newly-built desktop. These instructions have been moved to an appendix (). If you intend to setup your own login support you can consult that appendix or view the sample-kde-env-master.sh file included with the &kdesrc-build; source. Module Organization and selection KDE Software Organization &kde; software is split into different components, many of which can be built by &kdesrc-build;. Understanding this organization will help you properly select the software modules that you want built. At the lowest level comes the &Qt; library, which is a very powerful, cross-platform toolkit library. &kde; is based on &Qt;, and some of the non-&kde; libraries required by &kde; are also based on &Qt;. &kdesrc-build; can build &Qt;, or use the one already installed on your system if it is a recent enough version. On top of &Qt; are required libraries that are necessary for &kde; software to work. Some of these libraries are not considered part of &kde; itself due to their generic nature, but are still essential to the &kde; Platform. These libraries are collected under a kdesupport module grouping but are not considered part of the Frameworks libraries. On top of these essential libraries come the &kde; Frameworks, sometimes abbreviated as KF5, which are essential libraries for the &kde; Plasma desktop, &kde; Applications, and other third-party software. On top of the Frameworks, come several different things: Third-party applications. These are applications that use the &kde; Frameworks or are designed to run under &kde; Plasma but are not authored by or in association with the &kde; project. Plasma, which is a full workspace desktop environment. This is what users normally see when they log-in to &kde;. The &kde; Application suite. This is a collection of useful software included with the Platform and &plasma; Desktop, grouped into individual modules, including utilities like &dolphin;, games like KSudoku, and productivity software released by &kde; such as &kontact;. Finally, there is a collection of software (also collected in modules) whose development is supported by &kde; resources (such as translation, source control, bug tracking, &etc;) but is not released by &kde; as part of Plasma or the Application suite. These modules are known as Extragear. Selecting modules to build Selecting which of the possible modules to build is controlled by the configuration file. After the global section is a list of modules to build, bracketed by module ... end module lines. An example entry for a module is shown in . Example module entry in the configuration file module kdesrc-build-git # Options for this module go here, example: repository kde:kdesrc-build make-options -j4 # Run 4 compiles at a time end module In practice, this module construct is not usually used directly. Instead most modules are specified via module-sets as described below. When using only module entries, &kdesrc-build; builds them in the order you list, and does not attempt to download any other repositories other than what you specify directly. Module Sets The &kde; source code is decomposed into a great number of relatively small Git-based repositories. To make it easier to manage the large number of repositories involved in any useful &kde;-based install, &kdesrc-build; supports grouping multiple modules and treating the group as a module set. The basic module set concept By using a module set, you can quickly declare many Git modules to be downloaded and built, as if you'd typed out a separate module declaration for each one. The repository option is handled specially to setup where each module is downloaded from, and every other option contained in the module set is copied to every module generated in this fashion. Using module sets global kde-git kde: end global module qt # Options removed for brevity end module module-set kde-support-libs kde-git automoc attica akonadi end module-set # Other modules as necessary... module kdesupport end module In a brief module set is shown. When &kdesrc-build; encounters this module set, it acts as if, for every module given in , that an individual module has been declared, with its equal to the module-set's followed immediately by the given module name. In addition, other options can be passed in a module set, which are copied to every new module that is created this way. By using module-set it is possible to quickly declare many Git modules that are all based on the same repository URL. In addition, it is possible to give module-sets a name (as shown in the example), which allows you to quickly refer to the entire group of modules from the command line. Special Support for KDE module sets The module set support described so far is general to any Git-based modules. For the &kde; Git repositories, &kdesrc-build; includes additional features to make things easier for users and developers. This support is enabled by specifying kde-projects as the for the module set. &kdesrc-build; normally only builds the modules you have listed in your configuration file, in the order you list them. But with a kde-projects module set, &kdesrc-build; can do dependency resolution of &kde;-specific modules, and in addition automatically include modules into the build even if only indirectly specified. Using kde-projects module sets # Only adds a module for juk (the kde/kdemultimedia/juk repo) module-set juk-set kde-projects juk end module-set # Adds all modules that are in kde/multimedia/*, including juk, # but no other dependencies module-set multimedia-set kde-projects kde/multimedia end module-set # Adds all modules that are in kde/multimedia/*, and all kde-projects # dependencies from outside of kde/kdemultimedia module-set multimedia-deps-set kde-projects kde/multimedia true end module-set # All modules created out of these three module sets are automatically put in # proper dependency order, regardless of the setting for include-dependencies This kde-projects module set construct is the main method of declaring which modules you want to build. All module sets use the repository and use-modules options. kde-projects module sets have a predefined value, but other types of module sets also will use the git-repository-base option. The official &kde; module database &kde;'s Git repositories allow for grouping related Git modules into collections of related modules (e.g. kdegraphics). Git doesn't recognize these groupings, but &kdesrc-build; can understand these groups, using module sets with a option set to kde-projects. &kdesrc-build; will recognize that the kde-projects repository requires special handling, and adjust the build process appropriately. Among other things, &kdesrc-build; will: Download the latest module database from the &kde; git archive. Try to find a module with the name given in the module set's setting in that database. For every module that is found, &kdesrc-build; will lookup the appropriate repository in the database, based upon the branch-group setting in effect. If a repository exists and is active for the branch group, &kdesrc-build; will automatically use that to download or update the source code. In the current database, some module groups not only have a collection of modules, but they also declare their own &git; repository. In these situations &kdesrc-build; will currently prefer the group's &git; repository instead of including the childrens' repositories. The following example shows how to use the &kde; module database to install the Phonon multimedia library. module-set media-support # This option must be kde-projects to use the module database. kde-projects # This option chooses what modules to look for in the database. phonon/phonon phonon-gstreamer phonon-vlc end module-set phonon/phonon is used since (with the current project database) &kdesrc-build; would otherwise have to decide between the group of projects called phonon or the individual project named phonon. Currently &kdesrc-build; would pick the former, which would build many more backends than needed. The following example is perhaps more realistic, and shows a feature only available with the &kde; module database: Building all of the &kde; graphics applications with only a single declaration. module-set kdegraphics # This option must be kde-projects to use the module database. kde-projects # This option chooses what modules to look for in the database. kdegraphics/libs kdegraphics/* end module-set There are two important abilities demonstrated here: &kdesrc-build; allows you to specify modules that are descendents of a given module, without building the parent module, by using the syntax module-name/*. It is actually required in this case since the base module, kdegraphics, is marked as inactive so that it is not accidentally built along with its children modules. Specifying the descendent modules allows &kdesrc-build; to skip around the disabled module. &kdesrc-build; will also not add a given module to the build list more than once. This allows us to manually set kdegraphics/libs to build first, before the rest of kdegraphics, without trying to build kdegraphics/libs twice. This used to be required for proper dependency handling, and today remains a fallback option in case the &kde; project database is missing dependency metadata. Filtering out &kde; project modules You might decide that you'd like to build all programs within a &kde; module grouping except for a given program. For instance, the kdeutils group includes a program named kremotecontrol. If your computer does not have the proper hardware to receive the signals sent by remote controls then you may decide that you'd rather not download, build, and install kremotecontrol every time you update kdeutils. You can achieve this by using the ignore-modules configuration option. Example for ignoring a kde-project module in a group module-set utils kde-projects # This option chooses what modules to look for in the database. kdeutils # This option "subtracts out" modules from the modules chosen by use-modules, above. kremotecontrol end module-set module-set graphics kde-projects # This option chooses what modules to look for in the database. extragear/graphics # This option "subtracts out" modules from the modules chosen by use-modules, above. # In this case, *both* extragear/graphics/kipi-plugins and # extragear/graphics/kipi-plugins/kipi-plugins-docs are ignored extragear/graphics/kipi-plugins end module-set Getting Started Conclusion These are the major features and concepts needed to get started with &kdesrc-build; For additional information, you could keep reading through this documentation. In particular, the list of command-line options and the table of configuration file options are useful references. The &kde; Community also maintains an online Wiki reference for how to build the source code, which refers to &kdesrc-build; and includes tips and other guidelines on how to use the tool. Script Features Feature Overview &kdesrc-build; features include: You can pretend to do the operations. If you pass or on the command line, the script will give a verbose description of the commands it is about to execute, without actually executing it. However if you've never run &kdesrc-build;, you would want to run the kdesrc-build command first in order for to work. For an even more verbose description of what &kdesrc-build; is doing, try using the option. &kdesrc-build; can (with the assistance of the &kde; FTP server) allow for speedy checkouts of some Subversion modules. If the module you are checking out has already been packaged at the website, then &kdesrc-build; will download the snapshot and prepare it for use on your computer. There is generally no need for any special preparation to perform the initial checkout of a Git module, as the entire Git repository must be downloaded anyways, so it is easy for the server to determine what to send. This is faster for you, and helps to ease the load on the kde.org anonymous &subversion; servers. Another speedup is provided by starting the build process for a module as soon as the source code for that module has been downloaded. (Available since version 1.6) Excellent support for building the &Qt; library (in case the &kde; software you are trying to build depends on a recent &Qt; not available in your distribution). &kdesrc-build; does not require a GUI present to operate. So, you can build &kde; software without needing a graphical environment. Supports setting default options for all modules (such as the compilation settings or the configuration options). Such options can normally be changed for specific modules as well. Also, &kdesrc-build; will add standard flags as appropriate to save you the trouble and possible -errors from typing them yourself. +errors from typing them yourself. Nota Bene: this does not apply when a (custom) +toolchain is configured through e.g.: +cmake-toolchain &kdesrc-build; can checkout a specific branch or tag of a module. You can also ensure that a specific revision is checked out of a module. &kdesrc-build; can automatically switch a source directory to checkout from a different repository, branch, or tag. This happens automatically when you change an option that changes what the repository &url; should be, but you must use the --src-only option to let &kdesrc-build; know that it is acceptable to perform the switch. For developers: &kdesrc-build; will remind you if you use svn+ssh:// but ssh-agent is not running, as this will lead to repeated password requests from &ssh;. Can delete the build directory of a module after its installation to save space at the expense of future compilation time. The locations for the directories used by &kdesrc-build; are configurable (even per module). Can use &sudo;, or a different user-specified command to install modules so that &kdesrc-build; does not need to be run as the super user. &kdesrc-build; runs with reduced priority by default to allow you to still use your computer while &kdesrc-build; is working. Has support for using &kde;'s tags and branches. There is support for resuming a build from a given module. You can even ignore some modules temporarily for a given build. &kdesrc-build; will show the progress of your build when using &cmake;, and will always time the build process so you know after the fact how long it took. Comes built-in with a sane set of default options appropriate for building a base &kde; single-user installation from the anonymous source repositories. Tilde-expansion for your configuration options. For example, you can specify: qtdir ~/kdesrc/build/qt Automatically sets up a build system, with the source directory not the same as the build directory, in order to keep the source directory pristine. You can specify global options to apply to every module to check out, and you can specify options to apply to individual modules as well. Forced full rebuilds, by running &kdesrc-build; with the option. You can specify various environment values to be used during the build, including KDEDIR, QTDIR, DO_NOT_COMPILE, and CXXFLAGS. Command logging. Logs are dated and numbered so that you always have a log of a script run. Also, a special symlink called latest is created to always point to the most recent log entry in the log directory. You can check out only a portion of a &kde; &subversion; module. For example, you could check out only the taglib from kdesupport. &kdesrc-build;'s build logging Logging overview Logging is a &kdesrc-build; feature whereby the output from every command that &kdesrc-build; runs is saved to a file for examination later, if necessary. This is done because it is often necessary to have the output of these programs when there is a build failure, because there are so many reasons why a build can fail in the first place. Logging directory layout The logs are always stored under the log directory. The destination of the log directory is controlled by the log-dir option, which defaults to ${source-dir}/log (where ${source-dir} is the value of the source-dir option. The in rest of this section, this value will be referred to as ${log-dir}). Under ${log-dir}, is a set of directories, one for every time that &kdesrc-build; was run. Each directory is named with the date, and the run number. For instance, the second time that &kdesrc-build; is run on May 26, 2004, it would create a directory called 2004-05-26-02, where the 2004-05-26 is for the date, and the -02 is the run number. For your convenience, &kdesrc-build; will also create a link to the logs for your latest run, called latest. So the logs for the most recent &kdesrc-build; run should always be under ${log-dir}/latest. Now, each directory for a &kdesrc-build; run will itself contain a set of directories, one for every &kde; module that &kdesrc-build; tries to build. Also, a file called build-status will be contained in the directory, which will allow you to determine which modules built and which failed. If a module itself has a submodule (such as extragear/multimedia, playground/utils, or KDE/kdelibs), then there would actually be a matching layout in the log directory. For example, the logs for KDE/kdelibs after the last &kdesrc-build; run would be found in ${log-dir}/latest/KDE/kdelibs, and not under ${log-dir}/latest/kdelibs. In each module log directory, you will find a set of files for each operation that &kdesrc-build; performs. If &kdesrc-build; updates a module, you may see filenames such as svn-co.log (for a module checkout) or svn-up.log (when updating a module that has already been checked out). If the configure command was run, then you would expect to see a configure.log in that directory. If an error occurred, you should be able to see an explanation of why in one of the files. To help you determine which file contains the error, &kdesrc-build; will create a link from the file containing the error (such as build-1.log to a file called error.log). The upshot to all of this is that to see why a module failed to build after your last &kdesrc-build;, the file you should look at first is ${log-dir}/latest/module-name/error.log. If the file error.log is empty (especially after an installation), then perhaps there was no error. Some of the tools used by the &kde; build system will sometimes mistakenly report an error when there was none. Also, some commands will evade &kdesrc-build;'s output redirection and bypass the log file in certain circumstances (normally when performing the first &subversion; checkout), and the error output in that case is not in the log file but is instead at the &konsole; or terminal where you ran &kdesrc-build;. Configuring &kdesrc-build; Overview of &kdesrc-build; configuration To use the script, you must have a file in your home directory called .kdesrc-buildrc, which describes the modules you would like to download and build, and any options or configuration parameters to use for these modules. Layout of the configuration file Global configuration The configuration file starts with the global options, specified like the following: global option-name option-value [...] end global Module configuration It is then followed by one or more module sections, specified in one of the following two forms: module module-name option-name option-value [...] end module module-set module-set-name repository kde-projects or git://host.org/path/to/repo.git use-modules module-names # Other options may also be set option-name option-value [...] end module-set Note that the second form, module sets, only works for Git-based modules. For Subversion modules, module-name must be a module from the &kde; &subversion; repository (for example, kdeartwork or kde-wallpapers), although it is possible to get around this if you manually specify the &subversion; URL. For Git modules, the module name can be essentially whatever you'd like, as long as it does not duplicate any other module name in the configuration. Keep in mind the source and build directory layout will be based on the module name if you do not use the dest-dir option. However, for Git module sets the module-names must correspond with actual git modules in the chosen . See git-repository-base or use-modules for more information. <quote>options</quote> modules There is a final type of configuration file entry, options groups, which may be given wherever a module or module-set may be used. options module-name option-name option-value [...] end options An options group may have options set for it just like a module declaration, and is associated with an existing module. Any options set these way will be used to override options set for the associated module. The associated module name must match the name given in the options declaration. Be careful of mis-typing the name. This is useful to allow for declaring an entire module-set worth of modules, all using the same options, and then using options groups to make individual changes. options groups can also apply to named module sets. This allows expert users to use a common configuration file (which includes module-set declarations) as a baseline, and then make changes to the options used by those module-sets in configuration files that use the include command to reference the base configuration. Example of using options In this example we choose to build all modules from the &kde; multimedia software grouping. However we want to use a different version of the &kmix; application (perhaps for testing a bug fix). It works as follows: module-set kde-multimedia-set repository kde-projects use-modules kde/kdemultimedia branch master end module-set # kmix is a part of kde/kdemultimedia group, even though we never named # kmix earlier in this file, &kdesrc-build; will figure out the change. options kmix branch KDE/4.12 end options Now when you run &kdesrc-build;, all of the &kde; multimedia programs will be built from the master branch of the source repository, but &kmix; will be built from the older KDE/4.12 branch. By using options you didn't have to individually list all the other &kde; multimedia programs to give them the right branch option. Note that this feature is only available in &kdesrc-build; from version 1.16, or using the development version of &kdesrc-build; after 2014-01-12. Including other configuration files Within the configuration file, you may reference other files by using the include keyword with a file, which will act as if the file referenced had been inserted into the configuration file at that point. For example, you could have something like this: global include ~/common-kdesrc-build-options # Insert specific options here. end global If you don't specify the full path to the file to include, then the file will be searched for starting from the directory containing the source file. This works recursively as well. Commonly used configuration options The following is a list of commonly-used options. Click on the option to find out more about it. To see the full list of options, see . cmake-options to define what flags to configure a module with using &cmake;. branch, to checkout from a branch instead of /trunk (for &subversion;) or master (for Git). configure-flags to define what flags to configure &Qt; with. kdedir, to set the directory to install &kde; to. make-options, to pass options to the &make; program (such as number of CPUs to use). qtdir, to set the path to &Qt;. source-dir, to change where to download the source code to. Table of available configuration options Here is a table of the various options, containing the following information: The option name A description of how &kdesrc-build; responds if the option is set in both the global section, and the module section of the configuration file while building a module. Special comments on the purpose and usage of the option. Table of Options Option-name Module -> Global Behavior Notes apidox This option was used to allow for building KDE module API documentation. It was removed in &kdesrc-build; 1.6.3 due to lack of support. Online API documentation is available from kde.org. In addition it is possible to build KDE API documentation using the kdedoxygen.sh script included in the kde-dev-scripts module. See KDE TechBase for more details. apply-qt-patches This option was removed in kdesrc-build 1.10. To get the same effect, see and the repository option. async Cannot be overridden This option used to control the asynchronous mode of operation, permitting source code updates to happen at the same time as the build process when possible, instead of waiting for all of the source code updates before starting the build process. Asynchronous is the only supported mode of operation, though you can still use separate --no-src and --no-build script runs to achieve the same effect. This option was available since the 1.6 release and removed late 2018. binpath Module setting overrides global Set this option to set the environment variable PATH while building. You cannot override this setting in a module option. The default value is the $PATH that is set when the script starts. This environment variable should include the colon-separated paths of your development toolchain. The paths $KDEDIR/bin and $QTDIR/bin are automatically added. You may use the tilde (~) for any paths you add using this option. branch Module setting overrides global Set this option to checkout from a branch of &kde; instead of the default of master (for &git; modules) or trunk (for &subversion;), where &kde; development occurs. For instance, to checkout &kde; 4.6 branch, you would set this option to 4.6. If &kdesrc-build; fails to properly download a branch with this option, you may have to manually specify the &url; to download from using the module-base-path or override-url options. For most &kde; modules you probably wish to use the branch-group option instead and use this option for case-by-case exceptions. branch-group Module setting overrides global Set this option to a general group from which you want modules to be chosen. For supported &git; module types, &kdesrc-build; will determine the actual branch to use automatically based on rules encoded by the &kde; developers (these rules may be viewed in the kde-build-metadata source repository in your source directory). After a branch is determined that branch is used as if you had specified it yourself using the branch option. This is useful if you're just trying to maintain up-to-date on some normal development track without having to pay attention to all the branch name changes. The current branch groups (as of 2013-08-11) are: stable-qt4, for tracking bugfixes to the &Qt; 4-based &kde; libraries and applications. latest-qt4, for tracking development and new features for the &Qt; 4-based &kde; libraries and applications. kf5-qt5, for tracking bleeding-edge development for the upcoming &Qt; 5-based &kde; Frameworks 5, &plasma; Workspace 2, &etc; Note that if you do choose a branch yourself, that it will override this setting. The same is true of other specific branch selection options such as tag. This option was added in &kdesrc-build; 1.16-pre2. This option only applies to kde-projects &git; modules (the common case). See also . build-dir Module setting overrides global Use this option to change the directory to contain the built sources. There are three different ways to use it: Relative to the &kde; &subversion; source directory (see the source-dir option). This is the default, and is selected if you type a directory name that does not start with a tilde (~) or a slash (/). The default value is build. Absolute path. If you specify a path that begins with a /, then that path is used directly. For example, /tmp/kde-obj-dir/. Relative to your home directory. If you specify a path that begins with a ~, then the path is used relative to your home directory, analogous to the shell's tilde-expansion. For example, ~/builddir would set the build directory to /home/user-name/builddir. Perhaps surprisingly, this option can be changed per module. build-when-unchanged Module setting overrides global Use this option in order to control whether &kdesrc-build; always tries to build a module that has not had any source code updates. By setting to true, &kdesrc-build; always attempts the build phase for a module, even if the module did not have any source code updates. This is the default setting since it is more likely to lead to a correct build. By setting to false, &kdesrc-build; will only attempt to run the build phase for a module if the module has a source code update, or in other situations where it is likely that a rebuild is actually required. This can save time, especially if you run &kdesrc-build; daily, or more frequently. This feature is provided as an optimization only. Like many other optimizations, there are trade-offs for the correctness of your installation. For instance, changes to the qt or kdelibs modules may cause a rebuild of other modules to be necessary, even if the source code doesn't change at all. checkout-only Module setting overrides global This option has been removed in &kdesrc-build; 18.10, as the use cases it supported are no longer significant issues for &kde; contributors. cmake-generator Module setting overrides global Use this option to specify which generator to use with &cmake;. Currently both Ninja and Unix Makefiles are supported. Invalid (unsupported) values are ignored and treated as if unset. If not set Unix Makefiles will be used by default. Note that if a valid generator is also specified through cmake-options it will override the value for cmake-generator. + +cmake-toolchain +Module setting overrides global +Use this option to specify a toolchain file to use with &cmake;. + +When a valid toolchain file is configured, &kdesrc-build; will +no longer set environment variables automatically. +You can use &set-env;, &binpath; and &libpath; to fix up the environment +manually if your toolchain file does not work out of the box with +&kdesrc-build;. Refer to the overview +of standard flags added by &kdesrc-build; for more information. + +Note that if a valid toolchain is also specified through +cmake-options it will override the +value for cmake-toolchain. + + cmake-options Appends to global options for the default buildsystem, overrides global for other buildsystems. Use this option to specify what flags to pass to &cmake; when creating the build system for the module. When this is used as a global option, it is applied to all modules that this script builds. When used as a module option, it is added to the end of the global options. This allows you to specify common &cmake; options in the global section. This option does not apply to qt (which does not use &cmake;). Use configure-flags instead. If a valid generator is specified among the listed options it will override the value of cmake-generator. Invalid (unsupported) generators are ignored and will not be passed to &cmake;. +If a valid toolchain file is specified among the listed options it will +override the value of +cmake-toolchain. Invalid +toolchains are ignored and will not be passed to &cmake;. + + Since these options are passed directly to the &cmake; command line, they should be given as they would be typed into &cmake;. For example: cmake-options -DCMAKE_BUILD_TYPE=RelWithDebInfo Since this is a hassle, &kdesrc-build; takes pains to ensure that as long as the rest of the options are set correctly, you should be able to leave this option blank. (In other words, required &cmake; parameters are set for you automatically) colorful-output Cannot be overridden Set this option to false to disable the colorful output of &kdesrc-build;. This option defaults to true. Note that &kdesrc-build; will not output the color codes to anything but a terminal (such as xterm, &konsole;, or the normal &Linux; console). configure-flags Appends to global options for the default buildsystem, overrides global for other buildsystems. Use this option to specify what flags to pass to ./configure when creating the build system for the module. When this is used as a global-option, it is applied to all modules that this script builds. This option only works for qt. To change configuration settings for KDE modules, see cmake-options. custom-build-command Module setting overrides global (build system option) This option can be set to run a different command (other than make, for example) in order to perform the build process. &kdesrc-build; should in general do the right thing, so you should not need to set this option. However it can be useful to use alternate build systems. The value of this option is used as the command line to run, modified by the make-options option as normal. cxxflags Appends to global options for the default buildsystem, overrides global for other buildsystems. Use this option to specify what flags to use for building the module. This option is specified here instead of with configure-flags or cmake-options because this option will also set the environment variable CXXFLAGS during the build process. Note that for &kde; 4 and any other modules that use &cmake;, it is necessary to set the CMAKE_BUILD_TYPE option to none when configuring the module. This can be done using the cmake-options option. dest-dir Module setting overrides global Use this option to change the name a module is given on disk. For example, if your module was extragear/network, you could rename it to extragear-network using this option. Note that although this changes the name of the module on disk, it is not a good idea to include directories or directory separators in the name as this will interfere with any build-dir or source-dir options. disable-agent-check Cannot be overridden Normally if you are using &ssh; to download the &subversion; sources (such as if you are using the svn+ssh protocol), &kdesrc-build; will try and make sure that if you are using ssh-agent, it is actually managing some &ssh; identities. This is to try and prevent &ssh; from asking for your pass phrase for every module. You can disable this check by setting to true. do-not-compile Module setting overrides global Use this option to select a specific set of directories not to be built in a module (instead of all of them). The directories not to build should be space-separated. Note that the sources to the programs will still be downloaded. This option currently only works for &cmake;-based modules. It works by passing appropriate -DBUILD_foo options when running &cmake;, but this can also be done using cmake-options. email-address Cannot be overridden This option was removed in &kdesrc-build; 1.14. email-on-compile-error Cannot be overridden This option was removed in &kdesrc-build; 1.14. inst-apps This option was removed in version 1.10 git-desired-protocol Cannot be overridden This option only applies to modules from a &kde; project repository. What this option actually does is configure which network protocol to -prefer when updating source code for these modules. Normally the very-efficient +prefer when pushing source code for these modules. Normally the very-efficient git protocol is used, but this may be blocked in some networks (e.g. corporate intranets, public Wi-Fi). An alternative protocol -which is much better supported is the HTTP protocol used for +which is much better supported is the https protocol used for Internet web sites. If you are using one of these constrained networks you can set this -option to http to prefer HTTP +option to http to prefer https communications instead. You may also need the http-proxy option if an HTTP proxy is also needed for network traffic. In any other situation you should not set this option as the default protocol is most efficient. -This option was added in &kdesrc-build; 1.16. +This option was added in &kdesrc-build; 1.16. Prior to 20.06 this option +was used to configure the fetch URL instead of the push URL. As of 20.06 +https is always used when updating KDE projects. git-repository-base Cannot be overridden This option, added in version 1.12.1, is used to create a short name to reference a specific Git repository base URL in later module set declarations, which is useful for quickly declaring many Git modules to build. You must specify two things (separated by a space): The name to assign to the base URL, and the actual base URL itself. For example: global # other options # This is the common path to all anonymous Git server modules. git-repository-base kde-git kde: end global # Module declarations module-set # Now you can use the alias you defined earlier, but only # in a module-set. repository kde-git use-modules module1.git module2.git end module-set The module-set's use-modules option created two modules internally, with &kdesrc-build; behaving as if it had read: module module1 repository kde:module1.git end module module module2 repository kde:module2.git end module The kde: &git; repository prefix used above is a shortcut which will be setup by &kdesrc-build; automatically. See the TechBase URL Renaming article for more information. Note that unlike most other options, this option can be specified multiple times in order to create as many aliases as necessary. It is not required to use this option to take advantage of module-set, this option exists to make it easy to use the same repository across many different module sets. git-user Module setting overrides global This option is intended for &kde; developers. If set, it will be used to automatically setup identity information for the &git; source control software for newly downloaded &git; modules (including the vast majority of &kde; modules). Specifically, the user's name and email fields for each new &git; repository are filled in to the values set by this option. The value must be specified in the form . For instance, a developer named Foo Barbaz with the email address foo@abc.xyz would use: git-user Foo Barbaz <foo@abc.xyz> This option was introduced in &kdesrc-build; 15.09. http-proxy Module setting overrides global This option, if set, uses the specified URL as a proxy server to use for any HTTP network communications (for example, when downloading snapshots for new modules, or the KDE project database). In addition, &kdesrc-build; will try to ensure that the tools it depends on also use that proxy server, if possible, by setting the http_proxy environment variable to the indicated server, if that environment variable is not already set. This option was introduced with &kdesrc-build; 1.16. ignore-kde-structure Module setting overrides global This option is used to store the source and the build files directly in the name of the module. For example, source/extragear/network/telepathy/ktp-text-ui becomes source/ktp-text-ui. This option is disabled by default. If you want to enable this option you need to set it to true. This option was introduced with &kdesrc-build; 1.16. ignore-modules Can't be overridden Modules named by this option, which would be chosen by &kdesrc-build; due to a use-modules option, are instead skipped entirely. Use this option when you want to build an entire kde-projects project grouping except for some specific modules. The option value does not necessarily have to name the module directly. Any module that has full consecutive parts of its &kde; projects module path match one of the option values will be ignored, so you can ignore multiple modules this way. For example, an option value of libs would result in both kde/kdegraphics/libs and playground/libs being excluded (though not kde/kdelibs since the full part kdelibs is what is compared). See also . This option was introduced with &kdesrc-build; 1.16. include-dependencies Module setting overrides global This option, when set to true requests that &kdesrc-build; also include known dependencies of this module in its build, without requiring you to mention those dependencies (even indirectly). This option only works for kde-project-based modules, and requires that the metadata maintained by the &kde; developers is accurate for your selected branch-group. This option is enabled by default, to support building applications that need versions of &Qt; or &plasma; more recent than supported on common operating systems. install-after-build Module setting overrides global This option is used to install the package after it successfully builds. This option is enabled by default. If you want to disable this, you need to set this option to false in the configuration file. You can also use the command line flag. install-environment-driver Cannot be overridden By default, &kdesrc-build; will install a shell script that can be sourced in a user's profile setup scripts to easily establish needed environment variables to run the Plasma desktop built by &kdesrc-build;. This driver will alter the following files: $XDG_CONFIG_HOME/kde-env-master.sh (normally found at ~/.config/kde-env-master.sh). $XDG_CONFIG_HOME/kde-env-user.sh (normally found at ~/.config/kde-env-user.sh). The kde-env-user.sh is optional. It is intended for user customizations (see the Troubleshooting and Debugging section of the &kde; UserBase for examples of customizable settings), but these settings can be set elsewhere by the user in their existing profile setup scripts. You can disable this feature by setting this option to false, and ensuring that the install-session-driver option is also disabled. This option was introduced with &kdesrc-build; 17.08. &kdesrc-build; will not overwrite your existing files (if present) unless you also pass the command-line option. install-session-driver Cannot be overridden If enabled, &kdesrc-build; will try to install a driver for the graphical login manager that allows you to login to your &kdesrc-build;-built &kde; desktop. This driver will alter the following files: ~/.xsession $XDG_CONFIG_HOME/kde-env-master.sh (normally found at ~/.config/kde-env-master.sh). $XDG_CONFIG_HOME/kde-env-user.sh (normally found at ~/.config/kde-env-user.sh). If you maintain your own login driver then you can disable this feature by setting this option to false. If enabled, this feature also enables the install-environment-driver feature. This option was introduced with &kdesrc-build; 1.16. &kdesrc-build; will not overwrite your existing files (if present) unless you also pass the command-line option. kdedir Module setting overrides global This option sets the directory that &kde; will be installed to after it is built. It defaults to ~/kde. If you change this to a directory needing root access, you may want to read about the make-install-prefix option as well. kde-languages Cannot be overridden This option was removed in &kdesrc-build; 18.10 due to inability to maintain the underlying &subversion; support that is required. libpath Module setting overrides global Set this option to set the environment variable LD_LIBRARY_PATH while building. You cannot override this setting in a module option. The default value is blank, but the paths $KDEDIR/lib and $QTDIR/lib are automatically added. You may use the tilde (~) for any paths you add using this option. log-dir Module setting overrides global Use this option to change the directory used to hold the log files generated by the script. make-install-prefix Module setting overrides global Set this variable to a space-separated list, which is interpreted as a command and its options to precede the make command used to install modules. This is useful for installing packages with &sudo; for example, but please be careful while dealing with root privileges. make-options Module setting overrides global (build system option) Set this variable in order to pass command line options to the make command. This is useful for programs such as distcc or systems with more than one processor core. Note that not all supported build systems use make. For build systems that use ninja for build (such as the Meson build system), see the ninja-options setting. manual-build Module setting overrides global Set the option value to true to keep the build process from attempting to build this module. It will still be kept up-to-date when updating from &subversion;. This option is exactly equivalent to the command line option. manual-update Module setting overrides global Set the option value to true to keep the build process from attempting to update (and by extension, build or install) this module. If you set this option for a module, then you have essentially commented it out. module-base-path Module setting overrides global Set this option to override &kdesrc-build;'s default directory path to the module in question. This can be used, for example, to pull specific branches or tagged versions of libraries. The &kde; Source Viewer is invaluable in helping to pick the right path. Note that &kdesrc-build; constructs the final path according to the following template: $svn-server/home/kde/$module-base-path. The default value is either trunk/$module or trunk/KDE/$module, depending on the module name. Use the branch or tag options instead whenever they are applicable. niceness Cannot be overridden Set this option to a number between 20 and 0. The higher the number, the lower a priority &kdesrc-build; will set for itself, i.e. the higher the number, the "nicer" the program is. The default is 10. ninja-options Module setting overrides global (build system option) Set this variable in order to pass command line options to the ninja build command. This can be useful to enable verbose output or to manually reduce the number of parallel build jobs that ninja would use. Note that this setting only controls ninja when used by &kdesrc-build;. The &Qt; webengine module uses ninja indirectly, but only officially supports being built by make. In this situation, you can set NINJAFLAGS as a way to have make pass the appropriate flags when it later calls ninja, by using make-options. options qtwebengine # Restrict make and ninja to using no more than 6 separate compile jobs even # when more CPU is available, to avoid running out of memory -j6 NINJAFLAGS=-j6 end options no-svn Module setting overrides global If this option is set to true then &kdesrc-build; will not update the source code for the module automatically. It will still try to build the module if it normally would have tried anyways. no-rebuild-on-fail This option was removed in version 1.10, since this behavior no longer helps due to fixes in the underlying build system. override-build-system Module setting overrides global This is an advanced option, added in &kdesrc-build; 1.16. Normally &kdesrc-build; will detect the appropriate build system to use for a module after it is downloaded. This is done by checking for the existence of specific files in the module's source directory. Some modules may include more than one required set of files, which could confuse the auto-detection. In this case you can manually specify the correct build type. Currently supported build types that can be set are: KDE Used to build &kde; modules. In reality it can be used to build almost any module that uses &cmake; but it is best not to rely on this. Qt Used to build the &Qt; library itself. qmake Used to build &Qt; modules that use qmake-style .pro files. generic Used to build modules that use plain Makefiles and that do not require any special configuration. autotools This is the standard configuration tool used for most Free and open-source software not in any of the other categories. meson This is a relatively new tool gaining popularity as a replacement for the autotools and may be required for some non-&kde; modules. override-url Module setting overrides global If you set this option, &kdesrc-build; will use its value as the &url; to pass to &subversion; completely unchanged. You should generally use this if you want to download a specific release but &kdesrc-build; cannot figure out what you mean using branch. persistent-data-file Cannot be overridden Use this option to change where &kdesrc-build; stores its persistent data. The default is to store this data in a file called .kdesrc-build-data placed in the same directory as the configuration file in use. If you have multiple available configurations in the same directory you may want to manually set this option so that the different configurations do not end up with conflicting persistent data. This option was added with &kdesrc-build; 1.15. prefix Module setting overrides global This option controls where to install the module (normally the setting is used). Using this option allows you to install a module to a different directory than where the KDE Platform libraries are installed, such as if you were using &kdesrc-build; only to build applications. You can use ${MODULE} or $MODULE in the path to have them expanded to the module's name. purge-old-logs Module setting overrides global This option controls whether old log directories are automatically deleted or not. The default value is true. qmake-options Module setting overrides global Any options specified here are passed to the qmake command, for modules that use the qmake build system. For instance, you can use the PREFIX=/path/to/qt option to qmake to override where it installs the module. This option was added to &kdesrc-build; 1.16. qtdir Module setting overrides global Set this option to set the environment variable QTDIR while building. If you do not specify this option, &kdesrc-build; will assume that &Qt; is provided by the operating system. remove-after-install Module setting overrides global If you are low on hard disk space, you may want to use this option in order to automatically delete the build directory (or both the source and build directories for one-time installs) after the module is successfully installed. Possible values for this option are: none - Do not delete anything (This is the default). builddir - Delete the build directory, but not the source. all - Delete both the source code and build directory. Note that using this option can have a significant detrimental impact on both your bandwidth usage (if you use all) and the time taken to compile &kde; software, since &kdesrc-build; will be unable to perform incremental builds. repository Module setting overrides global This option was introduced with version 1.10, and is used to specify the &git; repository to download the source code for the module. &Qt; (and therefore qt) would need this option, as well as various &kde; modules that are in the process of conversion to use &git;. revision Module setting overrides global If this option is set to a value other than 0 (zero), &kdesrc-build; will force the source update to bring the module to the exact revision given, even if options like branch are in effect. If the module is already at the given revision then it will not be updated further unless this option is changed or removed from the configuration. This option did not work for git-based modules (including kde-projects modules) until &kdesrc-build; version 1.16. run-tests Module setting overrides global (build system option) If set to true, then the module will be built with support for running its test suite, and the test suite will be executed as part of the build process. &kdesrc-build; will show a simple report of the test results. This is useful for developers or those who want to ensure their system is setup correctly. set-env Module setting overrides global This option accepts a space-separated set of values, where the first value is the environment variable to set, and the rest of the values is what you want the variable set to. For example, to set the variable RONALD to McDonald, you would put in the appropriate section this command: set-env RONALD McDonald This option is special in that it can be repeated without overriding earlier set-env settings in the same section of the configuration file. This way you can set more than one environment variable per module (or globally). source-dir Module setting overrides global This option is used to set the directory on your computer to store the &kde; &subversion; sources at. If you do not specify this value, the default is ~/kdesrc. You may use the tilde (~) to represent the home directory if using this option. ssh-identity-file Cannot be overridden Set this option to control which private SSH key file is passed to the ssh-add command when &kdesrc-build; is downloading source code from repositories that require authentication. See also: . This option was added in version 1.14.2. stop-on-failure Module setting overrides global Set this option value to true to cause the script to stop execution after an error occurs during the build or install process. This option is off by default. svn-server Module setting overrides global This option is used to set the server used to check out from &subversion;. The default is the anonymous &subversion; repository, svn://anonsvn.kde.org/ If you are developing for KDE, use the &subversion; repository that was provided to you when you received your developer account, instead of the anonymous repository. tag Module setting overrides global Use this option to download a specific release of a module. Note: The odds are very good that you do not want to use this option. &kde; releases are available in tarball form from the &kde; download site. This option has only been supported for git-based modules since &kdesrc-build; 1.16. use-clean-install Module setting overrides global (build system option) Set this option to true in order to have &kdesrc-build; run make uninstall directly before running make install. This can be useful in ensuring that there are not stray old library files, &cmake; metadata, etc. that can cause issues in long-lived &kde; installations. However this only works on build systems that support make uninstall. This option was added with &kdesrc-build; 1.12, but was not documented until &kdesrc-build; 1.16. use-cmake This option was removed in &kdesrc-build; 1.4 as all &kde; 4 modules require &cmake;, and &cmake; use is not permitted on any other modules. use-idle-io-priority Cannot be overridden This option, added in &kdesrc-build; 1.12, will cause a lower priority to be used for disk and other I/O usage, which can significantly improve the responsiveness of the rest of the system at the expense of slightly longer running times for &kdesrc-build;. The default is to be disabled, to enable the lower disk priority set this to true. use-modules Can only use in module-set This option, added in &kdesrc-build; 1.12.1, allows you to easily specify many different modules to build at the same point in the configuration file. This option must be used within a module-set. Every identifier passed to this option is internally converted to a &kdesrc-build; module, with a option set to the module-set's repository combined with the identifier name in order to setup the final repository to download from. All other options that are assigned in the module-set are also copied to the generated modules unaltered. The order that modules are defined in this option is important, because that is also the order that &kdesrc-build; will process the generated modules when updating, building, and installing. All modules defined in the given module-set will be handled before &kdesrc-build; moves to the next module after the module-set. If you need to change the options for a generated module, simply declare the module again after it is defined in the module-set, and set your options as needed. Although you will change the options set for the module this way, the module will still be updated and built in the order set by the module-set (i.e. you can't reorder the build sequence doing this). The name to use for the module if you do this is simply the name that you passed to , with the exception that any .git is removed. See and git-repository-base for a description of its use and an example. use-qt-builddir-hack Module setting overrides global This option has been removed due to improvements in the &Qt; build system. use-stable-kde Can't be overridden This option is deprecated and will be removed (again) in a future release. Please use the instead, which offers more selectivity.
Command Line Options and Environment Variables Command Line Usage &kdesrc-build; is designed to be run as follows: kdesrc-build --options modules to build If no modules to build are specified on the command line, then kdesrc-build will build all modules defined in its configuration file, in the order listed in that file (although this can be modified by various configuration file options). Commonly used command line options The full list of command line options is given in . The most-commonly used options include: (or ) This option causes &kdesrc-build; to indicate what actions it would take, without actually really implementing them. This can be useful to make sure that the modules you think you are building will actually get built. This option forces &kdesrc-build; to build the given modules from an absolutely fresh start point. Any existing build directory for that module is removed and it is rebuilt. This option is useful if you have errors building a module, and sometimes is required when &Qt; or &kde; libraries change. This option skips the source update process. You might use it if you have very recently updated the source code (perhaps you did it manually or recently ran &kdesrc-build;) but still want to rebuild some modules. This option is similar to above, but this time the build process is skipped. Specifying modules to build In general, specifying modules to build is as simple as passing their module name as you defined it in the configuration file. You can also pass modules that are part of a module set, either as named on use-modules, or the name of the entire module set itself, if you have given it a name. In the specific case of module sets based against the KDE project database, &kdesrc-build; will expand module name components to determine the exact module you want. For example, &kdesrc-build;'s KDE project entry locates the project in extragear/utils/kdesrc-build. You could specify any of the following to build &kdesrc-build;: % kdesrc-build % kdesrc-build % kdesrc-build The commands in the previous example preceded the module-name with a +. This forces the module name to be interpreted as a module from the KDE project database, even if that module hasn't been defined in your configuration file. Be careful about specifying very generic projects (e.g. extragear/utils by itself), as this can lead to a large amount of modules being built. You should use the option before building a new module set to ensure it is only building the modules you want. Supported Environment Variables &kdesrc-build; does not use environment variables. If you need to set environment variables for the build or install process, please see the set-env option. Supported command-line parameters The script accepts the following command-line options: --help Only display simple help on this script. --version (or -v) Display the program version. --show-info Displays information about &kdesrc-build; and the operating system, that may prove useful in bug reports or when asking for help in forums or mailing lists. Available since version 18.11. --initial-setup Has &kdesrc-build; perform the one-time initial setup necessary to prepare the system for &kdesrc-build; to operate, and for the newly-installed &kde; software to run. This includes: Installing known dependencies (on supported &Linux; distributions) Adding required environment variables to ~/.bashrc Setting up a configuration file Available since version 18.11. --author Display contact information for the author. --color Enable colorful output. (This is the default for interactive terminals). --nice=value This value adjusts the computer CPU priority requested by &kdesrc-build;, and should be in the range of 0-20. 0 is highest priority (because it is the least nice), 20 is lowest priority. &kdesrc-build; defaults to 10. --no-async This option used to disables the asynchronous mode of updating. Due to lack of testing and the reliability of the asynchronous mode, this option has been disabled and asynchronous builds are always in effect. --no-color Disable colorful output. --pretend (or -p) &kdesrc-build; will run through the update and build process, but instead of performing any actions to update or build, will instead output what the script would have done (e.g. what commands to run, general steps being taken, etc.). Simple read-only commands (such as reading file information) may still be run to make the output more relevant (such as correctly simulating whether source code would be checked out or updated). This option requires that some needed metadata is available, which is normally automatically downloaded, but downloads are disabled in pretend mode. If you've never run &kdesrc-build; (and therefore, don't have this metadata), you should run kdesrc-build to download the required metadata first. --quiet (or -q) Do not be as noisy with the output. With this switch only the basics are output. --really-quiet Only output warnings and errors. --verbose Be very descriptive about what is going on, and what &kdesrc-build; is doing. --src-only (or --svn-only) Only perform the source update. (The --svn-only is only supported for compatibility with older scripts). --build-only Only perform the build process. --install-only Only perform the install process. --metadata-only Only perform the metadata download process. &kdesrc-build; normally handles this automatically, but you might manually use this to allow the command line option to work. --rebuild-failures Use this option to build only those modules which failed to build on a previous &kdesrc-build; run. This is useful if a significant number of failures occurred mixed with successful builds. After fixing the issue causing the build failures you can then easily build only the modules that failed previously. Note that the list of previously-failed modules is reset every time a &kdesrc-build; run finishes with some module failures. However it is not reset by a completely successful build, so you can successfully rebuild a module or two and this flag will still work. This option was added for &kdesrc-build; 15.09. --include-dependencies --no-include-dependencies This option causes &kdesrc-build; to automatically include other &kde; and &Qt; modules in the build, if required for the modules you have requested to build on the command line or in your configuration file. The modules that are added are as recorded within the &kde; source code management system. See . The corresponding configure file option is include-dependencies. You can also use --no-include-dependencies, which turns off automatic inclusion of additional dependency modules. --ignore-modules Do not include the modules passed on the rest of the command line in the update/build process (this is useful if you want to build most of the modules in your configuration file and just skip a few). --no-src (or --no-svn) Skip contacting the &subversion; server. (The --no-svn parameter is only supported for compatibility with older versions of the script). --no-build Skip the build process. --no-metadata Do not automatically download the extra metadata needed for &kde; git modules. The source updates for the modules themselves will still occur unless you pass --no-src as well. This can be useful if you are frequently re-running &kdesrc-build; since the metadata does not change very often. But note that many other features require the metadata to be available. You might want to consider running &kdesrc-build; with the --metadata-only option one time and then using this option for subsequent runs. --no-install Do not automatically install packages after they are built. --no-build-when-unchanged --force-build This option explicitly disables skipping the build process (an optimization controlled by the build-when-unchanged option). This is useful for making &kdesrc-build; run the build when you have changed something that &kdesrc-build; cannot check. --force-build performs the exact same function, and is perhaps easier to remember. --debug Enables debug mode for the script. Currently this means that all output will be dumped to the standard output in addition to being logged in the log directory like normal. Also, many functions are much more verbose about what they are doing in debugging mode. --query=mode This command causes &kdesrc-build; to query a parameter of the modules in the build list (either passed on the command line or read in from the configuration file), outputting the result to screen (one module per line). This option must be provided with a query mode, which should be one of the following: , which causes &kdesrc-build; to output the full path to where the module's source code is stored. , which causes &kdesrc-build; to output the full path to where the module build process occurs. , which causes &kdesrc-build; to output the full path to where the module will be installed. , which causes &kdesrc-build; to output the location of the module within the hierarchy of KDE source code repositories. See for more information on this hierarchy. , which causes &kdesrc-build; to output the resolved git branch that will be used for each module, based on the tag, branch and branch-group settings in effect. Otherwise, option names that are valid for modules in the configuration file can be used, the resolved value of which will be listed for each module. If a single module is passed on the command line, then the output is simply the value of the parameter being queried. If multiple (or no) modules are passed on the command line, then each line is prefixed by the name of the module. Either way, &kdesrc-build; stops running once each value is output. This option was added with &kdesrc-build; 16.05. For example, the command kdesrc-build --query branch kactivities kdepim might end up with output like: kactivities: master kdepim: master --no-rebuild-on-fail Do not try to rebuild modules that have failed building from scratch. &kdesrc-build; will never try to do this to a module that already was tried to be built from scratch. --refresh-build Recreate the build system and make from scratch. --reconfigure Run cmake (for &kde; modules) or configure (for &Qt;) again, without cleaning the build directory. You should not normally have to specify this, as &kdesrc-build; will detect when you change the relevant options and automatically re-run the build setup. This option is implied if --refresh-build is used. --resume-from This option is used to resume the build starting from the given module, which should be the next option on the command line. You should not specify other module names on the command line. This option formerly added --no-src, but does not any longer (since &kdesrc-build; 1.13). If you want to avoid source updates when resuming, simply pass in addition to the other options. See also: and . You would prefer to use this command line option if you have fixed the build error and want &kdesrc-build; to complete the build. --resume-after This option is used to resume the build starting after the given module, which should be the next option on the command line. You should not specify other module names on the command line. This option formerly added --no-src, but does not any longer (since &kdesrc-build; 1.13). If you want to avoid source updates when resuming, simply pass in addition to the other options. See also: and . You would prefer to use this command line option if you have fixed the build error and have also built and installed the module yourself, and want &kdesrc-build; to start again with the next module. --resume This option can be used to run &kdesrc-build; after it has had a build failure. It resumes the build from the module that failed, using the list of modules that were waiting to be built before, and disables source and metadata updates as well. The use case is when a simple mistake or missing dependency causes the build failure. Once you correct the error you can quickly get back into building the modules you were building before, without fiddling with and . This is even handier with the command line argument, especially if you're initially setting up your development environment. This option was added with &kdesrc-build; 1.16. --stop-before This command line option is used to stop the normal build process just before a module would ordinarily be built. For example, if the normal build list was moduleAmoduleBmoduleC, then would cause &kdesrc-build; to only build moduleA. This command line option was added with &kdesrc-build; 1.16. --stop-after This command line option is used to stop the normal build process just after a module would ordinarily be built. For example, if the normal build list was moduleAmoduleBmoduleC, then would cause &kdesrc-build; to build moduleA and moduleB. This command line option was added with &kdesrc-build; 1.16. --stop-on-failure This option causes the build to abort as soon as a failure occurs. Useful when you're setting up your initial development environment. Without this flag, &kdesrc-build; will try to press on with the rest of the modules in the build to avoid wasting time in case the problem is with a single module. This option was added with &kdesrc-build; 1.16. See also the stop-on-failure option. --rc-file This interprets the next command line parameter as the file to read the configuration options from. The default value for this parameter is kdesrc-buildrc (checked in the current directory) if it is present, or ~/.kdesrc-buildrc otherwise. See also . --print-modules Takes all actions up to and including dependency reordering of the modules specified on the command line (or configuration file), prints the modules that would be processed one per line, and then exits without further action. The kde-project metadata is downloaded first (though, see or ). The output is not fully compatible with usage by scripts as other output messages may be generated until the module list is shown. This is mostly just useful for quickly determining what &kdesrc-build; understands a module's dependencies to be, which means it's only useful for kde-projects modules. This option is also compatible with , , , . --list-build Lists the modules that would be built, in the order in which they would be built. If applicable, the output listing also mentions which commit/branch/tag would be selected for checkout. This option is similar to . For more detailed information on how modules relate to each other, see also: . --dependency-tree Prints out dependency information on the modules that would be built using a tree format (recursive). Listed information also includes which specific commit/branch/tag is depended on and whether or not the dependency would be built. Note: the generated output may become quite large for applications with many dependencies. --run This option interprets the next item on the command line as a program to run, and &kdesrc-build; will then finish reading the configuration file, update the environment as normal, and then execute the given program. This will not work to start a shell with the &kdesrc-build; environment in most cases however, since interactive shells typically reset at least part of the environment variables (such as PATH and KDEDIRS) in the startup sequence. If you want to see the environment used by &kdesrc-build;, you can run the printenv command: $ kdesrc-build --run printenv KDE_SESSION_VERSION=4 SDL_AUDIODRIVER=alsa LANGUAGE= XCURSOR_THEME=Oxygen_Blue LESS=-R -M --shift 5 QMAIL_CONTROLDIR=/var/qmail/control ... etc. --prefix=</path/to/kde> This allows you to change the directory that &kde; will be installed to from the command line. This option implies --reconfigure, but using --refresh-build may still be required. --revision This option causes &kdesrc-build; to checkout a specific numbered revision for each &subversion; module, overriding any branch, tag, or revision options already set for these modules. This option is likely not a good idea, and is only supported for compatibility with older scripts. --build-system-only This option causes &kdesrc-build; to abort building a module just before the make command would have been run. This is supported for compatibility with older versions only, this effect is not helpful for the current &kde; build system. --install If this is the only command-line option, it tries to install all of the modules contained in log/latest/build-status. If command-line options are specified after --install, they are all assumed to be modules to install (even if they did not successfully build on the last run). --no-snapshots Supplying this option causes &kdesrc-build; to always perform a normal initial checkout of a module instead of using a quick-start snapshot (only available for Git modules from the kde-projects repository). Note that this option should only be used if there is a failure using snapshots, as the quick-start snapshot reduces load on the KDE source repositories. Module snapshots are real checkouts. You should not need to specify this option, it is only a troubleshooting aid. --delete-my-patches This option is used to let &kdesrc-build; delete source directories that may contain user data, so that the module can be re-downloaded. This would normally only be useful for &kde; developers (who might have local changes that would be deleted). This is currently only used to checkout modules that have been converted from &subversion; to &git;. You should not use this option normally, &kdesrc-build; will prompt to be re-run with it if it is needed. --delete-my-settings This option is used to let &kdesrc-build; overwrite existing files which may contain user data. This is currently only used for xsession setup for the login manager. You should not use this option normally, &kdesrc-build; will prompt to be re-run with it if it is needed. --<option-name>= You can use this option to override an option in your configuration file for every module. For instance, to override the log-dir option, you would do: --log-dir=/path/to/dir. This feature can only be used for option names already recognized by &kdesrc-build;, that are not already supported by relevant command line options. --set-module-option-value=<module-name>,<option-name>,<option-value> You can use this option to override an option in your configuration file for a specific module. Any other command-line options are assumed to be modules to update and build. Please, do not mix building with installing. Using &kdesrc-build; Preface Normally using &kdesrc-build; after you have gone through is as easy as doing the following from a terminal prompt: % kdesrc-build &kdesrc-build; will then download the sources for &kde;, try to configure and build them, and then install them. Read on to discover how &kdesrc-build; does this, and what else you can do with this tool. Basic &kdesrc-build; features qt support &kdesrc-build; supports building the &Qt; toolkit used by &kde; software as a convenience to users. This support is handled by a special module named qt. &Qt; is developed under a separate repository from &kde; software located at http://code.qt.io/cgit/qt/. In order to build &Qt;, you should make sure that the qtdir setting is set to the directory you'd like to install &Qt; to, as described in . You should then ensure that the qt module is added to your .kdesrc-buildrc, before any other modules in the file. If you are using the sample configuration file, you can simply uncomment the existing qt module entry. Now you should verify the repository option and branch options are set appropriately: The first option is to build &Qt; using a mirror maintained on the &kde; source repositories (no other changes are applied, it is simply a clone of the official source). This is highly recommended due to occasional issues with cloning the full &Qt; module from its official repository. You can set the option for the qt module to kde:qt to use this option. Otherwise, to build the standard &Qt;, set your option to git://gitorious.org/qt/qt.git. Note that you may experience problems performing the initial clone of &Qt; from this repository. In both cases, the branch option should be set to master (unless you'd like to build a different branch). Standard flags added by &kdesrc-build; +Nota Bene: this section does not apply to modules for which you have +configured a custom toolchain, using e.g. +cmake-toolchain. + To save you time, &kdesrc-build; adds some standard paths to your environment for you: The path to the &kde; and &Qt; libraries is added to the LD_LIBRARY_PATH variable automatically. This means that you do not need to edit &libpath; to include them. The path to the &kde; and &Qt; development support programs are added to the PATH variable automatically. This means that you do not need to edit &binpath; to include them. The path to the &kde;-provided pkg-config is added automatically to PKG_CONFIG_PATH. This means that you do not need to use &set-env; to add these. The setting for &kdedir; is automatically propagated to the KDEDIR environment variable while building. (KDEDIRS is not affected). The setting for &qtdir; is automatically propagated to the QTDIR environment variable while building. Changing &kdesrc-build;'s build priority Programs can run with different priority levels on Operating Systems, including &Linux; and &BSD;. This allows the system to allocate time for the different programs in accordance with how important they are. &kdesrc-build; will normally allocate itself a low priority so that the rest of the programs on your system are unaffected and can run normally. Using this technique, &kdesrc-build; will use extra CPU when it is available. &kdesrc-build; will still maintain a high enough priority level so that it runs before routine batch processes and before CPU donation programs such as Seti@Home. To alter &kdesrc-build; so that it uses a higher (or lower) priority level permanently, then you need to adjust the &niceness; setting in the configuration file. The &niceness; setting controls how nice &kdesrc-build; is to other programs. In other words, having a higher &niceness; gives &kdesrc-build; a lower priority. So to give &kdesrc-build; a higher priority, reduce the &niceness; (and vice versa). The &niceness; can go from 0 (not nice at all, highest priority) to 20 (super nice, lowest priority). You can also temporarily change the priority for &kdesrc-build; by using the &cmd-nice; command line option. The value to the option is used exactly the same as for &niceness;. It is possible for some programs run by the super user to have a negative nice value, with a correspondingly even higher priority for such programs. Setting a negative (or even 0) &niceness; for &kdesrc-build; is not a great idea, as it will not help run time significantly, but will make your computer seem very sluggish should you still need to use it. To run &kdesrc-build; with a niceness of 15 (a lower priority than normal): % kdesrc-build Or, you can edit the configuration file to make the change permanent: &niceness; 15 The niceness option only affects the usage of the computer's processor(s). One other major affect on computer performance relates to how much data input or output (I/O) a program uses. In order to control how much I/O a program can use, modern &Linux; operating systems support a similar tool called ionice. &kdesrc-build; supports ionice, (but only to enable or disable it completely) using the use-idle-io-priority option, since &kdesrc-build; version 1.12. Installation as the superuser You may wish to have &kdesrc-build; run the installation with super user privileges. This may be for the unrecommended system-wide installation. This is also useful when using a recommended single user &kde; build, however. This is because some modules (especially kdebase) install programs that will briefly need elevated permissions when run. They are not able to achieve these permission levels unless they are installed with the elevated permissions. You could simply run &kdesrc-build; as the super user directly, but this is not recommended, since the program has not been audited for that kind of use. Although it should be safe to run the program in this fashion, it is better to avoid running as the super user when possible. To take care of this, &kdesrc-build; provides the &make-install-prefix; option. You can use this option to specify a command to use to perform the installation as another user. The recommended way to use this command is with the &sudo; program, which will run the install command as the super user. For example, to install all modules using &sudo;, you could do something like this: global &make-install-prefix; sudo # Other options end global To use &make-install-prefix; for only a single module, this would work: module svn-module-name &make-install-prefix; sudo end module Showing the progress of a module build This feature is always available, and is automatically enabled when possible. What this does is display an estimated build progress while building a module; that way you know about how much longer it will take to build a module. Advanced features Branching and tagging support for &kdesrc-build; What are branches and tags? &subversion; supports managing the history of the &kde; source code. &kde; uses this support to create branches for development, and to tag the repository every so often with a new version release. For example, the &kmail; developers may be working on a new feature in a different branch in order to avoid breaking the version being used by most developers. This branch has development ongoing inside it, even while the main branch (called /trunk) may have development going on inside of it. A tag, on the other hand, is a snapshot of the source code repository at a position in time. This is used by the &kde; administration team to mark off a version of code suitable for release and still allow the developers to work on the code. In &subversion;, there is no difference between branches, tags, or trunk within the code. It is only a convention used by the developers. This makes it difficult to properly support branches and tags within &kdesrc-build;. However, there are some things that can be done. How to use branches and tags Support for branches and tags is handled by a set of options, which range from a generic request for a version, to a specific &url; to download for advanced users. The easiest method is to use the &branch; and &tag; options. You simply use the option along with the name of the desired branch or tag for a module, and &kdesrc-build; will try to determine the appropriate location within the &kde; repository to download from. For most &kde; modules this works very well. To download kdelibs from &kde; 4.6 (which is simply known as the 4.6 branch): module kdelibs branch 4.6 # other options... end module Or, to download kdemultimedia as it was released with &kde; 4.6.1: module kdemultimedia tag 4.6.1 # other options... end module You can specify a global branch value. But if you do so, do not forget to specify a different branch for modules that should not use the global branch! Advanced branch support options &kdesrc-build; supports two options for situations where &branch; and &tag; guess the correct path improperly: &module-base-path; and &override-url;. &module-base-path; is used to help &kdesrc-build; fill in the missing part of a module's path. In the &kde; repository, all of the paths are of the form svnRoot/module-base-path/module-name. Normally &kdesrc-build; can figure out the appropriate middle part by itself. When it cannot, you can use &module-base-path;, like this: module kdesupport # kdesupport supports various tags to easily organize the required # software for a given KDE Platform release. module-base-path tags/kdesupport-for-4.5 end module This would cause &kdesrc-build; to download kdesupport from (in this example), svn://anonsvn.kde.org/home/kde/tags/kdesupport-for-4.5. In previous versions of &kdesrc-build;, the &module-base-path; was handled differently. If you encounter trouble using an old module-base-path definition perhaps you should verify that the actual path is as &kdesrc-build; expects by using the --pretend option. The &override-url; option, on the other hand, requires you to specify the exact path to download from. However, this allows you to pull from paths that previous versions of &kdesrc-build; would have no hope of downloading from. Currently, the &module-base-path; option should be sufficient for any Subversion source URL. &kdesrc-build; will not touch or correct the value you specify for &override-url; at all, so if you change your &svn-server; setting, you may need to update this as well. How &kdesrc-build; tries to ensure a successful build Automatic rebuilds &kdesrc-build; used to include features to automatically attempt to rebuild the module after a failure (as sometimes this re-attempt would work, due to bugs in the build system at that time). Thanks to switching to &cmake; the build system no longer suffers from these bugs, and so &kdesrc-build; will not try to build a module more than once. There are situations where &kdesrc-build; will automatically take action though: If you change configure-flags or cmake-options for a module, then &kdesrc-build; will detect that and automatically re-run configure or cmake for that module. If the buildsystem does not exist (even if &kdesrc-build; did not delete it) then &kdesrc-build; will automatically re-create it. This is useful to allow for performing a full --refresh-build for a specific module without having that performed on other modules. Manually rebuilding a module If you make a change to a module's option settings, or the module's source code changes in a way &kdesrc-build; does not recognize, you may need to manually rebuild the module. You can do this by simply running kdesrc-build . If you would like to have &kdesrc-build; automatically rebuild the module during the next normal build update instead, you can create a special file. Every module has a build directory. If you create a file called .refresh-me in the build directory for a module, &kdesrc-build; will rebuild the module next time the build process occurs, even if it would normally perform the faster incremental build. By default, the build directory is ~/kdesrc/build/module/. If you change the setting of the &build-dir; option, then use that instead of ~/kdesrc/build. Rebuild using .refresh-me for module kdelibs: % touch ~/kdesrc/build/kdelibs/.refresh-me % kdesrc-build Changing environment variable settings Normally &kdesrc-build; uses the environment that is present when starting up when running programs to perform updates and builds. This is useful for when you are running &kdesrc-build; from the command line. However, you may want to change the setting for environment variables that &kdesrc-build; does not provide an option for directly. (For instance, to setup any required environment variables when running &kdesrc-build; on a timer such as &cron;) This is possible with the &set-env; option. Unlike most options, it can be set more than once, and it accepts two entries, separated by a space. The first one is the name of the environment variable to set, and the remainder of the line is the value. Set DISTRO=BSD for all modules: global set-env DISTRO BSD end global Resuming builds Resuming a failed or canceled build You can tell &kdesrc-build; to start building from a different module than it normally would. This can be useful when a set of modules failed, or if you canceled a build run in the middle. You can control this using the &cmd-resume-from; option and the &cmd-resume-after; option. Older versions of &kdesrc-build; would skip the source update when resuming a build. This is no longer done by default, but you can always use the command line option to skip the source update. Resuming the build starting from kdebase: % kdesrc-build Resuming the build starting after kdebase (in case you manually fixed the issue and installed the module yourself): % kdesrc-build If the last &kdesrc-build; build ended with a build failure, you can also use the --resume command line option, which resumes the last build starting at the module that failed. The source and metadata updates are skipped as well (but if you need these, it's generally better to use --resume-from instead). Ignoring modules in a build Similar to the way you can resume the build from a module, you can instead choose to update and build everything normally, but ignore a set of modules. You can do this using the &cmd-ignore-modules; option. This option tells &kdesrc-build; to ignore all the modules on the command line when performing the update and build. Ignoring extragear/multimedia and kdereview during a full run: % kdesrc-build extragear/multimedia kdereview Changing options from the command line Changing global options You can change the setting of options read from the configuration file directly from the command line. This change will override the configuration file setting, but is only temporary. It only takes effect as long as it is still present on the command line. &kdesrc-build; allows you to change options named like option-name by passing an argument on the command line in the form . &kdesrc-build; will recognize whether it does not know what the option is, and search for the name in its list of option names. If it does not recognize the name, it will warn you, otherwise it will remember the value you set it to and override any setting from the configuration file. Setting the &source-dir; option to /dev/null for testing: % kdesrc-build Changing module options It is also possible to change options only for a specific module. The syntax is similar: --module,option-name=value. This change overrides any duplicate setting for the module found in the configuration file, and applies only while the option is passed on the command line. Using a different build directory for the kdeedu module: % kdesrc-build Features for &kde; developers &ssh; Agent checks &kdesrc-build; can ensure that &kde; developers that use &ssh; to access the &kde; source repository do not accidentally forget to leave the &ssh; Agent tool enabled. This can cause &kdesrc-build; to hang indefinitely waiting for the developer to type in their &ssh; password, so by default &kdesrc-build; will check if the Agent is running before performing source updates. This is only done for &kde; developers using &ssh;. This is because no password is required for the default anonymous checkout. &subversion; will handle passwords for the second possible protocol for &kde; developers, https. You may wish to disable the &ssh; Agent check, in case of situations where &kdesrc-build; is mis-detecting the presence of an agent. To disable the agent check, set the option to true. Disabling the &ssh; agent check: global disable-agent-check true end global Other &kdesrc-build; features Changing the amount of output from &kdesrc-build; &kdesrc-build; has several options to control the amount of output the script generates. In any case, errors will always be output. The option (short form is ) causes &kdesrc-build; to be mostly silent. Only important messages, warnings, or errors will be shown. When available, build progress information is still shown. The option (no short form) causes &kdesrc-build; to only display important warnings or errors while it is running. The option (short form is ) causes &kdesrc-build; to be very detailed in its output. The option is for debugging purposes only, it causes &kdesrc-build; to act as if was turned on, causes commands to also output to the terminal, and will display debugging information for many functions. Color output When being run from &konsole; or a different terminal, &kdesrc-build; will normally display with colorized text. You can disable this by using the on the command line, or by setting the &colorful-output; option in the configuration file to false. Disabling color output in the configuration file: global colorful-output false end global Removing unneeded directories after a build Are you short on disk space but still want to run a bleeding-edge &kde; checkout? &kdesrc-build; can help reduce your disk usage when building &kde; from &subversion;. Be aware that building &kde; does take a lot of space. There are several major space-using pieces when using &kdesrc-build;: The actual source checkout can take up a fair amount of space. The default modules take up about 1.6 gigabytes of on-disk space. You can reduce this amount by making sure that you are only building as many modules as you actually want. &kdesrc-build; will not delete source code from disk even if you delete the entry from the configuration file, so make sure that you go and delete unused source checkouts from the source directory. Note that the source files are downloaded from the Internet, you should not delete them if you are actually using them, at least until you are done using &kdesrc-build;. Also, if you already have a &Qt; installed by your distribution (and the odds are good that you do), you probably do not need to install the qt module. That will shave about 200 megabytes off of the on-disk source size. One thing to note is that due to the way &subversion; works: there are actually two files on disk for every file checked-out from the repository. &kdesrc-build; does not have code at this point to try and minimize the source size when the source is not being used. &kdesrc-build; will create a separate build directory to build the source code in. Sometimes &kdesrc-build; will have to copy a source directory to create a fake build directory. When this happens, space-saving symlinks are used, so this should not be a hassle on disk space. The build directory will typically be much larger than the source directory for a module. For example, the build directory for kdebase is about 1050 megabytes, whereas kdebase's source is only around 550 megabytes. Luckily, the build directory is not required after a module has successfully been built and installed. &kdesrc-build; can automatically remove the build directory after installing a module, see the examples below for more information. Note that taking this step will make it impossible for &kdesrc-build; to perform the time-saving incremental builds. Finally, there is disk space required for the actual installation of &kde;, which does not run from the build directory. This typically takes less space than the build directory. It is harder to get exact figures however. How do you reduce the space requirements of &kde;? One way is to use the proper compiler flags, to optimize for space reduction instead of for speed. Another way, which can have a large effect, is to remove debugging information from your &kde; build. You should be very sure you know what you are doing before deciding to remove debugging information. Running bleeding-edge software means you are running software which is potentially much more likely to crash than a stable release. If you are running software without debugging information, it can be very hard to create a good bug report to get your bug resolved, and you will likely have to re-enable debugging information for the affected application and rebuild to help a developer fix the crash. So, remove debugging information at your own risk! Removing the build directory after installation of a module. The source directory is still kept, and debugging is enabled: global configure-flags --enable-debug remove-after-install builddir # Remove build directory after install end global Removing the build directory after installation, without debugging information, with size optimization. global cxxflags -Os # Optimize for size configure-flags --disable-debug remove-after-install builddir # Remove build directory after install end global &cmake;, the &kde; build system Introduction to &cmake; In March 2006, the &cmake; program beat out several competitors and was selected to be the build system for &kde; 4, replacing the autotools-based system that &kde; had used from the beginning. A introduction to &cmake; page is available on the &kde; Community Wiki. Basically, instead of running make Makefile.cvs, then configure, then &make;, we simply run &cmake; and then &make;. &kdesrc-build; has support for &cmake;. &cmake; options can be set for each module using cmake-options, and &kdesrc-build; will also pass required environment variable and command line options to &cmake; automatically. Credits And License &underFDL; &kde; modules and source code organization The <quote>Module</quote> &kde; groups its software into modules of various size. This was initially a loose grouping of a few large modules, but with the introduction of the Git-based source code repositories, these large modules were further split into many smaller modules. &kdesrc-build; uses this module concept as well. In essence, a module is a grouping of code that can be downloaded, built, tested, and installed. Individual modules It is easy to set &kdesrc-build; to build a single module. The following listing is an example of what a declaration for a Subversion-based module would look like in the configuration file. module kdefoo end module This is a Subversion-based module since it doesn't use a repository option. Also, the option is listed as an example only, it is not required. Groups of related modules Now most &kde; source modules are Git-based &kde;, and are normally combined into groups of modules. &kdesrc-build; therefore supports groups of modules as well, using module sets. An example: module-set base-modules kde-projects kde-runtime kde-workspace kde-baseapps end module-set You can leave the module set name (base-modules in this case) empty if you like. This setting tells &kdesrc-build; where to download the source from, but you can also use a git:// URL. One special feature of the kde-projects is that &kdesrc-build; will automatically include any Git modules that are grouped under the modules you list (in the KDE Project database). Module <quote>branch groups</quote> Taking the concept of a group of modules further, the &kde; developers eventually found that synchronizing the names of the Git branches across a large number of repositories was getting difficult, especially during the development push for the new &kde; Frameworks for &Qt; 5. So the concept of branch groups was developed, to allow users and developers to select one of only a few groups, and allow the script to automatically select the appropriate Git branch. &kdesrc-build; supports this feature as of version 1.16-pre2, via the branch-group option. Example of using branch-group branch-group can be used in the configuration file as follows: global # Select KDE Frameworks 5 and other Qt5-based apps kf5-qt5 # Other global options here ... end global module-set # branch-group only works for kde-projects kde-projects # branch-group is inherited from the one set globally, but could # specified here. kdelibs kde-workspace end module-set # kdelibs's branch will be "frameworks" # kde-workspace's branch will be "master" (as of August 2013) In this case the same branch-group gives different branch names for each Git module. This feature requires some data maintained by the &kde; developers in a Git repository named kde-build-metadata, however this module will be included automatically by &kdesrc-build; (though you may see it appear in the script output). &kde; modules that do not have a set branch name for the branch group you choose will default to an appropriate branch name, as if you had not specified branch-group at all. Superseded profile setup procedures Setting up a &kde; login profile These instructions cover how to setup the profile required to ensure your computer can login to your newly-built &kde; &plasma; desktop. &kdesrc-build; will normally try to do this automatically (see ). This appendix section can be useful for those who cannot use &kdesrc-build;'s support for login profile setup. However the instructions may not always be up-to-date, it can also be useful to consult the kde-env-master.sh file included with the &kdesrc-build; source. Changing your startup profile settings The .bash_profile is the login settings file for the popular bash shell used by many &Linux; distributions. If you use a different shell, then you may need to adjust the samples given in this section for your particular shell. Open or create the .bash_profile file in the home directory with your favorite editor, and add to the end of the file: If you are building the qt module (you are by default), add instead: QTDIR=(path to qtdir) # Such as ~/kdesrc/build/qt by default. KDEDIR=(path to kdedir) # Such as ~/kde by default. KDEDIRS=$KDEDIR PATH=$KDEDIR/bin:$QTDIR/bin:$PATH MANPATH=$QTDIR/doc/man:$MANPATH # Act appropriately if LD_LIBRARY_PATH is not already set. if [ -z $LD_LIBRARY_PATH ]; then LD_LIBRARY_PATH=$KDEDIR/lib:$QTDIR/lib else LD_LIBRARY_PATH=$KDEDIR/lib:$QTDIR/lib:$LD_LIBRARY_PATH fi export QTDIR KDEDIRS PATH MANPATH LD_LIBRARY_PATH or, if you are not building qt (and are using your system &Qt; instead), add this instead: KDEDIR=(path to kdedir) # Such as ~/kde by default. KDEDIRS=$KDEDIR PATH=$KDEDIR/bin:$QTDIR/bin:$PATH # Act appropriately if LD_LIBRARY_PATH is not already set. if [ -z $LD_LIBRARY_PATH ]; then LD_LIBRARY_PATH=$KDEDIR/lib else LD_LIBRARY_PATH=$KDEDIR/lib:$LD_LIBRARY_PATH fi export KDEDIRS PATH LD_LIBRARY_PATH If you are not using a dedicated user, set a different $KDEHOME for your new environment in your .bash_profile: export KDEHOME="${HOME}/.kde-svn" # Create it if needed [ ! -e ~/.kde-svn ] && mkdir ~/.kde-svn If later your K Menu is empty or too crowded with applications from your distribution, you may have to set the XDG environment variables in your .bash_profile: XDG_CONFIG_DIRS="/etc/xdg" XDG_DATA_DIRS="${KDEDIR}/share:/usr/share" export XDG_CONFIG_DIRS XDG_DATA_DIRS Starting &kde; Now that you have adjusted your environment settings to use the correct &kde;, it is important to ensure that the correct startkde script is used as well. Open the .xinitrc text file from the home directory, or create it if necessary. Add the line: exec On some distributions, it may be necessary to perform the same steps with the .xsession file, also in the home directory. This is especially true when using graphical login managers such as &kdm;, gdm, or xdm. Now start your fresh &kde;: in &BSD; and &Linux; systems with virtual terminal support, &Ctrl;&Alt;F1 ... &Ctrl;&Alt;F12 keystroke combinations are used to switch to Virtual Console 1 through 12. This allows you to run more than one desktop environment at the same time. The fist six are text terminals and the following six are graphical displays. If when you start your computer you are presented to the graphical display manager instead, you can use the new &kde; environment, even if it is not listed as an option. Most display managers, including &kdm;, have an option to use a Custom Session when you login. With this option, your session settings are loaded from the .xsession file in your home directory. If you have already modified this file as described above, this option should load you into your new &kde; installation. If it does not, there is something else you can try that should normally work: Press &Ctrl;&Alt;F2, and you will be presented to a text terminal. Log in using the dedicated user and type: startx You can run the &kde; from sources and the old &kde; at the same time! Log in using your regular user, start the stable &kde; desktop. Press &Ctrl;&Alt;F2 (or F1, F3, etc..), and you will be presented with a text terminal. Log in using the dedicated &kde; &subversion; user and type: startx You can go back to the &kde; desktop of your regular user by pressing the shortcut key for the already running desktop. This is normally &Ctrl;&Alt;F7, you may need to use F6 or F8 instead. To return to your &kdesrc-build;-compiled &kde;, you would use the same sequence, except with the next function key. For example, if you needed to enter &Ctrl;&Alt;F7 to switch to your regular &kde;, you would need to enter &Ctrl;&Alt;F8 to go back to your &kdesrc-build; &kde;. diff --git a/doc/kdesrc-build.desktop b/doc/kdesrc-build.desktop index 04269b1..7663593 100644 --- a/doc/kdesrc-build.desktop +++ b/doc/kdesrc-build.desktop @@ -1,89 +1,89 @@ # KDE Config File [Desktop Entry] Type=Application X-DocPath=kdesrc-build/index.html Name=KDE Source Builder Name[bs]=KDE graditelj izvornog koda Name[ca]=Compilació del codi font del KDE Name[ca@valencia]=Compilació del codi font del KDE Name[cs]=Sestavování KDE ze zdrojových kódů Name[da]=KDE Source Builder Name[de]=KDE-Source-Builder Name[el]=Κατασκευαστής πηγής του KDE Name[en_GB]=KDE Source Builder Name[es]=Constructor del código fuente de KDE Name[et]=KDE lähtekoodi ehitaja Name[fi]=KDE-lähdekoodin kääntämisohjelma Name[fr]=Compilateur de sources KDE Name[ga]=Tógálaí Foinse KDE Name[gl]=Compilador do código de KDE Name[hu]=KDE forrásfordító Name[it]=Generazione del sorgente di KDE Name[km]=កម្មវិធី​ស្ថាបនា​ប្រភព​​របស់​ KDE​ Name[ko]=KDE 소스 빌더 Name[lt]=KDE kodo kompiliuoklis Name[mr]=केडीई स्रोतबिल्डर Name[nb]=KDE kildebygger Name[nds]=KDE-Bornkode-Buumoduul Name[nl]=KDE Source Builder Name[nn]=Byggjar for KDE-kjeldekode Name[pa]=KDE ਸਰੋਤ ਬਿਲਡਰ Name[pl]=Budowanie KDE ze źródeł Name[pt]=Compilação do Código do KDE Name[pt_BR]=Compilação do código do KDE Name[ru]=Сборка KDE из исходного кода Name[sk]=Prekladač zdrojového kódu KDE Name[sl]=Izgrajevalnik izvorne kode za KDE Name[sr]=Градња КДЕ‑а из извора Name[sr@ijekavian]=Градња КДЕ‑а из извора Name[sr@ijekavianlatin]=Gradnja KDE‑a iz izvora Name[sr@latin]=Gradnja KDE‑a iz izvora Name[sv]=KDE-källkodsbyggare Name[tr]=KDE Kaynak Oluşturucu Name[ug]=KDE Source Builder Name[uk]=Програма для збирання KDE з початкових кодів Name[x-test]=xxKDE Source Builderxx Name[zh_TW]=KDE Source Builder Comment=Builds the KDE Platform and associated software from its source code. A command-line only program. Comment[bs]=Gradi platformu KDE‑a i pridruženi softver iz izvornog koda. Program komandne linije. Comment[ca]=Construeix la plataforma del KDE i el programari associat des del seu codi font. Un programa que només és per a la línia d'ordres. Comment[ca@valencia]=Construeix la plataforma del KDE i el programari associat des del seu codi font. Un programa que només és per a la línia d'ordres. Comment[cs]=Sestaví ze zdrojových kódů platformu KDE a přidružený software. Nástroj pouze pro příkazovou řádku. Comment[da]=Bygger KDE Platform og tilknyttet software fra kildekode. Et program kun til kommandolinjen. Comment[de]=Erstellt KDE und dazugehörige Software aus den Quelltexten. Es ist ein reines Befehlszeilenprogramm. Comment[el]=Δημιουργεί την πλατφόρμα του KDE και το αντίστοιχο λογισμικό από τον πηγαίο κώδικα. Ένα πρόγραμμα μόνο γραμμής εντολών. Comment[en_GB]=Builds the KDE Platform and associated software from its source code. A command-line only program. Comment[es]=Construye la Plataforma KDE y el software asociado desde su código fuente. Un programa solo para la línea de órdenes. Comment[et]=KDE platvormi ja sellega seotud tarkvara ehitamine lähtekoodist. Ainult käsureal kasutatav programm. Comment[fi]=Kääntää KDE-alustan ja siihen liittyvät ohjelmistot lähdekoodista. On vain komentoriviohjelma. Comment[fr]=Construit la plate-forme KDE et les logiciels associés à partir de son code source. Uniquement un programme en ligne de commandes. Comment[gl]=Compila a plataforma KDE e o software asociado a partires do código fonte. Programa de só liña de ordes. Comment[hu]=Lefordítja a KDE platformot és a kapcsolódó szoftvereket a forráskódjukból. Kizárólag parancssoros program. Comment[it]=Genera la piattaforma di KDE e il software ad essa associato dal codice sorgente. Un programma disponibile solo dalla riga di comando. Comment[km]=ស្ថាបនា​កម្មវិធី​របស់​ KDE និង​កម្មវិធី​​ដែល​ភ្ជាប់​​ពី​កូដ​ប្រភព​របស់​វា​​ ។​ កម្មវិធី​បន្ទាត់​ពាក្យ​បញ្ជា​តែ​​​ប៉ុណ្ណោះ​ ។​ -Comment[ko]=KDE 플랫폼과 소프트웨어를 원본 코드에서 빌드합니다. 명령행 전용 프로그램입니다. +Comment[ko]=KDE 플랫폼과 소프트웨어를 소스 코드에서 빌드합니다. 명령행 전용 프로그램입니다. Comment[lt]=Kompiliuoja KDE platformą ir susijusią programinę įrangą iš išeities kodo. Tik komandinės eilutės programa. Comment[nb]=Bygger KDE-plattformen og tilordnede programmer fra kildekoden. Dette er et program med bare kommandolinje. Comment[nds]=Buut de KDE-Systemümgeven un tohören Programmen ut den Bornkode. Bloots en Konsoolprogramm. Comment[nl]=Bouwt het KDE-platform en geassocieerde software uit zijn broncode. Werkt alleen op de opdrachtregel. Comment[nn]=Byggjer KDE-plattforma og tilhøyrande programvare frå kjeldekoden. Er eit kommandolinjebasert program. Comment[pl]=Budowanie Środowiska KDE i związanego z nim oprogramowania z kodu źródłowego. Program działa tylko w linii poleceń. Comment[pt]=Compila a Plataforma do KDE e os programas associados a partir do seu código-fonte. Um programa apenas para a linha de comandos. Comment[pt_BR]=Compila a Plataforma do KDE e os programas associados a partir do seu código-fonte. Um programa apenas para a linha de comando. Comment[ru]=Собирает платформу KDE и основанное на ней программное обеспечение из исходного кода. Программа предназначена только для запуска из командной строки. Comment[sk]=Vybuduje KDE platformu a asociovaný softvér z jeho zdrojového kódu. Program iba pre príkazový riadok. Comment[sl]=Izgradi okolje KDE in povezane programe iz izvorne kode. Gre za program ukazne vrstice. Comment[sr]=Гради платформу КДЕ‑а и придружени софтвер из изворног кода. Програм командне линије. Comment[sr@ijekavian]=Гради платформу КДЕ‑а и придружени софтвер из изворног кода. Програм командне линије. Comment[sr@ijekavianlatin]=Gradi platformu KDE‑a i pridruženi softver iz izvornog koda. Program komandne linije. Comment[sr@latin]=Gradi platformu KDE‑a i pridruženi softver iz izvornog koda. Program komandne linije. Comment[sv]=Bygger KDE-plattformen och tillhörande programvara från dess källkod. Ett program som bara använder kommandoraden. Comment[tr]=KDE Platformunu ve ilişkili yazılımı kaynak kodundan derler. Sadece komt satırı olan bir uygulama. Comment[ug]=KDE سۇپىسى ۋە مۇناسىۋەتلىك يۇمشاق دېتاللارنى ئەسلى كودىدىن ھاسىل قىلىش. پەقەتلا بۇيرۇق قۇرىدا ئىشلەيدۇ. Comment[uk]=Збирає Платформу KDE і пов’язане з нею програмне забезпечення з початкових кодів. Керується за допомогою командного рядка. Comment[x-test]=xxBuilds the KDE Platform and associated software from its source code. A command-line only program.xx Comment[zh_TW]=從源碼建立 KDE 平台與相關軟體。是一個只有命令列的程式。 Categories=Qt;KDE;Development; Exec=kdialog --sorry "kdesrc-build is a command-line only program. Please read the handbook at help:/kdesrc-build for more information." Terminal=true NoDisplay=true diff --git a/doc/man-kdesrc-build.1.docbook b/doc/man-kdesrc-build.1.docbook index 794431d..1bb6a07 100644 --- a/doc/man-kdesrc-build.1.docbook +++ b/doc/man-kdesrc-build.1.docbook @@ -1,1337 +1,1337 @@ kdesrc-build"> ]> kdesrc-build User's Manual MichaelPyne mpyne@kde.org Authored man page 2019-08-31 kdesrc-build 19.08 kdesrc-build 1 19.08 kdesrc-build Downloads, builds and installs &kde; software. kdesrc-build OPTIONS Module name | Module set name kdesrc-build --query VALUE TO QUERY Module name DESCRIPTION The kdesrc-build command is used in order to build &kde; software directly from its source repositories. It can download from Subversion or Git repositories, interfaces with the &kde; project database, and supports controlling which options are passed to make(1) and cmake(1). The operation of kdesrc-build is driven by a configuration file, typically ~/.kdesrc-buildrc. The module name or module set name as given on the command line should be as those names were defined in the configuration file (either in a module definition or declaration, or in a definition). In addition, it can be the name of a &kde; module listed in the &kde; project database (and you can precede the module name with + to force this). kdesrc-build is designed to be able to be completely headless (however, see ENVIRONMENT), and so typically ignores its input completely. Command output is logged instead of being shown on the kdesrc-build output. Modules are built one after the other. If a module fails to update then it is not built. kdesrc-build will not abort just because of a module failure, instead it will keep trying to update and build subsequent modules. By default, kdesrc-build will commence building a module as soon as the source code update is complete for that module, even if other updates are occurring concurrently. At the end kdesrc-build will show which modules failed to build, and where the logs were kept for that build run. OPTIONS NOTE: Some options have short forms, but the kdesrc-build option parser does not support combining short options into one at this point. (E.g. running would not be the same as ). Shows a brief synopsis and frequently-used command line options. Shows information about &kdesrc-build; and the operating system which may be useful in bug reports or when requesting help on forums or mailing lists. Performs one-time setup for users running &kdesrc-build; on common distributions. This includes installation of known system dependencies, a default configuration file setup, and changes to your ~/.bashrc to make the software installed by &kdesrc-build; accessible. Operate in a "dry run" mode. No network accesses are made, no log files are created, no modules are built, and no other permanent changes to disk are made. One important exception is that if you try to build a module that comes from the &kde; project database, and the database hasn't been downloaded yet, the database will be downloaded since the pretend-mode output may change significantly based on the database results. Skips the update and build phase and immediately attempts to install the modules given. Skips the update and build phase and immediately attempts to uninstall the modules given. NOTE: This is only supported for buildsystems that supports the make uninstall command (e.g. &kde; CMake-based). Skips the source update phase. Other phases are included as normal. is a deprecated alias for this option and will be removed in a future release. Skips the metadata update phase for KDE modules. Other phases (including the source update phase) are included as normal. If you wish to avoid all network updates you should also pass . This option can be useful if you are frequently running kdesrc-build since the metadata itself does not change very often. Skips the install phase from the build. Other phases are included as normal. Skips the build phase for the build. Internally the install phase depends on the build phase completing so this is effectively equivalent to , but the semantics may change in the future (e.g. when test suites are moved into their own phase). Disables running the test suite for CMake-based modules. To be fully effective this requires re-running CMake, which can be forced by using the or options. Only performs the source update phase, does not build or install. is a deprecated alias for this option and will be removed in a future release. Forces the build process to be performed without updating source code first. In addition, installation is not performed. (Testing is still performed if applicable, but this will change in a future release) Only updates the build metadata needed for KDE modules, then exits. This is useful to allow the option to work if you've never run kdesrc-build. See also . Removes the build directory for a module before the build phase starts. This has the desired side effect of forcing kdesrc-build to re-configure the module and build it from a "pristine" state with no existing temporary or intermediate output files. Use this option if you have problems getting a module to build but realize it will take longer (possibly much longer) for the build to complete as a result. When in doubt use this option for the entire kdesrc-build run. Force CMake to be re-run, but without deleting the build directory. Usually you actually want , but if you are 100% sure your change to will not invalidate your current intermediate output then this can save some time. Interrupts the build process for each module built: The build process consists of normal setup up to and including running cmake or configure (as appropriate), but make is not run and no installation is attempted. This is mostly only useful to get things like configure --help and cmake-gui to work. Normally you want or . Use this option to skip module processing until the module foo is encountered. foo and all subsequent modules will be processed normally as if they had been specified on the command line. If you use this option because of a build failure you may want to consider using in addition to skip the resultant source update phase. This is just like , except that the module foo is not included in the list of modules to consider. You might use this if you've manually built/installed foo after fixing the build and just want to resume from there. This option can be used to run kdesrc-build after it has had a build failure. It resumes the build from the module that failed, using the list of modules that were waiting to be built before, and disables source and metadata updates as well. The use case is when a simple mistake or missing dependency causes the build failure. Once you correct the error you can quickly get back into building the modules you were building before, without fiddling with and . This is even handier with the option, especially if you're initially setting up your development environment. This is similar to the flag. This option causes the module list for the given build to be truncated just before foo would normally have been built. foo is not built (but see ). This flag may be used with or . This is just like , except that the given module is included in the build. This flag may be used with or . This causes kdesrc-build to include not only the modules it would normally build (either because they were specified on the command line, or mentioned in the configuration file), but also to include known dependencies of those modules in the build. This is normally the default; you can use to disable this effect. Dependencies are known to kdesrc-build based on the contents of the special kde-build-metadata git repository, which is managed for you by the script (see also the option). The KDE community keeps the dependency information in that module up to date, so if kdesrc-build appears to show the wrong dependencies then it may be due to missing or incorrect dependency information. All known dependencies will be included, which may be more than you need. Consider using the option (and similar options) to control the build list when using this option. To see just the list of modules that would be built, use or . Use this option to build only those modules which failed to build on a previous kdesrc-build run. This is useful if a significant number of failures occurred mixed with successful builds. After fixing the issue causing the build failures you can then easily build only the modules that failed previously. Note that the list of previously-failed modules is reset every time a kdesrc-build run finishes with some module failures. However it is not reset by a completely successful build, so you can successfully rebuild a module or two and this flag will still work. This option was added for kdesrc-build 15.09. This option causes the build to abort as soon as a failure occurs. Useful when you're setting up your initial development environment. Without this flag, kdesrc-build will try to press on with the rest of the modules in the build to avoid wasting time in case the problem is with a single module. Forces ALL modules that follow this option to be excluded from consideration by kdesrc-build. This might be useful if you know you want to process all modules except for specific exceptions. Use the given file, foo, for the configuration instead of ~/.kdesrc-buildrc or ./kdesrc-buildrc. The file can be empty, but must exist. Overrides the setting to be foo for this run. In addition, implies . It does not actually perform the action you would think it does (overriding the option to change where modules are installed), although by default modules are installed to the setting if is not set. Changes the CPU priority given to kdesrc-build (and all processes used by kdesrc-build e.g. make(1)). foo should be an integer number between -20 and 19. Positive values are "nicer" to the rest of the system (i.e. lower priority). Note that the possible priorities available on your system may be different than listed here, see nice(2) for more information. Note also that this only changes CPU priority, often you want to change I/O priority on systems where that is supported. There is no command-line option for I/O priority adjustment, but there is a configuration file option: (although like all options, there is a generic way to set this from the command line). Runs the program named by foo using kdesrc-build's normal build environment. All command line arguments present after this option are passed to foo as it is run. This command causes kdesrc-build to query a parameter of the modules in the build list (either passed on the command line or read in from the configuration file), outputting the result to screen (one module per line). This option must be provided with a query mode, which should be one of the following: , which causes kdesrc-build to output the full path to where the module's source code is stored. , which causes kdesrc-build to output the full path to where the module build process occurs. , which causes kdesrc-build to output the full path to where the module will be installed. , which causes kdesrc-build to output the location of the module within the hierarchy of KDE source code repositories. , which causes kdesrc-build to output the resolved git branch that will be used for each module, based on the , and settings in effect. Otherwise, option names that are valid for modules in the configuration file can be used, the resolved value of which will be listed for each module. If a single module is passed on the command line, then the output is simply the value of the parameter being queried. If multiple (or no) modules are passed on the command line, then each line is prefixed by the name of the module. Either way, kdesrc-build stops running once each value is output. This option was added with kdesrc-build 16.05. For example, the command kdesrc-build --query branch kactivities kdepim might end up with output like: kactivities: master kdepim: master Takes all actions up to and including dependency reordering of the modules specified on the command line (or configuration file), prints the modules that would be processed one per line, and then exits without further action. The kde-project metadata is downloaded first (though, see or ). The output is not fully compatible with usage by scripts as other output messages may be generated until the module list is shown. This is mostly just useful for quickly determining what kdesrc-build understands a module's dependencies to be, which means it's only useful for kde-projects modules. This option is also compatible with , , , and . Takes all actions up to and including dependency reordering of the modules specified on the command line (or configuration file), prints the modules that would be processed one per line, and then exits without further action. This option is similar to but includes what specific commit/branch/tag would be built as well. Furthermore, the order in which the modules are listed corresponds to the order in which these would be built. The kde-project metadata is downloaded first (though, see or ). The output is not fully compatible with usage by scripts as other output messages may be generated until the module list is shown. Takes all actions up to and including dependency reordering of the modules specified on the command line (or configuration file), and prints dependency information for each selected module in a (recursive) tree output format. Generated information includes which specific commit/branch/tag is depended on, as well as whether the module would be built. Note that the output can become quite large for applications with many dependencies or when many modules are (implicitly) selected. The kde-project metadata is downloaded first (though, see or ). The output is not fully compatible with usage by scripts as other output messages may be generated until the module list is shown. Enables "colorful output". (Enabled by default). Disables "colorful output". This can be made permanent by setting the option to false (or 0) in your configuration file. Have kdesrc-build start the build process for a module as soon as the source code has finished downloading. Without this option kdesrc-build performs all source updates at once and only then starts with the build process. This option is enabled by default. Disables asynchronous building of modules. See for a more detailed description. Note that kdesrc-build's output will be slightly different in this mode. Increases the level of verbosity of kdesrc-build output (which is already fairly verbose!) Makes kdesrc-build less noisy. Only important messages are shown. Makes kdesrc-build even less noisy. Only warnings/errors are shown. This will fill your terminal with descriptions and debugging output, usually unintelligible, describing what kdesrc-build is doing (and thinks it should be doing). The flag is included since the output may sometimes prove useful for debugging. Normally when kdesrc-build notices that there is no source update on a module which was previously successfully installed, it does not attempt to build or install that module. You can pass this flag to disable that behavior and always run make. Normally kdesrc-build supports using source repository tarball snapshots to reduce load on &kde; infrastructure for git clones for some &kde; modules. Passing this option disables this feature. This option must be passed to allow kdesrc-build to remove conflicting source directories. Currently even this only happens when trying to clone a git-based module if an existing source directory is present. Never specify this option unless it is suggested by kdesrc-build, and only if you don't mind the source directories that are referenced being deleted and re-cloned. Any option not listed above is checked to see if it matches the list of possible configuration file options. If so, the configuration file option is temporarily set to for the duration of this run. Like above, but option is only set to for the module . This does not work for module sets yet, you must repeat this for each module you want to be affected. (Of course, you could simply edit your configuration file...) This option worked slightly differently prior to version 1.16. EXIT STATUS 0 Success 1 Normally this means some part of the update, build or install process failed, but is also used for any abnormal program end not otherwise covered below. 5 A signal was received that killed kdesrc-build, but it attempted to perform normal closedown. 8 Unknown option was passed on the command line. 99 An exception was raised that forced kdesrc-build to abort early. ENVIRONMENT HOME Used for tilde-expansion of file names, and is the default base for the source, build, and installation directories. PATH This environment variable controls the default search path for executables. You can use the configuration file option to add to this variable (e.g. for running from cron(8)). LC_* Environment variables starting with LC_ control the locale used by kdesrc-build. Although kdesrc-build is still not localizable at this point, many of the commands it uses are. kdesrc-build normally sets LC_ALL=C for commands that its must examine the output of but you can manually do this as well. If setting LC_ALL=C fixes a kdesrc-build problem please submit a bug report. SSH_AGENT_PID This environment variable is checked to see if ssh-agent(1) is running, but only if kdesrc-build determines that you are checking out a module that requires an SSH login (but you should know this as no module requires this by default). KDESRC_BUILD_USE_TTY If set, this variable forces kdesrc-build not to close its input while executing system processes. Normally kdesrc-build closes stdin since the stdout and stderr for its child processes are redirected and therefore the user would never see an input prompt anyways. KDESRC_BUILD_DUMP_CONTEXT If set, this variable prints out a description of its "build context" just after reading options and command line arguments and determining which modules to build. You pretty much never want to set this. others Many programs are used by kdesrc-build in the course of its execution, including svn(1), git(1), make(1), and cmake(1). Each of these programs may have their own response to environment variables being set. kdesrc-build will pass environment variables that are set when it is run onto these processes. You can ensure certain environment variables (e.g. CC or CXX) are set by using the configuration file option. FILES ~/.kdesrc-buildrc - Default global configuration file. ./kdesrc-buildrc - If this file (note there is no leading period (.) this time) is found in the current directory when kdesrc-build is run, this file will be used for the configuration instead of ~/.kdesrc-buildrc. ~/.kdesrc-build-data - kdesrc-build uses this file to store persistent data (such as last CMake options used, last revision successfully installed, etc.). It can be safely deleted. BUGS See https://bugs.kde.org/. Be sure to search against the kdesrc-build product. EXAMPLE $ kdesrc-build Downloads, builds and installs all modules listed in the configuration file, in the order defined therein. $ kdesrc-build Same as above, except no permanent actions are taken (specifically no log files are created, downloads performed, build processes run, etc.). EXCEPTION: If you are trying to build a module defined in the &kde; project database, and the database has not been downloaded yet, kdesrc-build will download the database since this can significantly affect the final build order. $ kdesrc-build kdebase Deletes the build directory for the kdebase module set () and then starts the build process again without updating the source code in-between. $ kdesrc-build /dev/null Forces kdesrc-build to read an empty configuration file and simulate the resultant build process. This shows what would happen by default with no configuration file, without an error message about a missing configuration file. $ kdesrc-build +kdebase/kde-baseapps Downloads, builds and installs the kde-baseapps module from the &kde; project database. Since the module name is preceded by a + it is assumed to defined in the &kde; project database even if this hasn't been specifically configured in the configuration file. The kdebase/ portion forces kdesrc-build to ignore any kde-baseapps modules that are not children of the kdebase supermodule in the project database (although it is contrived for this example). $ kdesrc-build Downloads, builds and installs all modules defined in the configuration file but overrides the cmake-options option to have the value given on the command line for this run only. Any further kdesrc-build runs will use the cmake-options given in the configuration file. SEE ALSO build-tool - A program by Michael Jansen which can build &kde; software based on included recipes. RESOURCES Main web site: https://kdesrc-build.kde.org/ Documentation: https://docs.kde.org/index.php?application=kdesrc-build Setup script: kdesrc-build-setup COPYING -Copyright (C) 2003-2019 Michael Pyne. +Copyright (C) 2003-2020 Michael Pyne. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA diff --git a/kdesrc-build b/kdesrc-build index 3e1886d..024e6df 100755 --- a/kdesrc-build +++ b/kdesrc-build @@ -1,449 +1,450 @@ #!/usr/bin/env perl # Script to handle building KDE from source code. All of the configuration is # stored in the file ./kdesrc-buildrc (or ~/.kdesrc-buildrc, if that's not # present). # # Please also see the documentation that should be included with this program, # in the doc/ directory. # -# Copyright © 2003 - 2019 Michael Pyne. # Home page: https://kdesrc-build.kde.org/ # +# Copyright © 2003 - 2020 Michael Pyne. +# Copyright © 2018 - 2020 Johan Ouwerkerk # Copyright © 2005, 2006, 2008 - 2011 David Faure # Copyright © 2005 Thiago Macieira # Copyright © 2006 Stephan Kulow # Copyright © 2006, 2008 Dirk Mueller # ... and possibly others. Check the git source repository for specifics. # # This program is free software; you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software # Foundation; either version 2 of the License, or (at your option) any later # version. # # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS # FOR A PARTICULAR PURPOSE. See the GNU General Public License for more # details. # # You should have received a copy of the GNU General Public License along with # this program; if not, write to the Free Software Foundation, Inc., 51 # Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA # Adding an option? Grep for 'defaultGlobalOptions' in ksb::BuildContext --mpyne use 5.014; # Require Perl 5.14 use strict; use warnings; # On many container-based distros, even FindBin is missing to conserve space. # But we can use File::Spec to do nearly the same. my $RealBin; my $modPath; # The File::Spec calls have to run when parsing (i.e. in BEGIN) to make the # 'use lib' below work (which itself implicitly uses BEGIN { }) BEGIN { use File::Spec; # resolve symlinks my $scriptPath = $0; for (1..16) { last unless -l $scriptPath; $scriptPath = readlink $scriptPath; } die "Too many symlinks followed looking for script" if -l $scriptPath; my ($volume, $directories, $script) = File::Spec->splitpath($scriptPath); $RealBin = File::Spec->catpath($volume, $directories, ''); die "Couldn't find base directory!" unless $RealBin; # Use modules in git repo if running from git dir, otherwise assume # system install $modPath = File::Spec->rel2abs('modules', $RealBin); $modPath = ($RealBin =~ s,/bin/?$,/share/kdesrc-build/modules,r) unless -d $modPath; die "Couldn't find modules for kdesrc-build!" unless $modPath; } use lib "$modPath"; # Make ksb:: modules available sub dumpError { my $err = shift; open my $fh, '>>', "error-$$.log" or return; my $time = localtime; say $fh $time; say $fh $@; } # When running in a limited environment, we might not be able to load # our modules although we can find them. In this case we should help user # by setting up system dependencies. eval { if (grep { $_ eq '--initial-setup' } @ARGV) { require ksb::FirstRun; require ksb::Debug; ksb::Debug::setColorfulOutput(1); exit ksb::FirstRun::setupUserSystem(File::Spec->rel2abs($RealBin)); } }; if ($@) { dumpError($@); say STDERR <import(); ksb::Util->import(); ksb::BuildException->import(); ksb::Version->import(qw(scriptVersion)); ksb::Application->import(); ksb::UserInterface::TTY->import(); web::BackendServer->import(); Mojo::IOLoop->import(); Mojo::Server::Daemon->import(); # Make Perl 'plain die' exceptions use Carp::confess instead of their core # support. This is not supported by the Perl 5 authors but assuming it works # will be better than the alternative backtrace we get (which is to say, none) $SIG{__DIE__} = \&Carp::confess; ksb::Version->path($RealBin); ### Script-global functions. # These functions might be called at runtime via log_command, using # log_command's support for symbolic execution of a named subroutine. Because # of that, they have been left in the top-level script. # # Everything else should be in an appropriate class. # Subroutine to recursively symlink a directory into another location, in a # similar fashion to how the XFree/X.org lndir() program does it. This is # reimplemented here since some systems lndir doesn't seem to work right. # # As a special exception to the GNU GPL, you may use and redistribute this # function however you would like (i.e. consider it public domain). # # The first parameter is the directory to symlink from. # The second parameter is the destination directory name. # # e.g. if you have $from/foo and $from/bar, lndir would create $to/foo and # $to/bar. # # All intervening directories will be created as needed. In addition, you # may safely run this function again if you only want to catch additional files # in the source directory. # # Note that this function will unconditionally output the files/directories # created, as it is meant to be a close match to lndir. # # RETURN VALUE: Boolean true (non-zero) if successful, Boolean false (0, "") # if unsuccessful. sub safe_lndir { my ($from, $to) = @_; # Create destination directory. if (not -e $to) { print "$to\n"; if (not pretending() and not super_mkdir($to)) { error ("Couldn't create directory r[$to]: b[r[$!]"); return 0; } } # Create closure callback subroutine. my $wanted = sub { my $dir = $File::Find::dir; my $file = $File::Find::fullname; $dir =~ s/$from/$to/; # Ignore the .svn directory and files. return if $dir =~ m,/\.svn,; # Create the directory. if (not -e $dir) { print "$dir\n"; if (not pretending()) { super_mkdir ($dir) or croak_runtime("Couldn't create directory $dir: $!"); } } # Symlink the file. Check if it's a regular file because File::Find # has no qualms about telling you you have a file called "foo/bar" # before pointing out that it was really a directory. if (-f $file and not -e "$dir/$_") { print "$dir/$_\n"; if (not pretending()) { symlink $File::Find::fullname, "$dir/$_" or croak_runtime("Couldn't create file $dir/$_: $!"); } } }; # Recursively descend from source dir using File::Find eval { find ({ 'wanted' => $wanted, 'follow_fast' => 1, 'follow_skip' => 2}, $from); }; if ($@) { error ("Unable to symlink $from to $to: $@"); return 0; } return 1; } # Subroutine to delete recursively, everything under the given directory, # unless we're in pretend mode. # # Used from ksb::BuildSystem to handle cleaning a build directory. # # i.e. the effect is similar to "rm -r $arg/* $arg/.*". # # This assumes we're called from a separate child process. Therefore the # normal logging routines are /not used/, since our output will be logged # by the parent kdesrc-build. # # The first parameter should be the absolute path to the directory to delete. # # Returns boolean true on success, boolean false on failure. sub prune_under_directory { my $dir = shift; my $errorRef; print "starting delete of $dir\n"; eval { remove_tree($dir, { keep_root => 1, error => \$errorRef }); }; if ($@ || @$errorRef) { error ("\tUnable to clean r[$dir]:\n\ty[b[$@]"); return 0; } return 1; } sub findMissingModules { # should be either strings of module names to be found or a listref containing # a list of modules where any one of which will work. my @requiredModules = ( 'HTTP::Tiny', 'IO::Socket::SSL', # Assume if Mojolicious::Lite is present, that whole framework is properly installed 'Mojolicious::Lite', 'Mojo::Promise', # That was a bad assumption 'Mojo::JSON', [qw(YAML::XS YAML::PP YAML::Syck)] ); my @missingModules; my $validateMod = sub { return eval "require $_[0]; 1;"; }; my $description; foreach my $neededModule (@requiredModules) { if (ref $neededModule) { # listref of options my @moduleOptions = @$neededModule; next if (ksb::Util::any (sub { $validateMod->($_); }, $neededModule)); $description = 'one of (' . join(', ', @moduleOptions) . ')'; } else { next if $validateMod->($neededModule); $description = $neededModule; } push @missingModules, $description; } return @missingModules; } # Rather than running an interactive build, launches a web server that can be # interacted with by and outside user interface, printing the URL to the server # on stdout and then remaining in the foreground until killed. sub launchBackend { # Manually setup the daemon so that we can figure out what port it # ends up on. my $daemon = Mojo::Server::Daemon->new( app => web::BackendServer->new, listen => ['http://localhost'], silent => 1, ); $daemon->start; # Grabs the socket to listen on my $port = $daemon->ports->[0] or do { say STDERR "Can't autodetect which TCP port was assigned!"; exit 1; }; say "http://localhost:$port"; Mojo::IOLoop->start unless Mojo::IOLoop->is_running; exit 0; } # Script starts. # Ensure some critical Perl modules are available so that the user isn't surprised # later with a Perl exception if(my @missingModuleDescriptions = findMissingModules()) { say <start() and then as an atexit handler app->ksb->finish) # Use some exception handling to avoid ucky error messages eval { my $app = web::BackendServer->new(@ARGV); my $ui = ksb::UserInterface::TTY->new($app); push @atexit_subs, sub { $app->ksb->finish(99) }; # TODO: Reimplement --print-modules, --query modes, which wouldn't go through ->start my $result = $ui->start(); @atexit_subs = (); # Clear exit handlers # env driver is just the ~/.config/kde-env-*.sh, session driver is that + ~/.xsession # TODO: Move to Backend or UserInterface as appropriate. my $ctx = $app->context; if ($ctx->getOption('install-environment-driver') || $ctx->getOption('install-session-driver')) { ksb::Application::_installCustomSessionDriver($ctx); } # Exits the script exit $result; }; if (my $err = $@) { if (had_an_exception()) { say "kdesrc-build encountered an exceptional error condition:"; say " ========"; say " $err"; say " ========"; say "\tCan't continue, so stopping now."; say "\nPlease submit a bug against kdesrc-build on https://bugs.kde.org/" if ($err->{exception_type} eq 'Internal'); } else { # An exception was raised, but not one that kdesrc-build generated say "Encountered an error in the execution of the script."; say "The error reported was $err"; say "Please submit a bug against kdesrc-build on https://bugs.kde.org/"; } exit 99; } # vim: set et sw=4 ts=4: diff --git a/kdesrc-build-setup b/kdesrc-build-setup index 564b92e..c72d17e 100755 --- a/kdesrc-build-setup +++ b/kdesrc-build-setup @@ -1,501 +1,501 @@ #!/usr/bin/env perl # Script to create a configuration file for kdesrc-build. # -# Copyright © 2011, 2019 Michael Pyne. +# Copyright © 2011, 2020 Michael Pyne. # Home page: https://kdesrc-build.kde.org/ # # This program is free software; you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software # Foundation; either version 2 of the License, or (at your option) any later # version. # # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS # FOR A PARTICULAR PURPOSE. See the GNU General Public License for more # details. # # You should have received a copy of the GNU General Public License along with # this program; if not, write to the Free Software Foundation, Inc., 51 # Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA use strict; use 5.018; use IO::Pipe; use File::Copy; use File::Temp qw/tempfile/; use File::Basename; use Cwd qw(abs_path); our $VERSION = 0.03; # Not user-visible yet. sub clearScreen { require POSIX; my $termios = POSIX::Termios->new(); $termios->getattr(1); # Get STDOUT attributes require Term::Cap; my $terminal = Term::Cap->Tgetent({OSPEED => $termios->getospeed}); # Force the clear characters to be output immediately. # Otherwise it might overlap with other output, like error messages. local $| = 1; print $terminal->Tputs('cl', 0); return 0; } sub runDialogExecutable { my (@args) = @_; # Allow for 2 more file descriptors (on top of the normally allowed 0, 1, # 2) to survive the upcoming exec # See "SYSTEM_FD_MAX" in perldoc:perlvar $^F = 4; my $pipe = new IO::Pipe; my $pid; if ($pid = fork()) { # Parent $pipe->reader(); my $output = <$pipe>; waitpid $pid, 0; my $result = ($? >> 8); $pipe->close(); # dialog uses -1 as an exit code, Perl gets just the standard 8 bits # the rest of UNIX uses... if ($? == -1) { clearScreen(); die "Failed to run dialog(1): $@"; } elsif ($result == 255) { clearScreen(); die "Canceled the dialog"; } return $output || $result; } elsif (defined $pid) { # Child $pipe->writer(); my $outputFd = $pipe->fileno(); print "Using fd $outputFd"; exec ('dialog', '--output-fd', $outputFd, '--backtitle', 'kdesrc-build setup', @args); } else { die "Unable to fork? $!"; } } sub getUserInput { my $prompt = shift; my $default = shift; my @args = qw/--inputbox 8 50/; splice @args, 1, 0, $prompt; push @args, $default if $default; return runDialogExecutable(@args); } sub getMenuOption { my ($prompt, @opts) = @_; @opts = @{$opts[0]} if ref $opts[0] eq 'ARRAY'; my @args = qw/--menu 20 70 18/; splice @args, 1, 0, $prompt; while(my ($k, $v) = splice (@opts, 0, 2)) { push @args, $k, $v; } return runDialogExecutable(@args); } sub showInfo { my $message = shift; my @args = qw/--msgbox 20 62/; splice @args, 1, 0, $message; return runDialogExecutable(@args); } sub getYesNoAnswer { my $prompt = shift; my @args = qw/--yesno 8 55/; splice @args, 1, 0, $prompt; return runDialogExecutable(@args) == 0; } sub getDirectory { my $dir = shift; my @args = qw/--dselect 10 70/; splice @args, 1, 0, $dir; return runDialogExecutable(@args); } sub getListOptions { my ($prompt, $opts, $enabled) = @_; die "\$opts not a hash ref" unless (ref $opts eq 'ARRAY'); die "\$enabled not a hash ref" unless (ref $enabled eq 'HASH'); my @args = qw/--checklist 20 70 18/; splice @args, 1, 0, $prompt; splice @args, 0, 0, '--output-separator', ','; while (my ($k, $v) = splice(@{$opts}, 0, 2)) { push (@args, $k, $v, (exists ${$enabled}{$k} ? 'on' : 'off')); } my $output = runDialogExecutable(@args); # Filter out empty results, remove quotes. my @items = split (/,/, $output); s/^"(.*)"$/$1/ foreach @items; @items = grep { length $_ } @items; return @items; } # The 'dialog(1)' program is required, verify it exists before going # further. # We use the --help option since it doesn't send weird terminal characters to the screen # and it's supported on dialog and Debian's dialog replacement called whiptail. system('dialog', '--help') == 0 or do { my $osError = "$!"; say "Unable to run the dialog(1) program, it is required for this setup script."; if ($? == -1) { say "\tThe program wouldn't even run, due to error: $osError"; } else { say "\tProgram ran, but exited with error: ", $? >> 8; } exit 1; }; showInfo(< "$ENV{HOME}/kde/usr (default)", custom => "Custom location, chosen next screen", ]); if ($installDir eq 'custom') { $installDir = getDirectory('/usr/local/kde'); } else { $installDir = "~/kde/usr"; } my $sourceDir = getMenuOption('Where do you want the source code to be saved?', [ home => "$ENV{HOME}/kde/src (default)", custom => "Custom location, chosen next screen", ]); if ($sourceDir eq 'custom') { $sourceDir = getDirectory('/usr/local/kde/src'); } else { $sourceDir = "~/kde/src"; } my $buildDir = getMenuOption('Where do you want temporary build files to be saved? (They might need lots of space)', [ home => "$ENV{HOME}/kde/build (default)", custom => "Custom location, chosen next screen", ]); if ($buildDir eq 'custom') { $buildDir = getDirectory('/usr/local/kde/build'); } else { $buildDir = "~/kde/build"; } my @chosenModules = getListOptions( "Which major module groups do you want to build?", [ qt5 => 'Qt 5 - Base support libraries (required unless distro supplies)', frameworks => 'KDE Frameworks 5 - Essential libraries/runtime (required)', workspace => 'KDE Plasma 5 Desktop and workspace', base => 'Assorted useful KF5-based applications', pim => 'Personal Information Management software', ], { qt5 => 1, frameworks => 1, workspace => 1, base => 1, }, ); my $numCpus = getUserInput( 'How many CPU cores do you wish to use for building?', '4'); my $outputFileName = "$ENV{HOME}/.kdesrc-buildrc"; my $output; # Will be output filehandle. while (-e $outputFileName) { (my $printableName = $outputFileName) =~ s/^$ENV{HOME}/~/; my $outputChoice = getMenuOption( "$printableName already exists, what do you want to do?", [ backup => 'Make a backup, then overwrite with the new configuration', custom => 'Write the new configuration to a different file', cancel => 'Cancel setup', ], ); if ($outputChoice eq 'cancel') { showInfo('Setup canceled'); exit 0; } if ($outputChoice eq 'custom') { $outputFileName = getUserInput('Enter desired configuration file name.'); $outputFileName =~ s/^~/$ENV{HOME}/; } if ($outputChoice eq 'backup') { copy($outputFileName, "$outputFileName~") or do { my $error = "$!"; showInfo(<', $outputFileName) or do { my $error = "$!"; showInfo (<= 5.018, 'warnings', 'experimental::smartmatch'; use ksb::Debug 0.30; use ksb::Util; use ksb::BuildContext 0.35; use ksb::BuildSystem::QMake; use ksb::BuildException 0.20; use ksb::FirstRun; use ksb::Module; use ksb::ModuleResolver 0.20; use ksb::ModuleSet 0.20; use ksb::ModuleSet::KDEProjects; use ksb::ModuleSet::Qt; use ksb::OSSupport; use ksb::PromiseChain; use ksb::RecursiveFH; use ksb::DependencyResolver 0.20; use ksb::DebugOrderHints; use ksb::Updater::Git; use ksb::Version qw(scriptVersion); use Mojo::IOLoop; use Mojo::Promise; use Fcntl; # For sysopen use List::Util qw(first min); use File::Basename; # basename, dirname use File::Glob ':glob'; use POSIX qw(:sys_wait_h _exit :errno_h); use Getopt::Long qw(GetOptionsFromArray :config gnu_getopt nobundling); use IO::Handle; use IO::Select; ### Package-specific variables (not shared outside this file). use constant { # We use a named remote to make some git commands work that don't accept the # full path. KDE_PROJECT_ID => 'kde-projects', # git-repository-base for sysadmin/repo-metadata QT_PROJECT_ID => 'qt-projects', # git-repository-base for qt.io Git repo }; ### Package methods sub new { my ($class, @options) = @_; # TODO: Do something with @options my $self = bless { context => ksb::BuildContext->new(), metadata_module => undef, run_mode => 'build', modules => undef, module_resolver => undef, # ksb::ModuleResolver but see below _base_pid => $$, # See finish() }, $class; # Default to colorized output if sending to TTY ksb::Debug::setColorfulOutput(-t STDOUT); return $self; } # Call after establishContext (to read in config file and do one-time metadata # reading), but before you call startHeadlessBuild. # # Parameter: # # - workload, a hashref containing the following entries: # { # selectedModules: listref with the selected ksb::Modules to build # dependencyInfo: reference to a dependency info object created by # ksb::DependencyResolver # build: a boolean indicating whether to go through with build or not # } sub setModulesToProcess { my ($self, $workLoad) = @_; croak_internal("Expecting workload object!") unless ref $workLoad eq 'HASH'; $self->{modules} = $workLoad->{selectedModules}; $self->{workLoad} = $workLoad; $self->context()->addModule($_) foreach @{$self->{modules}}; # i.e. niceness, ulimits, etc. $self->context()->setupOperatingEnvironment(); } # Sets the application to be non-interactive, intended to make this suitable as # a backend for a Mojolicious-based web server with a separate U/I. sub setHeadless { my $self = shift; $self->{run_mode} = 'headless'; return $self; } # Method: _readCommandLineOptionsAndSelectors # # Returns a list of module/module-set selectors, selected module/module-set # options, and global options, based on the command-line arguments passed to # this function. # # This is a package method, should be called as # $app->_readCommandLineOptionsAndSelectors # # Phase: # initialization - Do not call from this function. # # Parameters: # cmdlineOptions - hashref to hold parsed modules options to be applied later. # *Note* this must be done separately, it is not handled by this subroutine. # Global options will be stored in a hashref at $cmdlineOptions->{global}. # Module or module-set options will be stored in a hashref at # $cmdlineOptions->{$moduleName} (it will be necessary to disambiguate # later in the run whether it is a module set or a single module). # # If the global option 'start-program' is set, then the program to start and # its options will be found in a listref pointed to under the # 'start-program' option. # # selectors - listref to hold the list of module or module-set selectors to # build, in the order desired by the user. These will just be strings, the # caller will have to figure out whether the selector is a module or # module-set, and create any needed objects, and then set the recommended # options as listed in cmdlineOptions. # # ctx - to hold the global build state. # # @options - The remainder of the arguments are treated as command line # arguments to process. # # Returns: # Nothing. An exception will be raised on failure, or this function may quit # the program directly (e.g. to handle --help, --usage). sub _readCommandLineOptionsAndSelectors { my $self = shift; my ($cmdlineOptionsRef, $selectorsRef, $ctx, @options) = @_; my $phases = $ctx->phases(); my @savedOptions = @options; # Copied for use in debugging. my $os = ksb::OSSupport->new; my $version = "kdesrc-build " . scriptVersion(); my $author = < Many people have contributed code, bugfixes, and documentation. Please report bugs using the KDE Bugzilla, at https://bugs.kde.org/ DONE # Getopt::Long will store options in %foundOptions, since that is what we # pass in. To allow for custom subroutines to handle an option it is # required that the sub *also* be in %foundOptions... whereupon it will # promptly be overwritten if we're not careful. Instead we let the custom # subs save to %auxOptions, and read those in back over it later. my (%foundOptions, %auxOptions); %foundOptions = ( 'show-info' => sub { say $version; say "OS: ", $os->vendorID(); exit }, version => sub { say $version; exit }, author => sub { say $author; exit }, help => sub { _showHelpMessage(); exit 0 }, install => sub { $self->{run_mode} = 'install'; $phases->phases('install'); }, uninstall => sub { $self->{run_mode} = 'uninstall'; $phases->phases('uninstall'); }, 'no-src' => sub { $phases->filterOutPhase('update'); }, 'no-install' => sub { $phases->filterOutPhase('install'); }, 'no-snapshots' => sub { # The documented form of disable-snapshots $auxOptions{'disable-snapshots'} = 1; }, 'no-tests' => sub { # The "right thing" to do $phases->filterOutPhase('test'); # What actually works at this point. $foundOptions{'run-tests'} = 0; }, 'no-build' => sub { $phases->filterOutPhase('build'); }, # Mostly equivalent to the above 'src-only' => sub { $phases->phases('update'); # We have an auto-switching function that we only want to run # if --src-only was passed to the command line, so we still # need to set a flag for it. $foundOptions{'allow-auto-repo-move'} = 1; }, 'build-only' => sub { $phases->phases('build'); }, 'install-only' => sub { $self->{run_mode} = 'install'; $phases->phases('install'); }, prefix => sub { my ($optName, $arg) = @_; $auxOptions{prefix} = $arg; $foundOptions{kdedir} = $arg; #TODO: Still needed for compat? $foundOptions{reconfigure} = 1; }, query => sub { my (undef, $arg) = @_; my $validMode = qr/^[a-zA-Z0-9_][a-zA-Z0-9_-]*$/; die("Invalid query mode $arg") unless $arg =~ $validMode; # Add useful aliases $arg = 'source-dir' if $arg =~ /^src-?dir$/; $arg = 'build-dir' if $arg =~ /^build-?dir$/; $arg = 'install-dir' if $arg eq 'prefix'; $self->{run_mode} = 'query'; $auxOptions{query} = $arg; $auxOptions{pretend} = 1; # Implied pretend mode }, pretend => sub { # Set pretend mode but also force the build process to run. $auxOptions{pretend} = 1; $foundOptions{'build-when-unchanged'} = 1; }, resume => sub { $auxOptions{resume} = 1; $phases->filterOutPhase('update'); # Implied --no-src $foundOptions{'no-metadata'} = 1; # Implied --no-metadata }, verbose => sub { $foundOptions{'debug-level'} = ksb::Debug::WHISPER }, quiet => sub { $foundOptions{'debug-level'} = ksb::Debug::NOTE }, 'really-quiet' => sub { $foundOptions{'debug-level'} = ksb::Debug::WARNING }, debug => sub { $foundOptions{'debug-level'} = ksb::Debug::DEBUG; debug ("Commandline was: ", join(', ', @savedOptions)); }, # Hack to set module options 'set-module-option-value' => sub { my ($optName, $arg) = @_; my ($module, $option, $value) = split (',', $arg, 3); if ($module && $option) { $cmdlineOptionsRef->{$module} //= { }; $cmdlineOptionsRef->{$module}->{$option} = $value; } }, # Getopt::Long doesn't set these up for us even though we specify an # array. Set them up ourselves. 'start-program' => [ ], 'ignore-modules' => [ ], # Module selectors, the <> is Getopt::Long shortcut for an # unrecognized non-option value (i.e. an actual argument) '<>' => sub { my $arg = shift; push @{$selectorsRef}, $arg; }, ); # Handle any "cmdline-eligible" options not already covered. my $flagHandler = sub { my ($optName, $optValue) = @_; # Assume to set if nothing provided. $optValue = 1 if (!defined $optValue or $optValue eq ''); $optValue = 0 if lc($optValue) eq 'false'; $optValue = 0 if !$optValue; $auxOptions{$optName} = $optValue; }; foreach my $option (keys %ksb::BuildContext::defaultGlobalFlags) { if (!exists $foundOptions{$option}) { $foundOptions{$option} = $flagHandler; # A ref to a sub here! } } # Actually read the options. my $optsSuccess = GetOptionsFromArray(\@options, \%foundOptions, # Options here should not duplicate the flags and options defined below # from ksb::BuildContext! 'version|v', 'author', 'help', 'show-info', 'install', 'uninstall', 'no-src|no-svn', 'no-install', 'no-build', 'no-tests', 'build-when-unchanged|force-build', 'no-metadata', 'verbose', 'quiet|quite|q', 'really-quiet', 'debug', 'reconfigure', 'colorful-output|color!', 'src-only|svn-only', 'build-only', 'install-only', 'build-system-only', 'rc-file=s', 'prefix=s', 'niceness|nice:10', 'ignore-modules=s{,}', 'pretend|dry-run|p', 'refresh-build', 'query=s', 'start-program|run=s{,}', 'launch-browser', 'revision=i', 'resume-from=s', 'resume-after=s', 'rebuild-failures', 'resume', 'stop-after=s', 'stop-before=s', 'set-module-option-value=s', 'metadata-only', # Debug-only flags 'print-modules', 'list-build', 'dependency-tree', # Special sub used (see above), but have to tell Getopt::Long to look # for negatable boolean flags (map { "$_!" } (keys %ksb::BuildContext::defaultGlobalFlags)), # Default handling fine, still have to ask for strings. (map { "$_:s" } (keys %ksb::BuildContext::defaultGlobalOptions)), '<>', # Required to read non-option args ); if (!$optsSuccess) { croak_runtime("Error reading command-line options."); } # To store the values we found, need to strip out the values that are # subroutines, as those are the ones we created. Alternately, place the # subs inline as an argument to the appropriate option in the # GetOptionsFromArray call above, but that's ugly too. my @readOptionNames = grep { ref($foundOptions{$_}) ne 'CODE' } (keys %foundOptions); # Slice assignment: $left{$key} = $right{$key} foreach $key (@keys), but # with hashref syntax everywhere. @{ $cmdlineOptionsRef->{'global'} }{@readOptionNames} = @foundOptions{@readOptionNames}; @{ $cmdlineOptionsRef->{'global'} }{keys %auxOptions} = values %auxOptions; } # Generates the build context, builds various module, dependency and branch # group resolvers, and splits up the provided option/selector mix read from # cmdline into selectors (returned to caller, if any) and pre-built context and # resolvers. # # Use "modulesFromSelectors" to further generate the list of ksb::Modules in # dependency order. # # After this function is called all module set selectors will have been # expanded, and we will have downloaded kde-projects metadata. # # Returns: List of Selectors to build. sub establishContext { my $self = shift; my @argv = @_; # Note: Don't change the order around unless you're sure of what you're # doing. my $ctx = $self->context(); my $cmdlineOptions = { global => { }, }; my $cmdlineGlobalOptions = $cmdlineOptions->{global}; my $deferredOptions = { }; # 'options' blocks # Process --help, --install, etc. first. my @selectors; $self->_readCommandLineOptionsAndSelectors($cmdlineOptions, \@selectors, $ctx, @argv); # Convert list to hash for lookup my %ignoredSelectors = map { $_, 1 } @{$cmdlineGlobalOptions->{'ignore-modules'}}; # Set aside debug-related and other short-circuit cmdline options # for kdesrc-build CLI driver to handle my @debugFlags = qw(dependency-tree list-build print-modules); $self->{debugFlags} = { map { ($_, 1) } grep { defined $cmdlineGlobalOptions->{$_} } (@debugFlags) }; my @startProgramAndArgs = @{$cmdlineGlobalOptions->{'start-program'}}; delete @{$cmdlineGlobalOptions}{qw/ignore-modules start-program/}; # rc-file needs special handling. if (exists $cmdlineGlobalOptions->{'rc-file'} && $cmdlineGlobalOptions->{'rc-file'}) { $ctx->setRcFile($cmdlineGlobalOptions->{'rc-file'}); } my $fh = $ctx->loadRcFile(); $ctx->loadPersistentOptions(); if (exists $cmdlineGlobalOptions->{'resume'}) { my $moduleList = $ctx->getPersistentOption('global', 'resume-list'); if (!$moduleList) { error ("b[--resume] specified, but unable to find resume point!"); error ("Perhaps try b[--resume-from] or b[--resume-after]?"); croak_runtime("Invalid --resume flag"); } unshift @selectors, split(/,\s*/, $moduleList); } if (exists $cmdlineGlobalOptions->{'rebuild-failures'}) { my $moduleList = $ctx->getPersistentOption('global', 'last-failed-module-list'); if (!$moduleList) { error ("b[y[--rebuild-failures] was specified, but unable to determine"); error ("which modules have previously failed to build."); croak_runtime("Invalid --rebuild-failures flag"); } unshift @selectors, split(/,\s*/, $moduleList); } # _readConfigurationOptions will add pending global opts to ctx while ensuring # returned modules/sets have any such options stripped out. It will also add # module-specific options to any returned modules/sets. my @optionModulesAndSets = _readConfigurationOptions($ctx, $fh, $deferredOptions); close $fh; # Check if we're supposed to drop into an interactive shell instead. If so, # here's the stop off point. if (@startProgramAndArgs) { $ctx->setupEnvironment(); # Read options from set-env $ctx->commitEnvironmentChanges(); # Apply env options to environment _executeCommandLineProgram(@startProgramAndArgs); # noreturn } # Everything else in cmdlineOptions should be OK to apply directly as a module # or context option. $ctx->setOption(%{$cmdlineGlobalOptions}); # Selecting modules or module sets would requires having the KDE # build metadata (kde-build-metadata and sysadmin/repo-metadata) # available. $ctx->setKDEDependenciesMetadataModuleNeeded(); $ctx->setKDEProjectsMetadataModuleNeeded(); if (!exists $ENV{HARNESS_ACTIVE} && !$self->{run_mode} eq 'headless') { # In some modes (testing, acting as headless backend), we should avoid # downloading metadata automatically, so don't. - ksb::Updater::Git::verifyGitConfig(); + ksb::Updater::Git::verifyGitConfig($ctx); $self->_downloadKDEProjectMetadata(); } # At this point we have our list of candidate modules / module-sets (as read in # from rc-file). The module sets have not been expanded into modules. # We also might have cmdline "selectors" to determine which modules or # module-sets to choose. First let's select module sets, and expand them. my $moduleResolver = $self->{module_resolver} = ksb::ModuleResolver->new($ctx); $moduleResolver->setCmdlineOptions($cmdlineOptions); $moduleResolver->setDeferredOptions($deferredOptions); $moduleResolver->setInputModulesAndOptions(\@optionModulesAndSets); $moduleResolver->setIgnoredSelectors([keys %ignoredSelectors]); # The user might only want metadata to update to allow for a later # --pretend run, check for that here. if (exists $cmdlineGlobalOptions->{'metadata-only'}) { return; } return @selectors; } # Requires establishContext to have been called first. Converts string-based # "selectors" for modules or module-sets into a list of ksb::Modules (only # modules, no sets), and returns associated metadata including dependencies. # # After this function is called all module set selectors will have been # expanded, and we will have downloaded kde-projects metadata. # # The modules returned must still be added (using setModulesToProcess) to the # context if you intend to build. This is a separate step to allow for some # introspection prior to making choice to build. # # Returns: A hashref to a workload object (as described in setModulesToProcess) sub modulesFromSelectors { my ($self, @selectors) = @_; my $moduleResolver = $self->{module_resolver}; my $ctx = $self->context(); my @modules; if (@selectors) { @modules = $moduleResolver->resolveSelectorsIntoModules(@selectors); } else { # Build everything in the rc-file, in the order specified. my @rcfileModules = @{$moduleResolver->{inputModulesAndOptions}}; @modules = $moduleResolver->expandModuleSets(@rcfileModules); } # If modules were on the command line then they are effectively forced to # process unless overridden by command line options as well. If phases # *were* overridden on the command line, then no update pass is required # (all modules already have correct phases) @modules = _updateModulePhases(@modules) unless @selectors; # TODO: Verify this does anything still my $metadataModule = $ctx->getKDEDependenciesMetadataModule(); $ctx->addToIgnoreList($metadataModule->scm()->ignoredModules()); # Remove modules that are explicitly blanked out in their branch-group # i.e. those modules where they *have* a branch-group, and it's set to # be empty (""). my $resolver = $ctx->moduleBranchGroupResolver(); my $branchGroup = $ctx->effectiveBranchGroup(); @modules = grep { my $branch = $_->isKDEProject() ? $resolver->findModuleBranch($_->fullProjectPath(), $branchGroup) : 1; # Just a placeholder truthy value whisper ("Removing ", $_->fullProjectPath(), " due to branch-group") if (defined $branch and !$branch); (!defined $branch or $branch); # This is the actual test } (@modules); my @modulesFromCommand = @modules; my $moduleGraph = $self->_resolveModuleDependencyGraph(@modules); if (!$moduleGraph || !exists $moduleGraph->{graph}) { croak_runtime("Failed to resolve dependency graph"); } if (exists $self->{debugFlags}->{'dependency-tree'}) { # Save for later introspection $self->{debugFlags}->{'dependency-tree'} = $moduleGraph->{graph}; my $result = { dependencyInfo => $moduleGraph, modulesFromCommand => \@modulesFromCommand, selectedModules => [], build => 0 }; return $result; } @modules = ksb::DependencyResolver::sortModulesIntoBuildOrder( $moduleGraph->{graph} ); # Filter --resume-foo options. This might be a second pass, but that should # be OK since there's nothing different going on from the first pass (in # resolveSelectorsIntoModules) in that event. @modules = _applyModuleFilters($ctx, @modules); my $result = { dependencyInfo => $moduleGraph, modulesFromCommand => \@modulesFromCommand, selectedModules => \@modules, build => 1, }; # If debugging then don't build $result->{build} = 0 if exists $self->{debugFlags}->{'list-build'}; return $result; } # Causes kde-projects metadata to be downloaded (unless --pretend, --no-src, or # --no-metadata is in effect, although we'll download even in --pretend if # nothing is available). # # No return value. sub _downloadKDEProjectMetadata { my $self = shift; my $ctx = $self->context(); my $updateStillNeeded = 0; my $wasPretending = pretending(); eval { for my $metadataModule ( $ctx->getKDEDependenciesMetadataModule(), $ctx->getKDEProjectsMetadataModule()) { my $sourceDir = $metadataModule->getSourceDir(); super_mkdir($sourceDir); my $moduleSource = $metadataModule->fullpath('source'); my $updateDesired = !$ctx->getOption('no-metadata') && $ctx->phases()->has('update'); my $updateNeeded = (! -e $moduleSource) || is_dir_empty($moduleSource); my $lastUpdate = $ctx->getPersistentOption('global', 'last-metadata-update') // 0; $updateStillNeeded ||= $updateNeeded; if (!$updateDesired && $updateNeeded && (time - ($lastUpdate)) >= 7200) { warning (" r[b[*] Skipping build metadata update, but it hasn't been updated recently!"); } if ($updateNeeded && pretending()) { warning (" y[b[*] Ignoring y[b[--pretend] option to download required metadata\n" . " y[b[*] --pretend mode will resume after metadata is available."); ksb::Debug::setPretending(0); } if ($updateDesired && (!pretending() || $updateNeeded)) { $metadataModule->scm()->updateInternal(); $ctx->setPersistentOption('global', 'last-metadata-update', time); } ksb::Debug::setPretending($wasPretending); } }; my $err = $@; ksb::Debug::setPretending($wasPretending); if ($err) { die $err if $updateStillNeeded; # Assume previously-updated metadata will work if not updating warning (" b[r[*] Unable to download required metadata for build process"); warning (" b[r[*] Will attempt to press onward..."); warning (" b[r[*] Exception message: $@"); } } # Returns a graph of Modules according to the kde-build-metadata dependency # information. # # The kde-build-metadata repository must have already been updated, and the # module factory must be setup. The modules for which to calculate the graph # must be passed in as arguments sub _resolveModuleDependencyGraph { my $self = shift; my $ctx = $self->context(); my $metadataModule = $ctx->getKDEDependenciesMetadataModule(); my @modules = @_; my $graph = eval { my $moduleResolver = $self->{module_resolver}; my $dependencyResolver = ksb::DependencyResolver->new(sub { # Maps module names (what dep resolver has) to built ksb::Modules # (which we need), needs to include all option handling (cmdline, # rc-file, module-sets, etc) return $moduleResolver->resolveModuleIfPresent(shift); }); my $branchGroup = $ctx->effectiveBranchGroup(); for my $file ('dependency-data-common', "dependency-data-$branchGroup") { my $dependencyFile = $metadataModule->fullpath('source') . "/$file"; my $dependencies = pretend_open($dependencyFile) or die "Unable to open $dependencyFile: $!"; debug (" -- Reading dependencies from $dependencyFile"); $dependencyResolver->readDependencyData($dependencies); close $dependencies; } return $dependencyResolver->resolveToModuleGraph(@modules); }; if ($@) { warning (" r[b[*] Problems encountered trying to determing correct module graph:"); warning (" r[b[*] $@"); warning (" r[b[*] Will attempt to continue."); $graph = { graph => undef, syntaxErrors => 0, cycles => 0, trivialCycles => 0, pathErrors => 0, branchErrors => 0, exception => $@ }; } else { if (!$graph->{graph}) { warning (" r[b[*] Unable to determine correct module graph"); warning (" r[b[*] Will attempt to continue."); } } $graph->{exception} = undef; return $graph; } # Similar to the old interactive runAllModulePhases. Actually performs the # build for the modules selected by setModulesToProcess. # # Returns a Mojo::Promise that must be waited on. The promise resolves to # return a single success/failure result; use the event handler for now to get # more detail during a build. sub startHeadlessBuild { my $self = shift; my $ctx = $self->context(); $ctx->statusMonitor()->createBuildPlan($ctx); my $promiseChain = ksb::PromiseChain->new; my $startPromise = Mojo::Promise->new; # These succeed or die outright $startPromise = _handle_updates ($ctx, $promiseChain, $startPromise); $startPromise = _handle_build ($ctx, $promiseChain, $startPromise); die "Can't obtain build lock" unless $ctx->takeLock(); # Install signal handlers to ensure that the lockfile gets closed. _installSignalHandlers(sub { @main::atexit_subs = (); # Remove their finish, doin' it manually $self->finish(5); }); $startPromise->resolve; # allow build to start once control returned to evt loop my $promise = $promiseChain->makePromiseChain($startPromise)->finally(sub { my @results = @_; my $result = 0; # success, non-zero is failure # Must use ! here to make '0 but true' hack work $result = 1 if defined first { !($_->[0] // 1) } @results; $ctx->statusMonitor()->markBuildDone(); $ctx->closeLock(); my $failedModules = join(',', map { "$_" } $ctx->listFailedModules()); if ($failedModules) { # We don't clear the list of failed modules on success so that # someone can build one or two modules and still use # --rebuild-failures $ctx->setPersistentOption('global', 'last-failed-module-list', $failedModules); } # TODO: Anything to do with this info at this point? my $workLoad = $self->workLoad(); my $dependencyGraph = $workLoad->{dependencyInfo}->{graph}; $ctx->storePersistentOptions(); _cleanup_log_directory($ctx); return $result; }); return $promise; } # Method: finish # # Exits the script cleanly, including removing any lock files created. # # Parameters: # ctx - Required; BuildContext to use. # [exit] - Optional; if passed, is used as the exit code, otherwise 0 is used. sub finish { my $self = shift; my $ctx = $self->context(); my $exitcode = shift // 0; if (pretending() || $self->{_base_pid} != $$) { # Abort early if pretending or if we're not the same process # that was started by the user (for async mode) exit $exitcode; } $ctx->closeLock(); $ctx->storePersistentOptions(); my $logdir = $ctx->getLogDir(); note ("Your logs are saved in y[$logdir]"); exit $exitcode; } ### Package-internal helper functions. # Reads a "line" from a file. This line is stripped of comments and extraneous # whitespace. Also, backslash-continued multiple lines are merged into a single # line. # # First parameter is the reference to the filehandle to read from. # Returns the text of the line. sub _readNextLogicalLine { my $fileReader = shift; while($_ = $fileReader->readLine()) { # Remove trailing newline chomp; # Replace \ followed by optional space at EOL and try again. if(s/\\\s*$//) { $_ .= $fileReader->readLine(); redo; } s/#.*$//; # Remove comments next if /^\s*$/; # Skip blank lines return $_; } return undef; } # Takes an input line, and extracts it into an option name, and simplified # value. The value has "false" converted to 0, white space simplified (like in # Qt), and tildes (~) in what appear to be path-like entries are converted to # the home directory path. # # First parameter is the build context (used for translating option values). # Second parameter is the line to split. # Return value is (option-name, option-value) sub _splitOptionAndValue { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $input = shift; my $optionRE = qr/\$\{([a-zA-Z0-9-]+)\}/; # The option is the first word, followed by the # flags on the rest of the line. The interpretation # of the flags is dependent on the option. my ($option, $value) = ($input =~ /^\s* # Find all spaces ([-\w]+) # First match, alphanumeric, -, and _ # (?: ) means non-capturing group, so (.*) is $value # So, skip spaces and pick up the rest of the line. (?:\s+(.*))?$/x); $value = trimmed($value // ''); # Simplify whitespace. $value =~ s/\s+/ /g; # Check for false keyword and convert it to Perl false. $value = 0 if lc($value) eq 'false'; # Replace reference to global option with their value. # The regex basically just matches ${option-name}. my ($sub_var_name) = ($value =~ $optionRE); while ($sub_var_name) { my $sub_var_value = $ctx->getOption($sub_var_name) || ''; if(!$ctx->hasOption($sub_var_name)) { warning (" *\n * WARNING: $sub_var_name is not set at line y[$.]\n *"); ## TODO: filename is missing } debug ("Substituting \${$sub_var_name} with $sub_var_value"); $value =~ s/\$\{$sub_var_name\}/$sub_var_value/g; # Replace other references as well. Keep this RE up to date with # the other one. ($sub_var_name) = ($value =~ $optionRE); } # Replace tildes with home directory. 1 while ($value =~ s"(^|:|=)~/"$1$ENV{'HOME'}/"); return ($option, $value); } # Ensures that the given ModuleSet has at least a valid repository and # use-modules setting based on the given BuildContext. sub _validateModuleSet { my ($ctx, $moduleSet) = @_; my $name = $moduleSet->name() || 'unnamed'; my $rcSources = _getModuleSources($moduleSet); # re-read option from module set since it may be pre-set my $selectedRepo = $moduleSet->getOption('repository'); if (!$selectedRepo) { error (<getOption('git-repository-base'); if ($selectedRepo ne KDE_PROJECT_ID && $selectedRepo ne QT_PROJECT_ID && not exists $repoSet->{$selectedRepo}) { my $projectID = KDE_PROJECT_ID; my $moduleSetName = $moduleSet->name(); my $moduleSetId = $moduleSetName ? "module-set ($moduleSetName)" : "module-set"; error (<isa('ksb::BuildContext') ? 'global' : $module->isa('ksb::ModuleSet') ? 'module-set' : $module->isa('ksb::Module') ? 'module' : 'options'; # Just look for an end marker if terminator not provided. $endRE //= qr/^end[\w\s]*$/; _markModuleSource($module, $fileReader->currentFilename() . ":$."); # Read in each option while (($_ = _readNextLogicalLine($fileReader)) && ($_ !~ $endRE)) { my $current_file = $fileReader->currentFilename(); # Sanity check, make sure the section is correctly terminated if(/^(module\b|options\b)/) { error ("Invalid configuration file $current_file at line $.\nAdd an 'end $endWord' before " . "starting a new module.\n"); die make_exception('Config', "Invalid file $current_file"); } my ($option, $value) = _splitOptionAndValue($ctx, $_); eval { $module->setOption($option, $value); }; if (my $err = $@) { if (blessed($err) && $err->isa('ksb::BuildException::Config')) { my $msg = "$current_file:$.: " . $err->message(); my $explanation = $err->optionUsageExplanation(); $msg = $msg . "\n" . $explanation if $explanation; $err->setMessage($msg); } die; # re-throw } } return $module; } # Marks the given OptionsBase subclass (i.e. Module or ModuleSet) as being # read in from the given string (filename:line). An OptionsBase can be # tagged under multiple files. sub _markModuleSource { my ($optionsBase, $configSource) = @_; my $key = '#defined-at'; my $sourcesRef = $optionsBase->hasOption($key) ? $optionsBase->getOption($key) : []; push @$sourcesRef, $configSource; $optionsBase->setOption($key, $sourcesRef); } # Returns rcfile sources for given OptionsBase (comma-separated). sub _getModuleSources { my $optionsBase = shift; my $key = '#defined-at'; my $sourcesRef = $optionsBase->getOption($key) || []; return join(', ', @$sourcesRef); } # Reads in a "moduleset". # # First parameter is the build context. # Second parameter is the filehandle to the config file to read from. # Third parameter is the ksb::ModuleSet to use. # # Returns the ksb::ModuleSet passed in with read-in options set, which may need # to be further expanded (see ksb::ModuleSet::convertToModules). sub _parseModuleSetOptions { my ($ctx, $fileReader, $moduleSet) = @_; $moduleSet = _parseModuleOptions($ctx, $fileReader, $moduleSet, qr/^end\s+module(-?set)?$/); # Perl-specific note! re-blessing the module set into the right 'class' # You'd probably have to construct an entirely new object and copy the # members over in other languages. if ($moduleSet->getOption('repository') eq KDE_PROJECT_ID) { bless $moduleSet, 'ksb::ModuleSet::KDEProjects'; } elsif ($moduleSet->getOption('repository') eq QT_PROJECT_ID) { bless $moduleSet, 'ksb::ModuleSet::Qt'; } return $moduleSet; } # Function: _readConfigurationOptions # # Reads in the settings from the configuration, passed in as an open # filehandle. # # Phase: # initialization - Do not call from this function. # # Parameters: # ctx - The to update based on the configuration read and # any pending command-line options (see cmdlineGlobalOptions). # # filehandle - The I/O object to read from. Must handle _eof_ and _readline_ # methods (e.g. subclass). # # deferredOptions - An out parameter: a hashref holding the options set by any # 'options' blocks read in by this function. Each key (identified by the name # of the 'options' block) will point to a hashref value holding the options to # apply. # # Returns: # @module - Heterogeneous list of and defined in the # configuration file. No module sets will have been expanded out (either # kde-projects or standard sets). # # Throws: # - Config exceptions. sub _readConfigurationOptions { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $fh = shift; my $deferredOptionsRef = shift; my @module_list; my $rcfile = $ctx->rcFile(); my ($option, %readModules); my $fileReader = ksb::RecursiveFH->new($rcfile); $fileReader->addFile($fh, $rcfile); # Read in global settings while ($_ = $fileReader->readLine()) { s/#.*$//; # Remove comments s/^\s+//; # Remove leading whitespace next unless $_; # Skip blank lines # First command in .kdesrc-buildrc should be a global # options declaration, even if none are defined. if (not /^global\s*$/) { error ("Invalid configuration file: $rcfile."); error ("Expecting global settings section at b[r[line $.]!"); die make_exception('Config', 'Missing global section'); } # Now read in each global option. _parseModuleOptions($ctx, $fileReader, $ctx); last; } my $using_default = 1; + my $creation_order = 0; my %seenModules; # NOTE! *not* module-sets, *just* modules. my %seenModuleSets; # and vice versa -- named sets only though! my %seenModuleSetItems; # To track option override modules. # Now read in module settings while ($_ = $fileReader->readLine()) { s/#.*$//; # Remove comments s/^\s*//; # Remove leading whitespace next if (/^\s*$/); # Skip blank lines # Get modulename (has dash, dots, slashes, or letters/numbers) my ($type, $modulename) = /^(options|module)\s+([-\/\.\w]+)\s*$/; my $newModule; # 'include' directives can change the current file, so check where we're at $rcfile = $fileReader->currentFilename(); # Module-set? if (not $modulename) { my $moduleSetRE = qr/^module-set\s*([-\/\.\w]+)?\s*$/; ($modulename) = m/$moduleSetRE/; # modulename may be blank -- use the regex directly to match if (not /$moduleSetRE/) { error ("Invalid configuration file $rcfile!"); error ("Expecting a start of module section at r[b[line $.]."); die make_exception('Config', 'Ungrouped/Unknown option'); } if ($modulename && exists $seenModuleSets{$modulename}) { error ("Duplicate module-set $modulename at $rcfile:$."); die make_exception('Config', "Duplicate module-set $modulename defined at $rcfile:$."); } if ($modulename && exists $seenModules{$modulename}) { error ("Name $modulename for module-set at $rcfile:$. is already in use on a module"); die make_exception('Config', "Can't re-use name $modulename for module-set defined at $rcfile:$."); } # A moduleset can give us more than one module to add. $newModule = _parseModuleSetOptions($ctx, $fileReader, ksb::ModuleSet->new($ctx, $modulename || "")); + $newModule->{'#create-id'} = ++$creation_order; # Save 'use-modules' entries so we can see if later module decls # are overriding/overlaying their options. my @moduleSetItems = $newModule->moduleNamesToFind(); @seenModuleSetItems{@moduleSetItems} = ($newModule) x scalar @moduleSetItems; $seenModuleSets{$modulename} = $newModule if $modulename; } # Duplicate module entry? (Note, this must be checked before the check # below for 'options' sets) elsif (exists $seenModules{$modulename} && $type ne 'options') { error ("Duplicate module declaration b[r[$modulename] on line $. of $rcfile"); die make_exception('Config', "Duplicate module $modulename declared at $rcfile:$."); } # Module/module-set options overrides elsif ($type eq 'options') { my $options = _parseModuleOptions($ctx, $fileReader, ksb::OptionsBase->new()); $deferredOptionsRef->{$modulename} = $options->{options}; next; # Don't add to module list } # Must follow 'options' handling elsif (exists $seenModuleSets{$modulename}) { error ("Name $modulename for module at $rcfile:$. is already in use on a module-set"); die make_exception('Config', "Can't re-use name $modulename for module defined at $rcfile:$."); } else { $newModule = _parseModuleOptions($ctx, $fileReader, ksb::Module->new($ctx, $modulename)); + $newModule->{'#create-id'} = ++$creation_order; $seenModules{$modulename} = $newModule; } push @module_list, $newModule; $using_default = 0; } while (my ($name, $moduleSet) = each %seenModuleSets) { _validateModuleSet($ctx, $moduleSet); } # If the user doesn't ask to build any modules, build a default set. # The good question is what exactly should be built, but oh well. if ($using_default) { warning (" b[y[*] There do not seem to be any modules to build in your configuration."); return (); } return @module_list; } # Exits out of kdesrc-build, executing the user's preferred shell instead. The # difference is that the environment variables should be as set in kdesrc-build # instead of as read from .bashrc and friends. # # You should pass in the options to run the program with as a list. # # Meant to implement the --run command line option. sub _executeCommandLineProgram { my ($program, @args) = @_; if (!$program) { error ("You need to specify a program with the --run option."); exit 1; # Can't use finish here. } if (($< != $>) && ($> == 0)) { error ("kdesrc-build will not run a program as root unless you really are root."); exit 1; } debug ("Executing b[r[$program] ", join(' ', @args)); exit 0 if pretending(); exec $program, @args or do { # If we get to here, that sucks, but don't continue. error ("Error executing $program: $!"); exit 1; }; } # Function: _split_url # # Subroutine to split a url into a protocol and host sub _split_url { my $url = shift; my ($proto, $host) = ($url =~ m|([^:]*)://([^/]*)/|); return ($proto, $host); } # Function: _check_for_ssh_agent # # Checks if we are supposed to use ssh agent by examining the environment, and # if so checks if ssh-agent has a list of identities. If it doesn't, we run # ssh-add (with no arguments) and inform the user. This can be controlled with # the disable-agent-check parameter. # # Parameters: # 1. Build context sub _check_for_ssh_agent { my $ctx = assert_isa(shift, 'ksb::BuildContext'); # Don't bother with all this if the user isn't even using SSH. return 1 if pretending(); my @svnServers = grep { $_->scmType() eq 'svn' } ($ctx->modulesInPhase('update')); my @gitServers = grep { $_->scmType() eq 'git' } ($ctx->modulesInPhase('update')); my @sshServers = grep { my ($proto, $host) = _split_url($_->getOption('svn-server')); # Check if ssh is explicitly used in the proto, or if the host is the # developer main svn. (defined $proto && $proto =~ /ssh/) || (defined $host && $host =~ /^svn\.kde\.org/); } @svnServers; push @sshServers, grep { # Check for git+ssh:// or git@git.kde.org:/path/etc. my $repo = $_->getOption('repository'); ($repo =~ /^git\+ssh:\/\//) || ($repo =~ /^[a-zA-Z0-9_.]+@.*:\//); } @gitServers; return 1 if (not @sshServers) or $ctx->getOption('disable-agent-check'); whisper ("\tChecking for SSH Agent") if (scalar @sshServers); # We're using ssh to download, see if ssh-agent is running. return 1 unless exists $ENV{'SSH_AGENT_PID'}; my $pid = $ENV{'SSH_AGENT_PID'}; # It's supposed to be running, let's see if there exists the program with # that pid (this check is linux-specific at the moment). if (-d "/proc" and not -e "/proc/$pid") { warning ("r[ *] SSH Agent is enabled, but y[doesn't seem to be running]."); warning ("Since SSH is used to download from Subversion you may want to see why"); warning ("SSH Agent is not working, or correct the environment variable settings."); return 0; } # The agent is running, but does it have any keys? We can't be more specific # with this check because we don't know what key is required. my $noKeys = 0; filter_program_output(sub { $noKeys ||= /no identities/ }, 'ssh-add', '-l'); if ($noKeys) { # Use print so user can't inadvertently keep us quiet about this. print ksb::Debug::colorize (<getOption('ssh-identity-file'); push (@commandLine, $identFile) if $identFile; my $result = system (@commandLine); if ($result) # Run this code for both death-by-signal and nonzero return { my $rcfile = $ctx->rcFile(); print "\nUnable to add SSH identity, aborting.\n"; print "If you don't want kdesrc-build to check in the future,\n"; print ksb::Debug::colorize ("Set the g[disable-agent-check] option to g[true] in your $rcfile.\n\n"); return 0; } } return 1; } # Function: _handle_updates # # Subroutine to update a list of modules. Uses a Mojolicious event loop # to run each update in a subprocess to avoid blocking the script. Only # one update process will exist at a given time. # # Parameters: # 1. Build Context, which will be used to determine the module update list. # 2. A PromiseChain for adding work items and dependencies. # 3. A "start promise" that can be waited on for pre-update steps. # # This function accounts for every module in $ctx's update phase. # # Returns an updated start promise and can also throw exception on error sub _handle_updates { my ($ctx, $promiseChain, $start_promise) = @_; my $kdesrc = $ctx->getSourceDir(); my @update_list = $ctx->modulesInPhase('update'); return $start_promise unless @update_list; croak_runtime("SSH agent is not running but should be") unless _check_for_ssh_agent($ctx); # TODO: Extract this to a setup function that all updates/build depend upon whisper ("Creating source directory") unless -e $kdesrc; croak_runtime ("Unable to make directory r[$kdesrc]! $!") if (! -e $kdesrc && !super_mkdir ($kdesrc)); for my $module (@update_list) { # sub must be defined here to capture $module in the loop my $updateSub = sub { return $module->runPhase_p('update', # called in child process, can block sub { return $module->update($ctx) }, # called in this process, with results sub { my (undef, $was_successful, $extras) = @_; $module->setOption('#numUpdates', $extras->{update_count}); return $was_successful; } ); }; $promiseChain->addItem("$module/update", "network-queue", $updateSub); } return $start_promise; } # Throws an exception if essential build programs are missing as a sanity check. sub _checkForEarlyBuildExit { my $ctx = shift; my @modules = $ctx->modulesInPhase('build'); # Check for absolutely essential programs now. if (!_checkForEssentialBuildPrograms($ctx) && !exists $ENV{KDESRC_BUILD_IGNORE_MISSING_PROGRAMS}) { error (" r[b[*] Aborting now to save a lot of wasted time."); error (" y[b[*] export KDESRC_BUILD_IGNORE_MISSING_PROGRAMS=1 and re-run (perhaps with --no-src)"); error (" r[b[*] to continue anyways. If this check was in error please report a bug against"); error (" y[b[*] kdesrc-build at https://bugs.kde.org/"); croak_runtime ("Essential build programs are missing!"); } } sub _openStatusFileHandle { my $ctx = shift; my $outfile = pretending() ? '/dev/null' : $ctx->getLogDir() . '/build-status'; my $statusFile; open $statusFile, '>', $outfile or do { error (<modulesInPhase('build'); my $result = 0; _checkForEarlyBuildExit($ctx); # exception-thrower my $num_modules = scalar @modules; my ($statusFile, $outfile) = _openStatusFileHandle($ctx); my $everFailed = 0; # This generates a bunch of subs but doesn't call them yet foreach my $module (@modules) { # Needs to happen in this loop to capture $module my $buildSub = sub { return if ($everFailed && $module->getOption('stop-on-failure')); my $fail_count = $module->getPersistentOption('failure-count') // 0; my $num_updates = int ($module->getOption('#numUpdates', 'module') // 1); # check for skipped updates, --no-src forces build-when-unchanged # even when ordinarily disabled if ($num_updates == 0 && !$module->getOption('build-when-unchanged') && $fail_count == 0) { # TODO: Why is the param order reversed for these two? $ctx->statusMonitor()->markPhaseStart("$module", 'build'); $ctx->markModulePhaseSucceeded('build', $module); return 'skipped'; } # Can't build w/out blocking so return a promise instead, which ->build # already supplies return $module->build()->catch(sub { my $failureReason = shift; if (!$everFailed) { # No failures yet, mark this as resume point $everFailed = 1; my $moduleList = join(', ', map { "$_" } ($module, @modules)); $ctx->setPersistentOption('global', 'resume-list', $moduleList); } ++$fail_count; # Force this promise chain to stay dead return Mojo::Promise->new->reject('build'); })->then(sub { $fail_count = 0; })->finally(sub { $module->setPersistentOption('failure-count', $fail_count); }); }; $promiseChain->addItem("$module/build", 'cpu-queue', $buildSub); # If there's an update phase we need to depend on it and show status if (my $updatePromise = $promiseChain->promiseFor("$module/update")) { $promiseChain->addDep("$module/build", "$module/update"); } }; # Add to the build 'queue' for promise chain so that this runs only after all # other build jobs $promiseChain->addDep('@postBuild', 'cpu-queue', sub { if ($statusFile) { close $statusFile; # Update the symlink in latest to point to this file. my $logdir = $ctx->getSubdirPath('log-dir'); if (-l "$logdir/latest/build-status") { safe_unlink("$logdir/latest/build-status"); } symlink($outfile, "$logdir/latest/build-status"); } return Mojo::Promise->new->reject if $everFailed; return 0; }); return $start_promise->then( sub { $ctx->unsetPersistentOption('global', 'resume-list') }); } # Function: _handle_async_build # # This subroutine special-cases the handling of the update and build phases, by # performing them concurrently using forked processes and non-blocking I/O. # See Mojo::Promise and Mojo::IOLoop::Subprocess # # This procedure will use multiple processes (the main process and separate # processes for each update or build as they occur). # # Parameters: # 1. Build Context to use, from which the module lists will be determined. # # Returns 0 on success, non-zero on failure. sub _handle_async_build { my ($ctx) = @_; my $result = 0; $ctx->statusMonitor()->createBuildPlan($ctx); my $promiseChain = ksb::PromiseChain->new; my $start_promise = Mojo::Promise->new; # These succeed or die outright eval { $start_promise = _handle_updates ($ctx, $promiseChain, $start_promise); $start_promise = _handle_build ($ctx, $promiseChain, $start_promise); }; if ($@) { error ("Caught an error $@ setting up to build"); return 1; } my $chain = $promiseChain->makePromiseChain($start_promise) ->finally(sub { # Fail if we had a zero-valued result (indicates error) my @results = @_; # Must use ! here to make '0 but true' hack work $result = 1 if defined first { !($_->[0] // 1) } @results; $ctx->statusMonitor()->markBuildDone(); }); # Start the update/build process $start_promise->resolve; Mojo::IOLoop->stop; # Force the wait below to block $chain->wait; return $result; } # Function: _handle_install # # Handles the installation process. Simply calls 'make install' in the build # directory, though there is also provision for cleaning the build directory # afterwards, or stopping immediately if there is a build failure (normally # every built module is attempted to be installed). # # Parameters: # 1. Build Context, from which the install list is generated. # # Return value is a shell-style success code (0 == success) sub _handle_install { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my @modules = $ctx->modulesInPhase('install'); @modules = grep { $_->buildSystem()->needsInstalled() } (@modules); my $result = 0; for my $module (@modules) { $ctx->resetEnvironment(); $result = $module->install() || $result; if ($result && $module->getOption('stop-on-failure')) { note ("y[Stopping here]."); return 1; # Error } } return $result; } # Function: _handle_uninstall # # Handles the uninstal process. Simply calls 'make uninstall' in the build # directory, while assuming that Qt or CMake actually handles it. # # The order of the modules is often significant, and it may work better to # uninstall modules in reverse order from how they were installed. However this # code does not automatically reverse the order; modules are uninstalled in the # order determined by the build context. # # This function obeys the 'stop-on-failure' option supported by _handle_install. # # Parameters: # 1. Build Context, from which the uninstall list is generated. # # Return value is a shell-style success code (0 == success) sub _handle_uninstall { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my @modules = $ctx->modulesInPhase('uninstall'); @modules = grep { $_->buildSystem()->needsInstalled() } (@modules); my $result = 0; for my $module (@modules) { $ctx->resetEnvironment(); $result = $module->uninstall() || $result; if ($result && $module->getOption('stop-on-failure')) { note ("y[Stopping here]."); return 1; # Error } } return $result; } # Function: _applyModuleFilters # # Applies any module-specific filtering that is necessary after reading command # line and rc-file options. (This is as opposed to phase filters, which leave # each module as-is but change the phases they operate as part of, this # function could remove a module entirely from the build). # # Used for --resume-{from,after} and --stop-{before,after}, but more could be # added in theory. # This subroutine supports --{resume,stop}-* for both modules and module-sets. # # Parameters: # ctx - in use. # @modules - List of or to apply filters on. # # Returns: # list of or with any inclusion/exclusion filters # applied. Do not assume this list will be a strict subset of the input list, # however the order will not change amongst the input modules. sub _applyModuleFilters { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my @moduleList = @_; if (!$ctx->getOption('resume-from') && !$ctx->getOption('resume-after') && !$ctx->getOption('stop-before') && !$ctx->getOption('stop-after')) { debug ("No command-line filter seems to be present."); return @moduleList; } if ($ctx->getOption('resume-from') && $ctx->getOption('resume-after')) { # This one's an error. error (<getOption('stop-before') && $ctx->getOption('stop-after')) { # This one's an error. error (<getOption('resume-from') || $ctx->getOption('resume-after'); my $startIndex = scalar @moduleList; if ($resumePoint) { debug ("Looking for $resumePoint for --resume-* option"); # || 0 is a hack to force Boolean context. my $filterInclusive = $ctx->getOption('resume-from') || 0; my $found = 0; for (my $i = 0; $i < scalar @moduleList; $i++) { my $module = $moduleList[$i]; $found = $module->name() eq $resumePoint; if ($found) { $startIndex = $filterInclusive ? $i : $i + 1; $startIndex = min($startIndex, scalar @moduleList - 1); last; } } } else { $startIndex = 0; } my $stopPoint = $ctx->getOption('stop-before') || $ctx->getOption('stop-after'); my $stopIndex = 0; if ($stopPoint) { debug ("Looking for $stopPoint for --stop-* option"); # || 0 is a hack to force Boolean context. my $filterInclusive = $ctx->getOption('stop-before') || 0; my $found = 0; for (my $i = $startIndex; $i < scalar @moduleList; $i++) { my $module = $moduleList[$i]; $found = $module->name() eq $stopPoint; if ($found) { $stopIndex = $i - ($filterInclusive ? 1 : 0); last; } } } else { $stopIndex = scalar @moduleList - 1; } if ($startIndex > $stopIndex || scalar @moduleList == 0) { # Lost all modules somehow. croak_runtime("Unknown resume -> stop point $resumePoint -> $stopPoint."); } return @moduleList[$startIndex .. $stopIndex]; } # Updates the built-in phase list for all Modules passed into this function in # accordance with the options set by the user. sub _updateModulePhases { whisper ("Filtering out module phases."); for my $module (@_) { if ($module->getOption('manual-update') || $module->getOption('no-svn') || $module->getOption('no-src')) { $module->phases()->clear(); next; } if ($module->getOption('manual-build')) { $module->phases()->filterOutPhase('build'); $module->phases()->filterOutPhase('test'); $module->phases()->filterOutPhase('install'); } $module->phases()->filterOutPhase('install') unless $module->getOption('install-after-build'); $module->phases()->addPhase('test') if $module->getOption('run-tests'); } return @_; } # This subroutine extract the value from options of the form --option=value, # which can also be expressed as --option value. # # The first parameter is the option that the user passed to the cmd line (e.g. # --prefix=/opt/foo). # The second parameter is a reference to the list of command line options. # # The return value is the value of the option (the list of options might be # shorter by 1, copy it if you don't want it to change), or undef if no value # was provided. sub _extractOptionValue { my ($option, $options_ref) = @_; if ($option =~ /=/) { my @value = split(/=/, $option); shift @value; # We don't need the first one, that the --option part. return if (scalar @value == 0); # If we have more than one element left in @value it's because the # option itself has an = in it, make sure it goes back in the answer. return join('=', @value); } return if scalar @{$options_ref} == 0; return shift @{$options_ref}; } # Like _extractOptionValue, but throws an exception if the value is not # actually present, so you don't have to check for it yourself. If you do get a # return value, it will be defined to something. sub _extractOptionValueRequired { my ($option, $options_ref) = @_; my $returnValue = _extractOptionValue($option, $options_ref); if (not defined $returnValue) { croak_runtime("Option $option needs to be set to some value instead of left blank"); } return $returnValue; } # Function: _cleanup_log_directory # # This function removes log directories from old kdesrc-build runs. All log # directories not referenced by $log_dir/latest somehow are made to go away. # # Parameters: # 1. Build context. # # No return value. sub _cleanup_log_directory { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $logdir = $ctx->getSubdirPath('log-dir'); return 0 if ! -e "$logdir/latest"; # Could happen for error on first run... # This glob relies on the date being in the specific format YYYY-MM-DD-ID my @dirs = bsd_glob("$logdir/????-??-??-??/", GLOB_NOSORT); my @needed = _reachableModuleLogs("$logdir/latest"); # Convert a list to a hash lookup since Perl lacks a "list-has" my %needed_table; @needed_table{@needed} = (1) x @needed; for my $dir (@dirs) { my ($id) = ($dir =~ m/(\d\d\d\d-\d\d-\d\d-\d\d)/); safe_rmtree($dir) unless $needed_table{$id}; } } # Function: _output_failed_module_list # # Print out an error message, and a list of modules that match that error # message. It will also display the log file name if one can be determined. # The message will be displayed all in uppercase, with PACKAGES prepended, so # all you have to do is give a descriptive message of what this list of # packages failed at doing. # # No message is printed out if the list of failed modules is empty, so this # function can be called unconditionally. # # Parameters: # 1. Build Context # 2. Message to print (e.g. 'failed to foo') # 3. List of ksb::Modules that had failed to foo # # No return value. sub _output_failed_module_list { my ($ctx, $message, @fail_list) = @_; assert_isa($ctx, 'ksb::BuildContext'); $message = uc $message; # Be annoying if (@fail_list) { debug ("Message is $message"); debug ("\tfor ", join(', ', @fail_list)); } if (scalar @fail_list > 0) { my $homedir = $ENV{'HOME'}; my $logfile; warning ("\nr[b[<<< PACKAGES $message >>>]"); for my $module (@fail_list) { $logfile = $module->getOption('#error-log-file'); # async updates may cause us not to have a error log file stored # (though this should now only happen due to other bugs). Since # there's only one place it should be, take advantage of # side-effect of log_command() to find it. if (not $logfile) { my $logdir = $module->getLogDir() . "/error.log"; $logfile = $logdir if -e $logdir; } $logfile = "No log file" unless $logfile; $logfile =~ s|$homedir|~|; warning ("r[$module]") if pretending(); warning ("r[$module] - g[$logfile]") if not pretending(); } } } # Function: _output_failed_module_lists # # This subroutine reads the list of failed modules for each phase in the build # context and calls _output_failed_module_list for all the module failures. # # Parameters: # 1. Build context # # Return value: # None sub _output_failed_module_lists { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $moduleGraph = shift; my $extraDebugInfo = { phases => {}, failCount => {} }; my @actualFailures = (); # This list should correspond to the possible phase names (although # it doesn't yet since the old code didn't, TODO) for my $phase ($ctx->phases()->phases()) { my @failures = $ctx->failedModulesInPhase($phase); for my $failure (@failures) { # we already tagged the failure before, should not happen but # make sure to check to avoid spurious duplicate output next if $extraDebugInfo->{phases}->{$failure}; $extraDebugInfo->{phases}->{$failure} = $phase; push @actualFailures, $failure; } _output_failed_module_list($ctx, "failed to $phase", @failures); } # See if any modules fail continuously and warn specifically for them. my @super_fail = grep { ($_->getPersistentOption('failure-count') // 0) > 3 } (@{$ctx->moduleList()}); if (@super_fail) { warning ("\nThe following modules have failed to build 3 or more times in a row:"); warning ("\tr[b[$_]") foreach @super_fail; warning ("\nThere is probably a local error causing this kind of consistent failure, it"); warning ("is recommended to verify no issues on the system.\n"); } my $top = 5; my $numSuggestedModules = scalar @actualFailures; # # Omit listing $top modules if there are that many or fewer anyway. # Not much point ranking 4 out of 4 failures, # this feature is meant for 5 out of 65 # if ($numSuggestedModules > $top) { my @sortedForDebug = ksb::DebugOrderHints::sortFailuresInDebugOrder( $moduleGraph, $extraDebugInfo, \@actualFailures ); info ("\nThe following top $top may be the most important to fix to " . "get the build to work, listed in order of 'probably most " . "interesting' to 'probably least interesting' failure:\n"); info ("\tr[b[$_]") foreach (@sortedForDebug[0..($top - 1)]); } } # Function: _installTemplatedFile # # This function takes a given file and a build context, and installs it to a # given location while expanding out template entries within the source file. # # The template language is *extremely* simple: <% foo %> is replaced entirely # with the result of $ctx->getOption(foo, 'no-inherit'). If the result # evaluates false for any reason than an exception is thrown. No quoting of # any sort is used in the result, and there is no way to prevent expansion of # something that resembles the template format. # # Multiple template entries on a line will be replaced. # # The destination file will be created if it does not exist. If the file # already exists then an exception will be thrown. # # Error handling: Any errors will result in an exception being thrown. # # Parameters: # 1. Pathname to the source file (use absolute paths) # 2. Pathname to the destination file (use absolute paths) # 3. Build context to use for looking up template values # # Return value: There is no return value. sub _installTemplatedFile { my ($sourcePath, $destinationPath, $ctx) = @_; assert_isa($ctx, 'ksb::BuildContext'); open (my $input, '<', $sourcePath) or croak_runtime("Unable to open template source $sourcePath: $!"); open (my $output, '>', $destinationPath) or croak_runtime("Unable to open template output $destinationPath: $!"); while (!eof ($input)) { my $line = readline($input); if (!defined ($line)) { croak_runtime("Failed to read from $sourcePath at line $.: $!"); unlink($destinationPath); } # Some lines should only be present in the source as they aid with testing. next if $line =~ /kdesrc-build: filter/; $line =~ s { <% \s* # Template bracket and whitespace ([^\s%]+) # Capture variable name \s*%> # remaining whitespace and closing bracket } { $ctx->getOption($1, 'module') // croak_runtime("Invalid variable $1") }gxe; # Replace all matching expressions, use extended regexp w/ # comments, and replacement is Perl code to execute. (print $output $line) or croak_runtime("Unable to write line to $destinationPath at line $.: $!"); } } # Function: _installCustomFile # # This function installs a source file to a destination path, assuming the # source file is a "templated" source file (see also _installTemplatedFile), and # records a digest of the file actually installed. This function will overwrite # a destination if the destination is identical to the last-installed file. # # Error handling: Any errors will result in an exception being thrown. # # Parameters: # 1. Build context to use for looking up template values, # 2. The full path to the source file. # 3. The full path to the destination file (incl. name) # 4. The key name to use for searching/recording installed MD5 digest. # # Return value: There is no return value. sub _installCustomFile { use File::Copy qw(copy); my $ctx = assert_isa(shift, 'ksb::BuildContext'); my ($sourceFilePath, $destFilePath, $md5KeyName) = @_; my $baseName = basename($sourceFilePath); if (-e $destFilePath) { my $existingMD5 = $ctx->getPersistentOption('/digests', $md5KeyName) // ''; if (fileDigestMD5($destFilePath) ne $existingMD5) { if (!$ctx->getOption('#delete-my-settings')) { error ("\tr[*] Installing \"b[$baseName]\" would overwrite an existing file:"); error ("\tr[*] y[b[$destFilePath]"); error ("\tr[*] If this is acceptable, please delete the existing file and re-run,"); error ("\tr[*] or pass b[--delete-my-settings] and re-run."); return; } elsif (!pretending()) { copy ($destFilePath, "$destFilePath.kdesrc-build-backup"); } } } if (!pretending()) { _installTemplatedFile($sourceFilePath, $destFilePath, $ctx); $ctx->setPersistentOption('/digests', $md5KeyName, fileDigestMD5($destFilePath)); } } # Function: _installCustomSessionDriver # # This function installs the included sample .xsession and environment variable # setup files, and records the md5sum of the installed results. # # If a file already exists, then its md5sum is taken and if the same as what # was previously installed, is overwritten. If not the same, the original file # is left in place and the .xsession is instead installed to # .xsession-kdesrc-build # # Error handling: Any errors will result in an exception being thrown. # # Parameters: # 1. Build context to use for looking up template values, # # Return value: There is no return value. sub _installCustomSessionDriver { use FindBin qw($RealBin); use List::Util qw(first); use File::Copy qw(copy); my $ctx = assert_isa(shift, 'ksb::BuildContext'); my @xdgDataDirs = split(':', $ENV{XDG_DATA_DIRS} || '/usr/local/share/:/usr/share/'); my $xdgDataHome = $ENV{XDG_DATA_HOME} || "$ENV{HOME}/.local/share"; # First we have to find the source my @searchPaths = ($RealBin, map { "$_/apps/kdesrc-build" } ($xdgDataHome, @xdgDataDirs)); s{/+$}{} foreach @searchPaths; # Remove trailing slashes s{//+}{/}g foreach @searchPaths; # Remove duplicate slashes my $envScript = first { -f $_ } ( map { "$_/sample-kde-env-master.sh" } @searchPaths ); my $sessionScript = first { -f $_ } ( map { "$_/sample-xsession.sh" } @searchPaths ); if (!$envScript || !$sessionScript) { warning ("b[*] Unable to find helper files to setup a login session."); warning ("b[*] You will have to setup login yourself, or install kdesrc-build properly."); return; } my $destDir = $ENV{XDG_CONFIG_HOME} || "$ENV{HOME}/.config"; super_mkdir($destDir) unless -d $destDir; _installCustomFile($ctx, $envScript, "$destDir/kde-env-master.sh", 'kde-env-master-digest'); _installCustomFile($ctx, $sessionScript, "$ENV{HOME}/.xsession", 'xsession-digest') if $ctx->getOption('install-session-driver'); if (!pretending()) { if ($ctx->getOption('install-session-driver') && !chmod (0744, "$ENV{HOME}/.xsession")) { error ("\tb[r[*] Error making b[~/.xsession] executable: $!"); error ("\tb[r[*] If this file is not executable you may not be able to login!"); }; } } # Function: _checkForEssentialBuildPrograms # # This subroutine checks for programs which are absolutely essential to the # *build* process and returns false if they are not all present. Right now this # just means qmake and cmake (although this depends on what modules are # actually present in the build context). # # Parameters: # 1. Build context # # Return value: # None sub _checkForEssentialBuildPrograms { my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $kdedir = $ctx->getOption('kdedir'); my $qtdir = $ctx->getOption('qtdir'); my @preferred_paths = ("$kdedir/bin", "$qtdir/bin"); return 1 if pretending(); my @buildModules = $ctx->modulesInPhase('build'); my %requiredPrograms; my %modulesRequiringProgram; foreach my $module ($ctx->modulesInPhase('build')) { my @progs = $module->buildSystem()->requiredPrograms(); # Deliberately used @, since requiredPrograms can return a list. @requiredPrograms{@progs} = 1; foreach my $prog (@progs) { $modulesRequiringProgram{$prog} //= { }; $modulesRequiringProgram{$prog}->{$module->name()} = 1; } } my $wasError = 0; for my $prog (keys %requiredPrograms) { my %requiredPackages = ( qmake => 'Qt', cmake => 'CMake', meson => 'Meson', ); my $preferredPath = absPathToExecutable($prog, @preferred_paths); my $programPath = $preferredPath || absPathToExecutable($prog); # qmake is not necessarily named 'qmake' if (!$programPath && $prog eq 'qmake') { $programPath = ksb::BuildSystem::QMake::absPathToQMake(); } if (!$programPath) { # Don't complain about Qt if we're building it... if ($prog eq 'qmake' && ( grep { $_->buildSystemType() eq 'Qt' || $_->buildSystemType() eq 'Qt5' } (@buildModules)) || pretending() ) { next; } $wasError = 1; my $reqPackage = $requiredPackages{$prog} || $prog; my @modulesNeeding = keys %{$modulesRequiringProgram{$prog}}; local $, = ', '; # List separator in output error (<<"EOF"); Unable to find r[b[$prog]. This program is absolutely essential for building the modules: y[@modulesNeeding]. Please ensure the development packages for $reqPackage are installed by using your distribution's package manager. EOF } } return !$wasError; } # Function: _reachableModuleLogs # # Returns a list of module directory IDs that must be kept due to being # referenced from the "latest" symlink. # # This function may call itself recursively if needed. # # Parameters: # 1. The log directory under which to search for symlinks, including the "/latest" # part of the path. sub _reachableModuleLogs { my $logdir = shift; my @dirs; # A lexicalized var (my $foo) is required in face of recursiveness. opendir(my $fh, $logdir) or croak_runtime("Can't opendir $logdir: $!"); my $dir = readdir($fh); while(defined $dir) { if (-l "$logdir/$dir") { my $link = readlink("$logdir/$dir"); push @dirs, $link; } elsif ($dir !~ /^\.{1,2}$/) { # Skip . and .. directories (this is a great idea, trust me) push @dirs, _reachableModuleLogs("$logdir/$dir"); } $dir = readdir $fh; } closedir $fh; # Extract numeric IDs from directory names. @dirs = map { m/(\d{4}-\d\d-\d\d-\d\d)/ } (@dirs); # Convert to unique list by abusing hash keys. my %tempHash; @tempHash{@dirs} = (); return keys %tempHash; } # Installs the given subroutine as a signal handler for a set of signals which # could kill the program. # # First parameter is a reference to the sub to act as the handler. sub _installSignalHandlers { use Carp qw(confess); my $handlerRef = shift; my @signals = qw/HUP INT QUIT ABRT TERM PIPE/; foreach my $signal (@signals) { $SIG{$signal} = sub { confess ("Signal SIG$signal received, terminating.") unless $signal eq 'INT'; $handlerRef->(); }; } } # Ensures that basic one-time setup to actually *use* installed software is # performed, including .kdesrc-buildrc setup if necessary. # # Returns the appropriate exitcode to pass to the exit function sub performInitialUserSetup { my $self = shift; return ksb::FirstRun::setupUserSystem(); } # Shows a help message and version. Does not exit. sub _showHelpMessage { my $scriptVersion = scriptVersion(); say < and others, and is +Copyright (c) 2003 - 2020 Michael Pyne and others, and is distributed under the terms of the GNU GPL v2. This script automates the download, build, and install process for KDE software using the latest available source code. Configuration is controlled from "\$PWD/kdesrc-buildrc" or "~/.kdesrc-buildrc". See kdesrc-buildrc-sample for an example. Usage: \$ $0 [--options] [module names] All configured modules are built if none are listed. Important Options: --pretend Don't actually take major actions, instead describe what would be done. --list-build List what modules would be built in the order in which they would be built. --dependency-tree Print out dependency information on the modules that would be built, using a `tree` format. Very useful for learning how modules relate to each other. May generate a lot of output. --no-src Don't update source code, just build/install. --src-only Only update the source code --refresh-build Start the build from scratch. --rc-file= Read configuration from filename instead of default. --initial-setup Installs Plasma env vars (~/.bashrc), required system pkgs, and a base kdesrc-buildrc. --resume-from= Skips modules until just before or after the given --resume-after= package, then operates as normal. --stop-before= Stops just before or after the given package is --stop-after= reached. --include-dependencies Also builds KDE-based dependencies of given modules. (This is enabled by default; use --no-include-dependencies to disable) --stop-on-failure Stops the build as soon as a package fails to build. More docs at https://docs.kde.org/trunk5/en/extragear-utils/kdesrc-build/ Supported configuration options: https://go.kde.org/u/ksboptions Supported cmdline options: https://go.kde.org/u/ksbcmdline DONE # Look for indications this is the first run. if (! -e "./kdesrc-buildrc" && ! -e "$ENV{HOME}/.kdesrc-buildrc") { say <{context}; } sub metadataModule { my $self = shift; return $self->{metadata_module}; } sub runMode { my $self = shift; return $self->{run_mode}; } sub modules { my $self = shift; return @{$self->{modules}}; } sub workLoad { my $self = shift; return $self->{workLoad}; } 1; diff --git a/modules/ksb/BuildContext.pm b/modules/ksb/BuildContext.pm index 522e487..169904d 100644 --- a/modules/ksb/BuildContext.pm +++ b/modules/ksb/BuildContext.pm @@ -1,1058 +1,1053 @@ package ksb::BuildContext 0.36; # Class: BuildContext # # This contains the information needed about the build context, e.g. list of # modules, what phases each module is in, the various options, etc. use 5.014; use warnings; no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; use Carp 'confess'; use File::Basename; # dirname use IO::File; use POSIX qw(strftime); use Errno qw(:POSIX); use Mojo::JSON qw(encode_json decode_json); # We derive from ksb::Module so that BuildContext acts like the 'global' # ksb::Module, with some extra functionality. # TODO: Derive from OptionsBase directly and remove getOption override use parent qw(ksb::Module); use ksb::BuildException; use ksb::Debug; use ksb::Util; use ksb::PhaseList; use ksb::Module; use ksb::Module::BranchGroupResolver; use ksb::Updater::KDEProjectMetadata 0.20; use ksb::Version qw(scriptVersion); use ksb::StatusView 0.30; use ksb::StatusMonitor; use ksb::KDEProjectsReader 0.50; use File::Temp qw(tempfile); use File::Spec; # rel2abs use List::Util qw(sum); my @DefaultPhases = qw/update build install/; my @rcfiles = ("./kdesrc-buildrc", "$ENV{HOME}/.kdesrc-buildrc"); my $LOCKFILE_NAME = '.kdesrc-lock'; # The # will be replaced by the directory the rc File is stored in. my $PERSISTENT_FILE_NAME = '#/.kdesrc-build-data'; my $SCRIPT_VERSION = scriptVersion(); # Should be used for internal state that shouldn't be exposed as a hidden # cmdline option, or has other cmdline switches (e.g. debug/verbose handling). my %internalGlobalOptions = ( "async" => 1, "build-system-only" => "", "build-when-unchanged" => 1, # Safe default "colorful-output" => 1, # Use color by default. "debug-level" => ksb::Debug::INFO, "filter-out-phases" => '', "git-desired-protocol" => 'https', # protocol to grab from kde-projects "git-repository-base" => {}, # Base path template for use multiple times. "ignore-modules" => '', # See also: use-modules, kde-projects "include-dependencies" => 0, # Recursively include kde-projects module deps? "manual-build" => "", "manual-update" => "", "niceness" => "10", "no-svn" => "", # TODO: Rename to no-src "prefix" => "", # Override installation prefix. "pretend" => "", "reconfigure" => "", "refresh-build" => "", "repository" => '', # module's git repo "revision" => '', # Only useful for Subversion modules at cmdline "set-env" => { }, # Hash of environment vars to set "ssh-identity-file" => '', # If set, is passed to ssh-add. "use-modules" => "", + "x-invent-kde-push-urls" => 0, # 2020-12-04 temporary feature flag to use invent.kde.org for push URLs for KDE + # projects ); # Holds boolean flags that could be altered from cmdline. # These must be completely disjoint from the options provided in # ksb::Application to GetOptionsFromArray! our %defaultGlobalFlags = ( "delete-my-patches" => 0, # Should only be set from cmdline "delete-my-settings" => 0, # Should only be set from cmdline "disable-agent-check" => 0, # If true we don't check on ssh-agent "disable-snapshots" => 1, # 2016-07-31 Temp. disabled until kde.org fixed to supply snapshots "ignore-kde-structure" => 0, # Whether to use kde dir structure like extragear/network "include-dependencies" => 0, # 2019-08-31 Made negatable from cmdline (NB: false here but true in rcfile) "install-after-build" => 1, "install-environment-driver" => 1, # Setup ~/.config/kde-env-*.sh for login scripts "install-session-driver" => 0, # Above, + ~/.xsession "purge-old-logs" => 1, "run-tests" => 0, # 1 = make test, upload = make Experimental "stop-on-failure" => 0, "use-clean-install" => 0, "use-idle-io-priority" => 0, "use-stable-kde" => 0, ); # Holds other cmdline-accessible options that aren't simply binary flags. our %defaultGlobalOptions = ( "binpath" => '', "branch" => "", "branch-group" => "", # Overrides branch, uses JSON data. "build-dir" => "build", "cmake-options" => "", "configure-flags" => "", "custom-build-command" => '', "cxxflags" => "-pipe", "dest-dir" => '${MODULE}', # single quotes used on purpose! "do-not-compile" => "", "http-proxy" => '', # Proxy server to use for HTTP. "kdedir" => "$ENV{HOME}/kde", "libpath" => "", "log-dir" => "log", "make-install-prefix" => "", # Some people need sudo "make-options" => "", "module-base-path" => "", # Used for tags and branches "override-build-system"=> "", "override-url" => "", "persistent-data-file" => "", "qtdir" => "", "remove-after-install" => "none", # { none, builddir, all } "source-dir" => "$ENV{HOME}/kdesrc", "svn-server" => "svn://anonsvn.kde.org/home/kde", "tag" => "", ); sub new { my ($class, @args) = @_; # It is very important to use the ksb::Module:: syntax instead of ksb::Module->, # otherwise you can't pass $class and have it used as the classname. my $self = ksb::Module::new($class, undef, 'global'); my %newOpts = ( modules => [], context => $self, # Fix link to buildContext (i.e. $self) build_options => { global => { %internalGlobalOptions, %defaultGlobalFlags, %defaultGlobalOptions, }, # Module options are stored under here as well, keyed by module->name() }, # This one replaces ksb::Module::{phases} phases => ksb::PhaseList->new(@DefaultPhases), errors => { # A map from module *names* (as in modules[] above) to the # phase name at which they failed. }, logPaths=> { # Holds a hash table of log path bases as expanded by # getSubdirPath (e.g. [source-dir]/log) to the actual log dir # *this run*, with the date and unique id added. You must still # add the module name to use. }, rcFiles => [@rcfiles], rcFile => undef, env => { }, persistent_options => { }, # These are kept across multiple script runs ignore_list => [ ], # List of KDE project paths to ignore completely kde_projects_metadata => undef, # Enumeration of kde-projects kde_dependencies_metadata => undef, # Dependency resolution of kde-projects logical_module_resolver => undef, # For branch-group option status_monitor => ksb::StatusMonitor->new(), projects_db => undef, # See getProjectDataReader ); # Merge all new options into our self-hash. @{$self}{keys %newOpts} = values %newOpts; $self->{options} = $self->{build_options}{global}; assert_isa($self, 'ksb::Module'); assert_isa($self, 'ksb::BuildContext'); return $self; } # Gets the ksb::PhaseList for this context, and optionally sets it first to # the ksb::PhaseList passed in. sub phases { my ($self, $phases) = @_; if ($phases) { confess("Invalid type, expected PhaseList") unless $phases->isa('ksb::PhaseList'); $self->{phases} = $phases; } return $self->{phases}; } sub addModule { my ($self, $module) = @_; Carp::confess("No module to push") unless $module; my $path; if (list_has($self->{modules}, $module)) { debug("Skipping duplicate module ", $module->name()); } # TODO: Shouldn't this support all modules, not just 'proj' modules? elsif ($module->scmType() eq 'proj' && ($path = $module->fullProjectPath()) && # See if the name matches any given in the ignore list. any(sub { $path =~ /(^|\/)$_($|\/)/ }, $self->{ignore_list})) { debug("Skipping ignored module $module"); } else { debug("Adding $module to module list"); push @{$self->{modules}}, $module; } } # Returns a listref of the modules to build sub moduleList { my $self = shift; return $self->{modules}; } # Adds a list of modules to ignore processing on completely. # Parameters should simply be a list of KDE project paths to ignore, # e.g. 'extragear/utils/kdesrc-build'. Partial paths are acceptable, matches # are determined by comparing the path provided to the suffix of the full path # of modules being compared. See KDEProjectsReader::_projectPathMatchesWildcardSearch # # Existing items on the ignore list are not removed. sub addToIgnoreList { my $self = shift; push @{$self->{ignore_list}}, @_; } sub setupOperatingEnvironment { my $self = shift; # Set the process priority POSIX::nice(int $self->getOption('niceness')); # Set the IO priority if available. if ($self->getOption('use-idle-io-priority')) { # -p $$ is our PID, -c3 is idle priority # 0 return value means success if (safe_system('ionice', '-c3', '-p', $$) != 0) { warning (" b[y[*] Unable to lower I/O priority, continuing..."); } } # Get ready for logged output. ksb::Debug::setLogFile($self->getLogDirFor($self) . '/build-log'); # Propagate HTTP proxy through environment unless overridden. if ((my $proxy = $self->getOption('http-proxy')) && !defined $ENV{'http_proxy'}) { $self->queueEnvironmentVariable('http_proxy', $proxy); } } # Clears the list of environment variables to set for log_command runs. sub resetEnvironment { my $self = assert_isa(shift, 'ksb::BuildContext'); $self->{env} = { }; } # Adds an environment variable and value to the list of environment # variables to apply for the next subprocess execution. # # Note that these changes are /not/ reflected in the current environment, # so if you are doing something that requires that kind of update you # should do that yourself (but remember to have some way to restore the old # value if necessary). # # In order to keep compatibility with the old 'setenv' sub, no action is # taken if the value is not equivalent to boolean true. sub queueEnvironmentVariable { my $self = assert_isa(shift, 'ksb::BuildContext'); my ($key, $value) = @_; return unless $value; debug ("\tQueueing g[$key] to be set to y[$value]"); $self->{env}->{$key} = $value; } # Applies all changes queued by queueEnvironmentVariable to the actual # environment irretrievably. Use this before exec()'ing another child, for # instance. sub commitEnvironmentChanges { my $self = assert_isa(shift, 'ksb::BuildContext'); while (my ($key, $value) = each %{$self->{env}}) { $ENV{$key} = $value; debug ("\tSetting environment variable g[$key] to g[b[$value]"); } } # Adds the given library paths to the path already given in an environment # variable. In addition, detected "system paths" are stripped to ensure # that we don't inadvertently re-add a system path to be promoted over the # custom code we're compiling (for instance, when a system Qt is used and # installed to /usr). # # If the environment variable to be modified has already been queued using # queueEnvironmentVariable, then that (queued) value will be modified and # will take effect with the next forked subprocess. # # Otherwise, the current environment variable value will be used, and then # queued. Either way the current environment will be unmodified afterward. # # First parameter is the name of the environment variable to modify # All remaining parameters are prepended to the current environment path, in # the order given. (i.e. param1, param2, param3 -> # param1:param2:param3:existing) sub prependEnvironmentValue { my $self = assert_isa(shift, 'ksb::BuildContext'); my ($envName, @items) = @_; my @curPaths = split(':', $self->{env}->{$envName} // $ENV{$envName} // ''); # Filter out entries to add that are already in the environment from # the system. for my $path (grep { list_has(\@curPaths, $_) } (@items) ) { debug ("\tNot prepending y[$path] to y[$envName] as it appears " . "to already be defined in y[$envName]."); } @items = grep { not list_has(\@curPaths, $_); } (@items); my $envValue = join(':', @items, @curPaths); $envValue =~ s/^:*//; $envValue =~ s/:*$//; # Remove leading/trailing colons $envValue =~ s/:+/:/; # Remove duplicate colons $self->queueEnvironmentVariable($envName, $envValue); } # Tries to take the lock for our current base directory, which currently is # what passes for preventing people from accidentally running kdesrc-build # multiple times at once. The lock is based on the base directory instead # of being global to allow for motivated and/or brave users to properly # configure kdesrc-build to run simultaneously with different # configurations. # # Return value is a boolean success flag. sub takeLock { my $self = assert_isa(shift, 'ksb::BuildContext'); my $baseDir = $self->baseConfigDirectory(); my $lockfile = "$baseDir/$LOCKFILE_NAME"; $! = 0; # Force reset to non-error status sysopen LOCKFILE, $lockfile, O_WRONLY | O_CREAT | O_EXCL; my $errorCode = $!; # Save for later testing. if ($errorCode == EEXIST) { # Path already exists, read the PID and see if it belongs to a # running process. open (my $pidFile, "<", $lockfile) or do { # Lockfile is there but we can't open it?!? Maybe a race # condition but I have to give up somewhere. warning (" WARNING: Can't open or create lockfile r[$lockfile]"); return 1; }; my $pid = <$pidFile>; close $pidFile; if ($pid) { # Recent kdesrc-build; we wrote a PID in there. chomp $pid; # See if something's running with this PID. if (kill(0, $pid) == 1) { # Something *is* running, likely kdesrc-build. Don't use error, # it'll scan for $! print ksb::Debug::colorize(" r[*y[*r[*] kdesrc-build appears to be running. Do you want to:\n"); print ksb::Debug::colorize(" (b[Q])uit, (b[P])roceed anyways?: "); my $choice = // ''; chomp $choice; if (lc $choice ne 'p') { say ksb::Debug::colorize(" y[*] kdesrc-build run canceled."); return 0; } # We still can't grab the lockfile, let's just hope things # work out. note (" y[*] kdesrc-build run in progress by user request."); return 1; } # If we get here, then the program isn't running (or at least not # as the current user), so allow the flow of execution to fall # through below and unlink the lockfile. } # pid # No pid found, optimistically assume the user isn't running # twice. warning (" y[WARNING]: stale kdesrc-build lockfile found, deleting."); unlink $lockfile; sysopen (LOCKFILE, $lockfile, O_WRONLY | O_CREAT | O_EXCL) or do { error (" r[*] Still unable to lock $lockfile, proceeding anyways..."); return 1; }; # Hope the sysopen worked... fall-through } elsif ($errorCode == ENOTTY) { # Stupid bugs... normally sysopen will return ENOTTY, not sure who's to blame between # glibc and Perl but I know that setting PERLIO=:stdio in the environment "fixes" things. ; # pass } elsif ($errorCode != 0) # Some other error occurred. { warning (" r[*]: Error $errorCode while creating lock file (is $baseDir available?)"); warning (" r[*]: Continuing the script for now..."); # Even if we fail it's generally better to allow the script to proceed # without being a jerk about things, especially as more non-CLI-skilled # users start using kdesrc-build to build KDE. return 1; } say LOCKFILE "$$"; close LOCKFILE; return 1; } # Releases the lock obtained by takeLock. sub closeLock { my $self = assert_isa(shift, 'ksb::BuildContext'); my $baseDir = $self->baseConfigDirectory(); my $lockFile = "$baseDir/$LOCKFILE_NAME"; unlink ($lockFile) or warning(" y[*] Failed to close lock: $!"); } # This subroutine accepts a Module parameter, and returns the log directory # for it. You can also pass a BuildContext (including this one) to get the # default log directory. # # As part of setting up what path to use for the log directory, the # 'latest' symlink will also be setup to point to the returned log # directory. sub getLogDirFor { my ($self, $module) = @_; my $baseLogPath = $module->getSubdirPath('log-dir'); my $logDir; if (!exists $self->{logPaths}{$baseLogPath}) { # No log dir made for this base, do so now. my $id = '01'; my $date = strftime "%F", localtime; # ISO 8601 date $id++ while -e "$baseLogPath/$date-$id"; $self->{logPaths}{$baseLogPath} = "$baseLogPath/$date-$id"; } $logDir = $self->{logPaths}{$baseLogPath}; super_mkdir($logDir); # global logs go to basedir directly $logDir .= "/$module" unless $module->isa('ksb::BuildContext'); return $logDir; } # Constructs the appropriate full path to a log file based on the given # basename (including extensions). Use this instead of getLogDirFor when you # actually intend to create a log, as this function will also adjust the # 'latest' symlink properly. sub getLogPathFor { my ($self, $module, $path) = @_; my $baseLogPath = $module->getSubdirPath('log-dir'); my $logDir = $self->getLogDirFor($module); # We create this here to avoid needless empty module directories everywhere super_mkdir($logDir); # Add a symlink to the latest run for this module. 'latest' itself is # a directory under the base log directory that holds symlinks mapping # each module name to the specific log directory most recently used. my $latestPath = "$baseLogPath/latest"; # Handle stuff like playground/utils or KDE/kdelibs my ($moduleName, $modulePath) = fileparse($module->name()); $latestPath .= "/$modulePath" if $module->name() =~ m(/); super_mkdir($latestPath); my $symlink = "$latestPath/$moduleName"; if (-l $symlink and readlink($symlink) ne $logDir) { unlink($symlink); symlink($logDir, $symlink); } elsif(not -e $symlink) { # Create symlink initially if we've never done it before. symlink($logDir, $symlink); } return "$logDir/$path"; } # Returns rc file in use. Call loadRcFile first. sub rcFile { my $self = shift; return $self->{rcFile}; } # Forces the rc file to be read from to be that given by the first # parameter. sub setRcFile { my ($self, $file) = @_; $self->{rcFiles} = [$file]; $self->{rcFile} = undef; } # Returns an open filehandle to the user's chosen rc file. Use setRcFile # to choose a file to load before calling this function, otherwise # loadRcFile will search the default search path. After this function is # called, rcFile() can be used to determine which file was loaded. # # If unable to find or open the rc file an exception is raised. Empty rc # files are supported however. sub loadRcFile { my $self = shift; my @rcFiles = @{$self->{rcFiles}}; my $fh; for my $file (@rcFiles) { if (open ($fh, '<', "$file")) { $self->{rcFile} = File::Spec->rel2abs($file); return $fh; } } # No rc found, check if we can use default. if (scalar @rcFiles == 1) { # This can only happen if the user uses --rc-file, so if we fail to # load the file, we need to fail to load at all. my $failedFile = $rcFiles[0]; error (<rcFile() or croak_internal("Call to baseConfigDirectory before loadRcFile"); return dirname($rcfile); } sub modulesInPhase { my ($self, $phase) = @_; my @list = grep { list_has([$_->phases()->phases()], $phase) } (@{$self->moduleList()}); return @list; } # Searches for a module with a name that matches the provided parameter, # and returns its ksb::Module object. Returns undef if no match was found. # As a special-case, returns the BuildContext itself if the name passed is # 'global', since the BuildContext also is a (in the "is-a" OOP sense) # ksb::Module, specifically the 'global' one. sub lookupModule { my ($self, $moduleName) = @_; return $self if $moduleName eq 'global'; my @options = grep { $_->name() eq $moduleName } (@{$self->moduleList()}); return undef unless @options; if (scalar @options > 1) { croak_internal("Detected 2 or more $moduleName ksb::Module objects"); } return $options[0]; } sub markModulePhaseSucceeded { my ($self, $phase, $module, $extras) = @_; assert_isa($module, 'ksb::Module'); $extras //= { }; $extras->{elapsed} = sum values %{$module->{metrics}->{time_in_phase}}; $self->{status_monitor}->markPhaseComplete($module->name(), $phase, 'success', %{$extras}); } sub markModulePhaseFailed { my ($self, $phase, $module) = @_; assert_isa($module, 'ksb::Module'); my $name = $module->name(); my %details; if ((my $log = $module->getOption('#error-log-file', 'module'))) { $details{error_log} = "/error_log/$name"; $details{error_file} = $log; } $self->{status_monitor}->markPhaseComplete($name, $phase, 'error', %details); $self->{errors}->{$name} = $phase; } # Returns a list (i.e. not a reference to, but a real list) of Modules that failed to # complete the given phase. sub failedModulesInPhase { my ($self, $phase) = @_; my @failures = grep { ($self->{errors}->{$_->name()} // '') eq $phase } (@{$self->moduleList()}); return @failures; } # Returns true if the given module had a failure in any phase sub hasModuleFailed { my ($self, $module) = @_; return exists $self->{errors}->{$module->name()}; } # Returns a list of modules that had a failure of some sort, in the order the modules # are listed in our current module list. sub listFailedModules { my $self = shift; my @modules = @{$self->moduleList()}; # grepping for failures instead of returning error list directly maintains ordering @modules = grep { exists $self->{errors}->{$_->name()} } (@modules); return @modules; } # OVERRIDE: Our immediate parent class Module overrides this, but we actually # want the OptionsBase version to be used instead, until we break the recursive # use of Module's own getOption calls on our getOption. sub getOption { &ksb::OptionsBase::getOption; } # OVERRIDE: Overrides OptionsBase::setOption to handle some global-only options. sub setOption { my ($self, %options) = @_; # Special-case handling my $repoOption = 'git-repository-base'; if (exists $options{$repoOption}) { my $value = $options{$repoOption}; my ($repo, $url) = ($value =~ /^([a-zA-Z0-9_-]+)\s+(.+)$/); # This will be a hash reference instead of a scalar $value = $self->getOption($repoOption) || { }; if (!$repo || !$url) { die ksb::BuildException::Config->new($repoOption, "Invalid git-repository-base setting: $value"); } $value->{$repo} = $url; delete $options{$repoOption}; } # Actually set options. $self->SUPER::setOption(%options); # Automatically respond to various global option changes. while (my ($key, $value) = each %options) { my $normalizedKey = $key; $normalizedKey =~ s/^#//; # Remove sticky key modifier. given ($normalizedKey) { when ('colorful-output') { ksb::Debug::setColorfulOutput($value); } when ('debug-level') { ksb::Debug::setDebugLevel($value); } when ('pretend') { ksb::Debug::setPretending($value); } } } } # # Persistent option handling # # Returns the name of the file to use for persistent data. # Supports expanding '#' at the beginning of the filename to the directory # containing the rc-file in use, but only for the default name at this # point. sub persistentOptionFileName { my $self = shift; my $filename = $self->getOption('persistent-data-file'); if (!$filename) { $filename = $PERSISTENT_FILE_NAME; my $dir = $self->baseConfigDirectory(); $filename =~ s/^#/$dir/; } else { # Tilde-expand $filename =~ s/^~\//$ENV{HOME}\//; } return $filename; } # Reads in all persistent options from the file where they are kept # (.kdesrc-build-data) for use in the program. # # The directory used is the same directory that contains the rc file in use. sub loadPersistentOptions { my $self = assert_isa(shift, 'ksb::BuildContext'); my $fh = IO::File->new($self->persistentOptionFileName(), '<'); return unless $fh; # $persistent_data should be a JSON object which we can store directly as a # hash. my $persistent_data; { local $/ = undef; # Read in whole file with <> operator. $persistent_data = <$fh>; } my $persistent_options = decode_json($persistent_data); # We need to keep persistent data with the context instead of with the # applicable modules since otherwise we might forget to write out # persistent data for modules we didn't build in this run. So, we just # store it all. # Layout of this data: # $self->persistent_options = { # 'module-name' => { # option => value, # # foreach option/value pair # }, # # foreach module # } $persistent_options = {} if ref $persistent_options ne 'HASH'; $self->{persistent_options} = $persistent_options; } # Writes out the persistent options to the file .kdesrc-build-data. # # The directory used is the same directory that contains the rc file in use. sub storePersistentOptions { my $self = assert_isa(shift, 'ksb::BuildContext'); return if pretending(); my $fh = IO::File->new($self->persistentOptionFileName(), '>'); if (!$fh) { error ("Unable to save persistent module data: b[r[$!]"); return; } my $encodedJSON = encode_json ($self->{persistent_options}); print $fh $encodedJSON; undef $fh; # Closes the file } # Returns the value of a "persistent" option (normally read in as part of # startup), or undef if there is no value stored. # # First parameter is the module name to get the option for, or 'global' if # not for a module. # Note that unlike setOption/getOption, no inheritance is done at this # point so if an option is present globally but not for a module you # must check both if that's what you want. # Second parameter is the name of the value to retrieve (i.e. the key) sub getPersistentOption { my ($self, $moduleName, $key) = @_; my $persistent_opts = $self->{persistent_options}; # We must check at each level of indirection to avoid # "autovivification" return unless exists $persistent_opts->{$moduleName}; return unless exists $persistent_opts->{$moduleName}{$key}; return $persistent_opts->{$moduleName}{$key}; } # Clears a persistent option if set (for a given module and option-name). # # First parameter is the module name to get the option for, or 'global' for # the global options. # Second parameter is the name of the value to clear. # No return value. sub unsetPersistentOption { my ($self, $moduleName, $key) = @_; my $persistent_opts = $self->{persistent_options}; if (exists $persistent_opts->{$moduleName} && exists $persistent_opts->{$moduleName}->{$key}) { delete $persistent_opts->{$moduleName}->{$key}; } } # Sets a "persistent" option which will be read in for a module when # kdesrc-build starts up and written back out at (normal) program exit. # # First parameter is the module name to set the option for, or 'global'. # Second parameter is the name of the value to set (i.e. key) # Third parameter is the value to store, which must be a scalar. sub setPersistentOption { my ($self, $moduleName, $key, $value) = @_; my $persistent_opts = $self->{persistent_options}; # Initialize empty hash ref if nothing defined for this module. $persistent_opts->{$moduleName} //= { }; $persistent_opts->{$moduleName}{$key} = $value; } # Returns the ksb::Module (which has a 'metadata' scm type) that is used for # kde-build-metadata, so that other modules that need it can call into it if # necessary. # # Also may return undef if the metadata is unavailable or has not yet # been set by setKDEDependenciesMetadataModule (this method does not # automatically create the needed module). sub getKDEDependenciesMetadataModule { my $self = shift; return $self->{kde_dependencies_metadata}; } # Returns the ksb::Module (which has a 'metadata' scm type) that is used for # kde-project metadata, so that other modules that need it can call into it if # necessary. # # Also may return undef if the metadata is unavailable or has not yet # been set by setKDEProjectsMetadataModule (this method does not # automatically create the needed module). sub getKDEProjectsMetadataModule { my $self = shift; return $self->{kde_projects_metadata}; } # Call this method to force this build context to pull in the kde-build-metadata # module. This is a one-time action, subsequent calls to this method # are ignored. Use getKDEDependenciesMetadataModule to see if this build context is # using a metadata module. # # This method should be called before setModuleList. sub setKDEDependenciesMetadataModuleNeeded { my $self = assert_isa(shift, 'ksb::BuildContext'); return if defined $self->{kde_dependencies_metadata}; my $metadata = ksb::ModuleSet::KDEProjects::getDependenciesModule($self); debug ("Introducing dependency metadata into the build"); assert_isa($metadata->scm(), 'ksb::Updater::KDEProjectMetadata'); $self->{kde_dependencies_metadata} = $metadata; return; } # Call this method to force this build context to pull in the # sysadmin/repo-metadata module. This is a one-time action, # subsequent calls to this method are ignored. Use # getKDEProjectsMetadataModule to see if this build context is using # a metadata module. # # This method should be called before setModuleList. sub setKDEProjectsMetadataModuleNeeded { my $self = assert_isa(shift, 'ksb::BuildContext'); return if defined $self->{kde_projects_metadata}; my $metadata = ksb::ModuleSet::KDEProjects::getProjectMetadataModule($self); debug ("Introducing project enumeration metadata into the build"); assert_isa($metadata->scm(), 'ksb::Updater::KDEProjectMetadata'); $self->{kde_projects_metadata} = $metadata; return; } # Returns a KDEProjectsReader module, which has already read in the database and # is ready to be queried. Note that exceptions can be thrown in the process # of downloading and parsing the database information, so be ready for that. sub getProjectDataReader { my $self = shift; return $self->{projects_db} if $self->{projects_db}; my $projectDatabaseModule = $self->getKDEProjectsMetadataModule() or croak_runtime("kde-projects repository information could not be downloaded: $!"); - my $protocol = $self->getOption('git-desired-protocol') || 'git'; - if (!list_has(['git', 'http', 'https'], $protocol)) { - error (" b[y[*] Invalid b[git-desired-protocol] $protocol"); - error (" b[y[*] Try setting this option to 'git' if you're not using a proxy"); - croak_runtime ("Invalid git-desired-protocol: $protocol"); - } - - $self->{projects_db} = ksb::KDEProjectsReader->new($projectDatabaseModule, $protocol); + $self->{projects_db} = ksb::KDEProjectsReader->new($projectDatabaseModule); return $self->{projects_db}; } # Returns the effective branch group to use for modules. You should not call # this unless kde-build-metadata is also in use (see # setKDEDependenciesMetadataModule and moduleBranchGroupResolver). sub effectiveBranchGroup { my $self = shift; my $branchGroup = $self->getOption('branch-group', 'module') // ''; if (!$branchGroup) { $branchGroup = $self->getOption('use-stable-kde') ? 'latest-qt4' : ($self->hasOption('use-stable-kde') # Could also be false if unset ? 'kf5-qt5' # Really set to false : 'latest-qt4'); # Unset / this is default branch group if no option set } return $branchGroup; } # Returns a ksb::Module::BranchGroupResolver which can be used to efficiently # determine a git branch to use for a given kde-projects module (when the # branch-group option is in use), as specified at # https://community.kde.org/Infrastructure/Project_Metadata. sub moduleBranchGroupResolver { my $self = shift; if (!$self->{logical_module_resolver}) { my $metadataModule = $self->getKDEDependenciesMetadataModule(); croak_internal("Tried to use branch-group, but needed data wasn't loaded!") unless $metadataModule; my $resolver = ksb::Module::BranchGroupResolver->new( $metadataModule->scm()->logicalModuleGroups()); $self->{logical_module_resolver} = $resolver; } return $self->{logical_module_resolver}; } # An event-based aggregator for update events, to be used by user interfaces, # including remote interfaces. sub statusMonitor { my $self = shift; return $self->{status_monitor}; } 1; diff --git a/modules/ksb/BuildSystem.pm b/modules/ksb/BuildSystem.pm index d17d5fc..842d92d 100644 --- a/modules/ksb/BuildSystem.pm +++ b/modules/ksb/BuildSystem.pm @@ -1,441 +1,457 @@ package ksb::BuildSystem 0.30; # Base module for the various build systems, includes built-in implementations of # generic functions and supports hooks for subclasses to provide needed detailed # functionality. use strict; use warnings; use 5.014; use ksb::BuildException; use ksb::Debug 0.30; use ksb::Util; use ksb::StatusView; use List::Util qw(first); sub new { my ($class, $module) = @_; my $self = bless { module => $module }, $class; # This is simply the 'default' build system at this point, also used for # KF5. if ($class ne 'ksb::BuildSystem::KDE4') { _maskGlobalBuildSystemOptions($self); } return $self; } # Removes or masks global build system-related options, so that they aren't # accidentally picked up for use with our non-default build system. # Module-specific options are left intact. sub _maskGlobalBuildSystemOptions { my $self = shift; my $module = $self->module(); my $ctx = $module->buildContext(); my @buildSystemOptions = qw( cmake-options cmake-generator configure-flags custom-build-command cxxflags make-options run-tests use-clean-install ); for my $opt (@buildSystemOptions) { # If an option is present, and not set at module-level, it must be # global. Can't use getOption() method due to recursion. if ($ctx->{options}->{$opt} && !$module->{options}->{$opt}) { $module->{options}->{$opt} = ''; } } } +# +# Check if a (custom) toolchain is defined. +# If a build system is configured with a (custom) toolchain, it is assumed that +# +# a: the user knows what they are doing, or +# b: they are using an SDK that knows what it is about +# +# In either case, kdesrc-build will avoid touching the environment variables to +# give the custom configuration maximum 'power' (including foot shooting power). +# +sub hasToolchain +{ + my $self = shift; + return 0; +} + sub module { my $self = shift; return $self->{module}; } # Subroutine to determine if a given module needs to have the build system # recreated from scratch. # If so, it returns a non empty string sub needsRefreshed { my $self = assert_isa(shift, 'ksb::BuildSystem'); my $module = $self->module(); my $builddir = $module->fullpath('build'); my $confFileKey = $self->configuredModuleFileName(); if (not -e "$builddir") { return "the build directory doesn't exist"; } if (-e "$builddir/.refresh-me") { return "the last configure failed"; # see Module.pm } if ($module->getOption("refresh-build")) { return "the option refresh-build was set"; } if (not -e "$builddir/$confFileKey") { return "$builddir/$confFileKey is missing"; } return ""; } # Called by the module being built before it runs its build/install process. Should # setup any needed environment variables, build context settings, etc., in preparation -# for the build and install phases. +# for the build and install phases. Should take `hasToolchain()` into account here. sub prepareModuleBuildEnvironment { my ($self, $ctx, $module, $prefix) = @_; } # Returns true if the module should have make install run in order to be # used, or false if installation is not required or possible. sub needsInstalled { return 1; } # This should return a list of executable names that must be present to # even bother attempting to use this build system. An empty list should be # returned if there's no required programs. sub requiredPrograms { return; } sub name { return 'generic'; } # Returns a list of possible build commands to run, any one of which should # be supported by the build system. sub buildCommands { # Non Linux systems can sometimes fail to build when GNU Make would work, # so prefer GNU Make if present, otherwise try regular make. return 'gmake', 'make'; } sub defaultBuildCommand { my $self = shift; # Convert the path to an absolute path since I've encountered a sudo # that is apparently unable to guess. Maybe it's better that it # doesn't guess anyways from a security point-of-view. my $buildCommand = first { absPathToExecutable($_) } $self->buildCommands(); return $buildCommand; } # Return value style: hashref {was_successful => bool, warnings => int, ...} sub buildInternal { my $self = shift; my $optionsName = shift // 'make-options'; return $self->safe_make({ target => undef, message => 'Compiling...', 'make-options' => [ split(' ', $self->module()->getOption($optionsName)), ], logbase => 'build', }); } # Return value style: boolean sub configureInternal { # It is possible to make it here if there's no source dir and if we're # pretending. If we're not actually pretending then this should be a # bug... return 1 if pretending(); croak_internal('We were not supposed to get to this point...'); } # Returns name of file that should exist (relative to the module's build directory) # if the module has been configured. sub configuredModuleFileName { my $self = shift; return 'Makefile'; } # Runs the testsuite for the given module. # Returns true if a testsuite is present and all tests passed, false otherwise. sub runTestsuite { my $self = shift; my $module = $self->module(); whisper ("\ty[$module] does not support the b[run-tests] option"); return 0; } # Used to install a module (that has already been built, tested, etc.) # All options passed are prefixed to the eventual command to be run. # Returns boolean false if unable to install, true otherwise. sub installInternal { my $self = shift; my $module = $self->module(); my @cmdPrefix = @_; return $self->safe_make ({ target => 'install', message => 'Installing..', 'prefix-options' => [@cmdPrefix], })->{was_successful}; } # Used to uninstall a previously installed module. # All options passed are prefixed to the eventual command to be run. # Returns boolean false if unable to uninstall, true otherwise. sub uninstallInternal { my $self = shift; my $module = $self->module(); my @cmdPrefix = @_; return $self->safe_make ({ target => 'uninstall', message => "Uninstalling g[$module]", 'prefix-options' => [@cmdPrefix], })->{was_successful}; } # Subroutine to clean the build system for the given module. Works by # recursively deleting the directory and then recreating it. # Returns 0 for failure, non-zero for success. sub cleanBuildSystem { my $self = assert_isa(shift, 'ksb::BuildSystem'); my $module = $self->module(); my $srcdir = $module->fullpath('source'); my $builddir = $module->fullpath('build'); if (pretending()) { pretend ("\tWould have cleaned build system for g[$module]"); return 1; } # Use an existing directory if (-e $builddir && $builddir ne $srcdir) { whisper ("\tCleaning out build directory"); # This variant of log_command runs the sub prune_under_directory($builddir) # in a forked child, so that we can log its output. if (log_command($module, 'clean-builddir', [ 'kdesrc-build', 'main::prune_under_directory', $builddir ])) { error (" r[b[*]\tFailed to clean build directory. Verify the permissions are correct."); return 0; # False for this function. } # Let users know we're done so they don't wonder why rm -rf is taking so # long and oh yeah, why's my HD so active?... whisper ("\tOld build system cleaned, starting new build system."); } # or create the directory elsif (!super_mkdir ($builddir)) { error ("\tUnable to create directory r[$builddir]."); return 0; } return 1; } sub needsBuilddirHack { return 0; # By default all build systems are assumed to be sane } # Return convention: boolean sub createBuildSystem { my $self = assert_isa(shift, 'ksb::BuildSystem'); my $module = $self->module(); my $builddir = $module->fullpath('build'); my $srcdir = $module->fullpath('source'); if (! -e "$builddir" && !super_mkdir("$builddir")) { error ("\tUnable to create build directory for r[$module]!!"); return 0; } if ($builddir ne $srcdir && $self->needsBuilddirHack() && 0 != log_command($module, 'lndir', ['kdesrc-build', 'main::safe_lndir', $srcdir, $builddir])) { error ("\tUnable to setup symlinked build directory for r[$module]!!"); return 0; } return 1; } # Subroutine to run the build command with the arguments given by the # passed hash. # # Due to the various ways the build command is called by this script, it is # required to pass customization options in a hash: # { # target => undef, or a valid build target e.g. 'install', # message => 'Compiling.../Installing.../etc.' # make-options => [ list of command line arguments to pass to make. See # make-options ], # prefix-options => [ list of command line arguments to prefix *before* the # make command, used for make-install-prefix support for # e.g. sudo ], # logbase => 'base-log-filename', # } # # target and message are required. logbase is required if target is left # undefined, but otherwise defaults to the same value as target. # # Note that the make command is based on the results of the 'buildCommands' # subroutine which should be overridden if necessary by subclasses. Each # command should be the command name (i.e. no path). The user may override # the command used (for build only) by using the 'custom-build-command' # option. # # The first command name found which resolves to an executable on the # system will be used, if no command this function will fail. # # The first argument should be the ksb::Module object to be made. # The second argument should be the reference to the hash described above. # # Returns a hashref: # { # was_successful => $bool, (if successful) # } sub safe_make (@) { my ($self, $optsRef) = @_; assert_isa($self, 'ksb::BuildSystem'); my $module = $self->module(); my $buildCommand = $self->defaultBuildCommand(); my @buildCommandLine = $buildCommand; # Check for custom user command. We support command line options being # passed to the command as well. my $userCommand = $module->getOption('custom-build-command'); if ($userCommand) { @buildCommandLine = split_quoted_on_whitespace($userCommand); $buildCommand = absPathToExecutable($buildCommandLine[0]); } if (!$buildCommand) { $buildCommand = $userCommand || $self->buildCommands(); error (" r[b[*] Unable to find the g[$buildCommand] executable!"); return { was_successful => 0 }; } # Make it prettier if pretending (Remove leading directories). $buildCommand =~ s{^/.*/}{} if pretending(); shift @buildCommandLine; # $buildCommand is already the first entry. # Simplify code by forcing lists to exist. $optsRef->{'prefix-options'} //= [ ]; $optsRef->{'make-options'} //= [ ]; my @prefixOpts = @{$optsRef->{'prefix-options'}}; # If using sudo ensure that it doesn't wait on tty, but tries to read from # stdin (which should fail as we redirect that from /dev/null) if (@prefixOpts && $prefixOpts[0] eq 'sudo' && !grep { /^-S$/ } @prefixOpts) { splice (@prefixOpts, 1, 0, '-S'); # Add -S right after 'sudo' } # Assemble arguments my @args = (@prefixOpts, $buildCommand, @buildCommandLine); push @args, $optsRef->{target} if $optsRef->{target}; push @args, @{$optsRef->{'make-options'}}; # Will be output by _runBuildCommand my $buildMessage = $optsRef->{message}; my $logname = $optsRef->{logbase} // $optsRef->{logfile} // $optsRef->{target}; p_chdir ($module->fullpath('build')); return $self->_runBuildCommand($buildMessage, $logname, \@args); } # Subroutine to run make and process the build process output in order to # provide completion updates. This procedure takes the same arguments as # log_command() (described here as well), except that the callback argument # is not used. # # First parameter is the message to display to the user while the build # happens. # Second parameter is the name of the log file to use (relative to the log # directory). # Third parameter is a reference to an array with the command and its # arguments. i.e. ['command', 'arg1', 'arg2'] # # The return value is a hashref as defined by safe_make sub _runBuildCommand { my ($self, $message, $filename, $argRef) = @_; my $module = $self->module(); my $resultRef = { was_successful => 0 }; my $ctx = $module->buildContext(); # There are situations when we don't want progress output: # 1. If we're not printing to a terminal. # 2. When we're debugging (we'd interfere with debugging output). if (! -t STDERR || debugging()) { $resultRef->{was_successful} = (0 == log_command($module, $filename, $argRef)); return $resultRef; } # TODO More details my $warnings = 0; my $log_command_callback = sub { state $oldX = -1; state $oldY = -1; my $input = shift; return if not defined $input; $warnings++ if ($input =~ /warning: /); my ($x, $y); my ($percentage) = ($input =~ /^\[\s*([0-9]+)%]/); if ($percentage) { $x = int $percentage; $y = 100; } else { # ninja-syntax my ($newX, $newY) = ($input =~ /^\[([0-9]+)\/([0-9]+)] /); return unless ($newX && $newY); ($x, $y) = (int $newX, int $newY); } if ($x != $oldX || $y != $oldY) { ksb::Debug::reportProgressToParent($module, $x, $y); } }; $resultRef->{was_successful} = (0 == log_command($module, $filename, $argRef, { callback => $log_command_callback })); $resultRef->{warnings} = $warnings; $self->{module}->setPersistentOption('last-compile-warnings', $warnings); return $resultRef; } 1; diff --git a/modules/ksb/BuildSystem/KDE4.pm b/modules/ksb/BuildSystem/KDE4.pm index 0aee432..c69e753 100644 --- a/modules/ksb/BuildSystem/KDE4.pm +++ b/modules/ksb/BuildSystem/KDE4.pm @@ -1,378 +1,458 @@ package ksb::BuildSystem::KDE4 0.20; # Class responsible for building KDE4 CMake-based modules. use strict; use warnings; use 5.014; use parent qw(ksb::BuildSystem); use ksb::BuildContext 0.30; use ksb::Debug; use ksb::Util; use List::Util qw(first); my $GENERATOR_MAP = { 'Ninja' => { optionsName => 'ninja-options', installTarget => 'install', requiredPrograms => [ qw{ninja cmake qmake} ], buildCommands => [ qw{ninja} ] }, 'Unix Makefiles' => { optionsName => 'make-options', installTarget => 'install/fast', requiredPrograms => [ qw{cmake qmake} ], # Non Linux systems can sometimes fail to build when GNU Make would work, # so prefer GNU Make if present, otherwise try regular make. buildCommands => [ qw{gmake make} ] } }; sub _checkGeneratorIsWhitelisted { my $generator = shift; return exists ($GENERATOR_MAP->{$generator}); } sub _stripGeneratorFromCMakeOptions { my $nextShouldBeGenerator = 0; my @filtered = grep { my $accept = 1; if ($nextShouldBeGenerator) { $nextShouldBeGenerator = 0; $accept = 0; } else { my $maybeGenerator = $_; if ($maybeGenerator =~ /^-G(\S*(\s*\S)*)\s*/) { my $generator = $1 // ''; $nextShouldBeGenerator = 1 if ($generator eq ''); $accept = 0; } } $accept == 1; } (@_); return @filtered; } sub _findGeneratorInCMakeOptions { my $nextShouldBeGenerator = 0; my @filtered = grep { my $accept = 0; if ($nextShouldBeGenerator) { $nextShouldBeGenerator = 0; $accept = 1; } else { my $maybeGenerator = $_; if ($maybeGenerator =~ /^-G(\S*(\s*\S)*)\s*/) { my $generator = $1 // ''; if ($generator ne '') { $accept = 1; } else { $nextShouldBeGenerator = 1; } } } $accept == 1; } (@_); for my $found (@filtered) { if ($found =~ /^-G(\S*(\s*\S)*)\s*/) { $found = $1 // ''; } return $found unless ($found eq ''); } return ''; } +sub _checkToolchainOk +{ + my $toolchain = shift; + return $toolchain ne '' && -f $toolchain && -r $toolchain; +} + +sub _stripToolchainFromCMakeOptions +{ + my @filtered = grep { + my $accept = 1; + my $maybeToolchain = $_; + if ($maybeToolchain =~ /^-DCMAKE_TOOLCHAIN_FILE=(\S*(\s*\S)*)\s*/) { + $accept = 0; + } + + $accept == 1; + } (@_); + return @filtered; +} + +sub _findToolchainInCMakeOptions +{ + my $found = first { + my $accept = 0; + my $maybeToolchain = $_; + if ($maybeToolchain =~ /^-DCMAKE_TOOLCHAIN_FILE=(\S*(\s*\S)*)\s*/) { + my $file = $1 // ''; + $accept = 1 if (_checkToolchainOk($file)); + } + + $accept == 1; + } (@_); + + if ($found && $found =~ /^-DCMAKE_TOOLCHAIN_FILE=(\S*(\s*\S)*)\s*/) { + $found = $1 // ''; + return $found if (_checkToolchainOk($found)); + } + + return ''; +} + +sub _determineCmakeToolchain +{ + my $self = shift; + + my $module = $self->module(); + my @cmakeOptions = split_quoted_on_whitespace ($module->getOption('cmake-options')); + + my $toolchain = first { _checkToolchainOk($_); } ( + _findToolchainInCMakeOptions(@cmakeOptions), + $module->getOption('cmake-toolchain') + ); + + return $toolchain // ''; +} + +sub cmakeToolchain +{ + my $self = shift; + if (not (exists $self->{cmake_toolchain})) { + $self->{cmake_toolchain} = $self->_determineCmakeToolchain(); + } + return $self->{cmake_toolchain}; +} + +sub hasToolchain +{ + my $self = shift; + return $self->cmakeToolchain() ne ''; +} + sub _determineCmakeGenerator { my $self = shift; my $module = $self->module(); my @cmakeOptions = split_quoted_on_whitespace ($module->getOption('cmake-options')); my $generator = first { _checkGeneratorIsWhitelisted($_); } ( _findGeneratorInCMakeOptions(@cmakeOptions), $module->getOption('cmake-generator'), 'Unix Makefiles' ); croak_internal("Unable to determine CMake generator for: $module") unless $generator; return $generator; } sub cmakeGenerator { my $self = shift; if (not (exists $self->{cmake_generator})) { $self->{cmake_generator} = $self->_determineCmakeGenerator(); } return $self->{cmake_generator}; } - sub needsInstalled { my $self = shift; return 0 if $self->name() eq 'kde-common'; # Vestigial return 1; } sub name { return 'KDE'; } # Called by the module being built before it runs its build/install process. Should # setup any needed environment variables, build context settings, etc., in preparation # for the build and install phases. sub prepareModuleBuildEnvironment { my ($self, $ctx, $module, $prefix) = @_; + # + # Suppress injecting qtdir/kdedir related environment variables if a toolchain is also set + # Let the toolchain files/definitions take care of themselves. + # + return if $self->hasToolchain(); + # Avoid moving /usr up in env vars if ($prefix ne '/usr') { # Find the normal CMake "config" mode files for find_package() $ctx->prependEnvironmentValue('CMAKE_PREFIX_PATH', $prefix); # Try to ensure that older "module" mode find_package() calls also point to right directory $ctx->prependEnvironmentValue('CMAKE_MODULE_PATH', "$prefix/lib64/cmake:$prefix/lib/cmake"); $ctx->prependEnvironmentValue('XDG_DATA_DIRS', "$prefix/share"); } my $qtdir = $module->getOption('qtdir'); if ($qtdir && $qtdir ne $prefix) { # Ensure we can find Qt5's own CMake modules $ctx->prependEnvironmentValue('CMAKE_PREFIX_PATH', $qtdir); $ctx->prependEnvironmentValue('CMAKE_MODULE_PATH', "$qtdir/lib/cmake"); } } # This should return a list of executable names that must be present to # even bother attempting to use this build system. An empty list should be # returned if there's no required programs. sub requiredPrograms { my $self = shift; my $generator = $self->cmakeGenerator(); my @required = @{$GENERATOR_MAP->{$generator}->{requiredPrograms}}; return @required; } # Returns a list of possible build commands to run, any one of which should # be supported by the build system. sub buildCommands { my $self = shift; my $generator = $self->cmakeGenerator(); my @progs = @{$GENERATOR_MAP->{$generator}->{buildCommands}}; return @progs; } sub configuredModuleFileName { my $self = shift; return 'cmake_install.cmake'; } sub runTestsuite { my $self = assert_isa(shift, 'ksb::BuildSystem::KDE4'); my $module = $self->module(); # Note that we do not run safe_make, which should really be called # safe_compile at this point. # Step 1: Ensure the tests are built, oh wait we already did that when we ran # CMake :) my $make_target = 'test'; if ($module->getOption('run-tests') eq 'upload') { $make_target = 'Experimental'; } # Step 2: Run the tests. my $numTests = -1; my $countCallback = sub { if ($_ && /([0-9]+) tests failed out of/) { $numTests = $1; } }; my $buildCommand = $self->defaultBuildCommand(); my $result = log_command($module, 'test-results', [ $buildCommand, $make_target ], { callback => $countCallback, no_translate => 1}); if ($result != 0) { my $logDir = $module->getLogDir(); if ($numTests > 0) { warning ("\t$numTests tests failed for y[$module], consult $logDir/test-results.log for info"); } else { warning ("\tSome tests failed for y[$module], consult $logDir/test-results.log for info"); } return 0; } return 1; } # Re-implementing the one in BuildSystem since in CMake we want to call # make install/fast, so it only installs rather than building + installing sub installInternal { my $self = shift; my $module = $self->module(); my $generator = $self->cmakeGenerator(); my $target = $GENERATOR_MAP->{$generator}->{installTarget}; my @cmdPrefix = @_; $target = 'install' if $module->getOption('custom-build-command'); return $self->safe_make ({ target => $target, logfile => 'install', message => 'Installing..', 'prefix-options' => [@cmdPrefix], })->{was_successful}; } sub configureInternal { my $self = assert_isa(shift, 'ksb::BuildSystem::KDE4'); # Use cmake to create the build directory (sh script return value # semantics). return ($self->_safe_run_cmake() == 0); } # Return value style: boolean sub buildInternal { my $self = shift; my $generator = $self->cmakeGenerator(); my $defaultOptionsName = $GENERATOR_MAP->{$generator}->{optionsName}; my $optionsName = shift // "$defaultOptionsName"; return $self->safe_make({ target => undef, message => 'Compiling...', 'make-options' => [ split(' ', $self->module()->getOption($optionsName)), ], logbase => 'build', subdirs => [ split(' ', $self->module()->getOption("checkout-only")) ], }); } ### Internal package functions. # Subroutine to run CMake to create the build directory for a module. # CMake is not actually run if pretend mode is enabled. # # First parameter is the module to run cmake on. # Return value is the shell return value as returned by log_command(). i.e. # 0 for success, non-zero for failure. sub _safe_run_cmake { my $self = shift; my $module = $self->module(); my $generator = $self->cmakeGenerator(); + my $toolchain = $self->cmakeToolchain(); my $srcdir = $module->fullpath('source'); my @commands = split_quoted_on_whitespace ($module->getOption('cmake-options')); # grep out empty fields @commands = grep {!/^\s*$/} @commands; @commands = _stripGeneratorFromCMakeOptions(@commands); + @commands = _stripToolchainFromCMakeOptions(@commands); + + unshift @commands, "-DCMAKE_TOOLCHAIN_FILE=$toolchain" if $toolchain ne ''; # Add -DBUILD_foo=OFF options for the directories in do-not-compile. # This will only work if the CMakeLists.txt file uses macro_optional_add_subdirectory() my @masked_directories = split(' ', $module->getOption('do-not-compile')); push @commands, "-DBUILD_$_=OFF" foreach @masked_directories; # Get the user's CXXFLAGS, use them if specified and not already given # on the command line. my $cxxflags = $module->getOption('cxxflags'); if ($cxxflags and not grep { /^-DCMAKE_CXX_FLAGS(:\w+)?=/ } @commands) { push @commands, "-DCMAKE_CXX_FLAGS:STRING=$cxxflags"; } my $prefix = $module->installationPath(); push @commands, "-DCMAKE_INSTALL_PREFIX=$prefix"; # Add custom Qt to the prefix (but don't overwrite a user-set prefix) my $qtdir = $module->getOption('qtdir'); if ($qtdir && $qtdir ne $prefix && !grep { /^\s*-DCMAKE_PREFIX_PATH/ } (@commands) ) { push @commands, "-DCMAKE_PREFIX_PATH=$qtdir"; } if ($module->getOption('run-tests') && !grep { /^\s*-DKDE4_BUILD_TESTS(:BOOL)?=(ON|TRUE|1)\s*$/ } (@commands) ) { whisper ("Enabling tests"); push @commands, "-DKDE4_BUILD_TESTS:BOOL=ON"; # Also enable phonon tests. if ($module =~ /^phonon$/) { push @commands, "-DPHONON_BUILD_TESTS:BOOL=ON"; } } if ($module->getOption('run-tests') eq 'upload') { whisper ("Enabling upload of test results"); push @commands, "-DBUILD_experimental:BOOL=ON"; } unshift @commands, 'cmake', $srcdir, '-G', $generator; # Add to beginning of list. my $old_options = $module->getPersistentOption('last-cmake-options') || ''; my $builddir = $module->fullpath('build'); if (($old_options ne get_list_digest(@commands)) || $module->getOption('reconfigure') || ! -e "$builddir/CMakeCache.txt" # File should exist only on successful cmake run ) { # Remove any stray CMakeCache.txt safe_unlink ("$srcdir/CMakeCache.txt") if -e "$srcdir/CMakeCache.txt"; safe_unlink ("$builddir/CMakeCache.txt") if -e "$builddir/CMakeCache.txt"; $module->setPersistentOption('last-cmake-options', get_list_digest(@commands)); return log_command($module, "cmake", \@commands); } # Skip cmake run return 0; } 1; diff --git a/modules/ksb/DependencyResolver.pm b/modules/ksb/DependencyResolver.pm index 07fcfac..13afa33 100644 --- a/modules/ksb/DependencyResolver.pm +++ b/modules/ksb/DependencyResolver.pm @@ -1,764 +1,771 @@ package ksb::DependencyResolver; # Class: DependencyResolver # # This module handles resolving dependencies between modules. Each "module" # from the perspective of this resolver is simply a module full name, as # given by the KDE Project database. (e.g. extragear/utils/kdesrc-build) use strict; use warnings; use 5.014; our $VERSION = '0.20'; use ksb::BuildException; use ksb::Debug; use ksb::Util; use List::Util qw(first); sub uniq { my %seen; return grep { ++($seen{$_}) == 1 } @_; } # Constructor: new # # Constructs a new . # # Parameters: # # moduleFactoryRef - Reference to a sub that creates ksb::Modules from # kde-project module names. Used for ksb::Modules for which the user # requested recursive dependency inclusion. # # Synposis: # # > my $resolver = new DependencyResolver($modNewRef); # > $resolver->readDependencyData(open my $fh, '<', 'file.txt'); # > $resolver->resolveDependencies(@modules); sub new { my $class = shift; my $moduleFactoryRef = shift; my $self = { # hash table mapping short module names (m) to a hashref key by branch # name, the value of which is yet another hashref (see # readDependencyData). Note that this assumes KDE git infrastructure # ensures that all full module names (e.g. # kde/workspace/plasma-workspace) map to a *unique* short name (e.g. # plasma-workspace) by stripping leading path components dependenciesOf => { }, # hash table mapping a wildcarded module name with no branch to a # listref of module:branch dependencies. catchAllDependencies => { }, # reference to a sub that will properly create a ksb::Module from a # given kde-project module name. Used to support automatically adding # dependencies to a build. moduleFactoryRef => $moduleFactoryRef, }; return bless $self, $class; } # Function: shortenModuleName # # Internal: # # This method returns the 'short' module name of kde-project full project paths. # E.g. 'kde/kdelibs/foo' would be shortened to 'foo'. # # This is a static function, not an object method. # # Parameters: # # path - A string holding the full module virtual path # # Returns: # # The module name. sub _shortenModuleName { my $name = shift; $name =~ s{^.*/}{}; # Uses greedy capture by default return $name; } # Method: readDependencyData # # Reads in dependency data in a pseudo-Makefile format. # See kde-build-metadata/dependency-data. # # Parameters: # $self - The DependencyResolver object. # $fh - Filehandle to read dependencies from (should already be open). # # Exceptions: # Can throw an exception on I/O errors or malformed dependencies. sub readDependencyData { my $self = assert_isa(shift, 'ksb::DependencyResolver'); my $fh = shift; my $dependenciesOfRef = $self->{dependenciesOf}; my $dependencyAtom = qr/ ^\s* # Clear leading whitespace ([^\[:\s]+) # (1) Capture anything not a [, :, or whitespace (dependent item) \s* # Clear whitespace we didn't capture (?:\[ # Open a non-capture group... ([^\]:\s]+) # (2) Capture branch name without brackets ])?+ # Close group, make optional, no backtracking \s* # Clear whitespace we didn't capture : \s* ([^\s\[]+) # (3) Capture all non-whitespace (source item) (?:\s*\[ # Open a non-capture group... ([^\]\s]+) # (4) Capture branch name without brackets ])?+ # Close group, make optional, no backtracking \s*$ # Ensure no trailing cruft. Any whitespace should end line /x; # /x Enables extended whitespace mode while(my $line = <$fh>) { # Strip comments, skip empty lines. $line =~ s{#.*$}{}; next if $line =~ /^\s*$/; if ($line !~ $dependencyAtom) { croak_internal("Invalid line $line when reading dependency data."); } my ($dependentItem, $dependentBranch, $sourceItem, $sourceBranch) = $line =~ $dependencyAtom; # Ignore "catch-all" dependencies where the source is the catch-all if ($sourceItem =~ m,\*$,) { warning ("\tIgnoring dependency on wildcard module grouping " . "on line $. of kde-build-metadata/dependency-data"); next; } $dependentBranch ||= '*'; # If no branch, apply catch-all flag $sourceBranch ||= '*'; # _shortenModuleName may remove negation marker so check now my $depKey = (index($sourceItem, '-') == 0) ? '-' : '+'; $sourceItem =~ s/^-//; # remove negation marker if name already short # Source can never be a catch-all so we can shorten early. Also, # we *must* shorten early to avoid a dependency on a long path. $sourceItem = _shortenModuleName($sourceItem); # Handle catch-all dependent groupings if ($dependentItem =~ /\*$/) { $self->{catchAllDependencies}->{$dependentItem} //= [ ]; push @{$self->{catchAllDependencies}->{$dependentItem}}, "$sourceItem:$sourceBranch"; next; } $dependentItem = _shortenModuleName($dependentItem); # Initialize with hashref if not already defined. The hashref will hold # - => [ ] (list of explicit *NON* dependencies of item:$branch), # + => [ ] (list of dependencies of item:$branch) # # Each dependency item is tracked at the module:branch level, and there # is always at least an entry for module:*, where '*' means branch # is unspecified and should only be used to add dependencies, never # take them away. # # Finally, all (non-)dependencies in a list are also of the form # fullname:branch, where "*" is a valid branch. $dependenciesOfRef->{"$dependentItem:*"} //= { '-' => [ ], '+' => [ ], }; # Create actual branch entry if not present $dependenciesOfRef->{"$dependentItem:$dependentBranch"} //= { '-' => [ ], '+' => [ ], }; push @{$dependenciesOfRef->{"$dependentItem:$dependentBranch"}->{$depKey}}, "$sourceItem:$sourceBranch"; } $self->_canonicalizeDependencies(); } # Function: _canonicalizeDependencies # # Ensures that all stored dependencies are stored in a way that allows for # reproducable dependency ordering (assuming the same dependency items and same # selectors are used). # # Parameters: none # # Returns: none sub _canonicalizeDependencies { my $self = shift; my $dependenciesOfRef = $self->{dependenciesOf}; foreach my $dependenciesRef (values %{$dependenciesOfRef}) { @{$dependenciesRef->{'-'}} = sort @{$dependenciesRef->{'-'}}; @{$dependenciesRef->{'+'}} = sort @{$dependenciesRef->{'+'}}; } } sub _lookupDirectDependencies { my $self = assert_isa(shift, 'ksb::DependencyResolver'); my ($path, $branch) = @_; my $dependenciesOfRef = $self->{dependenciesOf}; my @directDeps = (); my @exclusions = (); my $item = _shortenModuleName($path); my $moduleDepEntryRef = $dependenciesOfRef->{"$item:*"}; if ($moduleDepEntryRef) { debug("handling dependencies for: $item without branch (*)"); push @directDeps, @{$moduleDepEntryRef->{'+'}}; push @exclusions, @{$moduleDepEntryRef->{'-'}}; } if ($branch && $branch ne '*') { $moduleDepEntryRef = $dependenciesOfRef->{"$item:$branch"}; if ($moduleDepEntryRef) { debug("handling dependencies for: $item with branch ($branch)"); push @directDeps, @{$moduleDepEntryRef->{'+'}}; push @exclusions, @{$moduleDepEntryRef->{'-'}}; } } while (my ($catchAll, $deps) = each %{$self->{catchAllDependencies}}) { my $prefix = $catchAll; $prefix =~ s/\*$//; if (($path =~ /^$prefix/) || !$prefix) { push @directDeps, @{$deps}; } } foreach my $exclusion (@exclusions) { # Remove only modules at the exact given branch as a dep. # However catch-alls can remove catch-alls. # But catch-alls cannot remove a specific branch, such exclusions have # to also be specific. @directDeps = grep { $_ ne $exclusion } (@directDeps); } my $result = { syntaxErrors => 0, trivialCycles => 0, dependencies => {} }; for my $dep (@directDeps) { my ($depPath, $depBranch) = ($dep =~ m/^([^:]+):(.*)$/); if (!$depPath) { error("r[Invalid dependency declaration: b[$dep]]"); ++($result->{syntaxErrors}); next; } my $depItem = _shortenModuleName($depPath); if ($depItem eq $item) { debug("\tBreaking trivial cycle of b[$depItem] to itself"); ++($result->{trivialCycles}); next; } if ($result->{dependencies}->{$depItem}) { debug("\tSkipping duplicate direct dependency b[$depItem] of b[$item]"); } else { $depBranch //= ''; # work-around: wildcard branches are a don't care, not an actual # branch name/value $depBranch = undef if ($depBranch eq '' || $depBranch eq '*'); $result->{dependencies}->{$depItem} = { item => $depItem, path => $depPath, branch => $depBranch }; } } return $result; } sub _runDependencyVote { my $moduleGraph = shift; for my $item (keys(%$moduleGraph)) { my @names = keys(%{$moduleGraph->{$item}->{allDeps}->{items}}); for my $name (@names) { ++($moduleGraph->{$name}->{votes}->{$item}); } } return $moduleGraph; } sub _detectDependencyCycle { my ($moduleGraph, $depItem, $item) = @_; my $depModuleGraph = $moduleGraph->{$depItem}; if ($depModuleGraph->{traces}->{status}) { if ($depModuleGraph->{traces}->{status} == 2) { debug("Already resolved $depItem -- skipping"); return $depModuleGraph->{traces}->{result}; } else { error("Found a dependency cycle at: $depItem while tracing $item"); $depModuleGraph->{traces}->{result} = 1; } } else { $depModuleGraph->{traces}->{status} = 1; $depModuleGraph->{traces}->{result} = 0; my @names = keys(%{$depModuleGraph->{deps}}); for my $name (@names) { if (_detectDependencyCycle($moduleGraph, $name, $item)) { $depModuleGraph->{traces}->{result} = 1; } } } $depModuleGraph->{traces}->{status} = 2; return $depModuleGraph->{traces}->{result}; } sub _checkDependencyCycles { my $moduleGraph = shift; my $errors = 0; for my $item (keys(%$moduleGraph)) { if(_detectDependencyCycle($moduleGraph, $item, $item)) { error("Somehow there is a circular dependency involving b[$item]! :("); error("Please file a bug against kde-build-metadata about this!"); ++$errors; } } return $errors; } sub _copyUpDependenciesForModule { my ($moduleGraph, $item) = @_; my $allDeps = $moduleGraph->{$item}->{allDeps}; if($allDeps->{done}) { debug("\tAlready copied up dependencies for b[$item] -- skipping"); } else { debug("\tCopying up dependencies and transitive dependencies for item: b[$item]"); $allDeps->{items} = {}; my @names = keys(%{$moduleGraph->{$item}->{deps}}); for my $name (@names) { if ($allDeps->{items}->{$name}) { debug("\tAlready copied up (transitive) dependency on b[$name] for b[$item] -- skipping"); } else { _copyUpDependenciesForModule($moduleGraph, $name); my @copied = keys(%{$moduleGraph->{$name}->{allDeps}->{items}}); for my $copy (@copied) { if ($allDeps->{items}->{$copy}) { debug("\tAlready copied up (transitive) dependency on b[$copy] for b[$item] -- skipping"); } else { ++($allDeps->{items}->{$copy}); } } ++($allDeps->{items}->{$name}); } } ++($allDeps->{done}); } } sub _copyUpDependencies { my $moduleGraph = shift; for my $item (keys(%$moduleGraph)) { _copyUpDependenciesForModule($moduleGraph, $item); } return $moduleGraph; } sub _detectBranchConflict { my ($moduleGraph, $item, $branch) = @_; if ($branch) { my $subGraph = $moduleGraph->{$item}; my $previouslySelectedBranch = $subGraph->{branch}; return $previouslySelectedBranch if($previouslySelectedBranch && $previouslySelectedBranch ne $branch); } return undef; } sub _getDependencyPathOf { my ($module, $item, $path) = @_; if ($module) { my $projectPath = $module->fullProjectPath(); $projectPath = "third-party/$projectPath" if(!$module->isKDEProject()); debug("\tUsing path: 'b[$projectPath]' for item: b[$item]"); return $projectPath; } debug("\tGuessing path: 'b[$path]' for item: b[$item]"); return $path; } sub _resolveDependenciesForModuleDescription { my $self = assert_isa(shift, 'ksb::DependencyResolver'); my ($moduleGraph, $moduleDesc) = @_; my $module = $moduleDesc->{module}; if($module) { assert_isa($module, 'ksb::Module'); } my $path = $moduleDesc->{path}; my $item = $moduleDesc->{item}; my $branch = $moduleDesc->{branch}; my $prettyBranch = $branch ? "$branch" : "*"; my $includeDependencies = $module ? $module->getOption('include-dependencies') : $moduleDesc->{includeDependencies}; my $errors = { syntaxErrors => 0, trivialCycles => 0, branchErrors => 0 }; debug("Resolving dependencies for module: b[$item]"); while (my ($depItem, $depInfo) = each %{$moduleGraph->{$item}->{deps}}) { my $depPath = $depInfo->{path}; my $depBranch = $depInfo->{branch}; my $prettyDepBranch = $depBranch ? "$depBranch" : "*"; debug ("\tdep-resolv: b[$item:$prettyBranch] depends on b[$depItem:$prettyDepBranch]"); my $depModuleGraph = $moduleGraph->{$depItem}; if($depModuleGraph) { my $previouslySelectedBranch = _detectBranchConflict($moduleGraph, $depItem, $depBranch); if($previouslySelectedBranch) { error("r[Found a dependency conflict in branches ('b[$previouslySelectedBranch]' is not 'b[$prettyDepBranch]') for b[$depItem]! :("); ++($errors->{branchErrors}); } else { if($depBranch) { $depModuleGraph->{branch} = $depBranch; } } } else { my $depModule = $self->{moduleFactoryRef}($depItem); my $resolvedPath = _getDependencyPathOf($depModule, $depItem, $depPath); # May not exist, e.g. misspellings or 'virtual' dependencies like kf5umbrella. if(!$depModule) { debug("\tdep-resolve: Will not build virtual or undefined module: b[$depItem]\n"); } my $depLookupResult = $self->_lookupDirectDependencies( $resolvedPath, $depBranch ); $errors->{trivialCycles} += $depLookupResult->{trivialCycles}; $errors->{syntaxErrors} += $depLookupResult->{syntaxErrors}; $moduleGraph->{$depItem} = { votes => {}, path => $resolvedPath, build => $depModule && $includeDependencies ? 1 : 0, branch => $depBranch, deps => $depLookupResult->{dependencies}, allDeps => {}, module => $depModule, traces => {} }; my $depModuleDesc = { includeDependencies => $includeDependencies, module => $depModule, item => $depItem, path => $resolvedPath, branch => $depBranch }; if (!$moduleGraph->{$depItem}->{build}) { debug (" y[b[*] $item depends on $depItem, but no module builds $depItem for this run.]"); } if($depModule && $depBranch && (_getBranchOf($depModule) // '') ne "$depBranch") { my $wrongBranch = _getBranchOf($depModule) // '?'; error(" r[b[*] $item needs $depItem:$prettyDepBranch, not $depItem:$wrongBranch]"); ++($errors->{branchErrors}); } debug("Resolving transitive dependencies for module: b[$item] (via: b[$depItem:$prettyDepBranch])"); my $resolvErrors = $self->_resolveDependenciesForModuleDescription( $moduleGraph, $depModuleDesc ); $errors->{branchErrors} += $resolvErrors->{branchErrors}; $errors->{syntaxErrors} += $resolvErrors->{syntaxErrors}; $errors->{trivialCycles} += $resolvErrors->{trivialCycles}; } } return $errors; } sub resolveToModuleGraph { my $self = assert_isa(shift, 'ksb::DependencyResolver'); my @modules = @_; my %graph; my $moduleGraph = \%graph; my $result = { graph => $moduleGraph, errors => { branchErrors => 0, pathErrors => 0, trivialCycles => 0, syntaxErrors => 0, cycles => 0 } }; my $errors = $result->{errors}; for my $module (@modules) { my $item = $module->name(); # _shortenModuleName($path); my $branch = _getBranchOf($module); my $path = _getDependencyPathOf($module, $item, ''); if (!$path) { error("r[Unable to determine project/dependency path of module: $item]"); ++($errors->{pathErrors}); next; } if($moduleGraph->{$item}) { debug("Module pulled in previously through (transitive) dependencies: $item"); my $previouslySelectedBranch = _detectBranchConflict($moduleGraph, $item, $branch); if($previouslySelectedBranch) { error("r[Found a dependency conflict in branches ('b[$previouslySelectedBranch]' is not 'b[$branch]') for b[$item]! :("); ++($errors->{branchErrors}); } elsif ($branch) { $moduleGraph->{$item}->{branch} = $branch; } # # May have been pulled in via dependencies but not yet marked for # build. Do so now, since it is listed explicitly in @modules # $moduleGraph->{$item}->{build} = 1; } else { my $depLookupResult = $self->_lookupDirectDependencies( $path, $branch ); $errors->{trivialCycles} += $depLookupResult->{trivialCycles}; $errors->{syntaxErrors} += $depLookupResult->{syntaxErrors}; $moduleGraph->{$item} = { votes => {}, path => $path, build => 1, branch => $branch, module => $module, deps => $depLookupResult->{dependencies}, allDeps => {}, traces => {} }; my $moduleDesc = { includeDependencies => $module->getOption('include-dependencies'), path => $path, item => $item, branch => $branch, module => $module }; my $resolvErrors = $self->_resolveDependenciesForModuleDescription( $moduleGraph, $moduleDesc ); $errors->{branchErrors} += $resolvErrors->{branchErrors}; $errors->{syntaxErrors} += $resolvErrors->{syntaxErrors}; $errors->{trivialCycles} += $resolvErrors->{trivialCycles}; } } my $pathErrors = $errors->{pathErrors}; if ($pathErrors) { error("Total of items which were not resolved due to path lookup failure: $pathErrors"); } my $branchErrors = $errors->{branchErrors}; if ($branchErrors) { error("Total of branch conflicts detected: $branchErrors"); } my $syntaxErrors = $errors->{syntaxErrors}; if ($syntaxErrors) { error("Total of encountered syntax errors: $syntaxErrors"); } if ($syntaxErrors || $pathErrors || $branchErrors) { error("Unable to resolve dependency graph"); $result->{graph} = undef; return $result; } my $trivialCycles = $errors->{trivialCycles}; my $cycles = _checkDependencyCycles($moduleGraph); if ($cycles) { error("Total of items with at least one circular dependency detected: $errors"); error("Unable to resolve dependency graph"); $result->{cycles} = $cycles; $result->{graph} = undef; return $result; } else { $result->{graph} = _runDependencyVote(_copyUpDependencies($moduleGraph)); return $result; } } sub hasErrors { my $info = shift; my $cycles = $info->{cycles} // 0; my $pathErrors = $info->{pathErrors} // 0; my $branchErrors = $info->{branchErrors} // 0; my $syntaxErrors = $info->{syntaxErrors} // 0; return $cycles || $pathErrors || $branchErrors || $syntaxErrors; } sub _compareBuildOrder { my ($moduleGraph, $a, $b) = @_; + my $aVotes = $moduleGraph->{$a}->{votes}; + my $bVotes = $moduleGraph->{$b}->{votes}; + # # Enforce a strict dependency ordering. # The case where both are true should never happen, since that would # amount to a cycle, and cycle detection is supposed to have been # performed beforehand. # - my $bDependsOnA = $moduleGraph->{$a}->{votes}->{$b} // 0; - my $aDependsOnB = $moduleGraph->{$b}->{votes}->{$a} // 0; + my $bDependsOnA = $aVotes->{$b} // 0; + my $aDependsOnB = $bVotes->{$a} // 0; my $order = $bDependsOnA ? -1 : ($aDependsOnB ? 1 : 0); return $order if $order; # # Assuming no dependency relation, next sort by 'popularity': # the item with the most votes (back edges) is depended on the most # so it is probably a good idea to build that one earlier to help # maximise the duration of time for which builds can be run in parallel # - my $voteA = scalar keys %{$moduleGraph->{$a}->{votes}}; - my $voteB = scalar keys %{$moduleGraph->{$b}->{votes}}; - my $votes = $voteB <=> $voteA; + my $votes = scalar keys %$bVotes <=> scalar keys %$aVotes; return $votes if $votes; # # If there is no good reason to perfer one module over another, - # simply sort by name to get a reproducible build order. - # That simplifies autotesting and/or reproducible builds. - # (The items to sort are supplied as a hash so the order of keys is by - # definition not guaranteed.) + # simply sort by the order contained within the configuration file (if + # present), which would be setup as the rc-file is read. # - my $name = ($a cmp $b); + my $aRcOrder = $moduleGraph->{$a}->{module}->{'#create-id'} // 0 ; + my $bRcOrder = $moduleGraph->{$b}->{module}->{'#create-id'} // 0 ; + my $configOrder = $aRcOrder <=> $bRcOrder; - return $name; + return $configOrder if $configOrder; + + # + # If the rc-file is not present then sort by name to ensure a reproducible + # build order that isn't influenced by randomization of the runtime. + # + return $a cmp $b; } sub sortModulesIntoBuildOrder { my $moduleGraph = shift; my @resolved = keys(%{$moduleGraph}); my @built = grep { $moduleGraph->{$_}->{build} && $moduleGraph->{$_}->{module} } (@resolved); my @prioritised = sort { _compareBuildOrder($moduleGraph, $a, $b); } (@built); my @modules = map { $moduleGraph->{$_}->{module} } (@prioritised); return @modules; } # Function: getBranchOf # # Internal: # # This function extracts the branch of the given Module by calling its # scm object's branch-determining method. It also ensures that the branch # returned was really intended to be a branch (as opposed to a detached HEAD); # undef is returned when the desired commit is not a branch name, otherwise # the user-requested branch name is returned. sub _getBranchOf { my $module = shift; my $scm = $module->scm(); # when the module's SCM is not git, # assume the default "no particular" branch wildcard return undef unless $scm->isa('ksb::Updater::Git'); my ($branch, $type) = $scm->_determinePreferredCheckoutSource($module); return ($type eq 'branch' ? $branch : undef); } 1; diff --git a/modules/ksb/FirstRun.pm b/modules/ksb/FirstRun.pm index c4ff92e..9447912 100644 --- a/modules/ksb/FirstRun.pm +++ b/modules/ksb/FirstRun.pm @@ -1,482 +1,542 @@ package ksb::FirstRun 0.10; use 5.014; use strict; use warnings; use File::Spec qw(splitpath); use ksb::BuildException; use ksb::Debug qw(colorize); use ksb::OSSupport; use ksb::Util; =head1 NAME ksb::FirstRun =head1 DESCRIPTION Performs initial-install setup, implementing the C<--initial-setup> option. B This module is supposed to be loadable even under minimal Perl environments as fielded in "minimal Docker container" forms of popular distros. =head1 SYNOPSIS my $exitcode = ksb::FirstRun::setupUserSystem(); exit $exitcode; =cut sub setupUserSystem { my $baseDir = shift; my $os = ksb::OSSupport->new; + my %shellRcMap = ( + 'bash' => '.bashrc', + 'zsh' => '.zshrc', + ); + my $envShell = $ENV{'SHELL'} // '/bin/bash'; + my $shellName = (split '/', $envShell)[-1]; + my $rcName = $shellRcMap{$shellName}; + + if (!defined($rcName)) { + $rcName = '.bashrc'; + say " - $envShell is not supported. Changes will go to ~/.bashrc."; + } eval { _installSystemPackages($os); _setupBaseConfiguration($baseDir); - _setupBashrcFile(); + _setupShellRcFile($rcName); }; if (had_an_exception($@)) { my $msg = $@->{message}; say colorize (" b[r[*] r[$msg]"); return 1; } return 0; } # Internal functions # Reads from the __DATA__ section below and dumps the contents in a hash keyed # by filename (the @@ part between each resource). my %packages; sub _readPackages { return \%packages if %packages; my $cur_file; my $cur_value; my $commit = sub { return unless $cur_file; $packages{$cur_file} = ($cur_value =~ s/ *$//r); $cur_value = ''; }; while(my $line = ) { next if $line =~ /^\s*#/ and $cur_file !~ /sample-rc/; chomp $line; my ($fname) = ($line =~ /^@@ *([^ ]+)$/); if ($fname) { $commit->(); $cur_file = $fname; } else { $cur_value .= "$line\n"; } } $commit->(); return \%packages; } sub _throw { my $msg = shift; die (make_exception('Setup', $msg)); } sub _installSystemPackages { my $os = shift; my $vendor = $os->vendorID; my $osVersion = $os->vendorVersion; print colorize(<> 8 == 0) { say colorize (" b[*] b[g[Looks like the necessary packages were successfully installed!]"); } else { say colorize (" r[b[*] Ran into an error with the installer!"); } } else { say colorize (" r[b[*] Packages could not be installed, because kdesrc-build does not know your linux distribution."); } } sub _setupBaseConfiguration { my $baseDir = shift; if (-e "kdesrc-buildrc" || -e "$ENV{HOME}/.kdesrc-buildrc") { print colorize(</dev/null` || 4; $sampleRc =~ s/%\{num_cpus}/$numCpus/g; $sampleRc =~ s/%\{base_dir}/$baseDir/g; open my $sampleFh, '>', "$ENV{HOME}/.kdesrc-buildrc" or _throw("Couldn't open new ~/.kdesrc-buildrc: $!"); print $sampleFh $sampleRc or _throw("Couldn't write to ~/.kdesrc-buildrc: $!"); close $sampleFh or _throw("Error closing ~/.kdesrc-buildrc: $!"); } } -sub _setupBashrcFile +sub _setupShellRcFile { - my $modifiedBashrc = 0; - + my $rc = shift; + my $rcFilepath = "$ENV{HOME}/$rc"; + my $rcModified = 0; + my $tag = '# auto-generated by kdesrc-build initial setup: do not remove!'; + # Add kdesrc-build path to PATH if not already in there - if (!ksb::Util::isInPath('src/kdesrc-build')) { - - say colorize(<>', "$ENV{HOME}/.bashrc") or _throw("Couldn't open ~/.bashrc: $!"); - - print $bashrc "\n# Adding the kdesrc-build directory to the path\n"; - print $bashrc 'export PATH="$HOME/kde/src/kdesrc-build:$PATH"'; - print $bashrc "\n"; - - $modifiedBashrc = 1; - } - # Create kdesrc-run alias for more convenient program execution - if (!ksb::Util::fileHasLine("$ENV{HOME}/.bashrc", "kdesrc-run ()")) { + if (!ksb::Util::fileHasLine($rcFilepath, $tag)) { say colorize(<>', "$ENV{HOME}/.bashrc") or _throw("Couldn't open ~/.bashrc: $!"); - - print $bashrc "\n# Creating alias for running software built with kdesrc-build\n"; - print $bashrc "kdesrc-run ()\n"; - print $bashrc "{\n"; - print $bashrc ' source "$HOME/kde/build/$1/prefix.sh" && "$HOME/kde/usr/bin/$1"'; - print $bashrc "\n}\n"; - - $modifiedBashrc = 1; + open(my $rcFh, '>>', "$rcFilepath") or _throw("Couldn't open ~/$rc: $!"); + + say $rcFh ''; + say $rcFh "$tag"; + say $rcFh '# Add the kdesrc-build directory to the path'; + say $rcFh 'export PATH="$HOME/kde/src/kdesrc-build:$PATH"'; + say $rcFh '# Create alias for running software built with kdesrc-build'; + say $rcFh 'kdesrc-run ()'; + say $rcFh '{'; + say $rcFh ' source "$HOME/kde/build/$1/prefix.sh" && "$HOME/kde/usr/bin/$1"'; + say $rcFh '}'; + + $rcModified = 1; } - - - if ($modifiedBashrc) { + if ($rcModified) { say colorize(<bestDistroMatch(@supportedDistros); say colorize (" Using installer for b[$bestVendor]"); my $version = $os->vendorVersion(); my @cmd; for my $opt ("$bestVendor/$version", "$bestVendor/unknown") { my $key = "cmd/install/$opt"; next unless exists $pkgsRef->{$key}; @cmd = split(' ', $pkgsRef->{$key}); last; } _throw("No installer for $bestVendor!") unless @cmd; # If not running as root already, add sudo unshift @cmd, 'sudo' if $> != 0; return @cmd; } sub _findBestVendorPackageList { my $os = shift; # Debian handles Ubuntu also my @supportedDistros = map { s{^pkg/([^/]+)/.*$}{$1}; $_ } grep { /^pkg\// } keys %{_readPackages()}; my $bestVendor = $os->bestDistroMatch(@supportedDistros); my $version = $os->vendorVersion(); say colorize (" Installing packages for b[$bestVendor]/b[$version]"); return _packagesForVendor($bestVendor, $version); } sub _packagesForVendor { my ($vendor, $version) = @_; my $packagesRef = _readPackages(); foreach my $opt ("pkg/$vendor/$version", "pkg/$vendor/unknown") { next unless exists $packagesRef->{$opt}; my @packages = split(' ', $packagesRef->{$opt}); return @packages; } return; } 1; __DATA__ @@ pkg/debian/unknown libyaml-libyaml-perl libio-socket-ssl-perl libjson-xs-perl git shared-mime-info cmake build-essential flex bison gperf libssl-dev intltool liburi-perl gettext @@ pkg/opensuse/unknown cmake docbook-xsl-stylesheets docbook_4 flex bison gettext-runtime gettext-tools giflib-devel git gperf intltool libboost_headers-devel libqt5-qtbase-common-devel libqt5-qtbase-private-headers-devel libqt5-qtimageformats-devel libQt5Core-private-headers-devel libQt5DesignerComponents5 libxml2-tools lmdb-devel make perl perl(IO::Socket::SSL) perl(JSON) perl(URI) perl(YAML::LibYAML) pkgconfig(libattr) pkgconfig(libical) pkgconfig(libpng) pkgconfig(libqrencode) pkgconfig(libudev) pkgconfig(libxml-2.0) pkgconfig(libxslt) pkgconfig(ModemManager) pkgconfig(NetworkManager) pkgconfig(openssl) pkgconfig(Qt5Core) pkgconfig(Qt5Multimedia) pkgconfig(Qt5Qml) pkgconfig(Qt5QuickControls2) pkgconfig(Qt5Script) pkgconfig(Qt5Svg) pkgconfig(Qt5UiTools) pkgconfig(Qt5WebKit) pkgconfig(Qt5WebKitWidgets) pkgconfig(Qt5X11Extras) pkgconfig(Qt5XmlPatterns) pkgconfig(sm) pkgconfig(wayland-server) pkgconfig(xcb-keysyms) pkgconfig(xrender) polkit-devel shared-mime-info @@ pkg/fedora/unknown bison boost-devel bzr cmake docbook-style-xsl docbook-utils doxygen flex gcc gcc-c++ gettext gettext-devel giflib-devel git gperf intltool libxml2 make pam-devel perl(IO::Socket::SSL) perl(IPC::Cmd) perl(JSON::PP) perl(URI) perl(YAML::LibYAML) pkgconfig(dbus-1) pkgconfig(gbm) pkgconfig(gl) pkgconfig(gstreamer-1.0) pkgconfig(libassuan) pkgconfig(libattr) pkgconfig(libnm) pkgconfig(libpng) pkgconfig(libqrencode) pkgconfig(libxml-2.0) pkgconfig(libxslt) pkgconfig(lmdb) pkgconfig(ModemManager) pkgconfig(openssl) pkgconfig(polkit-gobject-1) pkgconfig(sm) pkgconfig(wayland-client) pkgconfig(wayland-protocols) pkgconfig(xapian-core) pkgconfig(xcb-cursor) pkgconfig(xcb-ewmh) pkgconfig(xcb-keysyms) pkgconfig(xcb-util) pkgconfig(xfixes) pkgconfig(xrender) python shared-mime-info texinfo +systemd-devel @@ pkg/mageia/unknown bison boost cmake docbook-style-xsl docbook-utils flex gcc gcc-c++ gettext gettext-devel giflib git gperf intltool lib64lmdb-devel make perl(IO::Socket::SSL) perl(IPC::Cmd) perl(JSON::PP) perl(URI) perl(YAML::LibYAML) pkgconfig(dbus-1) pkgconfig(gl) pkgconfig(gstreamer-1.0) pkgconfig(libattr) pkgconfig(libnm) pkgconfig(libpng) pkgconfig(libqrencode) pkgconfig(libxml-2.0) pkgconfig(libxslt) pkgconfig(ModemManager) pkgconfig(openssl) pkgconfig(polkit-gobject-1) pkgconfig(sm) pkgconfig(wayland-client) pkgconfig(xcb-keysyms) pkgconfig(xrender) python shared-mime-info @@ pkg/gentoo/unknown dev-util/cmake dev-lang/perl @@ pkg/arch/unknown perl-json perl-yaml-libyaml perl-io-socket-ssl cmake gcc make qt5-base doxygen +@@ pkg/alpine/unknown +alpine-sdk +attr-dev +autoconf +automake +bison +boost-dev +cmake +doxygen +eudev-dev +flex +giflib-dev +gperf +gpgme-dev +grantlee-dev +gstreamer-dev +gst-plugins-base-dev +libdmtx-dev +libepoxy-dev +libgcrypt-dev +libinput-dev +libqrencode-dev +libxkbfile-dev +libxrender-dev +libxtst-dev +linux-pam-dev +lmdb-dev +networkmanager-dev +perl +perl-io-socket-ssl +perl-uri +perl-yaml-libyaml +polkit-elogind-dev +qt5-qtbase-dev +qt5-qtdeclarative-dev +qt5-qtquickcontrols2-dev +qt5-qtscript-dev +qt5-qtsensors-dev +qt5-qtsvg-dev +qt5-qttools-dev +qt5-qttools-static +qt5-qtwayland-dev +qt5-qtx11extras-dev +texinfo +wayland-protocols +xapian-core-dev +xcb-util-cursor-dev +xcb-util-image-dev +xcb-util-keysyms-dev +xcb-util-wm-dev + @@ cmd/install/debian/unknown apt-get -q -y --no-install-recommends install @@ cmd/install/opensuse/unknown zypper install -y --no-recommends @@ cmd/install/arch/unknown pacman -Syu --noconfirm --needed @@ cmd/install/fedora/unknown dnf -y install +@@ cmd/install/alpine/unknown +apk add + @@ sample-rc # This file controls options to apply when configuring/building modules, and # controls which modules are built in the first place. # List of all options: https://go.kde.org/u/ksboptions global # Paths kdedir ~/kde/usr # Where to install KF5-based software qtdir ~/kde/qt5 # Where to find Qt5 source-dir ~/kde/src # Where sources are downloaded build-dir ~/kde/build # Where the source build is run ignore-kde-structure true # Use flat structure # Will pull in KDE-based dependencies only, to save you the trouble of # listing them all below include-dependencies true cmake-options -DCMAKE_BUILD_TYPE=RelWithDebInfo make-options -j%{num_cpus} end global # With base options set, the remainder of the file is used to define modules to build, in the # desired order, and set any module-specific options. # # Modules may be grouped into sets, and this is the normal practice. # # You can include other files inline using the "include" command. We do this here # to include files which are updated with kdesrc-build. # Qt and some Qt-using middleware libraries include %{base_dir}/qt5-build-include include %{base_dir}/custom-qt5-libs-build-include # KF5 and Plasma :) include %{base_dir}/kf5-qt5-build-include # To change options for modules that have already been defined, use an # 'options' block options kcoreaddons make-options -j4 end options diff --git a/modules/ksb/KDEProjectsReader.pm b/modules/ksb/KDEProjectsReader.pm index 338496f..f1a9f99 100644 --- a/modules/ksb/KDEProjectsReader.pm +++ b/modules/ksb/KDEProjectsReader.pm @@ -1,213 +1,217 @@ package ksb::KDEProjectsReader 0.50; # Class: KDEProjectsReader # # Enumerates and provides basic metadata of KDE projects, based on # the YAML metadata included in sysadmin/repo-management. use strict; use warnings; use 5.014; use File::Find; sub _verifyYAMLModuleLoaded { # Load YAML-reading module if available without causing compile error if it # isn't. Note that YAML::Tiny and YAML do not work since some metadata files # use features it doesn't support my @YAML_Opts = qw(Dump Load LoadFile); my @YAML_Mods = qw(YAML::XS YAML::Syck YAML::PP); my $success = 0; foreach my $mod (@YAML_Mods) { $success ||= eval "require $mod; $mod->import(\@YAML_Opts); 1;"; last if $success; } if (!$success) { die "Unable to load one of " . join(', ', @YAML_Mods) . " modules, one of which is needed to handle KDE project data."; } } # Method: new # # Constructs a new KDEProjectsReader. This doesn't contradict any part of the class # documentation which claims this class is a singleton however. This should be # called as a method (e.g. KDEProjectsReader->new(...)). # # Parameters: # $projectMetadataModule - ksb::Module reference to the repo-metadata module. -# $desiredProtocol - Normally 'git', but other protocols like 'http' can also -# be preferred (e.g. for proxy compliance). sub new { my $class = shift; my $projectMetadataModule = shift; - my $desiredProtocol = shift; _verifyYAMLModuleLoaded(); my $self = { # Maps short names to repo info blocks - repositories => { }, - protocol => $desiredProtocol, + repositories => { } }; $self = bless ($self, $class); $self->_readProjectData($projectMetadataModule); return $self; } # The 'main' method for this class. Reads in *all* KDE projects and notes # their details for later queries. # Be careful, can throw exceptions. sub _readProjectData { my ($self, $projectMetadataModule) = @_; my $srcdir = $projectMetadataModule->fullpath('source'); my $file_searched = 0; File::Find::find(sub { $self->_readYAML($_) if $_ eq 'metadata.yaml'; $file_searched = 1; }, "$srcdir/projects"); } sub _readYAML { my ($self, $filename) = @_; my $proj_data = LoadFile($filename); if (!$proj_data->{repoactive} || !$proj_data->{hasrepo} || - ($proj_data->{type} ne 'project' && $proj_data->{type} ne 'module')) + # + # previously there used to be a check on 'type' which + # only caught two specific cases: everything else was + # already covered by the hasrepo/repoactive checks. + # + # replace the type assertion with a specific check on the + # projects that would otherwise have been ignored + # + ($proj_data->{projectpath} eq 'kde-build-metadata' || $proj_data->{projectpath} eq 'repo-management')) { return; }; - my $repoName = $proj_data->{repopath}; - my $repoPath = $self->{protocol} . "://anongit.kde.org/$repoName"; + my $repoPath = $proj_data->{repopath}; + my $repoName = $proj_data->{identifier} // $repoPath; my $curRepository = { 'fullName' => $proj_data->{projectpath}, - 'repo' => $repoPath, + 'repo' => "kde:$repoPath", 'name' => $repoName, 'active' => !!$proj_data->{repoactive}, 'found_by' => 'direct', # can be changed in getModulesForProject }; $self->{repositories}->{$repoName} = $curRepository; } # Note on $proj: A /-separated path is fine, in which case we look # for the right-most part of the full path which matches all of searchProject. # e.g. kde/kdebase/kde-runtime would be matched by a proj of either # "kdebase/kde-runtime" or simply "kde-runtime". sub getModulesForProject { my ($self, $proj) = @_; my $repositoryRef = $self->{repositories}; my @results; my $findResults = sub { my @matchList = grep { _projectPathMatchesWildcardSearch( $repositoryRef->{$_}->{'fullName'}, $proj) } (sort keys %{$repositoryRef}); if ($proj =~ m/\*/) { $repositoryRef->{$_}->{found_by} = 'wildcard' foreach @matchList; } push @results, @matchList; }; # Wildcard matches happen as specified if asked for. # Non-wildcard matches have an implicit "$proj/*" search as well for # compatibility with previous use-modules # Project specifiers ending in .git are forced to be non-wildcarded. if ($proj !~ /\*/ && $proj !~ /\.git$/) { # We have to do a search to account for over-specified module names # like phonon/phonon $findResults->(); # Now setup for a wildcard search to find things like kde/kdelibs/baloo # if just 'kdelibs' is asked for. $proj .= '/*'; } $proj =~ s/\.git$//; # If still no wildcard and no '/' then we can use direct lookup by module # name. if ($proj !~ /\*/ && $proj !~ /\// && exists $repositoryRef->{$proj}) { push @results, $proj; } else { $findResults->(); } return @{$repositoryRef}{@results}; } # Utility subroutine, returns true if the given kde-project full path (e.g. # kde/kdelibs/nepomuk-core) matches the given search item. # # The search item itself is based on path-components. Each path component in # the search item must be present in the equivalent path component in the # module's project path for a match. A '*' in a path component position for the # search item matches any project path component. # # Finally, the search is pinned to search for a common suffix. E.g. a search # item of 'kdelibs' would match a project path of 'kde/kdelibs' but not # 'kde/kdelibs/nepomuk-core'. However 'kdelibs/*' would match # 'kde/kdelibs/nepomuk-core'. # # First parameter is the full project path from the kde-projects database. # Second parameter is the search item. # Returns true if they match, false otherwise. sub _projectPathMatchesWildcardSearch { my ($projectPath, $searchItem) = @_; my @searchParts = split(m{/}, $searchItem); my @nameStack = split(m{/}, $projectPath); if (scalar @nameStack >= scalar @searchParts) { my $sizeDifference = scalar @nameStack - scalar @searchParts; # We might have to loop if we somehow find the wrong start point for our search. # E.g. looking for a/b/* against a/a/b/c, we'd need to start with the second a. my $i = 0; while ($i <= $sizeDifference) { # Find our common prefix, then ensure the remainder matches item-for-item. for (; $i <= $sizeDifference; $i++) { last if $nameStack[$i] eq $searchParts[0]; } return if $i > $sizeDifference; # Not enough room to find it now # At this point we have synched up nameStack to searchParts, ensure they # match item-for-item. my $found = 1; for (my $j = 0; $found && ($j < @searchParts); $j++) { return 1 if $searchParts[$j] eq '*'; # This always works $found = 0 if $searchParts[$j] ne $nameStack[$i + $j]; } return 1 if $found; # We matched every item to the substring we found. $i++; # Try again } } return; } 1; diff --git a/modules/ksb/Module.pm b/modules/ksb/Module.pm index 6e1def6..7a7ea8d 100644 --- a/modules/ksb/Module.pm +++ b/modules/ksb/Module.pm @@ -1,1181 +1,1191 @@ package ksb::Module 0.20; # Class: Module # # Represents a source code module of some sort, which can be updated, built, # and installed. Includes a stringifying overload and can be sorted amongst # other ksb::Modules. use 5.014; use warnings; no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; use parent qw(ksb::OptionsBase); use ksb::Debug; use ksb::Util; use ksb::Updater::Svn; use ksb::Updater::Git; use ksb::Updater::Bzr; use ksb::Updater::KDEProject; use ksb::Updater::KDEProjectMetadata; use ksb::Updater::Qt5; use ksb::BuildException 0.20; use ksb::BuildSystem 0.30; use ksb::BuildSystem::Autotools; use ksb::BuildSystem::QMake; use ksb::BuildSystem::Qt4; use ksb::BuildSystem::Qt5; use ksb::BuildSystem::KDE4; use ksb::BuildSystem::CMakeBootstrap; use ksb::BuildSystem::Meson; use ksb::ModuleSet::Null; use Mojo::Promise; use Mojo::IOLoop; use POSIX qw(_exit :errno_h); use Storable qw(dclone thaw); use Carp 'confess'; use Scalar::Util 'blessed'; use overload '""' => 'toString', # Add stringify operator. '<=>' => 'compare', ; sub new { my ($class, $ctx, $name) = @_; croak_internal ("Empty ksb::Module constructed") unless $name; my $self = ksb::OptionsBase::new($class); # If building a BuildContext instead of a ksb::Module, then the context # can't have been setup yet... my $contextClass = 'ksb::BuildContext'; if ($class ne $contextClass && (!blessed($ctx) || !$ctx->isa($contextClass))) { croak_internal ("Invalid context $ctx"); } # Clone the passed-in phases so we can be different. my $phases = dclone($ctx->phases()) if $ctx; my %newOptions = ( name => $name, scm_obj => undef, build_obj => undef, phases => $phases, context => $ctx, 'module-set' => undef, metrics => { time_in_phase => { }, }, ); $self->installPhasePromises() if $class eq 'ksb::Module'; # alias our options into the build context for global vis $ctx->{build_options}->{$name} = $self->{options}; @{$self}{keys %newOptions} = values %newOptions; return $self; } sub phases { my $self = shift; return $self->{phases}; } sub moduleSet { my ($self) = @_; $self->{'module-set'} //= ksb::ModuleSet::Null->new(); return $self->{'module-set'}; } sub setModuleSet { my ($self, $moduleSet) = @_; assert_isa($moduleSet, 'ksb::ModuleSet'); $self->{'module-set'} = $moduleSet; } # Subroutine to retrieve a subdirectory path with tilde-expansion and # relative path handling. # The parameter is the option key (e.g. build-dir or log-dir) to read and # interpret. sub getSubdirPath { my ($self, $subdirOption) = @_; my $dir = $self->getOption($subdirOption); # If build-dir starts with a slash, it is an absolute path. return $dir if $dir =~ /^\//; # Make sure we got a valid option result. if (!$dir) { confess ("Reading option for $subdirOption gave empty \$dir!"); } # If it starts with a tilde, expand it out. if ($dir =~ /^~/) { $dir =~ s/^~/$ENV{'HOME'}/; } else { # Relative directory, tack it on to the end of $kdesrcdir. my $kdesrcdir = $self->getOption('source-dir'); $dir = "$kdesrcdir/$dir"; } return $dir; } # Method: getInstallPathComponents # # Returns the directory that a module should be installed in. # # NOTE: The return value is a hash. The key 'module' will return the final # module name, the key 'path' will return the full path to the module. The # key 'fullpath' will return their concatenation. # # For example, with $module == 'KDE/kdelibs', and no change in the dest-dir # option, you'd get something like: # # > { # > 'path' => '/home/user/kdesrc/KDE', # > 'module' => 'kdelibs', # > 'fullpath' => '/home/user/kdesrc/KDE/kdelibs' # > } # # If dest-dir were changed to e.g. extragear-multimedia, you'd get: # # > { # > 'path' => '/home/user/kdesrc', # > 'module' => 'extragear-multimedia', # > 'fullpath' => '/home/user/kdesrc/extragear-multimedia' # > } # # Parameters: # pathType - Either 'source' or 'build'. # # Returns: # hash (Not a hashref; See description). sub getInstallPathComponents { my $module = assert_isa(shift, 'ksb::Module'); my $type = shift; my $destdir = $module->destDir(); my $srcbase = $module->getSourceDir(); $srcbase = $module->getSubdirPath('build-dir') if $type eq 'build'; my $combined = "$srcbase/$destdir"; # Remove dup // $combined =~ s/\/+/\//; my @parts = split(/\//, $combined); my %result = (); $result{'module'} = pop @parts; $result{'path'} = join('/', @parts); $result{'fullpath'} = "$result{path}/$result{module}"; my $compatDestDir = $module->destDir($module->name()); my $fullCompatPath = "$srcbase/$compatDestDir"; # We used to have code here to migrate very old directory layouts. It was # removed as of about 2013-09-29. return %result; } # Do note that this returns the *base* path to the source directory, # without the module name or kde_projects stuff appended. If you want that # use subroutine fullpath(). sub getSourceDir { my $self = shift; return $self->getSubdirPath('source-dir'); } sub name { my $self = shift; return $self->{name}; } sub scm { my $self = shift; return $self->{scm_obj} if $self->{scm_obj}; # Look for specific setting of repository and svn-server. If both is # set it's a bug, if one is set, that's the type (because the user says # so...). Don't use getOption($key) as it will try to fallback to # global options. my $svn_status = $self->getOption('svn-server', 'module'); my $repository = $self->getOption('repository', 'module') // ''; my $rcfile = $self->buildContext()->rcFile(); if ($svn_status && $repository) { error (<{scm_obj} = ksb::Updater::Bzr->new($self); } # If it needs a repo it's git. Everything else is svn for now. $self->{scm_obj} //= $repository ? ksb::Updater::Git->new($self) : ksb::Updater::Svn->new($self); return $self->{scm_obj}; } sub setScmType { my ($self, $scmType) = @_; my $newType; given($scmType) { when('git') { $newType = ksb::Updater::Git->new($self); } when('proj') { $newType = ksb::Updater::KDEProject->new($self); } when('metadata') { $newType = ksb::Updater::KDEProjectMetadata->new($self); } # when('l10n') { $newType = ksb::l10nSystem->new($self); } when('svn') { $newType = ksb::Updater::Svn->new($self); } when('bzr') { $newType = ksb::Updater::Bzr->new($self); } when('qt5') { $newType = ksb::Updater::Qt5->new($self); } default { $newType = undef; } } $self->{scm_obj} = $newType; } # Returns a string describing the scm platform of the given module. # Return value: 'git' or 'svn' at this point, as appropriate. sub scmType { my $self = shift; return $self->scm()->name(); } sub currentScmRevision { my $self = shift; return $self->scm()->currentRevisionInternal(); } # Returns a new build system object, given the appropriate name. # This is a sub-optimal way to fix the problem of allowing users to override # the detected build system (we could instead use introspection to figure out # available build systems at runtime). However, KISS... sub buildSystemFromName { my ($self, $name) = @_; my %buildSystemClasses = ( 'generic' => 'ksb::BuildSystem', 'qmake' => 'ksb::BuildSystem::QMake', 'cmake-bootstrap' => 'ksb::BuildSystem::CMakeBootstrap', 'kde' => 'ksb::BuildSystem::KDE4', 'qt' => 'ksb::BuildSystem::Qt4', 'qt5' => 'ksb::BuildSystem::Qt5', 'autotools' => 'ksb::BuildSystem::Autotools', 'meson' => 'ksb::BuildSystem::Meson', ); my $class = $buildSystemClasses{lc $name} // undef; return $class->new($self) if ($class); # Past here, no class found croak_runtime("Invalid build system $name requested"); } sub buildSystem { my $self = shift; if ($self->{build_obj} && $self->{build_obj}->name() ne 'generic') { return $self->{build_obj}; } if (my $userBuildSystem = $self->getOption('override-build-system')) { $self->{build_obj} = $self->buildSystemFromName($userBuildSystem); return $self->{build_obj}; } # If not set, let's guess. my $buildType; my $sourceDir = $self->fullpath('source'); if (($self->getOption('repository') =~ /gitorious\.org\/qt\//) || ($self->getOption('repository') =~ /^kde:qt$/) || (-e "$sourceDir/bin/syncqt")) { $buildType = ksb::BuildSystem::Qt4->new($self); } # This test must come before the KDE buildsystem's as cmake's own # bootstrap system also has CMakeLists.txt if (!$buildType && (-e "$sourceDir/CMakeLists.txt") && (-e "$sourceDir/bootstrap")) { $buildType = ksb::BuildSystem::CMakeBootstrap->new($self); } if (!$buildType && (-e "$sourceDir/CMakeLists.txt" || $self->getOption('#xml-full-path'))) { $buildType = ksb::BuildSystem::KDE4->new($self); } # We have to assign to an array to force glob to return all results, # otherwise it acts like a non-reentrant generator whose output depends on # how many times it's been called... if (!$buildType && (my @files = glob ("$sourceDir/*.pro"))) { $buildType = ksb::BuildSystem::QMake->new($self); } # 'configure' is a popular fall-back option even for other build # systems so ensure we check last for autotools. if (!$buildType && (-e "$sourceDir/configure" || -e "$sourceDir/autogen.sh")) { $buildType = ksb::BuildSystem::Autotools->new($self); } # Someday move this up, but for now ensure that Meson happens after # configure/autotools support is checked for. if (!$buildType && -e "$sourceDir/meson.build") { $buildType = ksb::BuildSystem::Meson->new($self); } # Don't just assume the build system is KDE-based... $buildType //= ksb::BuildSystem->new($self); $self->{build_obj} = $buildType; return $self->{build_obj}; } # Sets the build system **object**, although you can find the build system # type afterwards (see buildSystemType). sub setBuildSystem { my ($self, $obj) = @_; assert_isa($obj, 'ksb::BuildSystem'); $self->{build_obj} = $obj; } # Current possible build system types: # KDE (i.e. cmake), Qt, l10n (KDE language buildsystem), autotools (either # configure or autogen.sh). A final possibility is 'pendingSource' which # simply means that we don't know yet. # # If the build system type is not set ('pendingSource' counts as being # set!) when this function is called then it will be autodetected if # possible, but note that not all possible types will be detected this way. # If in doubt use setBuildSystemType sub buildSystemType { my $self = shift; return $self->buildSystem()->name(); } # Creates the Mojo::Promises corresponding to each named phase. # E.g. the 'update' phase would be mapped to a sequence of subs to be executed # for the update. sub installPhasePromises { # Make our normal "self" a name unlikely to be used in a closure # below by mistake my $misnamedSelf = shift; # Each phase either maps directly to a subroutine which can # block (and whose return value will have a default handler), # or can map to an array of exactly 2 subroutines. The first # sub would be the blocking handler, the second sub is a # custom handler for the result. # Both subs are passed the module as the first param my %phaseBuilders = ( # Always runs, always first, the appropriate subsequent phase will # be linked to this one (based on --no-src, --install-only, etc.) start => sub { my $self = shift; my %pathinfo = $self->getInstallPathComponents('build'); super_mkdir($pathinfo{'path'}); return { was_successful => 1 }; }, # update => [ # See Application.pm for its custom runPhase_p # ], buildsystem => [ sub { my $self = shift; return { was_successful => $self->setupBuildSystem() }; }, sub { my ($self, $was_successful) = @_; return Mojo::Promise->new->reject('Unable to setup build system') unless $was_successful; return $was_successful; } ], build => [ sub { # called in child process, can block my $self = shift; # already returns a hashref in proper schema return $self->buildSystem()->buildInternal(); }, sub { # called in this process, with results my ($self, $was_successful, $extras) = @_; $self->setPersistentOption('last-build-rev', $self->currentScmRevision()); # $extras has metadata on number of warnings, but it's already # been reported by the time we get here. return 1 if $was_successful; return Mojo::Promise->new->reject('Build failed'); }, ], test => sub { my $self = shift; $self->buildSystem()->runTestsuite() if $self->getOption('run-tests'); # TODO: Make test failure a blocker for install? return { was_successful => 1 }; }, install => sub { my $self = shift; my $success = 1; $success = 0 if $self->getOption('install-after-build') and !$self->install(); return { was_successful => $success }; }, ); $misnamedSelf->{builders} = \%phaseBuilders; } # Subroutine to build this module. # Returns a promise that resolves to true (on success) or rejects with a error # string sub build { my $self = assert_isa(shift, 'ksb::Module'); my $moduleName = $self->name(); my %pathinfo = $self->getInstallPathComponents('build'); my $builddir = $pathinfo{'fullpath'}; my $buildSystem = $self->buildSystem(); return Mojo::Promise->new->reject('There is no build system to use') if ($buildSystem->name() eq 'generic' && !pretending()); # Ensure we're in a known directory before we start; some options remove # the old build directory that a previous module might have been using. super_mkdir($pathinfo{'path'}); p_chdir($pathinfo{'path'}); my $buildSystemPromise = $self->runPhase_p('buildsystem'); return $buildSystemPromise if $self->getOption('build-system-only'); # If we don't stop with the build system only, then keep extending that # promise chain to complete the build, test, and install return $buildSystemPromise->then(sub { return $self->runPhase_p('build'); })->then(sub { return $self->runPhase_p('test'); })->then(sub { return $self->runPhase_p('install'); }); } # Subroutine to setup the build system in a directory. # Returns boolean true on success, boolean false (0) on failure. sub setupBuildSystem { my $self = assert_isa(shift, 'ksb::Module'); my $moduleName = $self->name(); my $buildSystem = $self->buildSystem(); if ($buildSystem->name() eq 'generic' && !pretending()) { croak_internal('Build system determination still pending when build attempted.'); } my $refreshReason = $buildSystem->needsRefreshed(); if ($refreshReason ne "") { # Check to see if we're actually supposed to go through the # cleaning process. if (!$self->getOption('#cancel-clean') && !$buildSystem->cleanBuildSystem()) { warning ("\tUnable to clean r[$self]!"); return 0; } } if (!$buildSystem->createBuildSystem()) { error ("\tError creating r[$self]'s build system!"); return 0; } # Now we're in the checkout directory # So, switch to the build dir. # builddir is automatically set to the right value for qt p_chdir ($self->fullpath('build')); if (!$buildSystem->configureInternal()) { error ("\tUnable to configure r[$self] with " . $self->buildSystemType()); # Add undocumented ".refresh-me" file to build directory to flag # for --refresh-build for this module on next run. See also the # "needsRefreshed" subroutine. if (open my $fh, '>', '.refresh-me') { say $fh "# Build directory will be re-generated next kdesrc-build run"; say $fh "# due to failing to complete configuration on the last run"; close $fh; }; return 0; } return 1; } # Responsible for installing the module (no update, build, etc.) # Return value: Boolean flag indicating whether module installed successfully or # not. # Exceptions may be thrown for abnormal conditions (e.g. no build dir exists) sub install { my $self = assert_isa(shift, 'ksb::Module'); my $builddir = $self->fullpath('build'); my $buildSysFile = $self->buildSystem()->configuredModuleFileName(); if (!pretending() && ! -e "$builddir/$buildSysFile") { warning ("\tThe build system doesn't exist for r[$self]."); warning ("\tTherefore, we can't install it. y[:-(]."); return 0; } $self->setupEnvironment(); my @makeInstallOpts = split(' ', $self->getOption('make-install-prefix')); # We can optionally uninstall prior to installing # to weed out old unused files. if ($self->getOption('use-clean-install') && $self->getPersistentOption('last-install-rev')) { if (!$self->buildSystem()->uninstallInternal(@makeInstallOpts)) { warning ("\tUnable to uninstall r[$self] before installing the new build."); warning ("\tContinuing anyways..."); } else { $self->unsetPersistentOption('last-install-rev'); } } if (!$self->buildSystem()->installInternal(@makeInstallOpts)) { error ("\tUnable to install r[$self]!"); $self->buildContext()->markModulePhaseFailed('install', $self); return 0; } if (pretending()) { pretend ("\tWould have installed g[$self]"); return 1; } # Past this point we know we've successfully installed, for real. $self->setPersistentOption('last-install-rev', $self->currentScmRevision()); my $remove_setting = $self->getOption('remove-after-install'); # Possibly remove the srcdir and builddir after install for users with # a little bit of HD space. if($remove_setting eq 'all') { # Remove srcdir my $srcdir = $self->fullpath('source'); note ("\tRemoving b[r[$self source]."); safe_rmtree($srcdir); } if($remove_setting eq 'builddir' || $remove_setting eq 'all') { # Remove builddir note ("\tRemoving b[r[$self build directory]."); safe_rmtree($builddir); # We're likely already in the builddir, so chdir back to the root p_chdir('/'); } return 1; } # Handles uninstalling this module (or its sub-directories as given by the checkout-only # option). # # Returns boolean false on failure, boolean true otherwise. sub uninstall { my $self = assert_isa(shift, 'ksb::Module'); my $builddir = $self->fullpath('build'); my $buildSysFile = $self->buildSystem()->configuredModuleFileName(); if (!pretending() && ! -e "$builddir/$buildSysFile") { warning ("\tThe build system doesn't exist for r[$self]."); warning ("\tTherefore, we can't uninstall it."); return 0; } $self->setupEnvironment(); my @makeInstallOpts = split(' ', $self->getOption('make-install-prefix')); if (!$self->buildSystem()->uninstallInternal(@makeInstallOpts)) { error ("\tUnable to uninstall r[$self]!"); $self->buildContext()->markModulePhaseFailed('install', $self); return 0; } if (pretending()) { pretend ("\tWould have uninstalled g[$self]"); return 1; } $self->unsetPersistentOption('last-install-rev'); return 1; } sub buildContext { my $self = shift; return $self->{context}; } # Integrates 'set-env' option to the build context environment sub applyUserEnvironment { my $self = assert_isa(shift, 'ksb::Module'); my $ctx = $self->buildContext(); # Let's see if the user has set env vars to be set. # Note the global set-env must be checked separately anyways, so # we limit inheritance when searching. my $env_hash_ref = $self->getOption('set-env', 'module'); while (my ($key, $value) = each %{$env_hash_ref}) { $ctx->queueEnvironmentVariable($key, $value); } } # Establishes proper build environment in the build context. Should be run # before forking off commands for e.g. updates, builds, installs, etc. sub setupEnvironment { my $self = assert_isa(shift, 'ksb::Module'); my $ctx = $self->buildContext(); - my $kdedir = $self->getOption('kdedir'); - my $qtdir = $self->getOption('qtdir'); my $prefix = $self->installationPath(); # Add global set-envs and context $self->buildContext()->applyUserEnvironment(); - # Ensure the platform libraries we're building can be found, as long as they - # are not the system's own libraries. - for my $platformDir ($qtdir, $kdedir) { - next unless $platformDir; # OK, assume system platform is usable - next if $platformDir eq '/usr'; # Don't 'fix' things if system platform - # manually set + # Build system's environment injection + my $buildSystem = $self->buildSystem(); - $ctx->prependEnvironmentValue('PKG_CONFIG_PATH', "$platformDir/lib/pkgconfig"); - $ctx->prependEnvironmentValue('LD_LIBRARY_PATH', "$platformDir/lib"); - $ctx->prependEnvironmentValue('PATH', "$platformDir/bin"); + # + # Suppress injecting qtdir/kdedir related environment variables if a toolchain is also set + # Let the toolchain files/definitions take care of themselves. + # + if ($buildSystem->hasToolchain()) { + note ("\tNot setting environment variables for b[$self]: a custom toolchain is used"); + } else { + my $kdedir = $self->getOption('kdedir'); + my $qtdir = $self->getOption('qtdir'); + + # Ensure the platform libraries we're building can be found, as long as they + # are not the system's own libraries. + for my $platformDir ($qtdir, $kdedir) { + next unless $platformDir; # OK, assume system platform is usable + next if $platformDir eq '/usr'; # Don't 'fix' things if system platform + # manually set + + $ctx->prependEnvironmentValue('PKG_CONFIG_PATH', "$platformDir/lib/pkgconfig"); + $ctx->prependEnvironmentValue('LD_LIBRARY_PATH', "$platformDir/lib"); + $ctx->prependEnvironmentValue('PATH', "$platformDir/bin"); + } } - # Build system's environment injection - my $buildSystem = $self->buildSystem(); $buildSystem->prepareModuleBuildEnvironment($ctx, $self, $prefix); # Read in user environment defines $self->applyUserEnvironment() unless $self == $ctx; } # Returns the path to the log directory used during this run for this # ksb::Module, based on an autogenerated unique id. The id doesn't change # once generated within a single run of the script. sub getLogDir { my ($self) = @_; return $self->buildContext()->getLogDirFor($self); } # Returns a full path that can be open()'d to write a log # file, based on the given basename (with extension). # Updates the 'latest' symlink as well, unlike getLogDir # Use when you know you're going to create a new log sub getLogPath { my ($self, $path) = @_; return $self->buildContext()->getLogPathFor($self, $path); } sub toString { my $self = shift; return $self->name(); } sub compare { my ($self, $other) = @_; return $self->name() cmp $other->name(); } # Throws an exception on error, otherwise returns number of updates known to # have occurred. Only returns 0 if we positively know from the scm that no # update occurred. sub update { my ($self, $ctx) = @_; my $module_src_dir = $self->getSourceDir(); my $kdesrc = $ctx->getSourceDir(); if ($kdesrc ne $module_src_dir && !super_mkdir($module_src_dir)) { # This module has a different source directory, ensure it exists. croak_runtime ("Unable to create separate source directory for $self at $module_src_dir"); } # Use 1 as default value to force a rebuild if we can't determine there # were truly *no* updates my $count = $self->scm()->updateInternal() // 1; return { was_successful => 1, update_count => $count }; } # OVERRIDE # # This calls OptionsBase::setOption and performs any Module-specific # handling. sub setOption { my ($self, %options) = @_; # Ensure we don't accidentally get fed module-set options for (qw(git-repository-base use-modules ignore-modules)) { if (exists $options{$_}) { error (" r[b[*] module b[$self] should be declared as module-set to use b[$_]"); die ksb::BuildException::Config->new($_, "Option $_ can only be used in module-set"); }; } # Special case handling. if (exists $options{'filter-out-phases'}) { for my $phase (split(' ', $options{'filter-out-phases'})) { $self->phases()->filterOutPhase($phase); } delete $options{'filter-out-phases'}; } # When running in a subprocess, option changes will be forgotten unless they are fed # back to parent process, so store those in a special array also. if (exists $self->buildContext()->{'#pending'}) { # Only forward 'plain' options, just a sanity check my @keys = grep { !ref($options{$_}) } (keys %options); $self->{options}->{'#pending'} //= { }; @{$self->{options}->{'#pending'}}{@keys} = @options{@keys}; } $self->SUPER::setOption(%options); } # OVERRIDE # # This subroutine returns an option value for a given module. Some globals # can't be overridden by a module's choice (but see 2nd parameter below). # If so, the module's choice will be ignored, and a warning will be issued. # # Option names are case-sensitive! # # Some options (e.g. cmake-options, configure-flags) have the global value # and then the module's own value appended together. To get the actual # module setting you must use the level limit parameter set to 'module'. # # Likewise, some qt module options do not obey the previous proviso since # Qt options are not likely to agree nicely with generic KDE buildsystem # options. # # 1st parameter: Name of option # 2nd parameter: Level limit (optional). If not present, then the value # 'allow-inherit' is used. Options: # - allow-inherit: Module value is used if present (with exceptions), # otherwise global is used. # - module: Only module value is used (if you want only global then use the # buildContext) NOTE: This overrides global "sticky" options as well! sub getOption { my ($self, $key, $levelLimit) = @_; my $ctx = $self->buildContext(); $levelLimit //= 'allow-inherit'; # Some global options would probably make no sense applied to Qt. my @qtCopyOverrides = qw(branch configure-flags tag cxxflags); if (list_has(\@qtCopyOverrides, $key) && $self->buildSystemType() eq 'Qt') { $levelLimit = 'module'; } assert_in($levelLimit, [qw(allow-inherit module)]); # If module-only, check that first. return $self->{options}{$key} if $levelLimit eq 'module'; my $ctxValue = $ctx->getOption($key); # we'll use this a lot from here # Some global options always override module options. return $ctxValue if $ctx->hasStickyOption($key); # Some options append to the global (e.g. conf flags) my @confFlags = qw(cmake-options configure-flags cxxflags); if (list_has(\@confFlags, $key) && $ctxValue) { return trimmed("$ctxValue " . ($self->{options}{$key} || '')); } # Everything else overrides the global option, unless it's simply not # set at all. return $self->{options}{$key} // $ctxValue; } # Gets persistent options set for this module. First parameter is the name # of the option to lookup. Undef is returned if the option is not set, # although even if the option is set, the value returned might be empty. # Note that ksb::BuildContext also has this function, with a slightly # different signature, which OVERRIDEs this function since Perl does not # have parameter-based method overloading. sub getPersistentOption { my ($self, $key) = @_; return $self->buildContext()->getPersistentOption($self->name(), $key); } # Sets a persistent option (i.e. survives between processes) for this module. # First parameter is the name of the persistent option. # Second parameter is its actual value. # See the warning for getPersistentOption above, it also applies for this # method vs. ksb::BuildContext::setPersistentOption sub setPersistentOption { my ($self, $key, $value) = @_; return $self->buildContext()->setPersistentOption($self->name(), $key, $value); } # Unsets a persistent option for this module. # Only parameter is the name of the option to unset. sub unsetPersistentOption { my ($self, $key) = @_; $self->buildContext()->unsetPersistentOption($self->name(), $key); } # Returns the path to the desired directory type (source or build), # including the module destination directory itself. sub fullpath { my ($self, $type) = @_; assert_in($type, [qw/build source/]); my %pathinfo = $self->getInstallPathComponents($type); return $pathinfo{'fullpath'}; } # Returns the "full kde-projects path" for the module. As should be obvious by # the description, this only works for modules with an scm type that is a # Updater::KDEProject (or its subclasses), but modules that don't fall into this # hierarchy will just return the module name (with no path components) anyways. sub fullProjectPath { my $self = shift; return ($self->getOption('#xml-full-path', 'module') || $self->name()); } # Returns true if this module is (or was derived from) a kde-projects module. sub isKDEProject { my $self = shift; return $self->hasOption('#xml-full-path'); } # Subroutine to return the name of the destination directory for the # checkout and build routines. Based on the dest-dir option. The return # value will be relative to the src/build dir. The user may use the # '$MODULE' or '${MODULE}' sequences, which will be replaced by the name of # the module in question. # # The first parameter is optional, but if provided will be used as the base # path to replace $MODULE entries in dest-dir. sub destDir { my $self = assert_isa(shift, 'ksb::Module'); my $destDir = $self->getOption('dest-dir'); my $basePath = ""; if ($self->getOption('ignore-kde-structure')) { $basePath = $self->name(); } else { $basePath = shift // $self->getOption('#xml-full-path'); $basePath ||= $self->name(); # Default if not provided in repo-metadata } $destDir =~ s/(\$\{MODULE})|(\$MODULE\b)/$basePath/g; return $destDir; } # Subroutine to return the installation path of a given module (the value # that is passed to the CMAKE_INSTALL_PREFIX CMake option). # It is based on the "prefix" and, if it is not set, the "kdedir" option. # The user may use '$MODULE' or '${MODULE}' in the "prefix" option to have # them replaced by the name of the module in question. sub installationPath { my $self = assert_isa(shift, 'ksb::Module'); my $path = $self->getOption('prefix'); if (!$path) { return $self->getOption('kdedir'); } my $moduleName = $self->name(); $path =~ s/(\$\{MODULE})|(\$MODULE\b)/$moduleName/g; return $path; } # Supports the subprocess handler in runPhase_p by reading messages that were # generated by the child process and sent back to us, and dispatching to # handler for real-time updates. Messages that can wait until the subprocess is # complete are handled separately. sub _readAndDispatchInProcessMessages { my ($self, $frozen_message, $monitor, $phaseName) = @_; my $linesRef = eval { thaw($frozen_message) }; croak_internal("Failed to read msg from child handler: $@") unless $linesRef; if (exists $linesRef->{message}) { $monitor->noteLogEvents("$self", $phaseName, [$linesRef->{message}]); } elsif (exists $linesRef->{progress}) { my ($x, $y) = @{$linesRef->{progress}}; $monitor->markPhaseProgress("$self", $phaseName, $x / $y) if $y > 0; } else { croak_internal("Couldn't handle message $frozen_message from child."); } } # Runs the given phase in a separate subprocess, using provided sub references. # Assumes use of promises for the provided sub references -- if launching the # subprocess fails, then a rejected promise is returned in the completion sub # reference # Returns a promise that yields the return value of the completion sub # reference. sub runPhase_p { my ($self, $phaseName, $blocking_coderef, $completion_coderef) = @_; my $phaseSubs = $self->{builders}->{$phaseName}; if (ref($phaseSubs) eq 'CODE') { $blocking_coderef //= $phaseSubs; } elsif (ref($phaseSubs) eq 'ARRAY') { ($blocking_coderef, $completion_coderef) = @{$phaseSubs}; croak_internal("Missing subs for $phaseName") unless ($blocking_coderef && $completion_coderef); } else { croak_internal("self->builders->{$phaseName} should not be set") if defined $phaseSubs; } # Default handler $completion_coderef //= sub { my ($module, $result, $extras) = (@_); return Mojo::Promise->new->reject unless $result; return $result; }; my $promise = Mojo::Promise->new; my $ctx = $self->buildContext(); pipe (my $reader, my $writer) or croak_runtime("Couldn't open pipe to subprocess for $phaseName: $!"); # Setup a pipe from child to parent so we can get updates as the phase # progresses, logs, etc. my $reactor = Mojo::IOLoop->singleton->reactor; my $monitor = $self->buildContext()->statusMonitor(); my $reactorPromise = Mojo::Promise->new; $reactor->io($reader => sub { my ($reactor) = @_; my $buffer; if ((my $lengthRead = $reader->sysread($buffer, 8192)) == 0) { # eof $reactor->remove($reader); close $reader; $reactorPromise->resolve; } elsif ($lengthRead > 0) { $self->_readAndDispatchInProcessMessages($buffer, $monitor, $phaseName); } else { croak_runtime("Error reading from child pipe: $!"); $reactorPromise->reject; } }); $reactor->watch($reader, 1, 0); # watch for pipe readability only $monitor->markPhaseStart($self->name(), $phaseName); my $start_time = time; Mojo::IOLoop->subprocess( sub { # blocks, runs in separate process $SIG{INT} = sub { POSIX::_exit(EINTR); }; $0 = "kdesrc-build[$phaseName]"; # This causes setOption to record changes, and is deliberately not # within ctx->{options} $ctx->{'#pending'} = { }; $writer->autoflush(1); close $reader; # we can't use this anyways ksb::Debug::setOutputHandle($writer); $self->buildContext->resetEnvironment(); $self->setupEnvironment(); # This coderef should return a hashref: { # was_successful => bool, # ... (other details) # } my $resultRef = $blocking_coderef->($self); my $result = $resultRef->{was_successful}; my %newOptions; # Grab any newly-set options to feed back to parent my @affectedMods = grep { exists $ctx->{build_options}->{$_}->{'#pending'}; } (keys %{$ctx->{build_options}}); foreach my $affected (@affectedMods) { $newOptions{$affected} = $ctx->{build_options}->{$affected}->{'#pending'}; } return { result => $result, newOptions => \%newOptions, extras => $resultRef, }; }, sub { # runs in this process once subprocess is done my ($subprocess, $err, $resultsRef) = @_; close $writer; # can't close it earlier because must be open at fork if ($err) { $ctx->markModulePhaseFailed($phaseName, $self); return $promise->reject($err); } $self->{metrics}->{time_in_phase}->{$phaseName} = time - $start_time; # Apply options that may have changed during child proc execution. if (%{$resultsRef->{newOptions}}) { while(my ($k, $v) = each %{$resultsRef->{newOptions}}) { my %modulesNewOptions = %{$v}; @{$ctx->{build_options}->{$k}}{keys %modulesNewOptions} = values %modulesNewOptions; } } my $result = $resultsRef->{result}; if ($result) { $ctx->markModulePhaseSucceeded($phaseName, $self, $resultsRef->{extras}); } else { $ctx->markModulePhaseFailed($phaseName, $self); } return $reactorPromise->then(sub { # This coderef should resolve or reject the promise, if used $result ? $promise->resolve($completion_coderef->($self, $result, $resultsRef)) : $promise->reject ($completion_coderef->($self, $result, $resultsRef)); }); } ); return $promise; } 1; diff --git a/modules/ksb/ModuleSet.pm b/modules/ksb/ModuleSet.pm index 328eabb..c1eb94d 100644 --- a/modules/ksb/ModuleSet.pm +++ b/modules/ksb/ModuleSet.pm @@ -1,214 +1,216 @@ package ksb::ModuleSet 0.20; # Class: ModuleSet # # This represents a collective grouping of modules that share common options, # and share a common repository (in this case, based on the git-repository-base # option, but see also the more common ModuleSet::KDEProjects which is used for # the special kde-projects repositories). # # This is parsed from module-set declarations in the rc-file. # # The major conceit here is several things: # # 1. A hash of options to set for each module read into this module set. # 2. A list of module search declarations to be used to construct modules for # this module set (in the case of kde-projects repository). For other # repository types we can still consider it a 'search', but with the # understanding that it's a 1:1 mapping to the 'found' module (which may not # exist for real). # 3. A list of module search declarations to *ignore* from this module set, # using the same syntax as used to search for them in 2. This is only really # useful at this point for kde-projects repository as everything else requires # you to manually specify modules one-by-one (module-sets are only useful here # for option grouping as per 1.). # 4. A name, which must not be empty, although user-specified names cannot be # assumed to be unique. # 5. A ksb::PhaseList describing what phases of the build a module should # participate in by default. # # See also: git-repository-base, ModuleSet::KDEProjects, use-modules use strict; use warnings; use 5.014; no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; use parent qw(ksb::OptionsBase); use ksb::Debug; use ksb::Util; use ksb::PhaseList; use ksb::Module; use Storable qw(dclone); sub new { my ($class, $ctx, $name) = @_; $name //= ''; my $self = ksb::OptionsBase::new($class); my %newOptions = ( name => $name, module_search_decls => [ ], module_ignore_decls => [ ], phase_list => ksb::PhaseList->new($ctx->phases()->phases()), ); @{$self}{keys %newOptions} = values %newOptions; return $self; } sub name { my $self = shift; return $self->{name}; } sub setName { my ($self, $name) = @_; $self->{name} = $name; return; } # Just returns a reference to the existing ksb::PhaseList, there's no way to # replace this, though you can alter the underlying phases through the # ksb::PhaseList object itself. sub phases { my $self = shift; return $self->{phase_list}; } sub modulesToFind { my $self = shift; return @{$self->{module_search_decls}}; } sub setModulesToFind { my ($self, @moduleDecls) = @_; $self->{module_search_decls} = [@moduleDecls]; return; } # Same as modulesToFind, but strips away any path components to leave just # module names. # E.g. a "use-modules kde/kdelibs juk.git" would give (kdelibs, juk) as the # result list. sub moduleNamesToFind { my $self = shift; return map { s{([^/]+)$}{$1}; s{\.git$}{}; $_ } ($self->modulesToFind()); } sub modulesToIgnore { my $self = shift; return @{$self->{module_ignore_decls}}; } sub addModulesToIgnore { my ($self, @moduleDecls) = @_; $self->{module_ignore_decls} //= []; push @{$self->{module_ignore_decls}}, @moduleDecls; return; } # Should be called for each new ksb::Module created in order to setup common # module options. sub _initializeNewModule { my ($self, $newModule) = @_; $newModule->setModuleSet($self); $newModule->setScmType('git'); $newModule->phases->phases($self->phases()->phases()); $newModule->mergeOptionsFrom($self); + $newModule->{'#create-id'} = $self->{'#create-id'} + if exists $self->{'#create-id'}; } # OVERRIDE # # Handles module-set specific options for OptionsBase's setOption # sub setOption { my ($self, %options) = @_; # Special-case handling if (exists $options{'use-modules'}) { my @modules = split(' ', $options{'use-modules'}); if (not @modules) { error ("No modules were selected for module-set " . $self->name()); error ("in the y[use-modules] entry."); die ksb::BuildException::Config->new('use-modules', 'Invalid use-modules'); } $self->setModulesToFind(@modules); delete $options{'use-modules'}; } if (exists $options{'ignore-modules'}) { my @modules = split(' ', $options{'ignore-modules'}); if (not @modules) { error ("No modules were selected for module-set " . $self->name()); error ("in the y[ignore-modules] entry."); die ksb::BuildException::Config->new('ignore-modules', 'Invalid ignore-modules'); } $self->addModulesToIgnore(@modules); delete $options{'ignore-modules'}; } # Actually set options. $self->SUPER::setOption(%options); } # This function should be called after options are read and build metadata is # available in order to convert this module set to a list of ksb::Module. # Any modules ignored by this module set are excluded from the returned list. # The modules returned have not been added to the build context. sub convertToModules { my ($self, $ctx) = @_; my @moduleList; # module names converted to ksb::Module objects. my $optionsRef = $self->{options}; # Note: This returns a hashref, not a string. my $repoSet = $ctx->getOption('git-repository-base'); # Setup default options for each module # If we're in this method, we must be using the git-repository-base method # of setting up a module-set, so there is no 'search' or 'ignore' to # handle, just create ksb::Module and dump options into them. for my $moduleItem ($self->modulesToFind()) { my $moduleName = $moduleItem; $moduleName =~ s/\.git$//; my $newModule = ksb::Module->new($ctx, $moduleName); $self->_initializeNewModule($newModule); push @moduleList, $newModule; # Setup the only feature actually specific to a module-set, which is # the repository handling. my $selectedRepo = $repoSet->{$optionsRef->{'repository'}}; $newModule->setOption('repository', $selectedRepo . $moduleItem); } if (not scalar $self->modulesToFind()) { warning ("No modules were defined for the module-set $self->name()"); warning ("You should use the g[b[use-modules] option to make the module-set useful."); } return @moduleList; } 1; diff --git a/modules/ksb/ModuleSet/KDEProjects.pm b/modules/ksb/ModuleSet/KDEProjects.pm index e8b1ab8..a50bad5 100644 --- a/modules/ksb/ModuleSet/KDEProjects.pm +++ b/modules/ksb/ModuleSet/KDEProjects.pm @@ -1,227 +1,223 @@ package ksb::ModuleSet::KDEProjects 0.30; # Class: ModuleSet::KDEProjects # # This represents a collective grouping of modules that share common options, # based on the KDE project repositories. Metadata for that repository is # itself housed in a dedicated KDE.org git repository "sysadmin/repo-metadata", # which this class uses to imbue ksb::Modules generated by this ModuleSet. # # The only changes here are to allow for expanding out module specifications # (except for ignored modules), by using KDEProjectsReader. # # See also: ModuleSet use strict; use warnings; use 5.014; use parent qw(ksb::ModuleSet); no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; use ksb::BuildContext 0.20; use ksb::BuildException; use ksb::Debug; use ksb::KDEProjectsReader 0.50; use ksb::Module; use ksb::Util; sub new { my $self = ksb::ModuleSet::new(@_); $self->{projectsDataReader} = undef; # Will be filled in when we get fh return $self; } # Simple utility subroutine. See List::Util's perldoc sub none_true { ($_ && return 0) for @_; return 1; } sub _createMetadataModule { my ($ctx, $moduleName) = @_; my $metadataModule = ksb::Module->new($ctx, $moduleName =~ s,/,-,r); # Hardcode the results instead of expanding out the project info $metadataModule->setOption('repository', "kde:$moduleName"); $metadataModule->setOption('#xml-full-path', $moduleName); $metadataModule->setOption('#branch:stable', 'master'); $metadataModule->setScmType('metadata'); $metadataModule->setOption('disable-snapshots', 1); $metadataModule->setOption('branch', 'master'); my $moduleSet = ksb::ModuleSet::KDEProjects->new($ctx, ''); $metadataModule->setModuleSet($moduleSet); # Ensure we only ever try to update source, not build. $metadataModule->phases()->phases('update'); return $metadataModule; } # Function: getDependenciesModule # # Static. Returns a that can be used to download the # 'kde-build-metadata' module, which itself contains module dependencies # in the KDE build system. The module is meant to be held by the # # Parameters: # ctx - the for this script execution. sub getDependenciesModule { my $ctx = assert_isa(shift, 'ksb::BuildContext'); return _createMetadataModule($ctx, 'kde-build-metadata'); } # Function: getProjectMetadataModule # # Static. Returns a that can be used to download the # 'repo-metadata' module, which itself contains information on each # repository in the KDE build system (though currently not # dependencies). The module is meant to be held by the # # Parameters: # ctx - the for this script execution. sub getProjectMetadataModule { my $ctx = assert_isa(shift, 'ksb::BuildContext'); return _createMetadataModule($ctx, 'sysadmin/repo-metadata'); } # Function: _expandModuleCandidates # # A class method which goes through the modules in our search list (assumed to # be found in kde-projects), expands them into their equivalent git modules, # and returns the fully expanded list. Non kde-projects modules cause an error, # as do modules that do not exist at all within the database. # # *Note*: Before calling this function, the kde-projects database itself must # have been downloaded first. See getProjectMetadataModule, which ties to the # BuildContext. # # Modules that are part of a module-set requiring a specific branch, that don't # have that branch, are still listed in the return result since there's no way # to tell that the branch won't be there. These should be removed later. # # Parameters: # ctx - The in use. # moduleSearchItem - The search description to expand in ksb::Modules. See # _projectPathMatchesWildcardSearch for a description of the syntax. # # Returns: # @modules - List of expanded git . # # Throws: # Runtime - if the kde-projects database was required but couldn't be # downloaded or read. # Runtime - if the git-desired-protocol is unsupported. # Runtime - if an "assumed" kde-projects module was not actually one. sub _expandModuleCandidates { my $self = assert_isa(shift, 'ksb::ModuleSet::KDEProjects'); my $ctx = assert_isa(shift, 'ksb::BuildContext'); my $moduleSearchItem = shift; my @allModuleResults = $ctx-> getProjectDataReader()-> getModulesForProject($moduleSearchItem); croak_internal ("Unknown KDE project: $moduleSearchItem") unless @allModuleResults; # It's possible to match modules which are marked as inactive on # projects.kde.org, elide those. my @activeResults = grep { $_->{'active'} } (@allModuleResults); if (!@activeResults) { warning (" y[b[*] Module y[$moduleSearchItem] is apparently a KDE collection, but contains no\n" . "active modules to build!"); my $count = scalar @allModuleResults; if ($count > 0) { warning ("\tAlthough no active modules are available, there were\n" . "\t$count inactive modules. Perhaps the git modules are not ready?"); } } # Setup module options. my @moduleList; my @ignoreList = $self->modulesToIgnore(); foreach (@activeResults) { my $result = $_; - my $repo = $result->{'repo'}; - - # Prefer kde: alias to normal clone URL. - $repo =~ s(^(git|https)://anongit\.kde\.org/)(kde:); my $newModule = ksb::Module->new($ctx, $result->{'name'}); $self->_initializeNewModule($newModule); - $newModule->setOption('repository', $repo); + $newModule->setOption('repository', $result->{'repo'}); $newModule->setOption('#xml-full-path', $result->{'fullName'}); $newModule->setOption('#branch:stable', undef); $newModule->setOption('#found-by', $result->{found_by}); $newModule->setScmType('proj'); if (none_true( map { ksb::KDEProjectsReader::_projectPathMatchesWildcardSearch( $result->{'fullName'}, $_ ) } (@ignoreList))) { push @moduleList, $newModule; } else { debug ("--- Ignoring matched active module $newModule in module set " . $self->name()); } }; return @moduleList; } # This function should be called after options are read and build metadata is # available in order to convert this module set to a list of ksb::Module. # Any modules ignored by this module set are excluded from the returned list. # The modules returned have not been added to the build context. sub convertToModules { my ($self, $ctx) = @_; my @moduleList; # module names converted to ksb::Module objects. my %foundModules; # Setup default options for each module # Extraction of relevant kde-project modules will be handled immediately # after this phase of execution. for my $moduleItem ($self->modulesToFind()) { # We might have already grabbed the right module recursively. next if exists $foundModules{$moduleItem}; # eval in case the YAML processor throws an exception. undef $@; my @candidateModules = eval { $self->_expandModuleCandidates($ctx, $moduleItem); }; if ($@) { die $@ if had_an_exception(); # Forward exception objects up croak_runtime("The KDE Project database could not be understood: $@"); } my @moduleNames = map { $_->name() } @candidateModules; @foundModules{@moduleNames} = (1) x @moduleNames; push @moduleList, @candidateModules; } if (not scalar @moduleList) { warning ("No modules were defined for the module-set " . $self->name()); warning ("You should use the g[b[use-modules] option to make the module-set useful."); } return @moduleList; } 1; diff --git a/modules/ksb/Updater/Git.pm b/modules/ksb/Updater/Git.pm index 8cb3825..b474e0a 100644 --- a/modules/ksb/Updater/Git.pm +++ b/modules/ksb/Updater/Git.pm @@ -1,835 +1,930 @@ package ksb::Updater::Git 0.15; # Module which is responsible for updating git-based source code modules. Can # have some features overridden by subclassing (see ksb::Updater::KDEProject # for an example). use strict; use warnings; use 5.014; use parent qw(ksb::Updater); use ksb::BuildException; use ksb::Debug; use ksb::Util; use File::Basename; # basename use File::Spec; # tmpdir use POSIX qw(strftime); use List::Util qw(first); use IPC::Cmd qw(run_forked); use constant { DEFAULT_GIT_REMOTE => 'origin', }; # scm-specific update procedure. # May change the current directory as necessary. sub updateInternal { my $self = assert_isa(shift, 'ksb::Updater::Git'); return $self->updateCheckout(); } sub name { return 'git'; } sub currentRevisionInternal { my $self = assert_isa(shift, 'ksb::Updater::Git'); return $self->commit_id('HEAD'); } # Returns the current sha1 of the given git "commit-ish". sub commit_id { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $commit = shift or croak_internal("Must specify git-commit to retrieve id for"); my $module = $self->module(); my $gitdir = $module->fullpath('source') . '/.git'; # Note that the --git-dir must come before the git command itself. my ($id, undef) = filter_program_output( undef, # No filter qw/git --git-dir/, $gitdir, 'rev-parse', $commit, ); chomp $id if $id; return $id; } sub _verifyRefPresent { my ($self, $module, $repo) = @_; my ($commitId, $commitType) = $self->_determinePreferredCheckoutSource($module); return 1 if pretending(); my $ref = $commitId; my $hashref = run_forked("git ls-remote --exit-code $repo $ref", { timeout => 10, discard_output => 1, terminate_on_parent_sudden_death => 1}); my $result = $hashref->{exit_code}; return 0 if ($result == 2); # Connection successful, but ref not found return 1 if ($result == 0); # Ref is present croak_runtime("git had error exit $result when verifying $ref present in repository at $repo"); } # Perform a git clone to checkout the latest branch of a given git module # # First parameter is the repository (typically URL) to use. # Throws an exception if it fails. sub _clone { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $git_repo = shift; my $module = $self->module(); my $srcdir = $module->fullpath('source'); my @args = ('--', $git_repo, $srcdir); note ("Cloning g[$module]"); p_chdir($module->getSourceDir()); my ($commitId, $commitType) = $self->_determinePreferredCheckoutSource($module); $commitId =~ s,^refs/tags/,,; # git-clone -b doesn't like refs/tags/ unshift @args, '-b', $commitId; # Checkout branch right away if (0 != log_command($module, 'git-clone', ['git', 'clone', @args])) { croak_runtime("Failed to make initial clone of $module"); } #$ipc->notifyPersistentOptionChange($module->name(), 'git-cloned-repository', $git_repo); p_chdir($srcdir); # Setup user configuration if (my $name = $module->getOption('git-user')) { my ($username, $email) = ($name =~ /^([^<]+) +<([^>]+)>$/); if (!$username || !$email) { croak_runtime("Invalid username or email for git-user option: $name". " (should be in format 'User Name '"); } whisper ("\tAdding git identity $name for new git module $module"); my $result = (safe_system(qw(git config --local user.name), $username) >> 8) == 0; $result = (safe_system(qw(git config --local user.email), $email) >> 8 == 0) || $result; if (!$result) { warning ("Unable to set user.name and user.email git config for y[b[$module]!"); } } return; } # Checks that the required source dir is either not already present or is empty. # Throws an exception if that's not true. sub _verifySafeToCloneIntoSourceDir { my ($module, $srcdir) = @_; if (-e "$srcdir" && !is_dir_empty($srcdir)) { if ($module->getOption('#delete-my-patches')) { warning ("\tRemoving conflicting source directory " . "as allowed by --delete-my-patches"); warning ("\tRemoving b[$srcdir]"); safe_rmtree($srcdir) or croak_internal("Unable to delete $srcdir!"); } else { error (<module(); my $srcdir = $module->fullpath('source'); if (-d "$srcdir/.git") { # Note that this function will throw an exception on failure. return $self->updateExistingClone(); } else { _verifySafeToCloneIntoSourceDir($module, $srcdir); my $git_repo = $module->getOption('repository'); if (!$git_repo) { croak_internal("Unable to checkout $module, you must specify a repository to use."); } if (!$self->_verifyRefPresent($module, $git_repo)) { croak_runtime( $self->_moduleIsNeeded() ? "$module build was requested, but it has no source code at the requested git branch" : "The required git branch does not exist at the source repository" ); } $self->_clone($git_repo); return 1 if pretending(); return count_command_output('git', '--git-dir', "$srcdir/.git", 'ls-files'); } return 0; } # Intended to be reimplemented sub _moduleIsNeeded { return 1; } +# +# Determine whether or not _setupRemote should manage the configuration of the git push URL for the repo. +# +# Return value: boolean indicating whether or not _setupRemote should assume control over the push URL. +# +sub isPushUrlManaged +{ + return 0; +} + +# +# Ensures the given remote is pre-configured for the module's git repository. +# The remote is either set up from scratch or its URLs are updated. +# +# Param $remote name (alias) of the remote to configure +# +# Throws an exception on error. +# +sub _setupRemote +{ + my $self = assert_isa(shift, 'ksb::Updater::Git'); + my $remote = shift; + + my $module = $self->module(); + my $repo = $module->getOption('repository'); + my $hasOldRemote = $self->hasRemote($remote); + + if ($hasOldRemote) { + whisper("\tUpdating the URL for git remote $remote of $module ($repo)"); + if (log_command($module, 'git-fix-remote', ['git', 'remote', 'set-url', $remote, $repo]) != 0) { + croak_runtime("Unable to update the URL for git remote $remote of $module ($repo)"); + } + } + elsif (log_command($module, 'git-add-remote', ['git', 'remote', 'add', $remote, $repo]) != 0) { + whisper("\tAdding new git remote $remote of $module ($repo)"); + croak_runtime("Unable to add new git remote $remote of $module ($repo)"); + } + + if ($self->isPushUrlManaged()) { + # + # pushInsteadOf does not work nicely with git remote set-url --push + # The result would be that the pushInsteadOf kde: prefix gets ignored. + # + # The next best thing is to remove any preconfigured pushurl and restore the kde: prefix mapping that way. + # This is effectively the same as updating the push URL directly because of the remote set-url executed + # previously by this function for the fetch URL. + # + chomp (my $existingPushUrl = qx"git config --get remote.$remote.pushurl"); + if ($existingPushUrl) { + info("\tRemoving preconfigured push URL for git remote $remote of $module: $existingPushUrl"); + + if (log_command($module, 'git-fix-remote', ['git', 'config', '--unset', "remote.$remote.pushurl"]) != 0) { + croak_runtime("Unable to remove preconfigured push URL for git remote $remote of $module: $existingPushUrl"); + } + } + } +} + # Selects a git remote for the user's selected repository (preferring a # defined remote if available, using 'origin' otherwise). # # Assumes the current directory is already set to the source directory. # # Throws an exception on error. # # Return value: Remote name that should be used for further updates. # # See also the 'repository' module option. sub _setupBestRemote { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $module = $self->module(); my $cur_repo = $module->getOption('repository'); # Search for an existing remote name first. If none, add our alias. - my @remoteNames = $self->bestRemoteName($cur_repo); - - if (!@remoteNames) { - # The desired repo doesn't have a named remote, this should be - # because the user switched it in the rc-file. We control the - # 'origin' remote to fix this. - if ($self->hasRemote(DEFAULT_GIT_REMOTE)) { - if (log_command($module, 'git-update-remote', - ['git', 'remote', 'set-url', DEFAULT_GIT_REMOTE, $cur_repo]) - != 0) - { - croak_runtime("Unable to update the fetch URL for existing remote alias for $module"); - } - } - elsif (log_command($module, 'git-remote-setup', - ['git', 'remote', 'add', DEFAULT_GIT_REMOTE, $cur_repo]) - != 0) - { - croak_runtime("Unable to add a git remote named " . DEFAULT_GIT_REMOTE . " for $cur_repo"); - } + my @remoteNames = $self->bestRemoteName(); + my $chosenRemote = @remoteNames ? $remoteNames[0] : DEFAULT_GIT_REMOTE; - push @remoteNames, DEFAULT_GIT_REMOTE; - } + $self->_setupRemote($chosenRemote); # Make a notice if the repository we're using has moved. my $old_repo = $module->getPersistentOption('git-cloned-repository'); if ($old_repo and ($cur_repo ne $old_repo)) { note (" y[b[*]\ty[$module]'s selected repository has changed"); note (" y[b[*]\tfrom y[$old_repo]"); note (" y[b[*]\tto b[$cur_repo]"); note (" y[b[*]\tThe git remote named b[", DEFAULT_GIT_REMOTE, "] has been updated"); # Update what we think is the current repository on-disk. #$ipc->notifyPersistentOptionChange($module->name(), 'git-cloned-repository', $cur_repo); } - return $remoteNames[0]; + return $chosenRemote; } # Completes the steps needed to update a git checkout to be checked-out to # a given remote-tracking branch. Any existing local branch with the given # branch set as upstream will be used if one exists, otherwise one will be # created. The given branch will be rebased into the local branch. # # No checkout is done, this should be performed first. # Assumes we're already in the needed source dir. # Assumes we're in a clean working directory (use git-stash to achieve # if necessary). # # First parameter is the remote to use. # Second parameter is the branch to update to. # Returns boolean success flag. # Exception may be thrown if unable to create a local branch. sub _updateToRemoteHead { my $self = shift; my ($remoteName, $branch) = @_; my $module = $self->module(); # The 'branch' option requests a given head in the user's selected # repository. Normally the remote head is mapped to a local branch, # which can have a different name. So, first we make sure the remote # head is actually available, and if it is we compare its SHA1 with # local branches to find a matching SHA1. Any local branches that are # found must also be remote-tracking. If this is all true we just # re-use that branch, otherwise we create our own remote-tracking # branch. my $branchName = $self->getRemoteBranchName($remoteName, $branch); if (!$branchName) { my $newName = $self->makeBranchname($remoteName, $branch); whisper ("\tUpdating g[$module] with new remote-tracking branch y[$newName]"); if (0 != log_command($module, 'git-checkout-branch', ['git', 'checkout', '-b', $newName, "$remoteName/$branch"])) { croak_runtime("Unable to perform a git checkout of $remoteName/$branch to a local branch of $newName"); } } else { whisper ("\tUpdating g[$module] using existing branch g[$branchName]"); if (0 != log_command($module, 'git-checkout-update', ['git', 'checkout', $branchName])) { croak_runtime("Unable to perform a git checkout to existing branch $branchName"); } # On the right branch, merge in changes. return 0 == log_command($module, 'git-rebase', ['git', 'rebase', "$remoteName/$branch"]); } return 1; } # Completes the steps needed to update a git checkout to be checked-out to # a given commit. The local checkout is left in a detached HEAD state, # even if there is a local branch which happens to be pointed to the # desired commit. Based the given commit is used directly, no rebase/merge # is performed. # # No checkout is done, this should be performed first. # Assumes we're already in the needed source dir. # Assumes we're in a clean working directory (use git-stash to achieve # if necessary). # # First parameter is the commit to update to. This can be in pretty # much any format that git itself will respect (e.g. tag, sha1, etc.). # It is recommended to use refs/$foo/$bar syntax for specificity. # Returns boolean success flag. sub _updateToDetachedHead { my ($self, $commit) = @_; my $module = $self->module(); info ("\tDetaching head to b[$commit]"); return 0 == log_command($module, 'git-checkout-commit', ['git', 'checkout', $commit]); } # Updates an already existing git checkout by running git pull. # # Throws an exception on error. # # Return parameter is the number of affected *commits*. sub updateExistingClone { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $module = $self->module(); my $cur_repo = $module->getOption('repository'); my $result; p_chdir($module->fullpath('source')); # Try to save the user if they are doing a merge or rebase if (-e '.git/MERGE_HEAD' || -e '.git/rebase-merge' || -e '.git/rebase-apply') { croak_runtime ("Aborting git update for $module, you appear to have a rebase or merge in progress!"); } my $remoteName = $self->_setupBestRemote(); # Download updated objects. This also updates remote heads so do this # before we start comparing branches and such. if (0 != log_command($module, 'git-fetch', ['git', 'fetch', '--tags', $remoteName])) { croak_runtime ("Unable to perform git fetch for $remoteName ($cur_repo)"); } # Now we need to figure out if we should update a branch, or simply # checkout a specific tag/SHA1/etc. my ($commitId, $commitType) = $self->_determinePreferredCheckoutSource($module); note ("Updating (to $commitType b[$commitId])") if ($commitType ne 'branch' || $commitId ne 'master'); my $start_commit = $self->commit_id('HEAD'); my $updateSub; if ($commitType eq 'branch') { $updateSub = sub { $self->_updateToRemoteHead($remoteName, $commitId) }; } else { $updateSub = sub { $self->_updateToDetachedHead($commitId); } } # With all remote branches fetched, and the checkout of our desired # branch completed, we can now use our update sub to complete the # changes. $self->stashAndUpdate($updateSub); return count_command_output('git', 'rev-list', "$start_commit..HEAD"); } # Goes through all the various combination of git checkout selection options in # various orders of priority. # # Returns a *list* containing: (the resultant symbolic ref/or SHA1,'branch' or # 'tag' (to determine if something like git-pull would be suitable or whether # you have a detached HEAD)). Since the sym-ref is returned first that should # be what you get in a scalar context, if that's all you want. sub _determinePreferredCheckoutSource { my ($self, $module) = @_; $module //= $self->module(); my @priorityOrderedSources = ( # option-name type getOption-inheritance-flag [qw(commit tag module)], [qw(revision tag module)], [qw(tag tag module)], [qw(branch branch module)], [qw(branch-group branch module)], [qw(use-stable-kde branch module)], # commit/rev/tag don't make sense for git as globals [qw(branch branch allow-inherit)], [qw(branch-group branch allow-inherit)], [qw(use-stable-kde branch allow-inherit)], ); # For modules that are not actually a 'proj' module we skip branch-group # and use-stable-kde entirely to allow for global/module branch selection # options to be selected... kind of complicated, but more DWIMy if (!$module->scm()->isa('ksb::Updater::KDEProject')) { @priorityOrderedSources = grep { $_->[0] ne 'branch-group' && $_->[0] ne 'use-stable-kde' } @priorityOrderedSources; } my $checkoutSource; # Sorry about the !!, easiest way to be clear that bool context is intended my $sourceTypeRef = first { !!($checkoutSource = ($module->getOption($_->[0], $_->[2]) // '')) } @priorityOrderedSources; if (!$sourceTypeRef) { return qw(master branch); } # One fixup is needed for use-stable-kde, to pull the actual branch name # from the right spot. Although if no branch name is set we use master, # without trying to search again. if ($sourceTypeRef->[0] eq 'use-stable-kde') { $checkoutSource = $module->getOption('#branch:stable', 'module') || 'master'; } # Likewise branch-group requires special handling. checkoutSource is # currently the branch-group to be resolved. if ($sourceTypeRef->[0] eq 'branch-group') { assert_isa($self, 'ksb::Updater::KDEProject'); $checkoutSource = $self->_resolveBranchGroup($checkoutSource); if (!$checkoutSource) { my $branchGroup = $module->getOption('branch-group'); whisper ("No specific branch set for $module and $branchGroup, using master!"); $checkoutSource = 'master'; } } if ($sourceTypeRef->[0] eq 'tag' && $checkoutSource !~ m{^refs/tags/}) { $checkoutSource = "refs/tags/$checkoutSource"; } return ($checkoutSource, $sourceTypeRef->[1]); } # Tries to check whether the git module is using submodules or not. Currently # we just check the .git/config file (using git-config) to determine whether # there are any 'active' submodules. # # MUST BE RUN FROM THE SOURCE DIR sub _hasSubmodules { # The git-config line shows all option names of the form submodule.foo.active, # filtering down to options for which the option is set to 'true' my @configLines = filter_program_output(undef, # accept all lines qw(git config --local --get-regexp ^submodule\..*\.active true)); return scalar @configLines > 0; } # Splits a URI up into its component parts. Taken from # http://search.cpan.org/~ether/URI-1.67/lib/URI.pm # Copyright Gisle Aas under the following terms: # "This program is free software; you can redistribute it and/or modify it # under the same terms as Perl itself." sub _splitUri { my($scheme, $authority, $path, $query, $fragment) = $_[0] =~ m|(?:([^:/?#]+):)?(?://([^/?#]*))?([^?#]*)(?:\?([^#]*))?(?:#(.*))?|; return ($scheme, $authority, $path, $query, $fragment); } # This stashes existing changes if necessary, and then runs a provided # update routine in order to advance the given module to the desired head. # Finally, if changes were stashed, they are applied and the stash stack is # popped. # # It is assumed that the required remote has been setup already, that we # are on the right branch, and that we are already in the correct # directory. # # First parameter is a reference to the subroutine to run. This subroutine # should need no parameters and return a boolean success indicator. It may # throw exceptions. # # Throws an exception on error. # # No return value. sub stashAndUpdate { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $updateSub = shift; my $module = $self->module(); my $date = strftime ("%F-%R", gmtime()); # ISO Date, hh:mm time - # To find out if we should stash, we use git-status in its short mode which + # To find out if we should stash, we use git-diff-index which # is intended to be scriptable and returns information on both the index # and the working dir in one command. my $status = 1; my $needsStash = - !pretending() && - (scalar filter_program_output( - undef, # don't filter output - qw(git status --short --untracked-files=no))) > 0; + !pretending() && (system('git', 'diff-index', '--quiet', 'HEAD') >> 8); + + log_command($module, 'git-status-before-update', [qw(git status)]); if ($needsStash) { info ("\tLocal changes detected (will stash for now and then restore)"); $status = log_command($module, 'git-stash-save', [ qw(git stash save --quiet), "kdesrc-build auto-stash at $date", ]); if ($status != 0) { + log_command($module, 'git-status-after-error', [qw(git status)]); croak_runtime("Unable to stash local changes for $module, aborting update."); } } if (!$updateSub->()) { error ("\tUnable to update the source code for r[b[$module]"); + log_command($module, 'git-status-after-error', [qw(git status)]); return; } # Update is performed and successful, re-apply the stashed changes if ($needsStash) { + info ("\tModule updated, reapplying your local changes."); + log_command($module, 'git-status-before-stash-pop', [qw(git status)]); $status = log_command($module, 'git-stash-pop', [ qw(git stash pop --index --quiet) ]); if ($status != 0) { error (<module(); + my $configuredUrl = $module->getOption('repository'); my @outputs; # The Repo URL isn't much good, let's find a remote name to use it with. # We'd have to escape the repo URL to pass it to Git, which I don't trust, # so we just look for all remotes and make sure the URL matches afterwards. eval { @outputs = slurp_git_config_output( qw/git config --null --get-regexp remote\..*\.url ./ ); }; if ($@) { error ("\tUnable to run git config, is there a setup error?"); return; } my @results; foreach my $output (@outputs) { # git config output between key/val is divided by newline. my ($remoteName, $url) = split(/\n/, $output); $remoteName =~ s/^remote\.//; - $remoteName =~ s/\.url$//; # Extract the cruft + $remoteName =~ s/\.url$//; # remove the cruft # Skip other remotes - next if $url ne $repoUrl; + next unless $self->_isPlausibleExistingRemote($remoteName, $url, $configuredUrl); # Try to avoid "weird" remote names. next if $remoteName !~ /^[\w-]*$/; # A winner is this one. push @results, $remoteName; } return @results; } # Generates a potential new branch name for the case where we have to setup # a new remote-tracking branch for a repository/branch. There are several # criteria that go into this: # * The local branch name will be equal to the remote branch name to match usual # Git convention. # * The name chosen must not already exist. This methods tests for that. # * The repo name chosen should be (ideally) a remote name that the user has # added. If not, we'll try to autogenerate a repo name (but not add a # remote!) based on the repository.git part of the URI. # # As with nearly all git support functions, we should be running in the # source directory of the git module. Don't call this function unless # you've already checked that a suitable remote-tracking branch doesn't # exist. # # First parameter: The name of a git remote to use. # Second parameter: The name of the remote head we need to make a branch name # of. # Returns: A useful branch name that doesn't already exist, or '' if no # name can be generated. sub makeBranchname { my $self = assert_isa(shift, 'ksb::Updater::Git'); my $remoteName = shift || 'origin'; my $branch = shift; my $module = $self->module(); my $chosenName; # Use "$branch" directly if not already used, otherwise try to prefix # with the remote name. for my $possibleBranch ($branch, "$remoteName-$branch", "ksdc-$remoteName-$branch") { my $result = system('git', 'show-ref', '--quiet', '--verify', '--', "refs/heads/$possibleBranch") >> 8; return $possibleBranch if $result == 1; } croak_runtime("Unable to find good branch name for $module branch name $branch"); } # Returns the number of lines in the output of the given command. The command # and all required arguments should be passed as a normal list, and the current # directory should already be set as appropriate. # # Return value is the number of lines of output. # Exceptions are raised if the command could not be run. sub count_command_output { # Don't call with $self->, all args are passed to filter_program_output my @args = @_; my $count = 0; filter_program_output(sub { $count++ if $_ }, @args); return $count; } # A simple wrapper that is used to split the output of 'git config --null' # correctly. All parameters are then passed to filter_program_output (so look # there for help on usage). sub slurp_git_config_output { # Don't call with $self->, all args are passed to filter_program_output local $/ = "\000"; # Split on null # This gets rid of the trailing nulls for single-line output. (chomp uses # $/ instead of hardcoding newline chomp(my @output = filter_program_output(undef, @_)); # No filter return @output; } # Returns true if the git module in the current directory has a remote of the # name given by the first parameter. sub hasRemote { my ($self, $remote) = @_; my $hasRemote = 0; eval { filter_program_output(sub { $hasRemote ||= ($_ && /^$remote/) }, 'git', 'remote'); }; return $hasRemote; } # Subroutine to add the 'kde:' alias to the user's git config if it's not # already set. # # Call this as a static class function, not as an object method # (i.e. ksb::Updater::Git::verifyGitConfig, not $foo->verifyGitConfig) # # Returns false on failure of any sort, true otherwise. sub verifyGitConfig { + my $contextOptions = shift; + my $useInvent = $contextOptions->getOption('x-invent-kde-push-urls'); + my $protocol = $contextOptions->getOption('git-desired-protocol') || 'git'; + my $pushUrlPrefix = 'git@git.kde.org:'; + + if ($useInvent) { + if ($protocol eq 'git' || $protocol eq 'https') { + $pushUrlPrefix = $protocol eq 'git' ? 'git@invent.kde.org:' : 'https://invent.kde.org/'; + } + else { + error(" b[y[*] Invalid b[git-desired-protocol] $protocol"); + error(" b[y[*] Try setting this option to 'git' if you're not using a proxy"); + croak_runtime("Invalid git-desired-protocol: $protocol"); + } + } + my $configOutput = qx'git config --global --get url.https://anongit.kde.org/.insteadOf kde:'; # 0 means no error, 1 means no such section exists -- which is OK if ((my $errNum = $? >> 8) >= 2) { my $error = "Code $errNum"; my %errors = ( 3 => 'Invalid config file (~/.gitconfig)', 4 => 'Could not write to ~/.gitconfig', 2 => 'No section was provided to git-config', 1 => 'Invalid section or key', 5 => 'Tried to set option that had no (or multiple) values', 6 => 'Invalid regexp with git-config', 128 => 'HOME environment variable is not set (?)', ); $error = $errors{$errNum} if exists $errors{$errNum}; error (" r[*] Unable to run b[git] command:\n\t$error"); return 0; } # If we make it here, I'm just going to assume git works from here on out # on this simple task. if ($configOutput !~ /^kde:\s*$/) { whisper ("\tAdding git download kde: alias"); my $result = safe_system( qw(git config --global --add url.https://anongit.kde.org/.insteadOf kde:) ) >> 8; return 0 if $result != 0; } $configOutput = - qx'git config --global --get url.git@git.kde.org:.pushInsteadOf kde:'; + qx"git config --global --get url.$pushUrlPrefix.pushInsteadOf kde:"; if ($configOutput !~ /^kde:\s*$/) { whisper ("\tAdding git upload kde: alias"); - my $result = safe_system( - qw(git config --global --add url.git@git.kde.org:.pushInsteadOf kde:) - ) >> 8; + my $result = safe_system("git config --global --add url.$pushUrlPrefix.pushInsteadOf kde:") >> 8; return 0 if $result != 0; } # Remove old kdesrc-build installed aliases (kde: -> git://anongit.kde.org/) $configOutput = qx'git config --global --get url.git://anongit.kde.org/.insteadOf kde:'; if ($configOutput =~ /^kde:\s*$/) { whisper ("\tRemoving outdated kde: alias"); my $result = safe_system( qw(git config --global --unset-all url.git://anongit.kde.org/.insteadOf kde:) ) >> 8; return 0 if $result != 0; } + if ($useInvent) { + $configOutput = + qx'git config --global --get url.git@git.kde.org:.pushInsteadOf kde:'; + + if ($configOutput =~ /^kde:\s*$/) { + whisper ("\tRemoving outdated kde: alias"); + my $result = safe_system( + qw(git config --global --unset-all url.git@git.kde.org:.pushInsteadOf kde:) + ) >> 8; + return 0 if $result != 0; + } + } + return 1; } 1; diff --git a/modules/ksb/Updater/KDEProject.pm b/modules/ksb/Updater/KDEProject.pm index c2ccba2..f298054 100644 --- a/modules/ksb/Updater/KDEProject.pm +++ b/modules/ksb/Updater/KDEProject.pm @@ -1,53 +1,66 @@ package ksb::Updater::KDEProject 0.20; # An update class for KDE Project modules (i.e. those that use "repository # kde-projects" in the configuration file). use strict; use warnings; use 5.014; use parent qw(ksb::Updater::Git); use ksb::Debug; sub name { return 'proj'; } # Resolves the requested branch-group for this Updater's module. # Returns the required branch name, or undef if none is set. sub _resolveBranchGroup { my ($self, $branchGroup) = @_; my $module = $self->module(); # If we're using a logical group we need to query the global build context # to resolve it. my $ctx = $module->buildContext(); my $resolver = $ctx->moduleBranchGroupResolver(); my $modulePath = $module->fullProjectPath(); return $resolver->findModuleBranch($modulePath, $branchGroup); } # Reimplementation sub _moduleIsNeeded { my $self = shift; my $module = $self->module(); # selected-by looks at cmdline options, found-by looks at how we read # module info from rc-file in first place to select it from cmdline. # Basically if user asks for it on cmdline directly or in rc-file directly # then we need to try to grab it... if (($module->getOption('#selected-by', 'module') // '') ne 'name' && ($module->getOption('#found-by', 'module') // '') eq 'wildcard') { return 0; } return 1; } +# Reimplementation +sub _isPlausibleExistingRemote +{ + my ($self, $name, $url, $configuredUrl)= @_; + return $url eq $configuredUrl || $url =~ /^kde:/; +} + +# Reimplementation +sub isPushUrlManaged +{ + return 1; +} + 1; diff --git a/modules/ksb/Util.pm b/modules/ksb/Util.pm index 78ce15f..29a470b 100644 --- a/modules/ksb/Util.pm +++ b/modules/ksb/Util.pm @@ -1,679 +1,676 @@ package ksb::Util 0.30; # Useful utilities, which are exported into the calling module's namespace by default. use 5.014; # Needed for state keyword use strict; use warnings; use Scalar::Util qw(blessed); use File::Path qw(make_path); use File::Find; use Cwd qw(getcwd); use Errno qw(:POSIX); use Digest::MD5; use ksb::Debug; use ksb::Version qw(scriptVersion); use ksb::BuildException; use Exporter qw(import); # Use Exporter's import method our @EXPORT = qw(list_has assert_isa assert_in any unique_items absPathToExecutable fileDigestMD5 log_command disable_locale_message_translation trimmed split_quoted_on_whitespace safe_unlink safe_system p_chdir pretend_open safe_rmtree get_list_digest is_dir_empty super_mkdir filter_program_output prettify_seconds); # Function to work around a Perl language limitation. # First parameter is a reference to the list to search. ALWAYS. # Second parameter is the value to search for. # Returns true if the value is in the list sub list_has { my ($listRef, $value) = @_; my @list = @{$listRef}; return scalar grep { "$_" eq "$value" } (@list); } # Subroutine to return the path to the given executable based on the # either the given paths or the current PATH. # E.g.: # absPathToExecutable('make') -> '/usr/bin/make' # absPathToExecutable('make', 'foo', 'bar') -> /foo/make # If the executable is not found undef is returned. # # This assumes that the module environment has already been updated since # binpath doesn't exactly correspond to $ENV{'PATH'}. sub absPathToExecutable { my ($prog, @preferred) = @_; # If it starts with a / the path is already absolute. return $prog if $prog =~ /^\//; my @paths = @preferred ? @preferred : split(/:/, $ENV{'PATH'}); for my $path (@paths) { return "$path/$prog" if (-x "$path/$prog"); } return undef; } # Throws an exception if the first parameter is not an object at all, or if # it is not an object of the type given by the second parameter (which # should be a string of the class name. There is no return value; sub assert_isa { my ($obj, $class) = @_; if (!blessed($obj) || !$obj->isa($class)) { croak_internal("$obj is not of type $class, but of type " . ref($obj)); } return $obj; } # Throws an exception if the first parameter is not included in the # provided list of possible alternatives. The list of alternatives must # be passed as a reference, as the second parameter. sub assert_in { my ($val, $listRef) = @_; if (!list_has($listRef, $val)) { croak_runtime("$val is not a permissible value for its argument"); } return $val; } # Subroutine to unlink the given symlink if global-pretend isn't set. sub safe_unlink { if (pretending()) { pretend ("\tWould have unlinked ", shift, "."); return 1; # Return true } return unlink (shift); } # Subroutine to execute the system call on the given list if the pretend # global option is not set. # # Returns the shell error code, so 0 means success, non-zero means failure. sub safe_system(@) { if (!pretending()) { whisper ("\tExecuting g['", join("' '", @_), "'"); return system (@_) >> 8; } pretend ("\tWould have run g['" . join("' '", @_) . "'"); return 0; # Return true } # Is exactly like "chdir", but it will also print out a message saying that # we're switching to the directory when debugging. sub p_chdir($) { my $dir = shift; debug ("\tcd g[$dir]\n"); chdir ($dir) or do { return 1 if pretending(); croak_runtime("Could not change to directory $dir: $!"); }; } # Helper subroutine to create a directory, including any parent # directories that may also need created. # Throws an exception on failure. See File::Path. sub super_mkdir { my $pathname = shift; state %createdPaths; if (pretending()) { if (!exists $createdPaths{$pathname} && ! -e $pathname) { pretend ("\tWould have created g[$pathname]"); } $createdPaths{$pathname} = 1; return 1; } else { make_path($pathname); return (-e $pathname) ? 1 : 0; } } # Calculates the MD5 digest of a file already on-disk. The digest is # returned as a hex string digest as from Digest::MD5::md5_hex # # First parameter: File name to read # Return value: hex string MD5 digest of file. # An exception is thrown if an error occurs reading the file. sub fileDigestMD5 { my $fileName = shift; my $md5 = Digest::MD5->new; open my $file, '<', $fileName or croak_runtime( "Unable to open $fileName: $!"); binmode($file); $md5->addfile($file); return $md5->hexdigest(); } # This function is intended to disable the message translation catalog # settings in the program environment, so that any child processes executed # will have their output untranslated (and therefore scrapeable). # # As such this should only be called for a forked child about to exec as # there is no easy way to undo this within the process. sub disable_locale_message_translation { # Ensure that program output is untranslated by setting 'C' locale. # We're really trying to affect the LC_MESSAGES locale category, but # LC_ALL is a catch-all for that (so needs to be unset if set). # # Note that the ONLY SUPPORTED way to pass file names, command-line # args, etc. to commands is under the UTF-8 encoding at this point, as # that is the only sane way for this en_US-based developer to handle # the task. Patches (likely using Encode::Locale) are accepted. :P $ENV{'LC_MESSAGES'} = 'C'; if ($ENV{'LC_ALL'}) { $ENV{'LANG'} = $ENV{'LC_ALL'}; # This is lower-priority "catch all" delete $ENV{'LC_ALL'}; } } # Returns an array of lines output from a program. Use this only if you # expect that the output will be short. # # Since there is no way to disambiguate no output from an error, this # function will call die on error, wrap in eval if this bugs you. # # First parameter is subroutine reference to use as a filter (this sub will # be passed a line at a time and should return true if the line should be # returned). If no filtering is desired pass 'undef'. # # Second parameter is the program to run (either full path or something # accessible in $PATH). # # All remaining arguments are passed to the program. # # Return value is an array of lines that were accepted by the filter. sub filter_program_output { my ($filterRef, $program, @args) = @_; $filterRef //= sub { return 1 }; # Default to all lines debug ("Slurping '$program' '", join("' '", @args), "'"); # Check early for whether an executable exists since otherwise # it is possible for our fork-open below to "succeed" (i.e. fork() # happens OK) and then fail when it gets to the exec(2) syscall. if (!absPathToExecutable($program)) { croak_runtime("Can't find $program in PATH!"); } my $execFailedError = "\t - kdesrc-build - exec failed!\n"; my $pid = open(my $childOutput, '-|'); croak_internal("Can't fork: $!") if ! defined($pid); if ($pid) { # parent my @lines = grep { &$filterRef; } (<$childOutput>); close $childOutput or do { # $! indicates a rather grievous error croak_internal("Unable to open pipe to read $program output: $!") if $!; # we can pass serious errors back to ourselves too. my $exitCode = $? >> 8; if ($exitCode == 99 && @lines >= 1 && $lines[0] eq $execFailedError) { croak_runtime("Failed to exec $program, is it installed?"); } # other errors might still be serious but don't need a backtrace if (pretending()) { whisper ("$program gave error exit code $exitCode"); } else { warning ("$program gave error exit code $exitCode"); } }; return @lines; } else { disable_locale_message_translation(); # We don't want stderr output on tty. open (STDERR, '>', '/dev/null') or close (STDERR); exec { $program } ($program, @args) or do { # Send a message back to parent print $execFailedError; exit 99; # Helper proc, so don't use finish(), just die }; } } # Subroutine to return a string suitable for displaying an elapsed time, # (like a stopwatch) would, in a fixed format HH:MM:SS. The first parameter is # the number of seconds elapsed. sub prettify_seconds { my $elapsed = shift; my ($hours, $minutes, $seconds); return "(00:00:00)" if $elapsed <= 0; $seconds = int $elapsed % 60; $elapsed = int $elapsed / 60; $minutes = $elapsed % 60; $hours = int $elapsed / 60; return sprintf("(%02d:%02d:%02d)", $hours, $minutes, $seconds); } # Subroutine to mark a file as being the error log for a module. This also # creates a symlink in the module log directory for easy viewing. # First parameter is the module in question. # Second parameter is the filename in the log directory of the error log. sub _setErrorLogfile { my $module = assert_isa(shift, 'ksb::Module'); my $logfile = shift; return unless $logfile; my $logdir = $module->getLogDir(); $module->setOption('#error-log-file', "$logdir/$logfile"); debug ("Logfile for $module is $logfile"); # Setup symlink in the module log directory pointing to the appropriate # file. Make sure to remove it first if it already exists. unlink("$logdir/error.log") if -l "$logdir/error.log"; if(-e "$logdir/error.log") { # Maybe it was a regular file? error ("r[b[ * Unable to create symlink to error log file]"); return; } symlink "$logfile", "$logdir/error.log"; } # Subroutine to run a command, optionally filtering on the output of the child # command. # # First parameter is the module object being built (for logging purposes # and such). # Second parameter is the name of the log file to use (relative to the log # directory). # Third parameter is a reference to an array with the command and its # arguments. i.e. ['command', 'arg1', 'arg2'] # # After the required three parameters you can pass a hash reference of # optional features: # 'callback' => a reference to a subroutine to have each line # of child output passed to. This output is not supposed to be printed # to the screen by the subroutine, normally the output is only logged. # However this is useful for e.g. munging out the progress of the build. # USEFUL: When there is no more output from the child, the callback will be # called with an undef string. (Not just empty, it is also undefined). # # 'no_translate' => any true value will cause a flag to be set to request # the executed child process to not translate (for locale purposes) its # output, so that it can be screen-scraped. # # The return value is the shell return code, so 0 is success, and non-zero is # failure. # # NOTE: This function has a special feature. If the command passed into the # argument reference is 'kdesrc-build', then log_command will, when it # forks, execute the subroutine named by the second parameter rather than # executing a child process. The subroutine should include the full package # name as well (otherwise the package containing log_command's implementation # is used). The remaining arguments in the list are passed to the # subroutine that is called. sub log_command { my ($module, $filename, $argRef, $optionsRef) = @_; assert_isa($module, 'ksb::Module'); my @command = @{$argRef}; $optionsRef //= { }; my $callbackRef = $optionsRef->{'callback'}; debug ("log_command(): Module $module, Command: ", join(' ', @command)); ksb_debug_inspect('log_command', "$module", $filename, $argRef, $optionsRef); if (pretending()) { pretend ("\tWould have run g['" . join ("' '", @command) . "'"); return 0; } # Do this before we fork so we can see errors my $logpath = $module->getLogPath("$filename.log"); # Fork a child, with its stdout connected to CHILD. my $pid = open(CHILD, '-|'); if ($pid) { # Parent if (!$callbackRef && debugging()) { # If no other callback given, pass to debug() if debug-mode is on. while () { print ($_) if $_; } } if ($callbackRef) { &{$callbackRef}($_) while (); # Let callback know there is no more output. &{$callbackRef}(undef); } # This implicitly does a waitpid() as well close CHILD or do { if ($! == 0) { _setErrorLogfile($module, "$filename.log"); return $?; } return 1; }; return 0; } else { # Child. Note here that we need to avoid running our exit cleanup # handlers in here. For that we need POSIX::_exit. # Apply altered environment variables. $module->buildContext()->commitEnvironmentChanges(); $SIG{PIPE} = "IGNORE"; $SIG{INT} = sub { close (STDOUT); # This should be a pipe close (STDERR); POSIX::_exit(EINTR); }; # Redirect STDIN to /dev/null so that the handle is open but fails when # being read from (to avoid waiting forever for e.g. a password prompt # that the user can't see. open (STDIN, '<', "/dev/null") unless exists $ENV{'KDESRC_BUILD_USE_TTY'}; if ($callbackRef || debugging()) { open (STDOUT, "|tee $logpath") or do { error ("Error opening pipe to tee command."); # Don't abort, hopefully STDOUT still works. }; } else { open (STDOUT, '>', $logpath) or do { error ("Error $! opening log to $logpath!"); }; } # Make sure we log everything. open (STDERR, ">&STDOUT"); # Call internal function, name given by $command[1] if ($command[0] eq 'kdesrc-build') { # No colors! ksb::Debug::setColorfulOutput(0); debug ("Calling $command[1]"); my $cmd = $command[1]; splice (@command, 0, 2); # Remove first two elements. no strict 'refs'; # Disable restriction on symbolic subroutines. if (! &{$cmd}(@command)) # Call sub { POSIX::_exit (EINVAL); } POSIX::_exit (0); # Exit child process successfully. } # Don't leave empty output files, give an indication of the particular # command run. Use print to go to stdout. say "# kdesrc-build running: '", join("' '", @command), "'"; say "# from directory: ", getcwd(); # If a callback is set assume no translation can be permitted. disable_locale_message_translation() if $optionsRef->{'no_translate'}; # External command. exec (@command) or do { my $cmd_string = join(' ', @command); error (<($_) && return 1) foreach @{$listRef}; return 0; } # Returns unique items of the list. Order not guaranteed. sub unique_items { # See perlfaq4 my %seen; my @results = grep { ! $seen{$_}++; } @_; return @results; } # Subroutine to delete a directory and all files and subdirectories within. # Does nothing in pretend mode. An analog to "rm -rf" from Linux. # Requires File::Find module. # # First parameter: Path to delete # Returns boolean true on success, boolean false for failure. sub safe_rmtree { my $path = shift; # Pretty user-visible path my $user_path = $path; $user_path =~ s/^$ENV{HOME}/~/; my $delete_file_or_dir = sub { # $_ is the filename/dirname. return if $_ eq '.' or $_ eq '..'; if (-f $_ || -l $_) { unlink ($_) or croak_runtime("Unable to delete $File::Find::name: $!"); } elsif (-d $_) { rmdir ($File::Find::name) or croak_runtime("Unable to remove directory $File::Find::name: $!"); } }; if (pretending()) { pretend ("Would have removed all files/folders in $user_path"); return 1; } # Error out because we probably have a logic error even though it would # delete just fine. if (not -d $path) { error ("Cannot recursively remove $user_path, as it is not a directory."); return 0; } eval { $@ = ''; finddepth( # finddepth does a postorder traversal. { wanted => $delete_file_or_dir, no_chdir => 1, # We'll end up deleting directories, so prevent this. }, $path); }; if ($@) { error ("Unable to remove directory $user_path: $@"); return 0; } return 1; } # Returns a hash digest of the given options in the list. The return value is # base64-encoded at this time. # # Note: Don't be dumb and pass data that depends on execution state as the # returned hash is almost certainly not useful for whatever you're doing with # it. (i.e. passing a reference to a list is not helpful, pass the list itself) # # Parameters: List of scalar values to hash. # Return value: base64-encoded hash value. sub get_list_digest { use Digest::MD5 "md5_base64"; # Included standard with Perl 5.8 return md5_base64(@_); } # Utility function to see if a directory path is empty or not sub is_dir_empty { my $dir = shift; opendir my $dirh, $dir or return; # while-readdir needs Perl 5.12 while (readdir $dirh) { next if ($_ eq '.' || $_ eq '..'); closedir ($dirh); return; # not empty } closedir ($dirh); return 1; } # Takes in a string and returns 1 if that string exists somewhere in the # path variable. sub isInPath { if (index($ENV{'PATH'}, $_[0]) != -1) { return 1; } else { return 0; } } -# Takes in a string and returns 1 if that string exists as a line in the -# ~/.bashrc file. +# Takes in a string and returns 1 if that string exists as a line in the given file. sub fileHasLine -{ - my $found = 0; - open(my $bashrc, '<', $_[0]) or _croak_runtime("Couldn't open ~/.bashrc: $!"); - - while (my $line = <$bashrc>) { - if (index($line, $_[1]) == 0){ - return 1; - } +{ + my ($file, $tag) = @_; + open(my $fh, '<', $file) or croak_runtime("Couldn't open $file: $!"); + + while (my $line = <$fh>) { + chomp $line; + return 1 if $line eq $tag; } - + return 0; } 1; - diff --git a/org.kde.kdesrc-build.metainfo.xml b/org.kde.kdesrc-build.metainfo.xml index 98391d0..d780063 100644 --- a/org.kde.kdesrc-build.metainfo.xml +++ b/org.kde.kdesrc-build.metainfo.xml @@ -1,289 +1,356 @@ org.kde.kdesrc_build FSFAP GPL-2.0-or-later kdesrc-build kdesrc-build + kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build + kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build kdesrc-build + kdesrc-build kdesrc-build kdesrc-build xxkdesrc-buildxx Tool to allow you to easily build KDE software from its source repositories Eina que permet construir amb facilitat el programari KDE des dels seus repositoris d'origen + Eina que permet construir amb facilitat el programari KDE des dels seus repositoris d'origen Εργαλείο που σας επιτρέπει να κατασκευάζετε εύκολα λογισμικό του KDE από τα αποθετήρια πηγαίου κώδικα Tool to allow you to easily build KDE software from its source repositories Herramienta para permitirle compilar software de KDE de forma fácil desde sus repositorios de código fuente Tööriist KDE tarkvara hõlpsaks ehitamiseks lähtekoodihoidlast Työkalu KDE-ohjelmien koostamiseksi lähdekoodilähteistä helposti Outil permettant de compiler facilement les logiciels KDE depuis leurs dépôts de code source Strumento che ti permette di compilare facilmente il software di KDE dai suoi depositi di sorgenti + 소스 저장소에서 KDE 소프트웨어를 쉽게 빌드할 수 있는 도구 Hulpmiddel om gemakkelijk KDE software vanuit de opslagruimten voor broncode te compileren. Narzędzie umożliwiające łatwą budowę oprogramowania KDE z repozytoriów kodu Ferramenta que lhe permite compilar aplicações do KDE facilmente a partir dos seus repositórios de código + Ferramenta que permite compilar facilmente software do KDE de seus repositórios fonte + Orodje, ki vam omogoča enostavno gradnjo programske opreme KDE iz svojih izvornih skladišč Verktyg för att enkelt låta dig bygga KDE-programvara från källkodsarkiven Інструмент, за допомогою якого ви зможете без проблем збирати програмне забезпечення KDE з коду, який зберігається у сховищах коду програм xxTool to allow you to easily build KDE software from its source repositoriesxx

kdesrc-build is a tool to allow you to easily build KDE from its source repositories.

El kdesrc-build és una eina que permet construir amb facilitat el KDE des dels seus repositoris d'origen.

+

El kdesrc-build és una eina que permet construir amb facilitat el KDE des dels seus repositoris d'origen.

Το kdesrc-build είναιένα εργαλείο που σας επιτρέπει να κατασκευάζετε εύκολα το KDE από τα αποθετήρια πηγαίου κώδικα.

kdesrc-build is a tool to allow you to easily build KDE from its source repositories.

kdesrc-build es una herramienta que le permite compilar KDE de forma fácil desde sus repositorios de código fuente.

kdesrc-build kujutab endast tööriista, mis võimaldab hõlpsalt ehitada KDE tarkvara otse lähtekoodihoidlast.

kdesrc-build on työkalu, jolla voi helposti koostaa KDE:n lähdekoodilähteistään.

kdesrc-build est un outil qui permet de compiler facilement KDE depuis ses dépôts de code source.

kdesrc-build è uno strumento che ti permette di compilare facilmente il software di KDE dai suoi depositi di sorgenti.

+

kdesrc-build는 소스 저장소에서 KDE 소프트웨어를 쉽게 빌드할 수 있는 도구입니다.

kdesrc-build is een hulpmiddel om gemakkelijk KDE te bouwen uit zijn opslagruimten voor broncode.

kdesrc-build jest narzędziem umożliwiającym łatwe budowanie KDE z jego repozytoriów kodu.

O kdesrc-build é uma ferramenta que lhe permite compilar facilmente o KDE a partir dos seus repositórios de código.

+

O kdesrc-build é uma ferramenta que permite compilar facilmente o KDE de seus repositórios fonte.

+

kdesrc-build je orodje, s katerim lahko preprosto sestavite KDE iz svojih izvornih skladišč.

kdesrc-build är ett verktyg för att enkelt låta dig bygga KDE-programvara från källkodsarkiven.

kdesrc-build — інструмент, за допомогою якого ви зможете без проблем збирати програмне забезпечення KDE з коду, який зберігається у сховищах коду програм.

xxkdesrc-build is a tool to allow you to easily build KDE from its source repositories.xx

Features:

Característiques:

+

Característiques:

Vlastnosti:

Χαρακτηριστικά:

Features:

Funcionalidades:

Omadused:

Ominaisuuksia:

Fonctionnalités :

+

Characteristicas:

Funzionalità:

+

기능:

Kenmerken:

Możliwości:

Funcionalidades:

Funcionalidades:

Funkcie:

+

Možnosti:

Funktioner:

Можливості:

xxFeatures:xx

  • Will automatically checkout and update the KDE source code from its source repository. Both Subversion and git repository types are supported.
  • Automàticament comprovarà i actualitzarà el codi font del KDE des del seu repositori d'origen. Admet tant els tipus de repositori de Subversion com els Git.
    • +
  • Automàticament comprovarà i actualitzarà el codi font del KDE des del seu repositori d'origen. Admet tant els tipus de repositori de Subversion com els Git.
  • Θα φέρει αυτόματα και θα ενημερώσει τον πηγαίο κώδικα του KDE από το αποθετήριο. Και οι δύο τύποι αποθετηρίου, svn και git υποστηρίζονται.
  • Will automatically checkout and update the KDE source code from its source repository. Both Subversion and git repository types are supported.
  • Descarga y actualiza automáticamente el código fuente de KDE desde su repositorio de código fuente. Permite usar repositorios de Subversion y de Git.
  • KDE lähtekoodi automaatne väljavõtmine ja uuendamine lähtekoodihoidla põhjal. Toetatud on nii Subversioni kui ka Giti hoidlad.
  • Tuo ja päivittää automaattisesti KDE-lähdekoodin lähdekoodilähteestään. Sekä Subversion- että Git-lähteitä tuetaan.
  • Réalisera l'extraction et la mise à jour automatique du code source KDE depuis le dépôt des sources. Les types de dépôts Subversion et git sont tous les deux pris en charge.
  • Controllo ed aggiornamento del codice sorgente di KDE dai suoi depositi di sorgenti. Sono supportati sia i tipi di deposito Subversion che git.
    • +
  • KDE 소스 코드를 소스 저장소에서 자동으로 체크아웃 및 업데이트합니다. SVN과 Git 저장소를 지원합니다.
  • Zal automatisch de checkout en update van KDE broncode laten doen uit zijn opslagruimten voor broncode. Zowel opslagruimten met het type subversion als git worden ondersteund.
  • Samoczynnie pobierze i uaktualni kod ze źródła kodu KDE. Obsługiwane są zarówno repozytoria git jak i Subversion.
  • Irá obter e actualizar automaticamente o código-fonte do KDE a partir do seu repositório de código. São suportados ambos os tipos de repositório Git e Subversion.
    • +
  • Irá automaticamente baixar e atualizar o código-fonte do KDE de seus repositórios fonte. Ambos os tipos Subversion e git são suportados.
    • +
  • Samodejno bo prevzel in posodobil izvirno kodo KDE iz svojega izvornega skladišča. Podprti sta obe skladišči subversion in git.
  • Checkar automatiskt ut och uppdaterar KDE-källkod från källkodsarkivet. Båda arkivtyperna Subversion och git stöds.
  • Автоматичне отримання та оновлення початкових кодів KDE зі сховищ початкового коду. Передбачено підтримку сховищ типів Subversion та git.
  • xxWill automatically checkout and update the KDE source code from its source repository. Both Subversion and git repository types are supported.xx
  • Integrates with the KDE source code infrastructure to automatically build modules in the required order, and using the appropriate branch.
  • S'integra amb la infraestructura de codi font del KDE per a construir automàticament els mòduls en l'ordre requerit, i usant la branca adequada.
    • +
  • S'integra amb la infraestructura de codi font del KDE per a construir automàticament els mòduls en l'ordre requerit, i usant la branca adequada.
  • Ενσωματώνεται με την υποδομή του πηγαίου κώδικα του KDE για την αυτόματη κατασκευή αρθρωμάτων στην απαιτούμενη σειρά και με χρήση του κατάλληλου κλάδου.
  • Integrates with the KDE source code infrastructure to automatically build modules in the required order, and using the appropriate branch.
  • Se integra con la infraestructura de código fuente de KDE para compilar automáticamente sus módulos en el orden necesario, usando la rama correcta.
  • Lõimimine KDE lähtekoodi infrastruktuuriga moodulite automaatseks ehitamiseks nõutavas järjekorras sobivat haru kasutades.
  • Integroituu KDE-lähdekoodin infrastruktuuriin koostaakseen moduulit automaattisesti vaaditussa järjestyksessä ja käyttääkseen oikeita haaroja.
  • S'intègre à l'infrastructure de code source de KDE afin de compiler automatiquement les modules dans l'ordre nécessaire, et d'utiliser la bonne branche.
  • Si integra con l'infrastruttura del codice sorgente di KDE compilando automaticamente i moduli nell'ordine richiesto, inoltre usa il ramo appropriato.
    • +
  • KDE 소스 코드 기반과 통합하여 자동으로 필요한 순서와 브랜치에 따라서 모듈을 빌드합니다.
  • Integreert met de infrastructuur van de broncode van KDE om automatisch modules in de vereiste volgorde te bouwen en de van toepassing zijnde branch te gebruiken.
  • Integruje się z infrastrukturą kodu KDE, aby samoczynnie budować moduły w wymaganej kolejności oraz przy użyciu odpowiedniej gałęzi.
  • Integra com a infra-estrutura de código do KDE para compilar automaticamente os módulos pela ordem correcta e usando as ramificações apropriadas.
    • +
  • Integra com a infraestrutura de código-fonte do KDE para compilar automaticamente módulos na ordem necessária e usando o ramo apropriado.
    • +
  • Integrira se z infrastrukturo izvorne kode KDE, da samodejno sestavi module v zahtevanem vrstnem redu in z uporabo ustrezne veje.
  • Integrerar med KDE:s källkodsinfrastruktur för att automatiskt bygga moduler i den ordning som krävs, och med användning av lämplig gren.
  • Інтеграція із інфраструктурою початкових кодів KDE для автоматичного збирання модулів у потрібному порядку із використанням відповідної гілки.
  • xxIntegrates with the KDE source code infrastructure to automatically build modules in the required order, and using the appropriate branch.xx
  • Easily supports building specific branches, tags, or even revisions of a module (or all of KDE software).
  • Admet amb facilitat la creació de branques, etiquetes o fins i tot revisions específiques d'un mòdul (o de tot el programari KDE).
    • +
  • Admet amb facilitat la creació de branques, etiquetes o fins i tot revisions específiques d'un mòdul (o de tot el programari KDE).
  • Υποστηρίζει εύκολα την κατασκευή συγκεκριμένων κλάδων, ετικετών ή και αναθεωρήσεων ενός αρθρώματος (ή όλου του KDE λογισμικού).
  • Easily supports building specific branches, tags, or even revisions of a module (or all of KDE software).
  • Permite compilar de forma sencilla las ramas, las etiquetas, e incluso las revisiones de un módulo que indique (o todo el software de KDE).
  • Konkreetsete moodulite (või lausa kogu KDE tarkvara) ehitamise toetus harude, siltide või lausa versioonide järgi.
  • Tukee moduulien määrähaaran, -tunnisteen tai jopa -aliversion helppoa koostamista (tai kaiken KDE-ohjelmiston).
  • Prend en charge facilement la compilation de branches, d'étiquettes ou même de versions spécifiques d'un module (ou de tous les logiciels KDE).
  • Supporta facilmente la compilazione di rami specifici, di tag o perfino le revisioni di un modulo (o di tutto il software di KDE).
    • +
  • 특정한 브랜치, 태그, 모듈이나 전체 KDE 소프트웨어의 리비전을 쉽게 빌드할 수 있습니다.
  • Ondersteunt gemakkelijk het bouwen van specifieke branches, tags of zelfs revisies van een module (of alles van de KDE software).
  • Łatwa obsługa budowania danych gałęzi, znaczników, a nawet rewizji modułów (lub całego oprogramowania KDE).
  • Suporta facilmente a compilação de ramificações, marcas ou mesmo versões específicas de um dado módulo (ou de todas as aplicações do KDE).
    • +
  • Suporta facilmente a compilação de ramos específicos, tags ou até mesmo revisões de um módulo (ou de todos os softwares do KDE)
    • +
  • Preprosto podpira gradnjo določenih vej, oznak ali celo revizij modula (ali vse programske opreme KDE).
  • Stöder att enkelt bygga särskilda grenar, etiketter, eller till och med versioner av en modul (eller hela KDE:s programvara).
  • Проста підтримка збирання певних гілок, міток або навіть модифікацій модуля (або усього програмного забезпечення KDE).
  • xxEasily supports building specific branches, tags, or even revisions of a module (or all of KDE software).xx
  • Supports many build systems. It's even possible to build many non-KDE software projects in a pinch (for instance, CMake can be built from its own git sources if your distribution does not have CMake packages).
  • Admet molts sistemes de construcció. En cas de necessitat, fins i tot és possible construir molts projectes de programari que no són del KDE (per exemple, el CMake es pot construir des del codi font de Git si la vostra distribució no té paquets per a CMake).
    • +
  • Admet molts sistemes de construcció. En cas de necessitat, fins i tot és possible construir molts projectes de programari que no són del KDE (per exemple, el CMake es pot construir des del codi font de Git si la vostra distribució no té paquets per a CMake).
  • Υποστηρίζει πολλά συστήματα κατασκευής. Είναι πιθανό ακόμη και να κατασκευαστούν πολλά έργα λογισμικού εκτός KDE αν είναι απαραίτητο (για παράδειγμα, το CMake μπορεί να κατασκευαστεί από το δικό του πηγαίο κώδικα στο git αν η διανομή σας δεν έχει πακέτα CMake).
  • Supports many build systems. It's even possible to build many non-KDE software projects in a pinch (for instance, CMake can be built from its own git sources if your distribution does not have CMake packages).
  • Permite usar diversos sistemas de compilación. Incluso puede compilar numerosos proyectos de software que no es de KDE si no hay más remedio (por ejemplo, puede compilar CMake desde su propia fuente Git si su distribución no posee paquetes de CMake).
  • Paljude ehitussüsteemide toetus. Võimalik on isegi ehitada KDE-väliseid tarkvaraprojekte (näiteks saab ehitada CMake'i Giti lähtekoodist, kui kasutaja distributsioon ei peaks mingil põhjusel CMake'i tarkvarapaketti pakkuma).
  • Tukee monia koostamisjärjestelmiä. Hädän tullen voi koostaa monia ei-KDE-ohjelmiakin (esim. CMaken voi koostaa Git-lähteistään, ellei jakelusi tarjoa CMake-pakettia).
  • Prend en charge de nombreux systèmes de compilation. Il est même possible de compiler de nombreux projets logiciels non-KDE en un clin d'œil (par exemple, vous pouvez compiler CMake depuis ses propres sources git si votre distribution ne comporte pas de paquets CMake).
  • Supporta molti sistemi di compilazione. È anche possibile compilare in un attimo molti progetti software non di KDE (per esempio, CMake può essere compilato dal suo sorgente git se la tua distribuzione non ha il relativo pacchetto).
    • +
  • 다양한 빌드 시스템을 지원합니다. KDE 외부 소프트웨어도 쉽게 빌드할 수 있습니다(예를 들어 배포판에 CMake 패키지가 없으면 CMake git에서 빌드할 수 있음).
  • Ondersteunt vele bouwsystemen. Het is zelfs mogelijk om veel niet-KDE softwareprojecten in een oogwenk (bijvoorbeeld, CMake kan gebouwd worden vanuit zijn eigen bronnen in git als uw distributie geen CMake pakketten heeft).
  • Obsługuje wiele systemów budowania. Można nawet zbudować wiele projektów oprogramowania nie-KDE w chwilę (na przykład, CMake może budować ze swoich własnych źródeł git jeśli dystrybucja nie ma pakietów CMake).
  • Suporta muitos sistemas de compilação. Até é possível compilar diversos projectos de 'software' fora do KDE de uma vez (por exemplo, o próprio CMake pode ser compilado a partir do seu repositório de Git se a sua distribuição não tiver pacotes do CMake).
    • +
  • Suporta múltiplos sistemas de compilação. É até possível compilar vários projetos de softwares que não sejam do KDE (por exemplo, o CMake pode ser compilado a partir de suas próprias fontes git, se a sua distribuição não tiver pacotes do CMake).
    • +
  • Podpira veliko sistemov izgradnje. Mogoče je celo zgraditi številne projekte programske opreme, ki niso KDE, (na primer, CMake je mogoče zgraditi iz lastnih virov git, če vaša distribucija nima paketov CMake).
  • Stöder många byggsystem. Det är till och med möjligt att bygga många programvaruprojekt som inte ingår i KDE om nödvändigt. CMake kan byggas från sin egen git-källkod om din distribution inte har CMake-paket.
  • Підтримка багатьох систем збирання. Це уможливлює навіть збирання без проблем широкого спектра програмного забезпечення, яке не є програмним забезпеченням KDE (наприклад, можна зібрати CMake з його власних початкових кодів git, якщо у вашому дистрибутиві немає пакунків CMake).
  • xxSupports many build systems. It's even possible to build many non-KDE software projects in a pinch (for instance, CMake can be built from its own git sources if your distribution does not have CMake packages).xx
  • Supports speedy initial checkouts of modules (KDE Git projects only) by using the snapshots already available on the KDE project network.
  • Admet comprovacions inicials ràpides dels mòduls (només projectes Git del KDE) mitjançant l'ús d'instantànies ja disponibles a la xarxa de projectes del KDE.
    • +
  • Admet comprovacions inicials ràpides dels mòduls (només projectes Git del KDE) mitjançant l'ús d'instantànies ja disponibles a la xarxa de projectes del KDE.
  • Υποστηρίζει ταχύτατες αρχικές λήψεις αρθρωμάτων (για έργα KDE git μόνο) με χρήση στιγμιοτύπου που είναι ήδη διαθέσιμο στο δίκτυο των έργων του KDE.
  • Supports speedy initial checkouts of modules (KDE Git projects only) by using the snapshots already available on the KDE project network.
  • Permite acelerar la descarga inicial de los módulos (solo de proyectos Git de KDE) usando instantáneas disponibles en la red de proyectos de KDE.
  • Moodulite algne kiire väljavõtmine (ainult KDE Giti projektide puhul) hetkvõtete abil, mis on KDE projekti võrgustikus juba olemas.
  • Tukee moduulien nopeaa alustavaa lähetystä (vain KDE:n Git-projektit) käyttämällä KDE-projektiverkossa jo olemassa olevia tilannekuvia.
  • Prise en charge de l'extraction initiale rapide de modules (projects Git KDE uniquement) à l'aide d'archives déjà disponibles sur le réseau du projet KDE.
  • Supporta il controllo rapido iniziale dei moduli (solo per i progetti Git di KDE) usando l'istantanea già disponibile nella rete di progetti KDE.
    • +
  • KDE 프로젝트 네트워크에 있는 스냅샷을 이용한 빠른 모듈 체크아웃(KDE Git 프로젝트만)을 지원합니다.
  • Ondersteunt snelle initiële checkouts van modules (alleen KDE Git projecten) door de snapshots te gebruiken die al beschikbaar zijn op het KDE projectnetwerk.
  • Obsługuje szybkie pobieranie kodu modułów (tylko dla projektów KDE Git) przy użyciu zrzutów już dostępnych w sieci projektów KDE.
  • Suporta a recepção rápida das transferências iniciais (apenas nos projectos do KDE em Git), usando as versões fixas já disponíveis na rede de projectos do KDE.
    • +
  • Suporte ao download inicial rápido de módulos (somente projetos KDE no Git) usando os instantâneos já disponíveis na rede do projeto KDE.
    • +
  • Podpira hitre začetne preglede modulov (samo projekti KDE Git) z uporabo posnetkov, ki so že na voljo v projektnem omrežju KDE.
  • Stöder snabb initial utcheckning av moduler (bara KDE git-projekt) genom att använda versionerna som redan är tillgängliga på KDE:s projektnätverk.
  • Підтримка пришвидшеного отримання початкових пакунків модулів (лише проєкти Git KDE) на основі використання знімків, які вже доступні у мережі проєкту KDE.
  • xxSupports speedy initial checkouts of modules (KDE Git projects only) by using the snapshots already available on the KDE project network.xx
  • Supports a "dry run" mode (the --pretend) option so that you can experiment with different settings non-destructively.
  • Admet una opció (la --pretend) al mode «simulacre» («dry run») perquè pugueu experimentar amb diferents ajustaments de forma no destructiva.
    • +
  • Admet una opció (la --pretend) al mode «simulacre» («dry run») perquè pugueu experimentar amb diferents ajustaments de forma no destructiva.
  • Υποστηρίζεται λειτουργία «πρόβας» (η επιλογή --pretend) για να πειραματιστείτε απρόσκοπτα με διαφορετικές ρυθμίσεις.
  • Supports a "dry run" mode (the --pretend) option so that you can experiment with different settings non-destructively.
  • Permite un modo de «ensayo» (la opción --pretend), con el que puede experimentar con distintas preferencias de forma no destructiva.
  • Katsetusrežiimi (võti --pretend) toetus, mis lubab eksperimenteerida mitmete seadistustega ilma halbu tagajärgi kaasa toomata.
  • Tukee ”teeskentelytilaa” (--pretend-valinta), joten voit kokeilla eri asetuksia muuttamatta mitään.
  • Prise en charge d'un mode « dry run » (l'option --pretend) afin de pouvoir essayer différents paramètres sans écrire les changements.
  • Supporta la modalità «simulazione» (l'opzione --pretend), in modo che tu possa fare degli esperimenti non distruttivi con le varie impostazioni.
    • +
  • 시스템을 망치지 않고 다른 설정을 시험해 볼 수 있는 "모의 실행" 모드(--pretend 옵션)을 지원합니다.
  • Ondersteunt een modus "dry run" (de optie --pretend) zodat u niet-destructief kunt experimenteren met verschillende instellingen.
  • Obsługuje tryb "na sucho" (--pretend) tak aby można było próbować wielu ustawień nie niszcząc niczego.
  • Suporta a opção (--pretend) para um modo de "testes", que poderá experimentar com várias configurações de forma não destrutiva.
    • +
  • Suporte ao modo "de teste" (a opção --pretend) para você experimentar com várias configurações de maneira não destrutiva.
    • +
  • Podpira način "suhega teka" (the --pretend), tako da lahko brez škode eksperimentirate z različnimi nastavitvami.
  • Stöder en väljare för "torrkörning" (--pretend) så att du kan experimentera med olika inställningar utan att förstöra något.
  • Підтримка режимів тестового запуску (параметр --pretend) для неруйнівного експериментування із різними параметрами.
  • xxSupports a "dry run" mode (the --pretend) option so that you can experiment with different settings non-destructively.xx
  • kdesrc-build can download modules (both initial checkout and updates) even while building modules that have already been updated.
  • El kdesrc-build pot descarregar els mòduls (comprovació inicial i actualitzacions) fins i tot mentre construeix els mòduls que ja s'han actualitzat.
    • +
  • El kdesrc-build pot descarregar els mòduls (comprovació inicial i actualitzacions) fins i tot mentre construeix els mòduls que ja s'han actualitzat.
  • Με το kdesrc-build μπορεί να γίνει λήψη αρθρωμάτων (και αρχικές λήψεις και ενημερώσεις) ακόμη και όταν κατασκευάζονται αρθρώματα τα οποία έχουν ήση ενημερωθεί.
  • kdesrc-build can download modules (both initial checkout and updates) even while building modules that have already been updated.
  • kdesrc-build puede descargar módulos (tanto descargas iniciales como actualizaciones) incluso cuando está compilando módulos que ya se han actualizado.
  • kdesrc-build võib alla laadida mooduleid (nii välja võtta kui ka uuendada) isegi siis, kui parajasti ehitatakse juba uuendatud mooduleid.
  • kdesrc-build voi ladata moduulit (niin alkuperäisen lähetyksen kuin päivityksetkin) verkosta jopa koostettaessa jo päivitettyjä moduuleja.
  • kdesrc-build peut télécharger des modules (extraction initiale et mises à jour) même lors de la compilation de modules déjà mis à jour.
  • kdesrc-build può scaricare i moduli (sia come scaricamento iniziale che come aggiornamenti) anche durante la compilazione dei moduli che sono già stati aggiornati.
    • +
  • kdesrc-build는 업데이트된 모듈을 빌드하는 동안에도 모듈을 다운로드(첫 체크아웃과 업데이트)할 수 있습니다.
  • kdesrc-build kan modules downloaden (zowel initiële checkout als updates) zelfs tijdens het bouwen van modules die al zijn bijgewerkt.
  • kdesrc-build może pobierać moduły (zarówno pierwsze pobranie jak i uaktualnienia) nawet, gdy moduły budowania już zostały uaktualnione.
  • O kdesrc-build consegue obter módulos (tanto a transferência inicial como as actualizações) mesmo enquanto compila módulos que já foram actualizados.
    • +
  • O kdesrc-build pode baixar módulos (ambos download inicial e atualizações) mesmo quando compilando módulos que já foram atualizados.
    • +
  • kdesrc-build lahko naloži module (tako začetne verzije kot tudi posodobitve), tudi ko gradi že iz posodobljenih modulov.
  • kdesrc-build kan ladda ner moduler (både initial utcheckning och uppdateringar) medan moduler som redan har uppdaterats byggs.
  • kdesrc-build може отримувати модулі (початкові варіанти і оновлення), навіть при збиранні модулів, які вже було оновлено.
  • xxkdesrc-build can download modules (both initial checkout and updates) even while building modules that have already been updated.xx
  • kdesrc-build supports uninstalling modules (manually, or prior to installing an already-installed module) in order to keep the install directory clean. Note that this feature requires CMake support and is still experimental.
  • El kdesrc-build admet la desinstal·lació dels mòduls (manualment o abans d'instal·lar un mòdul ja instal·lat) per a mantenir net el directori d'instal·lació. Recordeu que aquesta característica requereix suport de CMake i encara és experimental.
    • +
  • El kdesrc-build admet la desinstal·lació dels mòduls (manualment o abans d'instal·lar un mòdul ja instal·lat) per a mantindre net el directori d'instal·lació. Recordeu que aquesta característica requereix suport de CMake i encara és experimental.
  • Το kdesrc-build υποστηρίζει απεγκατάσταση αρθρωμάτων (χειροκίνητα, ή πριν την εγκατάσταση ενός ήδη εγκατεστημένου αρθρώματος) για να διατηρείται ο κατάλογος εγκατάστασης καθαρός. Σημειώστε ότι το χαρακτηριστικό αυτό απαιτεί υποστήριξη CMake και είναι ακόμη πειραματικό.
  • kdesrc-build supports uninstalling modules (manually, or prior to installing an already-installed module) in order to keep the install directory clean. Note that this feature requires CMake support and is still experimental.
  • kdesrc-build permite desinstalar módulos (de forma manual, o antes de instalar un módulo ya instalado) para mantener limpio el directorio de instalación. Tenga en cuenta que esta función necesita usar CMake y sigue siendo experimental.
  • kdesrc-build toetab moodulite eemaldamist (käsitsi või enne juba paigaldatud mooduli paigaldamist), mis aitab hoida paigalduskataloogi puhtana. Siiski tasub tähele panna, et see nõuab CMake'i toetust ja on esialgu eksperimentaalne.
  • kdesrc-build tukee moduulien asennuksen poistoa (käsin tai ennen jo asennetun moduulin uudelleenasennusta) asennuskansion pitämiseksi siistinä. Huomaa, että tämä ominaisuus vaatii CMake-tukea ja on yhä kokeellinen.
  • kdesrc-build prend en charge la désinstallation de modules (manuellement, ou avant l'installation d'un module déjà installé) afin que vous puissiez conserver un dossier d'installation propre. Remarquez que cette fonctionnalité nécessitant la prise en charge de CMake est encore expérimentale.
  • kdesrc-build supporta la disinstallazione dei moduli (manualmente o prima dell'installazione di un modulo già installato), in modo da mantenere pulita la cartella di installazione. Nota che questa funzionalità richiede il supporto di CMake ed è ancora sperimentale.
    • +
  • kdesrc-build는 설치 디렉터리를 깨끗하게 해 주는 모듈 삭제를 지원합니다(수동 또는 모듈 설치 전에 자동). 이 기능은 CMake 지원이 필요하며 아직까지는 실험적입니다.
  • kdesrc-build ondersteunt het ongedaan maken van geïnstalleerde modules (handmatig of voor het installeren van een al geïnstalleerde module) om de installatiemap schoon te houden. Merk op dat deze mogelijkheid ondersteuning vereist van CMake en nog steeds experimenteel is.
  • kdesrc-build obsługuje usuwanie modułów (ręcznie, lub przed wgraniem już wgranego modułu) aby utrzymać porządek w katalogu wgrywania. Ta możliwość wymaga wsparcia w CMake i nadal jest eksperymentalna.
  • O kdesrc-build suporta a desinstalação de módulos (manual ou antes de instalar um módulo já instalado) para manter a pasta de instalação limpa.Lembre-se que esta funcionalidade obriga ao suporte do CMake e ainda é experimental.
    • +
  • O kdesrc-build suporta a desinstalação de módulos (manualmente ou antes de instalar os módulos já instalados) para poder manter a pasta de instalação limpa. Note que este recurso precisa do suporta ao CMake e ainda é experimental.
    • +
  • kdesrc-build podpira odstranjevanje modulov (ročno ali pred namestitvijo že nameščenega modula), da bi mapa namestitve ostala čista. Upoštevajte, da ta funkcija zahteva podporo CMake in je še vedno eksperimentalna.
  • kdesrc-build stöder att avinstallera moduler (manuellt, eller innan en redan installerad modul installeras) för att hålla installationskatalogen ren. Observera att funktionen kräver stöd för CMake och är fortfarande experimentell.
  • У kdesrc-build передбачено підтримку вилучення модулів (вручну або перед встановленням вже встановленого модуля) для підтримання чистоти у каталозі встановлення. Зауважте, що ця можливість потребує підтримки з боку CMake та усе ще є експериментальною.
  • xxkdesrc-build supports uninstalling modules (manually, or prior to installing an already-installed module) in order to keep the install directory clean. Note that this feature requires CMake support and is still experimental.xx
  • kdesrc-build logs everything for easy perusal later, that way you can determine why things went wrong if a build fails. kdesrc-build automatically creates symlinks for easy access to the last log (log/latest). color-coding of the output (which can be turned off).
  • El kdesrc-build ho enregistra tot per a una lectura fàcil posterior, d'aquesta manera podreu determinar perquè les coses han sortit malament si falla una construcció. El kdesrc-build crearà automàticament enllaços simbòlics per a facilitar l'accés a l'últim registre (log/latest). Codificant amb colors la sortida (el qual es pot desactivar).
    • +
  • El kdesrc-build ho enregistra tot per a una lectura fàcil posterior, d'aquesta manera podreu determinar perquè les coses han eixit malament si falla una construcció. El kdesrc-build crearà automàticament enllaços simbòlics per a facilitar l'accés a l'últim registre (log/latest). Codificant amb colors l'eixida (el qual es pot desactivar).
  • Το kdesrc-build καταγράφει τα πάντα για εύκολη χρήση αργότερα, έτσι μπορείτε να να εξηγήσετε τι πήγε στραβά αν μια κατασκευή αποτύχει. Το kdesrc-build δημιουργεί αυτόματα συμβολικούς δεσμούς για εύκολη πρόσβαση στην τελευταία καταγραφή (καταγραφή/τελευταία) με χρώματα στην έξοδο (το οποίο μπορεί να απενεργοποιηθεί).
  • kdesrc-build logs everything for easy perusal later, that way you can determine why things went wrong if a build fails. kdesrc-build automatically creates symlinks for easy access to the last log (log/latest). colour-coding of the output (which can be turned off).
  • kdesrc-build registra todo para que pueda examinarlo posteriormente con atención, de modo que pueda determinar qué ha ido mal si falla una compilación. kdesrc-build crea enlaces simbólicos automáticamente para que pueda acceder al último registro con facilidad (log/latest) y usa un código de colores para la salida (que se puede desactivar).
  • kdesrc-build logib kõike hilisemaks uurimiseks, et saaksid selgusele jõuda, mis läks valesti, kui ehitamine peaks nurjuma. kdesrc-build loob automaatselt nimeviidad viimasele logile. Väljundit võib lasta esitada värvilisena.
  • kdesrc-build tekee kaikesta myöhempää tarkastelua varten lokimerkinnän, joten voit tarkistaa, mitä meni pieleen, ellei koostaminen onnistunut. kdesrc-build luo automaattisesti symlinkin viimeisimpään lokiin. Tuloste on värikoodattu (mutta sen saa myös pois käytöstä).
  • kdesrc-build enregistre toutes ses actions afin que vous puissiez facilement déterminer la cause racine du problème en cas d'échec de la compilation. kdesrc-build crée automatiquement des liens symboliques afin que vous puissiez rapidement accéder au dernier journal (log/latest). Coloration syntaxique de la sortie (que vous pouvez désactiver).
  • kdesrc-build registra tutte le operazioni per una lettura semplice in un secondo tempo, in modo che tu possa determinare cosa è andato storto se una compilazione non ha avuto successo. Crea inoltre automaticamente dei collegamenti simbolici per facilitare l'accesso all'ultima registrazione (log/latest). Codici di colore diversi per l'output (che possono essere disattivati).
    • +
  • kdesrc-build는 매 과정마다 로그를 생성하여 빌드 실패 시 문제를 쉽게 찾을 수 있도록 도와 줍니다. kdesrc-build는 마지막 로그에 쉽게 접근하는 심볼릭 링크(log/latest)를 생성합니다. 출력에 색깔을 지정할 수 있습니다(끌 수 있음).
  • kdesrc-build logt alles voor gemakkelijk later bekijken, op die manier kunt u bepalen waarom dingen fout gingen als het bouwen mislukte. kdesrc-build maakt automatisch symbolische koppelingen aan voor gemakkelijke toegang tot de laatste log (log/latest). kleurcodering van de uitvoer (die uit gezet kan worden).
  • kdesrc-build zapisuje każde swoje poczynanie, tak aby można było później stwierdzić dlaczego nie można czegoś zbudować. kdesrc-build sam tworzy dowiązania, aby móc odczytywać ostatni dziennik (log/latest). kolorowe kodowanie wyniku (które można wyłączyć).
  • O kdesrc-build regista tudo para uso pessoal simples mais tarde, de forma que possa perceber o que se passou de errado no caso de um problema na compilação. O kdesrc-build cria automaticamente ligações simbólicas para aceder rapidamente ao último registo (log/latest), bem como códigos de cores no resultado (que poderão ser desligados).
    • +
  • O kdesrc-build registra tudo para facilitar a leitura posterior, desta forma você pode determinar porque algo deu errado se uma compilação falhar. O kdesrc-build cria automaticamente links simbólicos para um fácil acesso ao último registro (log/último). Colorindo a saída (que pode ser desativado).
    • +
  • kdesrc-build zabeleži vse za lažji kasnejši pregled, tako lahko ugotovite, zakaj je kaj šlo narobe, če se izgradnja ne uspe. kdesrc-build samodejno ustvari simbolične povezave za enostaven dostop do zadnjega dnevnika (log/najnovejši). barvno kodiranje izhoda (ga je mogoče izklopiti).
  • kdesrc-build loggar allting för enkel genomläsning senare. På så sätt kan du avgöra varför något gick fel om ett bygge misslyckas. kdesrc-build skapar automatiskt symboliska länkar för att enkelt komma åt den senaste loggen (log/latest). Färgkodning av utmatningen (som kan stängas av).
  • kdesrc-build записує усі команди для спрощення повторного використання. За допомогою журналу ви можете визначити причину помилок, якщо щось не вдається зібрати. kdesrc-build автоматично створює символічні посилання для спрощення доступу до останнього журналу (log/latest). Передбачено використання розфарбовування виведених даних (розфарбовування можна вимкнути).
  • xxkdesrc-build logs everything for easy perusal later, that way you can determine why things went wrong if a build fails. kdesrc-build automatically creates symlinks for easy access to the last log (log/latest). color-coding of the output (which can be turned off).xx
  • Has support for building the Qt Project's Qt library, which is a prerequisite for KDE software. kdesrc-build is very customizable. You can control most options down to a module-by-module basis if you so desire, including configure-flags, CMake flags, and your C++ flags during compilation. Instead of trying to remember what configure line you used, you can set it once and forget it.
  • Admet construir la biblioteca de les Qt del Projecte Qt, el qual és un requisit previ per al programari KDE. El kdesrc-build és molt personalitzable. Podreu controlar la majoria de les opcions fins a una base de fer mòdul per mòdul, si així ho voleu, incloent indicadors de configuració, indicadors de CMake i els vostres indicadors de C++ durant la compilació. En lloc d'intentar recordar quina línia de configuració havíeu usat, podreu configurar-la un cop i oblidar-la.
    • +
  • Admet construir la biblioteca de les Qt del Projecte Qt, el qual és un requisit previ per al programari KDE. El kdesrc-build és molt personalitzable. Podreu controlar la majoria de les opcions fins a una base de fer mòdul per mòdul, si així ho voleu, incloent indicadors de configuració, indicadors de CMake i els vostres indicadors de C++ durant la compilació. En lloc d'intentar recordar quina línia de configuració havíeu usat, podreu configurar-la un cop i oblidar-la.
  • Υποστηρίζει την κατασκευή της βιβλιοθήκης Qt για το έργο Qt, η οποία προαπαιτείται για το λογισμικό του KDE. Το kdesrc-build είναι πολύ προσαρμόσιμο. Μπορείτε να ελέγξετε τις περισσότερες επιλογές ανά άρθρωμα αν το επιθυμείτε, μαζί και σημαίες διαμόρφωσης, σημαίες του CMake και τις δικές σας σημαίες C++ κατα τη μεταγλώττιση. Αντί να προσπαθείτε να θυμηθείτε ποια γραμμή διαμόρφωσης χρησιμοποιήσατε, τη ρυθμίζετε μία φορά και την ξεχνάτε.
  • Has support for building the Qt Project's Qt library, which is a prerequisite for KDE software. kdesrc-build is very customisable. You can control most options down to a module-by-module basis if you so desire, including configure-flags, CMake flags, and your C++ flags during compilation. Instead of trying to remember what configure line you used, you can set it once and forget it.
  • Permite compilar la biblioteca del proyecto Qt, que es un requisito previo para el software de KDE. kdesrc-build es muy personalizable. Puede controlar la mayoría de las opciones para cada módulo, si lo desea, incluyendo banderas de configuración, banderas de CMake y banderas de C++ durante la compilación. En lugar de tratar de recordar la línea de configuración que ha usado, puede definirla una única vez y olvidarse de ella.
  • Qt projekti Qt teegi ehitamise toetus, mis teatavasti on KDE tarkvara eeltingimus. kdesrc-build on igati kohandatav. Enamikku valikuid saab määrata soovi korral isegi mooduli tasandil, kaasa arvatud konfigureerimisvõtmed, CMake'i võtmed ja C++ võtmed kompileerimiseks. Selle asemel, et püüda pingsalt meelde jätta, millist konfigureerimiskäsku oled kasutanud, saab selle ühe ropsuga paika panna ja unustada.
  • Tuki Qt-projektin Qt-kirjaston koostamiseen, mitä KDE-ohjelmat edellyttävät. kdesrc-build on hyvin mukautettavissa. Useimpia valintoja voi hallita moduuleittain, jos niin halutaan, myös koostamisen aikaisia asetus-, CMake- ja C++-lippuja. Sen sijaan että yrittäisit aina muistaa mitä asetuksia tarvittiin, voit asettaa ne kerran ja unohtaa koko jutun.
  • Prend en charge la compilation de la librairie Qt du projet Qt, un pré-requis pour les logiciels KDE. kdesrc-build est entièrement personnalisable. Vous pouvez contrôler la majorité des options module par module si vous le désirez, notamment les drapeaux configure, les drapeaux CMake et même les drapeaux C++ lors de la compilation. Au lieu d'essayer de vous rappeler la ligne de configuration que vous utilisez, vous pouvez la définir une seule fois et l'oublier.
  • Ha il supporto per la compilazione delle librerie del progetto Qt, che sono un prerequisito per il software di KDE. kdesrc-build è molto personalizzabile: puoi controllare la maggior parte delle opzioni, anche modulo per modulo se lo desideri, inclusa la configurazione dei flag, dei flag di CMake e dei tuoi flag di C++ durante la compilazione. Invece di provare a ricordare quale riga di configurazione hai usato, puoi impostarla una volta per tutte e quindi dimenticarla.
    • +
  • Qt 프로젝트의 Qt 라이브러리 빌드 기능을 지원합니다. KDE 소프트웨어에 필수적인 구성 요소입니다. kdesrc-builid는 다양하게 설정할 수 있습니다. 필요한 경우 모듈별로 configure 플래그, CMake 플래그, 컴파일 시간 C++ 플래그를 설정할 수 있습니다. 한 번만 설정해 두고 다시 입력하지 마십시오.
  • Heeft ondersteuning voor bouwen van de Qt bibliotheek van het Qt project, die een vereiste is voor KDE software. kdesrc-build is erg goed aan te passen. U kunt de meeste opties controleren tot op een basis van module-per-module als u dat wenst, inclusief configuratie-vlaggen, CMake-vlaggen en uw C++ vlaggen tijdens compilatie. In plaats van proberen om te onthouden welke regel voor configuratie u gebruikt, kunt u het een keer instellen en vergeten.
  • Wspiera budowanie biblioteki Qt projektu Qt, co jest wymaganiem dla oprogramowania KDE. kdesrc-build można dowolnie dopasowywać. Można zmieniać większość opcji aż do poziomu modułu, flag ustawiania, flag CMake oraz flag C++. Zamiast zapamiętywać tekst do ustawień w wierszu poleceń, wystarczy ustawić je raz i zapomnieć o nich.
  • Tem suporte para compilar a biblioteca Qt do Qt Project, a qual é um pré-requisito para as aplicações do KDE. O kdesrc-build tem bastantes capacidades de personalização. Poderá controlar a maioria das opções ao nível de cada módulo individual, se o desejar, incluindo as opções do 'configure', do CMake e as suas opções de C++ durante a compilação. Em vez de tentar recordar qual a linha de configuração que precisa, podê-la-á definir apenas uma vez e não se preocupar mais.
    • +
  • Tem suporte para compilar a biblioteca Qt do Qt Project, que é um pré-requisito para os softwares do KDE. O kdesrc-build é um personalizável. Você pode controlar a maioria das opções módulo a módulo se deseja, incluindo suas flags do configure, CMake e C++ durante a compilação. Ao invés de tentar lembrar qual linha de configuração você usou, você pode configurar uma vez e esquecer.
    • +
  • Ima podporo za gradnjo knjižnice Qt Project Qt, ki je pogoj za programsko opremo KDE. kdesrc-build je zelo prilagodljiv. Če želite, lahko nadzirate večino možnosti do nivoja posameznega modula, vključno z zastavicami configure, CMake in vašimi C++ zastavicami med kompilacijo. Namesto da bi se spomnili, katero konfiguracijsko vrstico ste uporabili, jo lahko nastavite enkrat in pozabite nanjo.
  • Har stöd för att bygga Qt-projektets Qt-bibliotek, vilket är en förutsättning för KDE-programvara. kdesrc-build är mycket anpassningsbart. Du kan styra de flesta alternativ ner till modul för modul om man så önskar, inklusive konfigurationsargument, CMake-argument och C++ argument under kompilering. Istället för att försöka komma ihåg vilken konfigurationsrad som du använde, kan du ställa in den en gång för alla och sedan glömma den.
  • Передбачено підтримку збирання бібліотек Qt проєкту Qt, який є основою програмного забезпечення KDE. kdesrc-build є дуже гнучким у налаштовуванні. Якщо захочете, ви можете керувати параметрами кожного з модулів, зокрема прапорцями configure, прапорцями CMake та прапорцями C++ під час збирання. Замість запам'ятовування рядка налаштовування, ви можете просто визначити його один раз і забути про нього.
  • xxHas support for building the Qt Project's Qt library, which is a prerequisite for KDE software. kdesrc-build is very customizable. You can control most options down to a module-by-module basis if you so desire, including configure-flags, CMake flags, and your C++ flags during compilation. Instead of trying to remember what configure line you used, you can set it once and forget it.xx
  • Extensive documentation. Feel free to let me know if you need something explained (e-mail address at bottom), or contact the kde-devel mailing list.
  • Disposa d'una àmplia documentació. Trobeu-vos lliure d'avisar-me si necessiteu que us expliqui alguna cosa (l'adreça de correu electrònic es troba la part inferior) o poseu-vos amb la llista de correu de kde-devel.
    • +
  • Disposa d'una àmplia documentació. Trobeu-vos lliure d'avisar-me si necessiteu que vos expliqui alguna cosa (l'adreça de correu electrònic es troba la part inferior) o poseu-vos amb la llista de correu de kde-devel.
  • Εκτεταμένη τεκμηρίωση. Ενημερώστε με αν χρειάζεστε διευκρινίσεις (η διεύθυνση ηλεκτρονικού ταχυδρομείου στη βάση) ή ελάτε σε επαφή με τη λίστα ηλεκτρονικής αλληλογραφίας kde-devel.
  • Extensive documentation. Feel free to let me know if you need something explained (e-mail address at bottom), or contact the kde-devel mailing list.
  • Extensa documentación. Siéntase libre de hacerme saber si necesita alguna explicación (dirección de correo electrónico al final), o contacte con la lista de distribución de kde-devel.
  • Ulatuslik dokumentatsioon. Andke julgelt teada, kui on vaja midagi täpsemalt selgitada (e-posti aadressi leiab altpoolt), või võtke ühendust postiloendiga kde-devel.
  • Laaja ohjeistus. Jos haluat jotakin selitettävän, ota vapaasti yhteyttä minuun (sähköpostiosoite löytyy alta) tai kde-devel-sähköpostilistaan.
  • Documentation étendue. N'hésitez pas à me faire savoir si vous avez besoin que quelque chose soit expliqué (adresse de courriel disponible ci-dessous), ou en contactant la liste de diffusion kde-devel.
  • Ampia documentazione. Sentiti libero di farmi sapere se hai bisogno di qualche spiegazione (l'indirizzo di posta elettronica è in basso), o di contattare la lista kde-devel.
    • +
  • 폭넓은 문서화. 필요한 것이 있으면 아래에 있는 이메일 주소나 kde-devel 메일링 리스트로 연락하십시오.
  • Uitgebreide documentatie. Voel u vrij om me te laten weten of u het nodig hebt iets uit te leggen (e-mailadres onderaan) of neem contact op via de e=maillijst kde-devel.
  • Szeroka dokumentacja. W razie pytań można pisać na pocztę (adres podany poniżej) lub na listę dyskusyjną kde-devel.
  • Documentação abrangente. Sinta-se à vontade para nos consultar se precisa que algo seja explicado (endereço de e-mail no fundo) ou contacte a lista de correio kde-devel.
    • +
  • Documentação extensiva. Fique a vontade para me informar se você precisa de algo explicado (endereço de e-mail na parte inferior) ou contate a lista de discussão kde-devel.
    • +
  • Obsežna dokumentacija. Prosim, obvestite nas, če potrebujete še kako pojasnilo (e-poštni naslov na dnu) ali se obrnite na e-poštni seznam kde-devel.
  • Omfattande dokumentation. Tala gärna om för mig om något behöver förklaras (e-postadressen längst ner), eller kontakta e-postlistan kde-devel.
  • Повна документація. Повідомте авторам програми, якщо якась з її частин видасться вам незрозумілою (адресу електронної пошти наведено нижче), або напишіть про вашу проблему до списку листування kde-devel.
  • xxExtensive documentation. Feel free to let me know if you need something explained (e-mail address at bottom), or contact the kde-devel mailing list.xx
  • A detailed sample configuration file is included, usually you can just copy it to ~/.kdesrc-buildrc and be done with it.
  • S'inclou un fitxer de configuració detallat d'exemple, normalment podreu copiar-lo a ~/.kdesrc-buildrc i llest.
    • +
  • S'inclou un fitxer de configuració detallat d'exemple, normalment podreu copiar-lo a ~/.kdesrc-buildrc i llest.
  • Συμπεριλαμβάνεται ένα δείγμα λεπτομερούς αρχείου διαμόρφωσης, συνήθως μπορείτε να το αντιγράψετε στο ~/.kdesrc-buildrc και είστε έτοιμοι.
  • A detailed sample configuration file is included, usually you can just copy it to ~/.kdesrc-buildrc and be done with it.
  • Se incluye un detallado archivo de configuración de ejemplo, que puede copiar en ~/.kdesrc-buildrc para que esté listo para usar.
  • Kaasa on pandud üksikasjalik konfigureerimisfaili näidis, mille võib tavaliselt muretult asukohta ~/.kdesrc-buildrc kopeerida.
  • Mukana toimiteaan yksityiskohtainen esimerkkiasetustiedosto, jonka usein voi vain kopioida ~/.kdesrc-buildrc:ksi.
  • Un exemple de fichier de configuration détaillé est inclus, vous pouvez généralement le copier dans ~/.kdesrc-build sans avoir besoin d'effectuer d'actions supplémentaires.
  • È incluso un file di configurazione di esempio, di solito basta semplicemente copiarlo in ~/.kdesrc-buildrc.
    • +
  • 간단한 기본 설정 파일이 들어 있어서 ~/.kdesrc-buildrc 파일로 복사하기만 하면 빌드를 실행할 수 있습니다.
  • Een gedetailleerde voorbeeldconfiguratiebestand is ingevoegd, gewoonlijk kunt u het gewoon kopiëren naar ~/.kdesrc-buildrc en u bent gereed.
  • Załączony jest szczegółowa próbka pliku ustawień, wystarczy ją skopiować do ~/.kdesrc-buildrc.
  • Está incluído um ficheiro de configuração de exemplo detalhado, o qual bastará copiar para ~/.kdesrc-buildrc e não precisar de mais nada.
    • +
  • Um exemplo de arquivo de configuração detalhado está incluído, geralmente você pode apenas copiá-lo para ~/.kdesrc-buildrc.
    • +
  • Vključena je podrobna vzorčna konfiguracijska datoteka, običajno jo lahko preprosto kopirate v ~/.kdesrc-buildrc in jo uporabite.
  • En detaljerad inställningsfil är inkluderad, och oftast kan du bara kopiera den till ~/.kdesrc-buildrc och inte behöva göra något mer.
  • До складу пакунка включено зразок файла налаштувань із докладними поясненнями. Зазвичай, ви можете просто скопіювати його до ~/.kdesrc-buildrc і почати користуватися програмоюо.
  • xxA detailed sample configuration file is included, usually you can just copy it to ~/.kdesrc-buildrc and be done with it.xx
  • Much more!
  • Molt més!
    • +
  • Molt més!
  • Πολλά περισσότερα!
  • Much more!
  • ¡Y mucho más!
  • Veel palju rohkem!
  • Ja paljon muuta!
  • Et bien plus !
  • E molto altro.
    • +
  • 이외 더 많은 기능!
  • En nog veel meer!
  • Dużo więcej!
  • E muito mais!
    • +
  • E muito mais!
  • A oveľa viac!
    • +
  • In še mnogo več!
  • Mycket mer!
  • Та багато іншого!
  • xxMuch more!xx
 https://kdesrc-build.kde.org https://bugs.kde.org/enter_bug.cgi?format=guided&product=kdesrc-build https://www.kde.org/community/donations/?app=kdesrc-build kdesrc-build KDE https://cdn.kde.org/screenshots/kdesrc-build/kdesrc-build.png Development kdesrc-build diff --git a/qt5-build-include b/qt5-build-include index 01e8ba3..2eb6152 100644 --- a/qt5-build-include +++ b/qt5-build-include @@ -1,30 +1,30 @@ module-set qt5-set repository qt-projects - branch 5.13 # not the most recent but recent enough + branch 5.14 # not the most recent but recent enough # init-repository supports catch-alls like "default" as well, in which case # you will want to uncomment ignore-modules below to control which modules to # leave out use-modules qtbase qtdeclarative qtgraphicaleffects qtimageformats \ qtmultimedia qtquickcontrols qtquickcontrols2 qtscript qtsensors qtsvg \ qttools qtwayland qtwebchannel qtwebsockets qtwebview qtx11extras \ qtxmlpatterns # ignore-modules qtwebengine # install path. This *MUST* match your qtdir setting in kdesrc-buildrc! prefix ${qtdir} configure-flags -optimized-tools -reduce-relocations # make-options -j7 end module-set # qtwebengine is essentially the Chromium Embedded Framework with Qt bindings # and has source code of unusually large complexity for the compiler. # TL;DR: This will eat a *ton* of RAM and can lockup your system if you have a # lot of CPU cores. qtwebengine is disabled by default but if you enable it # also ensure you don't outstrip your available RAM with too high of a # parallelism (-j flag). options qtwebengine make-options NINJAFLAGS=-j4 end options diff --git a/t/smoke/read-module-options.t b/t/smoke/read-module-options.t index b2eaa69..06915f9 100644 --- a/t/smoke/read-module-options.t +++ b/t/smoke/read-module-options.t @@ -1,93 +1,96 @@ use 5.014; use strict; use warnings; # Test basic option reading from rc-files use Test::More; package ksb::test { use Carp qw(confess cluck); use Test::More; my %inspectors = ( log_command => \&_inspect_log_command, ); our @CMD; our %OPTS; sub inspect { my $mod = shift; my $sub = $inspectors{$mod} or return; goto $sub; }; sub _inspect_log_command { my ($module, $filename, $argRef, $optionsRef) = @_; confess "No arg to module" unless $argRef; my @command = @{$argRef}; if (grep { $_ eq 'cmake' } @command) { @CMD = @command; %OPTS = %{$optionsRef}; } }; 1; }; use ksb::Application; use ksb::Util qw(trimmed); my @args = qw(--pretend --rc-file t/data/sample-rc/kdesrc-buildrc); my $app = ksb::Application->new; my @selectors = $app->establishContext(@args); my $workload = $app->modulesFromSelectors(@selectors); $app->setModulesToProcess($workload); my @moduleList = $app->modules(); is(scalar @moduleList, 4, 'Right number of modules'); -is($moduleList[0]->name(), 'module2', 'Right module name'); -my $scm = $moduleList[0]->scm(); +# module2 is last in rc-file so should sort last +is($moduleList[3]->name(), 'module2', 'Right module name'); + +my $scm = $moduleList[3]->scm(); isa_ok($scm, 'ksb::Updater::Git'); my ($branch, $type) = $scm->_determinePreferredCheckoutSource(); is($branch, 'refs/tags/fake-tag5', 'Right tag name'); is($type, 'tag', 'Result came back as a tag'); -is($moduleList[2]->name(), 'setmod2', 'Right module name from module-set'); -($branch, $type) = $moduleList[2]->scm()->_determinePreferredCheckoutSource(); +# setmod2 is second module in set of 3 at start, should be second overall +is($moduleList[1]->name(), 'setmod2', 'Right module name from module-set'); +($branch, $type) = $moduleList[1]->scm()->_determinePreferredCheckoutSource(); is($branch, 'refs/tags/tag-setmod2', 'Right tag name (options block)'); is($type, 'tag', 'options block came back as tag'); # # Test some of the option parsing indirectly by seeing how the value is input # into build system. # # Override auto-detection since no source is downloaded -$moduleList[2]->setOption('override-build-system', 'kde'); +$moduleList[1]->setOption('override-build-system', 'kde'); # Should do nothing in --pretend -ok($moduleList[2]->setupBuildSystem(), 'setup fake build system'); +ok($moduleList[1]->setupBuildSystem(), 'setup fake build system'); ok(@ksb::test::CMD, 'log_command cmake was called'); is(scalar (@ksb::test::CMD), 8); is($ksb::test::CMD[0], 'cmake', 'CMake command should start with cmake'); is($ksb::test::CMD[1], '/tmp/setmod2', 'CMake command should specify source directory as first argument'); is($ksb::test::CMD[2], '-G', 'CMake generator should be specified explicitly'); is($ksb::test::CMD[3], 'Unix Makefiles', 'Expect the default CMake generator to be used'); is($ksb::test::CMD[4], '-DCMAKE_BUILD_TYPE=a b', 'CMake options can be quoted'); is($ksb::test::CMD[5], 'bar=c', 'CMake option quoting does not eat all options'); is($ksb::test::CMD[6], 'baz', 'Plain CMake options are preserved correctly'); is($ksb::test::CMD[7], "-DCMAKE_INSTALL_PREFIX=$ENV{HOME}/kde", 'Prefix is passed to cmake'); # See https://phabricator.kde.org/D18165 -is($moduleList[1]->getOption('cxxflags'), '', 'empty cxxflags renders with no whitespace in module'); +is($moduleList[0]->getOption('cxxflags'), '', 'empty cxxflags renders with no whitespace in module'); done_testing(); diff --git a/t/smoke/set-module-option-value.t b/t/smoke/set-module-option-value.t index 03d5e77..c1f9c8c 100644 --- a/t/smoke/set-module-option-value.t +++ b/t/smoke/set-module-option-value.t @@ -1,36 +1,38 @@ use 5.014; use strict; use warnings; # Test use of --set-module-option-value use Test::More; use ksb::Application; my @args = (qw(--pretend --rc-file t/data/sample-rc/kdesrc-buildrc --set-module-option-value), 'module2,tag,fake-tag10', '--set-module-option-value', 'setmod2,tag,tag-setmod10'); my $app = ksb::Application->new; my @selectors = $app->establishContext(@args); my $workload = $app->modulesFromSelectors(@selectors); $app->setModulesToProcess($workload); my @moduleList = $app->modules(); is(scalar @moduleList, 4, 'Right number of modules'); -my $scm = $moduleList[0]->scm(); +my ($module) = grep { "$_" eq 'module2' } @moduleList; +my $scm = $module->scm(); my ($branch, $type) = $scm->_determinePreferredCheckoutSource(); is($branch, 'refs/tags/fake-tag10', 'Right tag name'); is($type, 'tag', 'Result came back as a tag'); -($branch, $type) = $moduleList[2]->scm()->_determinePreferredCheckoutSource(); +($module) = grep { "$_" eq 'setmod2' } @moduleList; +($branch, $type) = $module->scm()->_determinePreferredCheckoutSource(); is($branch, 'refs/tags/tag-setmod10', 'Right tag name (options block from cmdline)'); is($type, 'tag', 'cmdline options block came back as tag'); -ok(!$moduleList[2]->isKDEProject(), 'setmod2 is *not* a "KDE" project'); -is($moduleList[2]->fullProjectPath(), 'setmod2', 'fullProjectPath on non-KDE modules returns name'); +ok(!$module->isKDEProject(), 'setmod2 is *not* a "KDE" project'); +is($module->fullProjectPath(), 'setmod2', 'fullProjectPath on non-KDE modules returns name'); done_testing(); diff --git a/t/unit/dependency-resolver/compare-build-order.t b/t/unit/dependency-resolver/compare-build-order.t index 6aaccdd..c04026f 100644 --- a/t/unit/dependency-resolver/compare-build-order.t +++ b/t/unit/dependency-resolver/compare-build-order.t @@ -1,55 +1,101 @@ use 5.014; use strict; use warnings; # Test comparison operation for sorting modules into build order use Test::More; use ksb::DependencyResolver; my $graph1 = { 'a' => { votes => { 'b' => 1, 'd' => 1 }, - module => 1 + module => { + name => 'a', + }, }, 'b' => { votes => {}, - module => 1 + module => { + name => 'b', + }, }, 'c' => { votes => { 'd' => 1 }, - module => 1 + module => { + name => 'c', + }, }, 'd' => { votes => {}, - module => 1 - } + module => { + name => 'd', + }, + }, + + 'e' => { # Here to test sorting by rc-file order + votes => { + 'b' => 1, + 'd' => 1, + }, + module => { + name => 'e', + '#create-id' => 2, + }, + }, + 'f' => { # Identical to 'e' except it's simulated earlier in rc-file + votes => { + 'b' => 1, + 'd' => 1, + }, + module => { + name => 'f', + '#create-id' => 1, + }, + }, }; -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'a'), 0, "'a' should be sorted at the same position as itself"); +# Test that tests are symmetric e.g. a > b => b < a. This permits us to only manually +# test one pair of these tests now that the test matrix is growing. +for my $l ('a'..'f') { + for my $r ('a'..'f') { + my $res = ksb::DependencyResolver::_compareBuildOrder($graph1, $l, $r); + + if ($l eq $r) { + is($res, 0, "'$l' should be sorted at the same position as itself"); + } + else { + is(abs($res), 1, "Different module items ('$l' and '$r') compare to 1 or -1 (but not 0)"); + is(ksb::DependencyResolver::_compareBuildOrder($graph1, $r, $l), -$res, + "Swapping order of operands should negate the result ('$r' vs '$l')"); + } + } +} + is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'b'), -1, "'a' should be sorted before 'b' by dependency ordering"); is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'c'), -1, "'a' should be sorted before 'c' by vote ordering"); is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'd'), -1, "'a' should be sorted before 'd' by dependency ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'e'), -1, "'a' should be sorted before 'e' by lexicographic ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'a', 'f'), -1, "'a' should be sorted before 'f' by lexicographic ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'a'), 1, "'b' should be sorted after 'a' by dependency ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'b'), 0, "'b' should be sorted at the same position as itself"); is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'c'), 1, "'b' should be sorted after 'c' by vote ordering"); is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'd'), -1, "'b' should be sorted before 'd' by lexicographic ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'e'), 1, "'b' should be sorted after 'e' by dependency ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'b', 'f'), 1, "'b' should be sorted after 'f' by dependency ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'a'), 1, "'c' should be sorted after 'c' by vote ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'b'), -1, "'c' should be sorted before 'b' by vote ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'c'), 0, "'c' should be sorted at the same position as itself"); is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'd'), -1, "'c' should be sorted before 'd' by dependency ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'e'), 1, "'c' should be sorted after 'e' by vote ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'c', 'f'), 1, "'c' should be sorted after 'f' by vote ordering"); + +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'e'), 1, "'d' should be sorted after 'e' by dependency ordering"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'f'), 1, "'d' should be sorted after 'f' by dependency ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'a'), 1, "'d' should be sorted after 'a' by dependency ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'b'), 1, "'d' should be sorted after 'b' by lexicographic ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'c'), 1, "'d' should be sorted after 'c' by dependency ordering"); -is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'd', 'd'), 0, "'d' should be sorted at the same position as itself"); +is(ksb::DependencyResolver::_compareBuildOrder($graph1, 'e', 'f'), 1, "'e' should be sorted after 'f' by rc-file ordering"); done_testing(); diff --git a/t/unit/dependency-resolver/sort-build-order.t b/t/unit/dependency-resolver/sort-build-order.t index f0cc742..a02c3d7 100644 --- a/t/unit/dependency-resolver/sort-build-order.t +++ b/t/unit/dependency-resolver/sort-build-order.t @@ -1,113 +1,96 @@ use 5.014; use strict; use warnings; # Test sorting modules into build order use Test::More; use ksb::DependencyResolver; my $graph1 = { 'a' => { votes => { 'b' => 1, 'd' => 1 }, build => 1, - module => 'a' + module => { + name => 'a', + '#create-id' => 1, + }, }, 'b' => { votes => {}, build => 1, - module => 'b' + module => { + name => 'b', + '#create-id' => 1, + }, }, 'c' => { votes => { 'd' => 1 }, build => 1, - module => 'c' + module => { + name => 'c', + '#create-id' => 1, + }, }, 'd' => { votes => {}, build => 1, - module => 'd' - } + module => { + name => 'd', + '#create-id' => 1, + }, + }, + 'e' => { + votes => {}, + build => 1, + module => { + name => 'e', + '#create-id' => 2, # Should come after everything else + }, + }, }; -my @expected1 = ('a', 'c', 'b', 'd'); -my @actual1 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph1); +my @expected1 = map { $graph1->{$_}->{module} } ('a', 'c', 'b', 'd', 'e'); +my @actual1 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph1); is_deeply(\@actual1, \@expected1, "should sort modules into the proper build order"); # use some random key strokes for names: # unlikely to yield keys in equivalent order as $graph1: key order *should not matter* my $graph2 = { - 'avdnrvrl' => { - votes => { - 'd' => 1 - }, - build => 1, - module => 'c' - }, - 'lexical1' => { - votes => {}, - build => 1, - module => 'b' - }, - 'nllfmvrb' => { - votes => { - 'b' => 1, - 'd' => 1 - }, - build => 1, - module => 'a' - }, - 'lexical2' => { - votes => {}, - build => 1, - module => 'd' - } + 'avdnrvrl' => $graph1->{c}, + 'lexical1' => $graph1->{b}, + 'lexicla3' => $graph1->{e}, + 'nllfmvrb' => $graph1->{a}, + 'lexical2' => $graph1->{d}, }; -my @expected2 = ('a', 'c', 'b', 'd'); -my @actual2 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph2); +# corresponds to same order as the test above +my @expected2 = map { $graph2->{$_}->{module} } qw(nllfmvrb avdnrvrl lexical1 lexical2 lexicla3); +my @actual2 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph2); is_deeply(\@actual2, \@expected2, "key order should not matter for build order"); my $graph3 = { - 'a' => { - votes => { - 'b' => 1, - 'd' => 1 - }, - build => 0, - module => 'a' - }, - 'b' => { - votes => {}, - build => 1, - module => undef - }, - 'c' => { - votes => { - 'd' => 1 - }, - build => 1, - module => 'c' - }, - 'd' => { - votes => {}, - build => 1, - module => 'd' - } + 'a' => $graph1->{a}, + 'b' => $graph1->{b}, + 'c' => $graph1->{c}, + 'd' => $graph1->{d}, + 'e' => $graph1->{e}, }; +$graph3->{a}->{build} = 0; +$graph3->{b}->{module} = undef; # Empty module blocks should be treated as build == 0 -my @expected3 = ('c', 'd'); -my @actual3 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph3); +my @expected3 = map { $graph3->{$_}->{module} } ('c', 'd', 'e'); +my @actual3 = ksb::DependencyResolver::sortModulesIntoBuildOrder($graph3); is_deeply(\@actual3, \@expected3, "modules that are not to be built should be omitted"); done_testing();