diff --git a/doc/source-reference/ksb/Module.adoc b/doc/source-reference/ksb/Module.adoc index e4e7576..1dd1d60 100644 --- a/doc/source-reference/ksb/Module.adoc +++ b/doc/source-reference/ksb/Module.adoc @@ -1,209 +1,216 @@ = ksb::Module == DESCRIPTION This is ksb::Module, one of the core classes within kdesrc-build. It represents any single "buildable" module that kdesrc-build can manage. It acts as a common interface to the multiple types of build systems and source control management systems that kdesrc-build supports. The many options available to the user are managed using setOption/getOption (but see also the ksb::OptionsBase class that this derives from). kdesrc-build manages persistent metadata for each module as well, see {set,get}PersistentOption == METHODS The basic description of each method is listed here for ease of reference. See the source code itself for more detail. === Perl integration These functions are used to integrate into the Perl runtime or for use from other Perl modules. * ``new``, creates a new ksb::Module, and sets any provided options. * ``toString``, for "stringifying" a module into a quoted string. * ``compare``, for sorting ksb::Modules amongst each other based on name. === CONFIGURATION These functions are used to configure what the ksb::Module object should do, change settings, etc. * ``setModuleSet``, optional, specifies the ksb::ModuleSet this module was spawned from. * ``setScmType``, sets the source control plugin (git, svn, kde-projects) based on the given scmType name. Normally auto-detection is used instead, this permits manual setup. * ``buildSystemFromName``, as with ``setScmType``, used to manually set the build system plugin. This is exposed to the user as *override-build-system*. * ``setBuildSystem``, like ``buildSystemFromName``, but passes the proper ksb::BuildSystem directly. * ``setOption``, sets a configuration option that can be checked later using getOption. Normally set from user input (cmdline or rc-file) but supports ways for kdesrc-build to internally override user settings or set hidden flags for action in later phases. Does not survive beyond the current execution. * ``setPersistentOption``, sets an option to a string value that will be read-in again on the next kdesrc-build run and can then be queried again. * ``unsetPersistentOption``, removes an existing persistent option. === INTROSPECTION These functions are generally just read-only accessors of information about the object. ==== BASIC INFORMATION * ``name``, returns the module name. Only one module with a given name can be present during a build. * ``buildContext``, returns the ksb::BuildContext (as set when the object was constructed) * ``phases``, returns the list of execution phases (update, buildsystem, test, etc.) that apply to this module in this execution. * ``moduleSet``, returns the ksb::ModuleSet that was assigned earlier as the source set. If no module set was assigned, returns a valid (but null) set. ==== PLUGIN HANDLERS * ``scm``, **autodetects** the appropriate scm plugin if not already done (or manually set), and then returns the ksb::Updater plugin. * ``buildSystem``, **autodetects** the appropriate build system plugin if not already done (or manually set) and then returns the ksb::BuildSystem|ksb/BuildSystem.pm plugin. * ``scmType``, returns the **name** of the scm plugin (as determined by scm(), which can itself cause an autodetection pass. * ``buildSystemType``, returns the **name** of the build system plugin (as determined by buildSystem(), which can itself cause an autodetection pass. * ``currentScmRevision``, returns a string with an scm-specific revision ID. Can be a Git-style SHA, SVN-style sequential ID, or something else entirely. Can case an autodetection of the scm plugin. ==== PATHS Various path-handling functions. These aren't always easy to tell what they do just from the method name, sadly. * ``getSubdirPath``, maps a path from the rc-file (based on the option-name to pass to getOption) to a potential absolute path (handling tilde expansion and relative paths). Does not handle colon-separated paths. * ``getInstallPathComponents``, returns information about the directory the module should be installed to. See the detailed docs for this method at its decl, but generally you can just call fullpath today. * ``getSourceDir``, returns absolute base path to the source directory (not including dest-dir, module name, or anything else specific to this module). * ``getLogDir``, returns the base path to use for logs for this module during this execution. **NOTE** Different modules can have different base paths. * ``getLogPath``, returns the absolute filename to open() for a log file for this module based on the given basename. Updates the 'latest' symlink, which can trigger clean up of old log dirs after all modules are built. Only use when you're really going to open a log file! * ``fullpath``, returns the absolute full path to the source or build directory, including any module name or dest-dir accoutrement. This is the directory you can git-clone to, cd to for build, etc. * ``destDir``, returns the 'dest-dir' for the module. dest-dir is effectively just a way to modify the on-disk module name. It used to be used more heavily to allow for having multiple build/source directories for a given CVS/SVN module (varying by branch or tag), but even with git this value may change for KDE-based repositories to set subdirectories that match KDE project paths. Supports expanding '$MODULE' or '${MODULE}' sequences to what otherwise would have been the dest-dir. * ``installationPath``, as labeled on the tin. Prefers the 'prefix' option but falls back to 'kdedir' if not set. ==== USER AND PERSISTENT OPTIONS * ``getOption``, returns the value of the given named option. If no such option exists, inherits the same value from the module's build context. If no such option exists there either, returns an empty string. Option values are used by this function only exist during this script's execution. There is magic to permit build jobs that run in a subprocess to feed option changes back to the parent process. * accepts an option name, normally as set in the rc-file. Can also accept a second parameter 'module', to prevent falling back to a global option. However doing this also permits ``undef`` to be returned so you must check whether the result is defined. * Options starting with '#' can only be set internally (i.e. not from rc-file or cmdline) so this can be used as a way to tag modules with data meant not to be user-accessible... but this should probably be factored into a dedicated parallel option stack. * The combination of module-specific and global options also contains a wee bit of magic to control things like whether option values combine ("$global-value $module-value" style) or whether a module setting completely masks a global setting. * ``getPersistentOption``, similar to ``getOption``, only without the module/global magic and the append/mask magic, and the subprocess-support magic. But this function can return options that have been set in a previous kdesrc-build run. kdesrc-build uses the location of the rc-file to determine where to look for data from prior runs. ==== KDE-SPECIFIC HANDLERS * ``fullProjectPath``, returns the logical module path in the git.kde.org infrastructure for the module, if it's defined from a kde-projects module set. E.g. for the 'juk' module, would return 'kde/kdemultimedia/juk'. * ``isKDEProject``, returns true if the module was sourced from the special ``kde-projects`` module set in the user's rc-file. In this case the module's ``moduleSet()`` function should return a ksb::ModuleSet that is-a ksb::ModuleSet::KDEProjects. === OPERATIONS * ``update``, which executes the update (or pretends to do so) using the appropriate source control system and returns a true/false value reflecting success. Note this can also throw exceptions and future code is moving more to this mode of error-handling. * ``build``, which executes the build **and** install (or pretends to in pretend mode) using the appropriate build system and returns a true/false value reflecting success. Can also run the testsuite as part of the build. Note this can also throw exceptions and future code is moving more to this as the error-handling mechanism. * ``setupBuildSystem``, which sets up the build system for the module to permit ``build`` to work, including creating build dir, running cmake/configure/etc. as appropriate. It is called automatically but will not take any action if the build system is already established. * ``install``, which installs (or pretends to install) the module. Called automatically by ``build``. * ``uninstall``, which uninstalls (or pretends to uninstall) the module. Not normally called but can be configured to be called. * ``applyUserEnvironment``, this adds ``set-env`` module-specific environment variable settings into the module's build context, called by ``setupEnvironment``. This is needed since $ENV is not actually updated by ksb::BuildContext until after a new child process is ``fork``'ed. * ``setupEnvironment``, called by the kdesrc-build build driver, running in a subprocess, before calling the appropriate update/build/install etc. method. + +* ``getPostBuildMessages``, which returns a list of messages intended to be shown + to the user at the end of the build because they are so important that they should + not be missed. These should be used lightly, if at all. + +* ``addPostBuildMessage``, which pairs with ``getPostBuildMessages`` to add a message + to show to the user at the end of the build. diff --git a/modules/ksb/IPC.pm b/modules/ksb/IPC.pm index ec7ba2b..c71a1e0 100644 --- a/modules/ksb/IPC.pm +++ b/modules/ksb/IPC.pm @@ -1,403 +1,432 @@ package ksb::IPC; # Handles the asynchronous communications needed to perform update and build # processes at the same time. This can be thought of as a partially-abstract # class, really you should use IPC::Null (which is fully synchronous) or # IPC::Pipe, which both fall back to common methods implemented here. use strict; use warnings; use 5.014; no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; our $VERSION = '0.20'; use ksb::BuildException; # make_exception use ksb::Debug; use ksb::Util; # list_has # IPC message types use constant { MODULE_SUCCESS => 1, # Used for a successful src checkout MODULE_FAILURE => 2, # Used for a failed src checkout MODULE_SKIPPED => 3, # Used for a skipped src checkout (i.e. build anyways) MODULE_UPTODATE => 4, # Used to skip building a module when had no code updates # One of these messages should be the first message placed on the queue. ALL_SKIPPED => 5, # Used to indicate a skipped update process (i.e. build anyways) ALL_FAILURE => 6, # Used to indicate a major update failure (don't build) ALL_UPDATING => 7, # Informational message, feel free to start the build. # Used to indicate specifically that a source conflict has occurred. MODULE_CONFLICT => 8, MODULE_LOGMSG => 9, # Tagged message should be put to TTY for module. MODULE_PERSIST_OPT => 10, # Change to a persistent module option ALL_DONE => 11, # Affirmatively flags that all updates are done + + MODULE_POSTBUILD_MSG => 12, # A message to print after all work done }; sub new { my $class = shift; my $defaultOpts = { no_update => 0, updated => { }, logged_module => 'global', messages => { }, # Holds log output from update process + postbuild_msg => { }, # Like above but for post-build msgs updates_done => 0, opt_update_handler => undef, # Callback for persistent option changes }; # Must bless a hash ref since subclasses expect it. return bless $defaultOpts, $class; } # Sends a message to the main/build process that a persistent option for the # given module name must be changed. For use by processes that do not control # the persistent option store upon shutdown. sub notifyPersistentOptionChange { my $self = shift; my ($moduleName, $optName, $optValue) = @_; $self->sendIPCMessage(ksb::IPC::MODULE_PERSIST_OPT, "$moduleName,$optName,$optValue"); } +# Sends a message to the main/build process that a given message should be +# shown to the user at the end of the build. +sub notifyNewPostBuildMessage +{ + my ($self, $moduleName, $msg) = @_; + + $self->sendIPCMessage(ksb::IPC::MODULE_POSTBUILD_MSG, "$moduleName,$msg"); +} + sub notifyUpdateSuccess { my $self = shift; my ($module, $msg) = @_; $self->sendIPCMessage(ksb::IPC::MODULE_SUCCESS, "$module,$msg"); } # Sets which module messages stored by sendLogMessage are supposed to be # associated with. sub setLoggedModule { my ($self, $moduleName) = @_; $self->{logged_module} = $moduleName; } # Sends a message to be logged by the process holding the TTY. # The logged message is associated with the module set by setLoggedModule. sub sendLogMessage { my ($self, $msg) = @_; my $loggedModule = $self->{logged_module}; $self->sendIPCMessage(MODULE_LOGMSG, "$loggedModule,$msg"); } # Prints the given message out (adjusting to have proper whitespace # if needed). For use with the log-message forwarding facility. sub _printLoggedMessage { my $msg = shift; $msg = "\t$msg" unless $msg =~ /^\s+/; ksb::Debug::print_clr($msg); } +# Called any time we're waiting for an IPC message from a sub process. This can +# occur during a module build (waiting for messages from update process) or +# while we're near the end of the script execution. There is no way to tell +# which module we'll be about to receive messages for from the other end. sub _updateSeenModulesFromMessage { my ($self, $ipcType, $buffer) = @_; my $updated = $self->{'updated'}; my $messagesRef = $self->{'messages'}; my $message; croak_runtime("IPC failure: no IPC mechanism defined") unless $ipcType; given ($ipcType) { when (ksb::IPC::MODULE_SUCCESS) { my ($ipcModuleName, $msg) = split(/,/, $buffer); $message = $msg; $updated->{$ipcModuleName} = 'success'; } when (ksb::IPC::MODULE_SKIPPED) { # The difference between success here and 'skipped' below # is that success means we should build even though we # didn't perform an update, while 'skipped' means the # *build* should be skipped even though there was no # failure. $message = 'skipped'; $updated->{$buffer} = 'success'; } when (ksb::IPC::MODULE_CONFLICT) { $message = 'conflicts present'; $updated->{$buffer} = 'failed'; } when (ksb::IPC::MODULE_FAILURE) { $message = 'update failed'; $updated->{$buffer} = 'failed'; } when (ksb::IPC::MODULE_UPTODATE) { # Although the module source hasn't changed, the user might be forcing a # rebuild, so our message should reflect what's actually going to happen. $message = 'no files affected'; my ($ipcModuleName, $refreshReason) = split(',', $buffer); if ($refreshReason) { $updated->{$ipcModuleName} = 'success'; note ("\tNo source update, but $refreshReason"); } else { $updated->{$ipcModuleName} = 'skipped'; } } when (ksb::IPC::MODULE_PERSIST_OPT) { my ($ipcModuleName, $optName, $value) = split(',', $buffer); if ($self->{opt_update_handler}) { # Call into callback to update persistent options $self->{opt_update_handler}->($ipcModuleName, $optName, $value); } } when (ksb::IPC::MODULE_LOGMSG) { my ($ipcModuleName, $logMessage) = split(',', $buffer, 2); # Save it for later if we can't print it yet. $messagesRef->{$ipcModuleName} //= [ ]; push @{$messagesRef->{$ipcModuleName}}, $logMessage; } when (ksb::IPC::ALL_DONE) { $self->{updates_done} = 1; } + when (ksb::IPC::MODULE_POSTBUILD_MSG) { + my ($ipcModuleName, $postBuildMsg) = split(',', $buffer, 2); + + $self->{postbuild_msg}->{$ipcModuleName} //= [ ]; + push @{$self->{postbuild_msg}->{$ipcModuleName}}, $postBuildMsg; + } default { croak_internal("Unhandled IPC type: $ipcType"); } }; return $message; } # Used to assign a callback / subroutine to use for updating persistent # options based on IPC update messages. The sub should itself take a # key and value pair. sub setPersistentOptionHandler { my ($self, $handler) = @_; $self->{opt_update_handler} = $handler; } sub waitForEnd { - my ($self, $module) = @_; + my $self = shift; $self->waitForStreamStart(); while(!$self->{no_update} && !$self->{updates_done}) { my $buffer; my $ipcType = $self->receiveIPCMessage(\$buffer); # We ignore the return value in favor of ->{updates_done} $self->_updateSeenModulesFromMessage($ipcType, $buffer); } } # Waits for an update for a module with the given name. # Returns a list containing whether the module was successfully updated, # and any specific string message (e.g. for module update success you get # number of files affected) # Will throw an exception for an IPC failure or if the module should not be # built. sub waitForModule { my ($self, $module) = @_; my $moduleName = $module->name(); my $updated = $self->{'updated'}; # Wait for the initial phase to complete, if it hasn't. $self->waitForStreamStart(); # No update? Just mark as successful if ($self->{'no_update'} || !$module->phases()->has('update')) { $updated->{$moduleName} = 'success'; return ('success', 'Skipped'); } my $message; while(! defined $updated->{$moduleName} && !$self->{updates_done}) { my $buffer; my $ipcType = $self->receiveIPCMessage(\$buffer); $message = $self->_updateSeenModulesFromMessage($ipcType, $buffer); } # If we have 'global' messages they are probably for the first module and # include standard setup messages, etc. Print first and then print module's # messages. my $messagesRef = $self->{'messages'}; for my $item ('global', $moduleName) { for my $msg (@{$messagesRef->{$item}}) { _printLoggedMessage($msg); } delete $messagesRef->{$item}; } + # We won't print post-build messages now but we need to save them for when + # they can be printed. + for my $msg (@{$self->{postbuild_msg}->{$moduleName}}) { + $module->addPostBuildMessage($msg); + } + delete $self->{postbuild_msg}->{$moduleName}; + return ($updated->{$moduleName}, $message); } # Just in case we somehow have messages to display after all modules are # processed, we have this function to show any available messages near the end # of the script run. sub outputPendingLoggedMessages { my $self = shift; my $messages = $self->{messages}; while (my ($module, $logMessages) = each %{$messages}) { my @nonEmptyMessages = grep { !!$_ } @{$logMessages}; if (@nonEmptyMessages) { debug ("Unhandled messages for module $module:"); ksb::Debug::print_clr($_) foreach @nonEmptyMessages; } } $self->{messages} = { }; } # Flags the given module as something that can be ignored from now on. For use # after the module has been waited on sub forgetModule { my ($self, $module) = @_; my $modulename = $module->name(); delete $self->{'updated'}->{$modulename}; } # Returns a hashref mapping module *names* to update statuses, for modules that # have not already been marked as ignorable using forgetModule() sub unacknowledgedModules { my $self = shift; return $self->{'updated'}; } # Waits on the IPC connection until one of the ALL_* IPC codes is returned. # If ksb::IPC::ALL_SKIPPED is returned then the 'no_update' entry will be set in # $self to flag that you shouldn't wait. # If ksb::IPC::ALL_FAILURE is returned then an exception will be thrown due to the # fatal error. # This method can be called multiple times, but only the first time will # result in a wait. sub waitForStreamStart { my $self = shift; state $waited = 0; return if $waited; my $buffer = ''; my $ipcType = 0; $waited = 1; while ($ipcType != ksb::IPC::ALL_UPDATING) { $ipcType = $self->receiveIPCMessage(\$buffer); if (!$ipcType) { croak_internal("IPC Failure waiting for stream start :( $!"); } if ($ipcType == ksb::IPC::ALL_FAILURE) { croak_runtime("Unable to perform source update for any module:\n\t$buffer"); } elsif ($ipcType == ksb::IPC::ALL_SKIPPED) { $self->{'no_update'} = 1; $self->{'updates_done'} = 1; } elsif ($ipcType == ksb::IPC::MODULE_LOGMSG) { my ($ipcModuleName, $logMessage) = split(',', $buffer); $self->{messages}->{$ipcModuleName} //= [ ]; push @{$self->{messages}->{$ipcModuleName}}, $logMessage; } elsif ($ipcType != ksb::IPC::ALL_UPDATING) { croak_runtime("IPC failure while expecting an update status: Incorrect type: $ipcType"); } } } # Sends an IPC message along with some IPC type information. # # First parameter is the IPC type to send. # Second parameter is the actual message. # All remaining parameters are sent to the object's sendMessage() # procedure. sub sendIPCMessage { my ($self, $ipcType, $msg) = @_; my $encodedMsg = pack("l! a*", $ipcType, $msg); return $self->sendMessage($encodedMsg); } # Static class function to unpack a message. # # First parameter is the message. # Second parameter is a reference to a scalar to store the result in. # # Returns the IPC message type. sub unpackMsg { my ($msg, $outBuffer) = @_; my $returnType; ($returnType, $$outBuffer) = unpack("l! a*", $msg); return $returnType; } # Receives an IPC message and decodes it into the message and its # associated type information. # # First parameter is a *reference* to a scalar to hold the message contents. # All remaining parameters are passed to the underlying receiveMessage() # procedure. # # Returns the IPC type, or undef on failure. sub receiveIPCMessage { my $self = shift; my $outBuffer = shift; croak_internal("Trying to pull message from closed IPC channel!") if $self->{updates_done}; my $msg = $self->receiveMessage(); return ($msg ? unpackMsg($msg, $outBuffer) : undef); } # These must be reimplemented. They must be able to handle scalars without # any extra frills. # # sendMessage should accept one parameter (the message to send) and return # true on success, or false on failure. $! should hold the error information # if false is returned. sub sendMessage { croak_internal("Unimplemented."); } # receiveMessage should return a message received from the other side, or # undef for EOF or error. On error, $! should be set to hold the error # information. sub receiveMessage { croak_internal("Unimplemented."); } # Should be reimplemented if default does not apply. sub supportsConcurrency { return 0; } # Should be reimplemented if default does not apply. sub close { } 1; diff --git a/modules/ksb/Module.pm b/modules/ksb/Module.pm index bac7202..0998fff 100644 --- a/modules/ksb/Module.pm +++ b/modules/ksb/Module.pm @@ -1,993 +1,1016 @@ package ksb::Module 0.20; # Class: Module # # Represents a source code module of some sort, which can be updated, built, # and installed. Includes a stringifying overload and can be sorted amongst # other ksb::Modules. use 5.014; use warnings; no if $] >= 5.018, 'warnings', 'experimental::smartmatch'; use parent qw(ksb::OptionsBase); use ksb::IPC; use ksb::Debug; use ksb::Util; use ksb::l10nSystem; use ksb::Updater::Svn; use ksb::Updater::Git; use ksb::Updater::Bzr; use ksb::Updater::KDEProject; use ksb::Updater::KDEProjectMetadata; use ksb::Updater::Qt5; use ksb::BuildException 0.20; use ksb::BuildSystem 0.30; use ksb::BuildSystem::Autotools; use ksb::BuildSystem::QMake; use ksb::BuildSystem::Qt4; use ksb::BuildSystem::Qt5; use ksb::BuildSystem::KDE4; use ksb::BuildSystem::CMakeBootstrap; use ksb::BuildSystem::Meson; use ksb::ModuleSet::Null; use Storable 'dclone'; use Carp 'confess'; use Scalar::Util 'blessed'; use overload '""' => 'toString', # Add stringify operator. '<=>' => 'compare', ; sub new { my ($class, $ctx, $name) = @_; croak_internal ("Empty ksb::Module constructed") unless $name; my $self = ksb::OptionsBase::new($class); # If building a BuildContext instead of a ksb::Module, then the context # can't have been setup yet... my $contextClass = 'ksb::BuildContext'; if ($class ne $contextClass && (!blessed($ctx) || !$ctx->isa($contextClass))) { croak_internal ("Invalid context $ctx"); } # Clone the passed-in phases so we can be different. my $phases = dclone($ctx->phases()) if $ctx; my %newOptions = ( name => $name, scm_obj => undef, build_obj => undef, phases => $phases, context => $ctx, 'module-set' => undef, + post_build_msgs => [ ], ); @{$self}{keys %newOptions} = values %newOptions; return $self; } sub phases { my $self = shift; return $self->{phases}; } sub moduleSet { my ($self) = @_; $self->{'module-set'} //= ksb::ModuleSet::Null->new(); return $self->{'module-set'}; } sub setModuleSet { my ($self, $moduleSet) = @_; assert_isa($moduleSet, 'ksb::ModuleSet'); $self->{'module-set'} = $moduleSet; } # Subroutine to retrieve a subdirectory path with tilde-expansion and # relative path handling. # The parameter is the option key (e.g. build-dir or log-dir) to read and # interpret. sub getSubdirPath { my ($self, $subdirOption) = @_; my $dir = $self->getOption($subdirOption); # If build-dir starts with a slash, it is an absolute path. return $dir if $dir =~ /^\//; # Make sure we got a valid option result. if (!$dir) { confess ("Reading option for $subdirOption gave empty \$dir!"); } # If it starts with a tilde, expand it out. if ($dir =~ /^~/) { $dir =~ s/^~/$ENV{'HOME'}/; } else { # Relative directory, tack it on to the end of $kdesrcdir. my $kdesrcdir = $self->getOption('source-dir'); $dir = "$kdesrcdir/$dir"; } return $dir; } # Method: getInstallPathComponents # # Returns the directory that a module should be installed in. # # NOTE: The return value is a hash. The key 'module' will return the final # module name, the key 'path' will return the full path to the module. The # key 'fullpath' will return their concatenation. # # For example, with $module == 'KDE/kdelibs', and no change in the dest-dir # option, you'd get something like: # # > { # > 'path' => '/home/user/kdesrc/KDE', # > 'module' => 'kdelibs', # > 'fullpath' => '/home/user/kdesrc/KDE/kdelibs' # > } # # If dest-dir were changed to e.g. extragear-multimedia, you'd get: # # > { # > 'path' => '/home/user/kdesrc', # > 'module' => 'extragear-multimedia', # > 'fullpath' => '/home/user/kdesrc/extragear-multimedia' # > } # # Parameters: # pathType - Either 'source' or 'build'. # # Returns: # hash (Not a hashref; See description). sub getInstallPathComponents { my $module = assert_isa(shift, 'ksb::Module'); my $type = shift; my $destdir = $module->destDir(); my $srcbase = $module->getSourceDir(); $srcbase = $module->getSubdirPath('build-dir') if $type eq 'build'; my $combined = "$srcbase/$destdir"; # Remove dup // $combined =~ s/\/+/\//; my @parts = split(/\//, $combined); my %result = (); $result{'module'} = pop @parts; $result{'path'} = join('/', @parts); $result{'fullpath'} = "$result{path}/$result{module}"; my $compatDestDir = $module->destDir($module->name()); my $fullCompatPath = "$srcbase/$compatDestDir"; # We used to have code here to migrate very old directory layouts. It was # removed as of about 2013-09-29. return %result; } # Do note that this returns the *base* path to the source directory, # without the module name or kde_projects stuff appended. If you want that # use subroutine fullpath(). sub getSourceDir { my $self = shift; return $self->getSubdirPath('source-dir'); } sub name { my $self = shift; return $self->{name}; } sub scm { my $self = shift; return $self->{scm_obj} if $self->{scm_obj}; # Look for specific setting of repository and svn-server. If both is # set it's a bug, if one is set, that's the type (because the user says # so...). Don't use getOption($key) as it will try to fallback to # global options. my $svn_status = $self->getOption('svn-server', 'module'); my $repository = $self->getOption('repository', 'module') // ''; my $rcfile = $self->buildContext()->rcFile(); if ($svn_status && $repository) { error (<{scm_obj} = ksb::Updater::Bzr->new($self); } # If it needs a repo it's git. Everything else is svn for now. $self->{scm_obj} //= $repository ? ksb::Updater::Git->new($self) : ksb::Updater::Svn->new($self); return $self->{scm_obj}; } sub setScmType { my ($self, $scmType) = @_; my $newType; given($scmType) { when('git') { $newType = ksb::Updater::Git->new($self); } when('proj') { $newType = ksb::Updater::KDEProject->new($self); } when('metadata') { $newType = ksb::Updater::KDEProjectMetadata->new($self); } when('l10n') { $newType = ksb::l10nSystem->new($self); } when('svn') { $newType = ksb::Updater::Svn->new($self); } when('bzr') { $newType = ksb::Updater::Bzr->new($self); } when('qt5') { $newType = ksb::Updater::Qt5->new($self); } default { $newType = undef; } } $self->{scm_obj} = $newType; } # Returns a string describing the scm platform of the given module. # Return value: 'git' or 'svn' at this point, as appropriate. sub scmType { my $self = shift; return $self->scm()->name(); } sub currentScmRevision { my $self = shift; return $self->scm()->currentRevisionInternal(); } # Returns a new build system object, given the appropriate name. # This is a sub-optimal way to fix the problem of allowing users to override # the detected build system (we could instead use introspection to figure out # available build systems at runtime). However, KISS... sub buildSystemFromName { my ($self, $name) = @_; my %buildSystemClasses = ( 'generic' => 'ksb::BuildSystem', 'qmake' => 'ksb::BuildSystem::QMake', 'cmake-bootstrap' => 'ksb::BuildSystem::CMakeBootstrap', 'kde' => 'ksb::BuildSystem::KDE4', 'qt' => 'ksb::BuildSystem::Qt4', 'qt5' => 'ksb::BuildSystem::Qt5', 'autotools' => 'ksb::BuildSystem::Autotools', 'meson' => 'ksb::BuildSystem::Meson', ); my $class = $buildSystemClasses{lc $name} // undef; return $class->new($self) if ($class); # Past here, no class found croak_runtime("Invalid build system $name requested"); } sub buildSystem { my $self = shift; if ($self->{build_obj} && $self->{build_obj}->name() ne 'generic') { return $self->{build_obj}; } if (my $userBuildSystem = $self->getOption('override-build-system')) { $self->{build_obj} = $self->buildSystemFromName($userBuildSystem); return $self->{build_obj}; } # If not set, let's guess. my $buildType; my $sourceDir = $self->fullpath('source'); if (($self->getOption('repository') =~ /gitorious\.org\/qt\//) || ($self->getOption('repository') =~ /^kde:qt$/) || (-e "$sourceDir/bin/syncqt")) { $buildType = ksb::BuildSystem::Qt4->new($self); } # This test must come before the KDE buildsystem's as cmake's own # bootstrap system also has CMakeLists.txt if (!$buildType && (-e "$sourceDir/CMakeLists.txt") && (-e "$sourceDir/bootstrap")) { $buildType = ksb::BuildSystem::CMakeBootstrap->new($self); } if (!$buildType && (-e "$sourceDir/CMakeLists.txt" || $self->getOption('#xml-full-path'))) { $buildType = ksb::BuildSystem::KDE4->new($self); } # We have to assign to an array to force glob to return all results, # otherwise it acts like a non-reentrant generator whose output depends on # how many times it's been called... if (!$buildType && (my @files = glob ("$sourceDir/*.pro"))) { $buildType = ksb::BuildSystem::QMake->new($self); } # 'configure' is a popular fall-back option even for other build # systems so ensure we check last for autotools. if (!$buildType && (-e "$sourceDir/configure" || -e "$sourceDir/autogen.sh")) { $buildType = ksb::BuildSystem::Autotools->new($self); } # Someday move this up, but for now ensure that Meson happens after # configure/autotools support is checked for. if (!$buildType && -e "$sourceDir/meson.build") { $buildType = ksb::BuildSystem::Meson->new($self); } # Don't just assume the build system is KDE-based... $buildType //= ksb::BuildSystem->new($self); $self->{build_obj} = $buildType; return $self->{build_obj}; } # Sets the build system **object**, although you can find the build system # type afterwards (see buildSystemType). sub setBuildSystem { my ($self, $obj) = @_; assert_isa($obj, 'ksb::BuildSystem'); $self->{build_obj} = $obj; } # Current possible build system types: # KDE (i.e. cmake), Qt, l10n (KDE language buildsystem), autotools (either # configure or autogen.sh). A final possibility is 'pendingSource' which # simply means that we don't know yet. # # If the build system type is not set ('pendingSource' counts as being # set!) when this function is called then it will be autodetected if # possible, but note that not all possible types will be detected this way. # If in doubt use setBuildSystemType sub buildSystemType { my $self = shift; return $self->buildSystem()->name(); } # Subroutine to build this module. # Returns boolean false on failure, boolean true on success. sub build { my $self = assert_isa(shift, 'ksb::Module'); my $moduleName = $self->name(); my %pathinfo = $self->getInstallPathComponents('build'); my $builddir = $pathinfo{'fullpath'}; my $buildSystem = $self->buildSystem(); if ($buildSystem->name() eq 'generic' && !pretending()) { error ("\tr[b[$self] does not seem to have a build system to use."); return 0; } # Ensure we're in a known directory before we start; some options remove # the old build directory that a previous module might have been using. super_mkdir($pathinfo{'path'}); p_chdir($pathinfo{'path'}); return 0 if !$self->setupBuildSystem(); return 1 if $self->getOption('build-system-only'); if (!$buildSystem->buildInternal()) { return 0; } $self->setPersistentOption('last-build-rev', $self->currentScmRevision()); # TODO: This should be a simple phase to run. if ($self->getOption('run-tests')) { $self->buildSystem()->runTestsuite(); } # TODO: Likewise this should be a phase to run. if ($self->getOption('install-after-build')) { return 0 if !$self->install(); } else { info ("\tSkipping install for y[$self]"); } return 1; } # Subroutine to setup the build system in a directory. # Returns boolean true on success, boolean false (0) on failure. sub setupBuildSystem { my $self = assert_isa(shift, 'ksb::Module'); my $moduleName = $self->name(); my $buildSystem = $self->buildSystem(); if ($buildSystem->name() eq 'generic' && !pretending()) { croak_internal('Build system determination still pending when build attempted.'); } my $refreshReason = $buildSystem->needsRefreshed(); if ($refreshReason ne "") { # The build system needs created, either because it doesn't exist, or # because the user has asked that it be completely rebuilt. info ("\tPreparing build system for y[$self]."); # Check to see if we're actually supposed to go through the # cleaning process. if (!$self->getOption('#cancel-clean') && !$buildSystem->cleanBuildSystem()) { warning ("\tUnable to clean r[$self]!"); return 0; } } if (!$buildSystem->createBuildSystem()) { error ("\tError creating r[$self]'s build system!"); return 0; } # Now we're in the checkout directory # So, switch to the build dir. # builddir is automatically set to the right value for qt p_chdir ($self->fullpath('build')); if (!$buildSystem->configureInternal()) { error ("\tUnable to configure r[$self] with " . $self->buildSystemType()); # Add undocumented ".refresh-me" file to build directory to flag # for --refresh-build for this module on next run. See also the # "needsRefreshed" subroutine. if (open my $fh, '>', '.refresh-me') { say $fh "# Build directory will be re-generated next kdesrc-build run"; say $fh "# due to failing to complete configuration on the last run"; close $fh; }; return 0; } return 1; } # Responsible for installing the module (no update, build, etc.) # Return value: Boolean flag indicating whether module installed successfully or # not. # Exceptions may be thrown for abnormal conditions (e.g. no build dir exists) sub install { my $self = assert_isa(shift, 'ksb::Module'); my $builddir = $self->fullpath('build'); my $buildSysFile = $self->buildSystem()->configuredModuleFileName(); if (!pretending() && ! -e "$builddir/$buildSysFile") { warning ("\tThe build system doesn't exist for r[$self]."); warning ("\tTherefore, we can't install it. y[:-(]."); return 0; } $self->setupEnvironment(); my @makeInstallOpts = split(' ', $self->getOption('make-install-prefix')); # We can optionally uninstall prior to installing # to weed out old unused files. if ($self->getOption('use-clean-install') && $self->getPersistentOption('last-install-rev')) { if (!$self->buildSystem()->uninstallInternal(@makeInstallOpts)) { warning ("\tUnable to uninstall r[$self] before installing the new build."); warning ("\tContinuing anyways..."); } else { $self->unsetPersistentOption('last-install-rev'); } } if (!$self->buildSystem()->installInternal(@makeInstallOpts)) { error ("\tUnable to install r[$self]!"); $self->buildContext()->markModulePhaseFailed('install', $self); return 0; } if (pretending()) { pretend ("\tWould have installed g[$self]"); return 1; } # Past this point we know we've successfully installed, for real. $self->setPersistentOption('last-install-rev', $self->currentScmRevision()); my $remove_setting = $self->getOption('remove-after-install'); # Possibly remove the srcdir and builddir after install for users with # a little bit of HD space. if($remove_setting eq 'all') { # Remove srcdir my $srcdir = $self->fullpath('source'); note ("\tRemoving b[r[$self source]."); safe_rmtree($srcdir); } if($remove_setting eq 'builddir' || $remove_setting eq 'all') { # Remove builddir note ("\tRemoving b[r[$self build directory]."); safe_rmtree($builddir); # We're likely already in the builddir, so chdir back to the root p_chdir('/'); } return 1; } # Handles uninstalling this module (or its sub-directories as given by the checkout-only # option). # # Returns boolean false on failure, boolean true otherwise. sub uninstall { my $self = assert_isa(shift, 'ksb::Module'); my $builddir = $self->fullpath('build'); my $buildSysFile = $self->buildSystem()->configuredModuleFileName(); if (!pretending() && ! -e "$builddir/$buildSysFile") { warning ("\tThe build system doesn't exist for r[$self]."); warning ("\tTherefore, we can't uninstall it."); return 0; } $self->setupEnvironment(); my @makeInstallOpts = split(' ', $self->getOption('make-install-prefix')); if (!$self->buildSystem()->uninstallInternal(@makeInstallOpts)) { error ("\tUnable to uninstall r[$self]!"); $self->buildContext()->markModulePhaseFailed('install', $self); return 0; } if (pretending()) { pretend ("\tWould have uninstalled g[$self]"); return 1; } $self->unsetPersistentOption('last-install-rev'); return 1; } sub buildContext { my $self = shift; return $self->{context}; } # Integrates 'set-env' option to the build context environment sub applyUserEnvironment { my $self = assert_isa(shift, 'ksb::Module'); my $ctx = $self->buildContext(); # Let's see if the user has set env vars to be set. # Note the global set-env must be checked separately anyways, so # we limit inheritance when searching. my $env_hash_ref = $self->getOption('set-env', 'module'); while (my ($key, $value) = each %{$env_hash_ref}) { $ctx->queueEnvironmentVariable($key, $value); } } # Establishes proper build environment in the build context. Should be run # before forking off commands for e.g. updates, builds, installs, etc. sub setupEnvironment { my $self = assert_isa(shift, 'ksb::Module'); my $ctx = $self->buildContext(); my $prefix = $self->installationPath(); # Add global set-envs and context $self->buildContext()->applyUserEnvironment(); # Build system's environment injection my $buildSystem = $self->buildSystem(); # # Suppress injecting qtdir/kdedir related environment variables if a toolchain is also set # Let the toolchain files/definitions take care of themselves. # if ($buildSystem->hasToolchain()) { note ("\tNot setting environment variables for b[$self]: a custom toolchain is used"); } else { my $kdedir = $self->getOption('kdedir'); my $qtdir = $self->getOption('qtdir'); # Ensure the platform libraries we're building can be found, as long as they # are not the system's own libraries. for my $platformDir ($qtdir, $kdedir) { next unless $platformDir; # OK, assume system platform is usable next if $platformDir eq '/usr'; # Don't 'fix' things if system platform # manually set $ctx->prependEnvironmentValue('PKG_CONFIG_PATH', "$platformDir/lib/pkgconfig"); $ctx->prependEnvironmentValue('LD_LIBRARY_PATH', "$platformDir/lib"); $ctx->prependEnvironmentValue('PATH', "$platformDir/bin"); } } $buildSystem->prepareModuleBuildEnvironment($ctx, $self, $prefix); # Read in user environment defines $self->applyUserEnvironment() unless $self == $ctx; } # Returns the path to the log directory used during this run for this # ksb::Module, based on an autogenerated unique id. The id doesn't change # once generated within a single run of the script. sub getLogDir { my ($self) = @_; return $self->buildContext()->getLogDirFor($self); } # Returns a full path that can be open()'d to write a log # file, based on the given basename (with extension). # Updates the 'latest' symlink as well, unlike getLogDir # Use when you know you're going to create a new log sub getLogPath { my ($self, $path) = @_; return $self->buildContext()->getLogPathFor($self, $path); } sub toString { my $self = shift; return $self->name(); } sub compare { my ($self, $other) = @_; return $self->name() cmp $other->name(); } sub update { my ($self, $ipc, $ctx) = @_; my $moduleName = $self->name(); my $module_src_dir = $self->getSourceDir(); my $kdesrc = $ctx->getSourceDir(); if ($kdesrc ne $module_src_dir) { # This module has a different source directory, ensure it exists. if (!super_mkdir($module_src_dir)) { error ("Unable to create separate source directory for r[$self]: $module_src_dir"); $ipc->sendIPCMessage(ksb::IPC::MODULE_FAILURE, $moduleName); next; } } my $fullpath = $self->fullpath('source'); my $count; my $returnValue; eval { $count = $self->scm()->updateInternal($ipc) }; if ($@) { my $reason = ksb::IPC::MODULE_FAILURE; if (had_an_exception()) { if ($@->{'exception_type'} eq 'ConflictPresent') { $reason = ksb::IPC::MODULE_CONFLICT; } else { $ctx->markModulePhaseFailed('build', $self); } $@ = $@->{'message'}; } error ("Error updating r[$self], removing from list of packages to build."); error (" > y[$@]"); $ipc->sendIPCMessage($reason, $moduleName); $self->phases()->filterOutPhase('build'); $returnValue = 0; } else { my $message; if (not defined $count) { $message = ksb::Debug::colorize ("b[y[Unknown changes]."); $ipc->notifyUpdateSuccess($moduleName, $message); } elsif ($count) { $message = "1 file affected." if $count == 1; $message = "$count files affected." if $count != 1; $ipc->notifyUpdateSuccess($moduleName, $message); } else { $message = "0 files affected."; my $refreshReason = $self->buildSystem()->needsRefreshed(); $ipc->sendIPCMessage(ksb::IPC::MODULE_UPTODATE, "$moduleName,$refreshReason"); } # We doing e.g. --src-only, the build phase that normally outputs # number of files updated doesn't get run, so manually mention it # here. if (!$ipc->supportsConcurrency()) { info ("\t$self update complete, $message"); } $returnValue = 1; } info (""); # Print empty line. return $returnValue; } # OVERRIDE # # This calls OptionsBase::setOption and performs any Module-specific # handling. sub setOption { my ($self, %options) = @_; # Ensure we don't accidentally get fed module-set options for (qw(git-repository-base use-modules ignore-modules)) { if (exists $options{$_}) { error (" r[b[*] module b[$self] should be declared as module-set to use b[$_]"); die ksb::BuildException::Config->new($_, "Option $_ can only be used in module-set"); }; } # Special case handling. if (exists $options{'filter-out-phases'}) { for my $phase (split(' ', $options{'filter-out-phases'})) { $self->phases()->filterOutPhase($phase); } delete $options{'filter-out-phases'}; } $self->SUPER::setOption(%options); } # OVERRIDE # # This subroutine returns an option value for a given module. Some globals # can't be overridden by a module's choice (but see 2nd parameter below). # If so, the module's choice will be ignored, and a warning will be issued. # # Option names are case-sensitive! # # Some options (e.g. cmake-options, configure-flags) have the global value # and then the module's own value appended together. To get the actual # module setting you must use the level limit parameter set to 'module'. # # Likewise, some qt module options do not obey the previous proviso since # Qt options are not likely to agree nicely with generic KDE buildsystem # options. # # 1st parameter: Name of option # 2nd parameter: Level limit (optional). If not present, then the value # 'allow-inherit' is used. Options: # - allow-inherit: Module value is used if present (with exceptions), # otherwise global is used. # - module: Only module value is used (if you want only global then use the # buildContext) NOTE: This overrides global "sticky" options as well! sub getOption { my ($self, $key, $levelLimit) = @_; my $ctx = $self->buildContext(); $levelLimit //= 'allow-inherit'; # Some global options would probably make no sense applied to Qt. my @qtCopyOverrides = qw(branch configure-flags tag cxxflags); if (list_has(\@qtCopyOverrides, $key) && $self->buildSystemType() eq 'Qt') { $levelLimit = 'module'; } assert_in($levelLimit, [qw(allow-inherit module)]); # If module-only, check that first. return $self->{options}{$key} if $levelLimit eq 'module'; my $ctxValue = $ctx->getOption($key); # we'll use this a lot from here # Some global options always override module options. return $ctxValue if $ctx->hasStickyOption($key); # Some options append to the global (e.g. conf flags) my @confFlags = qw(cmake-options configure-flags cxxflags); if (list_has(\@confFlags, $key) && $ctxValue) { return trimmed("$ctxValue " . ($self->{options}{$key} || '')); } # Everything else overrides the global option, unless it's simply not # set at all. return $self->{options}{$key} // $ctxValue; } # Gets persistent options set for this module. First parameter is the name # of the option to lookup. Undef is returned if the option is not set, # although even if the option is set, the value returned might be empty. # Note that ksb::BuildContext also has this function, with a slightly # different signature, which OVERRIDEs this function since Perl does not # have parameter-based method overloading. sub getPersistentOption { my ($self, $key) = @_; return $self->buildContext()->getPersistentOption($self->name(), $key); } # Sets a persistent option (i.e. survives between processes) for this module. # First parameter is the name of the persistent option. # Second parameter is its actual value. # See the warning for getPersistentOption above, it also applies for this # method vs. ksb::BuildContext::setPersistentOption sub setPersistentOption { my ($self, $key, $value) = @_; return $self->buildContext()->setPersistentOption($self->name(), $key, $value); } # Unsets a persistent option for this module. # Only parameter is the name of the option to unset. sub unsetPersistentOption { my ($self, $key) = @_; $self->buildContext()->unsetPersistentOption($self->name(), $key); } # Returns the path to the desired directory type (source or build), # including the module destination directory itself. sub fullpath { my ($self, $type) = @_; assert_in($type, [qw/build source/]); my %pathinfo = $self->getInstallPathComponents($type); return $pathinfo{'fullpath'}; } # Returns the "full kde-projects path" for the module. As should be obvious by # the description, this only works for modules with an scm type that is a # Updater::KDEProject (or its subclasses), but modules that don't fall into this # hierarchy will just return the module name (with no path components) anyways. sub fullProjectPath { my $self = shift; return ($self->getOption('#xml-full-path', 'module') || $self->name()); } # Returns true if this module is (or was derived from) a kde-projects module. sub isKDEProject { my $self = shift; return $self->hasOption('#xml-full-path'); } # Subroutine to return the name of the destination directory for the # checkout and build routines. Based on the dest-dir option. The return # value will be relative to the src/build dir. The user may use the # '$MODULE' or '${MODULE}' sequences, which will be replaced by the name of # the module in question. # # The first parameter is optional, but if provided will be used as the base # path to replace $MODULE entries in dest-dir. sub destDir { my $self = assert_isa(shift, 'ksb::Module'); my $destDir = $self->getOption('dest-dir'); my $basePath = ""; if ($self->getOption('ignore-kde-structure')) { $basePath = $self->name(); } else { $basePath = shift // $self->getOption('#xml-full-path'); $basePath ||= $self->name(); # Default if not provided in repo-metadata } $destDir =~ s/(\$\{MODULE})|(\$MODULE\b)/$basePath/g; return $destDir; } # Subroutine to return the installation path of a given module (the value # that is passed to the CMAKE_INSTALL_PREFIX CMake option). # It is based on the "prefix" and, if it is not set, the "kdedir" option. # The user may use '$MODULE' or '${MODULE}' in the "prefix" option to have # them replaced by the name of the module in question. sub installationPath { my $self = assert_isa(shift, 'ksb::Module'); my $path = $self->getOption('prefix'); if (!$path) { return $self->getOption('kdedir'); } my $moduleName = $self->name(); $path =~ s/(\$\{MODULE})|(\$MODULE\b)/$moduleName/g; return $path; } +# Returns a list of any 'post-build' messages that have been set for the module +# to show after the build has ended. These may be messages such as warning of a +# local source conflict that may have scrolled past or similar things the user +# needs to know about. +# +# Each entry in the list will be a text message that should be shown (perhaps +# with additional formatting). +sub getPostBuildMessages +{ + my $self = assert_isa(shift, 'ksb::Module'); + return @{$self->{post_build_msgs}}; +} + +# Adds the given message to the list of post-build messages to show to the user +sub addPostBuildMessage +{ + my ($self, $new_msg) = @_; + + push @{$self->{post_build_msgs}}, $new_msg; + return; +} + 1;