Iron Hand
Overall Idea
This is a plan to improve KA.
Right now in KA we have the ppa-build-status script, the very new ka-graph script and kubuntu-retry-builds which recently added support to display graphs pointing the packages being rebuilt. These 3 scripts have something in common: they need to retrieve the information from launchpad for a complete package set of frameworks/plasma/apps; getting this information takes a lot of time, so this ongoing operation called "Iron Hand" is about providing a program called 'ka-iron-hand' which would retrieve the information for Launchpad and would be able to do any or all of these things:
- Draw graphs (proposed-migration, build status, general status)
- Render status webpages, like that one provided by ppa-build-status
- Retry builds, providing logs in text and graphs (kubuntu-retry-builds added support for this recently)
- Provide a filtered update-excuses webpage showing only KDE packages
- Provide an installability status report
Current status
- Drawing graphs : the retry-builds graph doesn't work yet and we don't have the general status ones
- Render status webpages: nothing done yet
- Retry builds: nothing done yet
- Provide a filtered update-excuses webpage: created a 'grep-excuses' module; it could be used both for this feature and to improve the proposed-migration graph
- Provide an installability status report: nothing done yet
Implemetation progress
DONE:
- Design and create the configuration system: file structure and code to read it
- Create an initial implementation of class KAIronHand, supporting graph drawing
- Provide a webpage with the current ka-iron-hand reports: http://gpul.grupos.udc.es/ka-iron-hand_reports/
- Complete the default configuration - they are some missing profiles
- Add logging feature - the text file log must be configurable
- Allow to consider only a specific set of architectures
- Import the build status page of tritemio in KA (this is also related to the Kraken operation)
- Create a grep_excuses module to parse the excuses webpage. As a side effect there's now in KA a program called 'ka-grep-excuses' which would be useful to test this module.
- The proposed migration graph suports "update excuses" states
- Add a page showing what versions we have where - see: http://gpul.grupos.udc.es/ka-iron-hand_reports/version_status.html
TODO:
- Add support for retrying builds to ka-iron-hand (also note that the current retry_builds graphs are an alias for build_status ones, this must be fixed in the future)
- Create a KABuildLogParser class which would be used by the general status graphs
- Start to use the KABuildLogParser for the tritemio build status page to test ths code with real data (this is also related to the Kraken operation)
- Use KAIronHand in ka-graph and kubuntu-retry-builds, de-duplicating code to login into launchpad, retrieve package statuses and draw graphs
- Use jinja templates for the tritemio build status page, so we could then test and re-use it for the Iron Hand.
- KCI graphs? http://kci.pangea.pub/api/json
- Use the cache for api.launchpad.net, see what ppa-build-status does
Graphs issues/wishlist
Mark the packages with autopkgtests in progress in dark blueMark the valid candidates in green- The build status graph still doesn't deal well with binary-NEW packages
The graphs doesn't mark the packages with incorrect versions
Usage
Configuration
The default configuration embedded in the Kubuntu Automation code is in conf/ironhand_defaultrc; you may override these default settings making your own custom configuration file located at ($HOME/.ka-iron-hand.conf)
Running
Once you have configured the program it can be executed just typing 'ka-iron-hand' in your shell; you also may add a line like this to your /etc/crontab if you want to run the program each 15 minutes:
0-59/15 * * * * user /home/user/ka/ka-iron-hand