import new data based on appstream appdata

Description

import new data based on appstream appdata

the overall plan here is to eventually autogenerate the entire
data set on a regular basis through a jenkins job.
to allow for this to happen all data is now detangled from the
rest of the website and only used here. if someone uses the new
data that's their problem. the data here is only meant to stay
stable for as long as the applications are here and have their
appstream appid. input welcome on how to move forward.

the old data wasn't crawled automatically, it had no proper format
definition and usually wasn't kept up to date.
by relying on appdata (which we need to maintain anyway for discover
and gnome-software) our data has a much better chance of staying
up-to-date on all fronts.
with future expansion to the appdata format we'll also be able to
render additional UI if desired (such as version tables with
links to tarballs/exes/dmgs etc)

changes at large are as follows

  • switches from AppData class to AppData2
  • new AppData2 uses appstream based data blobs
  • AppData2 properly localizes based on appdata's localized strings (doesn't apply to screenshots, because I have some size concerns and also there don't seem to be [m]any)
  • frontend php was adjusted for new paths and got some additional cleanup but visually stays largely the same
  • old data still here for backwards compat, no clue where else AppData is used
  • new crawler in a git repo
  • icons are now in icons/ to detangle them from icons used elsewhere on the websites
  • icons are now preferrably svg, they are picked out of the breeze theme when available for consistent visuals with the desktop
  • thumbnails similarly are in thumbnails/
  • all assets are generally identified by their appstream appid
  • new compatibility system to keep old names working: old names are symlinked to their new appids. using this approach we should also be able to retain appid stability (not implemented as I can't get that generated into yaml as of right now)

various more advanced features of our own data set disappeared now,
while the data is generally kept for pre existing apps I still have
to talk to @mak about how to best deal with this for new data.
appstream should allow us to define random entries, those seem to not
get reflected into the yaml though.

app coverage is roughly the same with some long gone apps disappearing
and some newer ones appearing.
some however are currently missing due to limitations with how crawling
works. unfortunately not all may be easy to resolve as apps that aren't
covered by CI are sometimes impossible to properly build appdata for
(e.g. when the desktop file is cmake generated or no icon can be found
in breeze)

CCMAIL: kde-www@kde.org

Details

Committed
sitterMar 9 2018, 3:37 PM
Parents
R883:1512079: SVN_SILENT Automatic fixes for Norwegian Bokmål.
Branches
Unknown
Tags
Unknown
mak added a comment.Apr 27 2018, 6:09 PM

@sitter How do you generate this data? All fields from the custom tag should end up in the YAML, if raw libappstream is used. If appstream-generator is used, the custom entries that should end up in the final output need to be whitelisted via the AllowedCustomKeys list field in the asgen configuration file.