Create KDE lineage tree
Closed, ResolvedPublic

Description

Inspired by the "Desktop through the ages" task T14620: it could also be nice to look at the lineage tree of the KDE desktop stack. I.e. which parts/technologies/apps begat other ones, which parts did we "lose"?, how far back can we trace existing apps? E.g. xpdf begat kpdf begat okular

Visually, I think of a lineage tree like this one: https://upload.wikimedia.org/wikipedia/commons/3/33/Linux_Distro_Timeline.png, not a linear representation like timeline.kde.org... https://launchpad.net/gnuclad is a tool to generate cladogram trees.

Open questions:

  • How to compile the data and who knows how the KDE apps and technologies inspired and enabled each other? A good starting point should be https://apps.kde.org, but help from the individual project maintainers would be invaluable

Proposals:

johanneszarl updated the task description. (Show Details)Jun 27 2021, 9:02 PM
lydia added a subscriber: ervin.Jul 3 2021, 11:12 AM

currently looking up the historical sound effects on repositories and attic but its very hard to do it remotely. First tried: https://download.kde.org/Attic/2.2.2/src/kdemultimedia-2.2.2.tar.bz2
unpack, no *.ogg files there, hm.

https://invent.kde.org/historical may help.

If someone with direct access to the archives could generate file lists (simple text files listing file paths) of all the archive content available, and put it as a new Genealogy-Repo into https://invent.kde.org/historical group, that would be very helpful!
This could enable us to download the file lists and do further research/analysis and visualize the evolution somehow with latex or similar.

Something like:
https://invent.kde.org/historical/kde-genealogy.git/1.0/src/...
https://invent.kde.org/historical/kde-genealogy.git/2.2.2/src/kdemultimedia-2.2.2.tar.bz2.list
https://invent.kde.org/historical/kde-genealogy.git/3.0/src/.../kdemultimedia-3.0.tar.bz2.list
https://invent.kde.org/historical/kde-genealogy.git/4.0/src/.../kdemultimedia-4.0.tar.bz2.list
https://invent.kde.org/historical/kde-genealogy.git/5.0/...
would be awesome!

but to download all the archives is currently not possible for me. Who are the longterm gatherers & collectors?
@valorie once said in an interview that she is very into geneology!

PS: or do you have a file index of all the KDE-related historical files somewhere? It's funny I have to think of the Semantic Desktop Vision of KDE4

lydia added a subscriber: lydia.

Is the above something sysadmin can help with?

Restricted Application added a subscriber: sysadmin. · View Herald TranscriptSep 1 2021, 8:59 PM
lydia added a comment.Sep 1 2021, 9:00 PM

Maybe the Internet Archive has something?

If someone could write a script to create the above then I can certainly look at running it on a copy of download.kde.org.
(Bear in mind we likely have tar.gz, tar.bz2 and in more recent times tar.xz archives on download.kde.org)

Note that the contents of download.kde.org can be accessed via rsync via rsync.kde.org::kdeftp

domson added a comment.EditedSep 2 2021, 9:47 AM

@lydia thanks! I'll have a look into internet archive, but I am afraid they don't index compressed file archive content. But for further research its great! they have kde-files.org back to 2005 and kde-apps to 2004, kde.org even back to 1998 in the wayback machine. So reconstructing the kde evolution will surely need manual investigation beyond the source code history I guess.

@bcooksley thanks for redirecting me further! I prepare a shell script to run over all archives depending on archive type (gz, bz2, xz) to add an additional [archive-name].[archive-extension].tar-tv in the same directory. This enables me to rsync all *.tar-tv files to do further research. Do you have a repository for such helper scripts?

@bcooksley furthermore I could try to prepare a script to simulate the kde (v2-v4) evolution within a single generated git-repository, so that every version change will become a git-merge at a certain point in time (archive date) The challenge will be to determine path renamings or even file merges between versions, maybe this can be solved via "approximate matching algorithms". But the more I think about it, the more it seems like a huge investigative task. Nevertheless such a simulated git-repo can still (even if its partly inconsistent in terms of merge history) help to analyse the project structure changes over time by listing the git history.

As far as I can remember the biggest architectural changes happened between 4 and kf5. the successive change towards git versioning happened during kf5 migration right? Anyways first things first.

EDIT: @sandsmark you build the KDE/QT History Group https://invent.kde.org/historical, do you have advice to progress further? Should we use the last stable version of each KDE major release? Like eg KDE 3.10? Do you aim for modern-system-compatible KDE versions? Great opportunity to learn some history btw.

domson updated the task description. (Show Details)Sep 2 2021, 9:50 AM

@domson Please make it write the lists to an alternative tree - as the whole of download.kde.org is ~500GB i'd prefer to avoid having to rsync it somewhere else to do this.

domson added a comment.EditedSep 2 2021, 7:49 PM

@bcooksley Ok I added a helper script to download.kde.org. it has some options to define a target directory.

@domson Where did you save the helper script?

domson added a comment.EditedSep 2 2021, 8:09 PM

@bcooksley oh havent you seen my merge request? its in the /bin directory.

domson updated the task description. (Show Details)Sep 3 2021, 7:46 AM
domson updated the task description. (Show Details)
domson added a comment.Sep 3 2021, 6:48 PM

hi @bcooksley,
can you help me out with invent.kde.org? I am not allowed to create a '.gitmodules' file in my repository.
is my script contribution for the lists in the right place?

As mentioned in the other task (got a bit of overlap here) the listing has now been generated as requested.

Removing Sysadmin as I believe we've provided everything needed from our side here.

lydia added a comment.Sep 26 2021, 1:51 PM

Checking in: Do you think we will have something nice by the time of the anniversary? It would be good to know so we can include it in the landing page and social media planning as appropriate.

paulb closed this task as Resolved.Jan 12 2024, 11:50 AM