Maybe related, maybe not, but I thought today that maybe we should spin our deps build system into a separate repo with stable and unstable branches, and actual releases.

Maybe related, maybe not, but I thought today that maybe we should spin our deps build system into a separate repo with stable and unstable branches, and actual releases.

We should list pros and cons of this split. Right now I see one con:

• [con] Updating the deps will require one more step to link krita repo to the correct dep (perhaps we should use git submodules in the end?). This manual will become one step longer.

I'd actually say that's an advantage... Krita's unstable branch would always checkout the master branch of the deps repo, unstable the unstable branch (so no manual changes needed), and every release would be linked to the correct dependency release. So it's the release manual that gets a bit longer (but I removed the copy_po) step today, so that's fine :P)

Krita's unstable branch would always checkout the master branch of the deps repo

Well, that is a solution but... we have such a discussion before when thinking about the way we link to Qt5 repo. And the result was that we want to link via 'git-sha', because the changes in the Krita repo may be dependent on the changed to in Qt. Though I'm not sure how important it is.

Problems of the split repos:

1. If we split deps, we will have to update Krita's repo somehow with what deps it needs (it is required for complex changes that rely on changes in deps)
2. another question on dependencies, is it not a problem that we are depending more and more on custom (made by contributors) github/lab branches

I guess we will need policies and guidelines for the two cases

If we split deps, we will have to update Krita's repo somehow with what deps it needs (it is required for complex changes that rely on changes in deps)

Well, no. It won't be materially different because right now the deps build and the krita build are completely separate already. They might be in the same repo, but the build systems do not connect.

If we right now rebuild all deps, then kick off a krita build, it would still be the case here. For master, kick off a deps build (or rebuild a single dep) and then kick off a krita build that fetches the latest deps. For stable, ditto, but the latest stable deps.

The only difference is that we can properly release deps and for release builds of krita refer to the right released deps. That's much cleaner and easier to manage.

Well, no. It won't be materially different because right now the deps build and the krita build are completely separate already. They might be in the same repo, but the build systems do not connect.

We can at least see in the commit that "this deps change is related to that Krita change". And to keep that link we would need to manually update the sha1 of the deps needed for the current version of Krita (that is what we do with the Qt repo right now). I mean, it is not about the build we use, but about the build we are expected to use.

The only difference is that we can properly release deps and for release builds of krita refer to the right released deps. That's much cleaner and easier to manage.

Well, the only benefit of multiple repo approach in this case is that we can tag/rebuild deps a few days before the actual tag/release of Krita itself. If we store everything in one repo, we will have to rebuild the same deps again on tagging, even when then didn't change. That might count as a benefit, yet.

Though I would like to check the KDE scripts first. I'm not very sure we will require "rebuild everything" triggers that often...

So, what you're saying is that having deps commits mixed in with Krita commits helps somehow to figure out which dep change was related to which change in Krita. Though, of course, oftentimes, deps change without a corresponding change in Krita's code?

I don't think that's important at all.

We cannot in the current situation know, at all, which deps build we need to use for a certain commit in Krita -- and we cannot even reliably recreate a deps build that would work with that commit, because we can only build against the latest deps build.

This situation is untenable. Having a separate repo for the deps makes for a much cleaner development cycle.

I don't know why you say ""Well, the only benefit of multiple repo approach in this case is that we can tag/rebuild deps a few days before the actual tag/release of Krita itself" -- this is not the case. The real advantage is that we can make releases of dependencies, store the binary artefacts and re-use them when we need to rebuild a release.

I'm sorry, but I don't see any problem whatsoever, and only advantages.

So, what you're saying is that having deps commits mixed in with Krita commits helps somehow to figure out which dep change was related to which change in Krita <...> I don't think that's important at all.

Well, in some cases this info is rather useful for me. Though I'm not yet sure about its relative importance against other requirements. That is why I would like to postpone this decision until we do the actual port to the KDE's scripts.

I guess we both understand the points of each other and should just wait until we actually start the port.

To make a proper decision we need to write a set of manuals/guidelines for the KDE-script-based process (we need to write them anyway, because our processes there are rather random atm):

1. How to update a dependency that is built from the official release?
   • add patches
   • trigger builds (which ones and how?)
2. How to update a dependency that is built from a repo maintained by KDE?
   • add patches
   • trigger builds (which ones and how?)
   • make a MR upstream
   • merge MR and remove patches (who in responsible for removing the patches and how does it fit split/merged deps repo ideas?)
3. How to update a dependency that is built from a personal fork of someone in Krita team?
   • add patches
   • trigger builds (which ones and how?)
   • make a MR upstream
   • merge MR and remove patches (who in responsible for removing the patches and how does it fit split/merged deps repo ideas?)
4. If we do a common change to a dep and Krita, which depends on each other, how should we handle that? Should we add some versioning to deps? Increase Major deps version?
5. How should we do a release? Step-by-step
6. The person wants to work on modifying dependency "ext_foo" on "Bar OS", how should he/she proceed with that?

In T16430#291701, @dkazakov wrote:

So, what you're saying is that having deps commits mixed in with Krita commits helps somehow to figure out which dep change was related to which change in Krita <...> I don't think that's important at all.

Well, in some cases this info is rather useful for me.

You could, in those cases always make a note in the commit message in the krita repo... Or just check for commits in the deps repo before the date of the relevant commit in the krita repo.

Though I'm not yet sure about its relative importance against other requirements. That is why I would like to postpone this decision until we do the actual port to the KDE's scripts.

I guess we both understand the points of each other and should just wait until we actually start the port.

To make a proper decision we need to write a set of manuals/guidelines for the KDE-script-based process (we need to write them anyway, because our processes there are rather random atm):

1. How to update a dependency that is built from the official release?
   • add patches
   • trigger builds (which ones and how?)
2. How to update a dependency that is built from a repo maintained by KDE?
   • add patches
   • trigger builds (which ones and how?)
   • make a MR upstream
   • merge MR and remove patches (who in responsible for removing the patches and how does it fit split/merged deps repo ideas?)
3. How to update a dependency that is built from a personal fork of someone in Krita team?
   • add patches
   • trigger builds (which ones and how?)
   • make a MR upstream
   • merge MR and remove patches (who in responsible for removing the patches and how does it fit split/merged deps repo ideas?)
4. If we do a common change to a dep and Krita, which depends on each other, how should we handle that? Should we add some versioning to deps? Increase Major deps version?
5. How should we do a release? Step-by-step
6. The person wants to work on modifying dependency "ext_foo" on "Bar OS", how should he/she proceed with that?

Well, sure, but that's orthagonal to my suggestion.

In any case, I've put my reasoning on the table, so I don't think I have to repeat that until we get down to actually implementing CI based deps building.

Oh, another manual needed:

7. How to make a MR like the MLT one, which modifies both, a dependency and Krita. We had multiple cases of that recently.

In any case, I've put my reasoning on the table, so I don't think I have to repeat that until we get down to actually implementing CI based deps building.

Yes, I understand your points, and kind of 70% agree with them :)

Just a note:

When we consider splitting the deps repo, we should think how we are going to handle the MRs like that:

https://invent.kde.org/graphics/krita/-/merge_requests/1748

The MR touches both, deps and Krita code, so the changes should be somewhat in sync...

(not that is "blocks" the split, we should just model and document how to do such MRs)

In T16430#292419, @dkazakov wrote:

(not that is "blocks" the split, we should just model and document how to do such MRs)

I'm kinda tempted, and this is just tentative, to want to first update the deps, verify Krita still builds with it, and then have an MR for Krita. I'm not sure why, but I think it would give me a bit of a safe feeling...

first update the deps, verify Krita still builds with it, and then have an MR for Krita. I'm not sure why, but I think it would give me a bit of a safe feeling...

Sounds sensible

Just a note: we need some better way to manage git repositories for the deps. The thing that happened today was that @lsegovia removed a branch in their personal repo (I guess it was renamed, though I'm not sure), and it broke our deps builds.

We have the following options:

1. All deps are stored either in tarballs (externally or on files.kde.org) or in non-personal repos on invent.kde.org with (main?) developers having access to that (to allow merging MRs)
2. All deps are stored in tarballs unconditionally
   • it will make caching easier
   • will make updating deps a total hell for the developers

We should probably plan some period of upstreaming our patches before each release to make sure our forks are not diverged too much.

A note from bcooksley: I would suggest we use something like teams/krita/ [namespace] for hosting those [forked dependencies]