Making settting up a development environment quick and easy
Open, Needs TriagePublic

Description

During the goals office hour that took place on IRC on 15 March 2018, the issue came up of making it as easy as possible for new contributors to setup a development environment so they can start working on all things KDE.

The main idea that surfaced during the initial discussion was to use some sort of virtual environment, you will find the relevant chat text at 17:21:48 to 17:43:28 of the log. The suggestions were about using either a virtual machine or a docker image. The discussion continued on the mailing list, and this time it was about using janitor. There is an ongoing effort to update the KDE container offered there.

What's more, at Akademy 2017, @ervin and @dfaure talked about making it easy for newcomers to build and test Applications. They discussed removing the need for make install and implementing solutions around conan.io and flatpak.org. Detailed summary, video and slides can be found here.

I'm starting this issue so we can continue the discussion and properly track our progress. What is the current status? Can we distinguish any requirements that an ideal implementation would fulfill?

Pinging @ngraham, @Fuchs, @sebas and @adridg as they are the ones I recognize from the people that took part in the discussions so far. Please further ping people I missed or might be interested.

richardbowen added a subscriber: richardbowen.EditedApr 10 2018, 4:37 PM

On the web side of things, a Dockerfile per web{site, application} would be helpful. That way, those who use docker can do isolated development from containers.

I finally had an "ah-hah!" moment a few weeks ago (thanks to the patience of @hein) when I finally got my development setup properly configured. Eike's gentle but direct question of "didn't we cover this on your last patch?" made me sit back down and follow all the documented steps one by one. With the addition of the right environment variables, I can now download and compile virtually any source module I need.

There are good, detailed directions already on the Community wiki, but the big picture of how everything interlocks remained a little mysterious.

Perhaps I can contribute some kind of flow chart or dependency diagram to illustrate how tools like kdesrc-build operate. The concepts aren't hard, but I clearly "had a moment" when it all suddenly snapped into focus. I'd like to share that for other incoming members.

I'd like to point out again the talk we delivered at Akademy 2017 with @dfaure (it is referred to in the parent task of that one). For most cases we should strive at not forcing people to use a VM or a container. Especially if we want to make it easier for people on non-Linux platforms to contribute. The highest potential solution is the one using conan but since @fredrik prototype no activity has happened in that direction.

@ervin I think you are confusing me with someone else here.

ervin added a subscriber: gladhorn.Apr 21 2018, 8:06 PM

Good point, I was thinking @gladhorn in fact. Sorry. :-)

Indeed, I did a small proof of concept with Conan, it looks promising, but someone would need to put real work into this. Ideally we get Qt and dependencies into Conan, at that point the whole magic would hopefully just work, the dependencies could be built as needed and would be shared thanks to they way Conan caches packages. This would add one step to the build, something like "make conan" or so, so all in all quite reasonable. I'd like to see this happen, but I will not have the time to implement this.

neofytosk updated the task description. (Show Details)Apr 26 2018, 4:37 PM

I repost here the comment previously posted in T7116 as requested by @neofytosk on my personal experience with the setup of the development environment.

I have been using KDE for a decade and I wrote some applications for KDE in the past. For the past 5? years I have mainly contributed with bug reports. Only recently I started to contribute with some patches to existing projects (okular, kdevelop, kio, kile).

As reported by others, the main wall to get started is setting up the development environment. I found the information quite scattered usually. I report here my experience to try to highlight some problems I think we should try to address.

Examples:
Patch to the "Open with" dialog.
To patch the "Open with" dialog, which is part of KIO I figured out I also had to build dolphin to open it.

Was this necessary? Could have I only built KIO and called the Open with dialog from the system installed dolphin? Not clear to me.

So I decided to create a new virtual machine and build dolphin, kio and their dependencies with kdesrc-build following this guide:
To build dolphin it was necessary to build a tons of other KDE component and I had to install a tons of development packages. The previous guide links to this other guide, which is outdated.
So with a painful trial and error, which involved a lot of google searches, I managed to install all the required packages.

In this case I haven't even tried to use KDevelop because of all the dependencies, so I just used kate to edit and konsole to compile. Not very practical for debugging.

I think I did not have too much problems with the environment variables in this case.

Bottom line: Building all those packages with kdesrc-build was a lot of work and I had to use a separate development environment (virtual machine) because I did not want to install all the dependencies in my own system. I still have the doubt if buliding dolphin and all its dependencies was really necessary. Debugging was difficult.

Patches to Okular
In this case I worked on my own system (solution I prefer), without creating a separate development environment. I downloaded okular via git and started to configure KDevelop.
I looked up for the git repository via the Okular website, and only in a second moment I discovered a feature of KDevelop that allows to clone a KDE project directly from within KDevelop. This feature should be advertised more I think Bug 392550

I would loved to have a guide that describe how to configure KDevelop to build okular. In particular:

How to create a run configuration
Should I pick one of the Okular targets or should I choose compiled binary? I did not have idea about what are the "targets" when I started. I used CMake in the past but I still deal with it like it is black magic (slowly I am trying to understand it more and more).
Is it enough to just build and execute? Or do I need to build install, and execute?

How to configure the environment variables
Setting up the environment variables is very important and usually is a mess. For example Okular is a KPart and if the env variables are not set, when I launch the compiled Okular it uses the system part so I could not see my modifications.
How to setup the env variables is explained in Okular website (rare case where this is). Another problem is that in KDevelop I cannot reuse a defined variable in another variable, so for example the variables as given in this guide won't work. Maybe this is a KDevelop bug (I should notify this).

Moreover in the KDevelop manual the "Environment" feature to setup multiple sets of environment variables is not even explained.

How to debug in KDevelop
By default KDevelop was using LLDB as debugger, so I had first to figure out what LLDB is, which led me to LLVM, so I had to figure out what LLVM is. Then given that is was giving me just problems I figured out I could use GDB by installing okteta that was not a dependency of KDevelop (maybe a problem of my distribution). Not really a KDE problem, but part of my odissey.

Debug messages
Another problem I often encounter when I want to debug an application to submit a bug report is how to enable the debug messages. I know I need to edit an *.ini file to enable the logging categories or export an env variable, but every time I forget. A guide on this topic would be useful.

Summary
To sum up I think that each project should have a page that contains the documentation for the developers with:

Guide to configure KDevelop to compile and run the project, which includes
How to clone it
How to configure a KDevelop launch (or more than one if there are multiple for different purposes)
How to setup the environment variables within KDevelop
If instead the project is not viable to be built within KDevelop, for example because it requires a lot of other dependencies, it should have:

Link to guide on how to setup kdesrc-build
List of environment variables required
Guide on how to run the compiled binary
This post is way longer then I expected, sorry for the long story but I hope that explaining in detail my experience can be useful to generate some ideas on how to make it easier to start writing code for the ones that come after me.

obogdan added a subscriber: obogdan.Wed, Aug 1, 6:16 AM

Hello world,

I've been MIA in KDE for a couple of years because of work. In this time I've worked heavily with porting C++ applications and libraries to CMake and Conan, and have become a Conan contributor in the meantime. I'd like to give back my Conan experience to the KDE community, and help write Conan recipies for as many frameworks as possible. I've looked over the proof of concept @gladhorn has done and I believe that is a good start, but there is so much more that can be done.

Here's what I've done so far:

  • Created a package for Extra-CMake-Modules
  • Created a package for KConfig

Both of these packages have a package test that tests the conan package. The recipies for each of those are in a branch use-conan.io derived from master.
Both of these packages have one binary uploaded to the Conan. binary storage in Bintray.

@gladhorn has created the KDE and Qt organizations in Bintray and the frameworks repository in KDE: https://bintray.com/kde/frameworks


Bincrafters is a community derived from the Conan community that has takes uppon themselves to package every or as many OpenSource libraries as possible.

They have also packaged Qt: https://bintray.com/bincrafters/public-conan/Qt%3Abincrafters

Their Qt package could use some improvements, but it's a starting point.


In their talk, @ervin and @dfaure have mentioned about generating the Conan recipies from the yaml files. While that might work, I'd like for starters to write them by hand so as to better control what is happening. This requires some complexity to make the conan recipie read some information from the YAML and CMakeLists.txt files.


There is a helper tool from Conan that can help with building a package for a variety of different settings: build type, OS, arch, compiler, etc. https://github.com/conan-io/conan-package-tools

I'd love to write some Jenkinsfiles that takes advantage of these tools, and builds each framework for everything that we want.


P.S. I'll be at this year's Akademy, and would like to talk to you guys about this more, maybe even do a workshop. But I'll be leaving on the 14th. Is there a room somewhere where we can organize this?

I've reserved a room for a Workshop/BoF at Akademy: Monday 13 August, Room 121 from 10:30 till 12:30.

https://community.kde.org/Akademy/2018/Monday#Room_121_.2837_people.2C_priority_for_workshops.29_-_13th_August

neofytosk added a comment.EditedThu, Aug 2, 9:59 AM

Thanks @obogdan! I've reserved a spot for this objective on Tuesday 14th. What time are you leaving? It would be great if you were around.

https://community.kde.org/Akademy/2018/Tuesday#Room_121_.2837_people.2C_priority_for_workshops.29_-_14th_August

Thanks @obogdan! I've reserved a spot for this objective on Tuesday 14th. What time are you leaving? It would be great if you were around.

https://community.kde.org/Akademy/2018/Tuesday#Room_121_.2837_people.2C_priority_for_workshops.29_-_14th_August

Can we have this in the morning? Or sometime before Lunch?

I've talked with @bcooksley about how we can automate the build of conan pacakges in our CI system and here are some ideas we've discussed:

I've added an example CI pipeline script and a Conan MultiPackage script that show the basic example on how we can leverage the Conan tooling to build for multiple targets (Arch, Build type, OS, etc...).

What we need to discuss further is:

  • Which compilers do we want to have binaries for
  • Which OSs
  • Which build types
  • Which Architectures

With this information we need to create/update docker images for building in which the building will happen.

Then we'll draft the pipeline and the tooling to be used in order to create the jobs that will only build releases (for the beginning). Later after a bigger adoption of Conan we'll look into allocating more time/resources to build from master on Nightly.

Just 2 cents : I'm in the middle of committing my first patch to KDE (and I'll tell my experience before I forget it, 'cos once the patcfh will be done, eveything will lokk simple I guess).

My experience is this. Following some community page, I tried to use kdssrcbuild on a platform that is not well supported (Debian Stable). As (not expected), the build didn't go well and unfortunately kdesrcbuild didn't warn me. Result 4 full days of desperately trying to fix all cmake related quirks. It would have been so cool to have a proper warning saying "you're trying to build on a system we don't support, be ready for problems" (without that, all build issues looks like something you can fix, except you can't).

Later on @ngraham suggested to use a Neon image in VirtualBox (as explained here too). Result : up and running in an hour or so. Downside : that requires a good computer to run (mine is 8 years old, 4 gigs of memory, that's barely sufficient).

Other things:

  • the page explaining how to submit a patch doesn't explain what will happen after submitting to Phabricator.
  • you don't know how much time you have to wait.
  • Phabricator ask for reviewers which you don't know about.
  • Phabricator also complains about "lint" and "test" that were not run and you don't know what it means ("Lint was skipped when generating these changes". "Unit tests were skipped when generating these changes")