Streamlined onboarding of new contributors
Open, Needs TriagePublic

Description

Merged with T6832 authored by @ngraham

Description

Getting people involved is vital for the long term sustainability of a project and its continuous development and growth. Of equal importance is for new contributors to find an environment in which they can thrive and a community and project which they can be part of for a long time.
KDE is a big organization of hundreds of contributors working on dozens of projects. Even though documentation exists on the basics of getting involved, this is sometimes incomplete, outdated, or impersonal. It can be difficult and overwhelming for a newcomer to feel comfortable and confident enough to step up and get more involved without knowing the related context, balances, relationships etc.
The KDE Community can do more to facilitate this, by motivating and nurturing involvement. We can be more proactive and effective, both in attracting new contributors, as well as in making them feel welcome and valued in the community and the projects they show interest in working on.

What it will take

Research
We will need to form a team that will try and understand the current status of the requirements and expectations for getting involved and gain a good overview of the processes in play. This team will have to hold or acquire good knowledge of KDE’s community and projects, as well as of how current contributors are intra- and inter-connected among them.
Holding discussions/interviews with current contributors is key here, both with people that recently joined as well as with individuals that have been around for a while , to gain a better insight and overview of the situation. Surveys could also prove useful.

Questions that we should be aiming at answering may include:

  • What are the possibilities for involvement (teams and projects, tools used, skills required)?
  • What are the available resources in terms of people and time that can be invested in this effort?
  • Are we making the most out of available potential of people interested in contributing?
  • Can some onboarding procedures be identified as essential?
  • Can we direct contributors towards areas that KDE could use more contributors in?
  • Are all the possibilities and pathways for getting involved properly documented and visible/reachable on the website?
  • To what extend are valuable resources spent on helping/training people that end up not joining the project or not making long term contributions?
  • What are the feelings of both new and older contributors in regards to the procedures of getting involved? What did they enjoy, what did they not like, what kept them around, what could we improve?

Implementation
The next step is to propose a plan for attracting and integrating newcomers into the community. This will largely depend on the outcome of the research.

I assume that the forming of a dedicated team of contributors might be needed, which would be responsible for:

  • Welcoming and introducing newcomers to the community and procedures, by taking in mind their skills, interests and experience.
  • Actively supporting newcomers’ involvement until they are fully engaged with their projects and integrated with the community in both a micro and macro level.
  • Monitoring new contributors’ progress and satisfaction.
  • Identifying contributors within each KDE team that are willing to act as the first port of call and mentor newcomers. Provide these mentors with guidelines on getting people involved based on KDEs principles, values and workflows.
  • Holding events (could very well be online) targeted at newcomers where they can be informed, ask questions, participate in discussions, or even get a chance to get their hands dirty with some tasks.

Specific ideas:

Bugzilla Done, check T6832

  • Collapse the UNCONFIRMED and CONFIRMED statuses into a new status "NEW" to avoid user confusion ("Why isn't this confirmed after 7 years and 5 duplicate reports!?").
  • Mine other projects' Bugzillas for ideas regarding additional or replacement statuses we can add, particularly for closed tickets. Examples from other projects include NOTABUG, NOTOURBUG, NOTGNOME, ASDESIGNED, CANTFIX. https://bugs.kde.org/show_bug.cgi?id=383753
  • New Bugzilla tickets should have a default template explaining how to file a good bug, what kind of information is required, and mentioning that patches should be submitted to Phabricator and not attached to the bug. https://bugs.kde.org/show_bug.cgi?id=383169
  • Each project can override the standard template with a project-specific one that can list additional information needed, and even offer information about common bugs that don't need to be re-reported and workarounds for common issues.
  • Change the attachments page to notify users that patches should be submitted to Phabricator instead, along with a link to a how-to page
  • Bug screeners who identify reporters of above-average technical competence should encourage them to dig into the code and submit patches on Phabricator.

Documentation

  • Make sure that the "how to contribute" pages on the top-level of kde.org are fully up-to-date
  • Closely document a "minimum viable product" approach to producing a user's first patch. We should aim for this process to be as painless as possible.
  • Improve documentation regarding how to check out and compile sources, how to set up a good repeatable dev environment, etc.
  • Document the process of becoming a KDE bug screener, along with helpful information
  • Improve documentation for how to translate strings

How we know we succeeded

Quantitative statistics and qualitative feedback can act as indicators.

An increase should be observed over time in the number of:

  • People showing initial interest
  • People ending up getting involved
  • New contributors that stay around for the long-term
  • Actionable bugzilla tickets
  • Phabricator patches from first-time submitters

New contributors and mentors should report higher levels of satisfaction with the related procedures and acknowledge a positive effect as a result of our efforts.
The sense of community should increase between the first days someone gets involved and after some time has passed.

What are the risks?

The obvious risk here is spending valuable resources of current contributors without any tangible outcome for the KDE community and its projects.
Yet, even if we fail to deliver changes, KDE will have gathered a lot of useful feedback about the current status of the community and its procedures.

Related Links

https://community.kde.org/Get_Involved
https://www.kde.org/get-involved
https://en.wikipedia.org/wiki/Onboarding
https://en.wikipedia.org/wiki/Human_resources
https://en.wikipedia.org/wiki/Sense_of_community
http://lydiapintscher.de/papers/mentoring.pdf

For Application Developers

At Akademy 2017, @ervin and @dfaure talked about making it easy for newcomers to build and test Applications. They discussed removing the need for make install and implementing solutions around conan.io and flatpak.org. Detailed summary, video and slides can be found here:
https://conf.kde.org/en/akademy2017/public/events/372

For financial contributors

Consider the use of Patreon/Liberapay

I am willing to put work into this

Neofytos Kolokotronis (@neofytosk)
Nate Graham (@ngraham) - writing and deploying documentation and the new Bugzilla template and statuses
Ivana Isadora Devcic (@skadinna)
Vijay Krishnavanshi (@vijaykrishnavanshi)
Łukasz Sawicki (@Lucas)
Elvis Angelaccio (@elvisangelaccio) - writing/updating docs (Wiki/Apidox)
John Samuel (@jsamuel) - writing/updating docs (Wiki) and updating external sites like Wikidata
Paul Brown (@paulb) - writing/updating docs, working on upgrading usability and friendliness of sites and protocols.
Matija Šuklje (@hook) - anything FLA, contributor agreement or licensing related
Aleix Pol (@apol)
Dimitris Kardarakos (@dkardarakos)
Ovidiu-Florin Bogdan (@obogdan) - Writing Conan packages and managing Conan packages + anything CMake

I am interested

Jeff Huang (@s8321414)
Gregor Mi (@gregormi)
Adrián Chaves (@adrianchavesfernandez)
Ken Plaha (@Kalpha)
Juan Carlos Torres (@jucato)

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
mmustac added a subscriber: mmustac.EditedJan 28 2018, 4:39 PM

What about editing the "Get involved" page on kde.org as first step?
I am missing an up2date step by step guide to get a working development environment.
Because I really would like to get started.

@mmustac Yup, I've proposed that and already gotten buy-in and permission to do it. Just working on getting my access to change the page.

Is it going to be a redirect to the existing pages? The example of page shown is a bit too much developer-oriented compared to https://community.kde.org/Get_Involved

@ngraham Great to hear that. I will be one of the first to test your explanations if they are also ready for noobs like me :-D And provide my feedback of course.

@mmustac this is done now!

Hi everyone, I feel bad for being quiet over the last couple of weeks. Lots of changes happening in Jan-Feb for me, and also FOSDEM and the KDE Promo sprint are coming up. I'll try to push this goal forward in a consistent way once all these have passed.

I do follow everything related and I've already organized some ideas and plan to start dedicated tasks for each.

My current plan is to hold a sprint for this goal in May, but we need to check what suits most, so I'll open another task for that later on.

I also posted a introductory blog post regarding this goal a month ago which I never shared here:
http://neofytosk.com/post/kde-community-goal-streamlined-onboarding-of-new-contributors---introduction/

Do keep the comments and ideas coming, and kudos to those already putting work into this!

Recently, I tried to find out KDE best practices to work with Docbook.

Two findings:

In search of general information for developers I found:

On thing we could use is an official policy regarding whether the stable or master branch is more appropriate for any given patch. Right now it's often not clear for edge cases such as D10651: Fix thumbnail hover icon show-in-fullscreen action. In that patch, a bug was fixed that led to a behavior change. Stable or master? We committed to master to be safe, but it wasn't as clear as we would have preferred.

Kalpha updated the task description. (Show Details)Mar 13 2018, 6:50 AM
Kalpha added a subscriber: Kalpha.
rkflx added a subscriber: rkflx.Mar 13 2018, 8:33 PM

As a newcomer and junior-jobs guy, I'm happy to see this initiative being pushed forward by the Top People(tm) of the KDE organization. I've been stumbling my way through a few patches with the help of @rkflx and @ngraham, the latter of which agreed to mentor me in the ways of KDE. Henrik has been patient with me as I learn C++ and has given me a few assignments for Spectacle bugfixing. Nate has been a good guide to the environment and community. I'm sure there are many more like these two out there. Busy people, but willing to share their knowledge and expertise.

If there's more I can do as a newcomer, please ask. If nothing else, I can share what's it been like for me so far, where I've found things easy, and where I've found things difficult, confusing, or intimidating. I'm not young (49 next month), and I'm just learning C++, but I've been a techie and programmer since I was small. I've always wanted to contribute to something open-source, and when I fixed my first bug here at KDE, the feeling was enormous. Knowing my minor fix for the Media Frame plasmoid will ship as part of Plasma 5.13 feels awesome.

But I spent at least a day or two asking around on mailing lists and IRC for "permission" to work on the bug. Someone finally told me I was free to work on anything I wanted. I think that bit of "documentation" is missing, or needs more emphasis. Submitting my code through Phabricator was also a nail-biting experience. I had limited exposure to what a "diff" was, and it was a disconcerting feeling to pull a copy of a KDE project from the official repository and boldly make changes and corrections. @davidedmundson reviewed the patch and pointed out a few missteps. But I got them fixed and he pushed my patch into the real code stream. That was really cool.

I'm looking for more things to fix and more patches to contribute. My goal is to progress far enough to be granted a developer account, a badge I'd wear proudly (but carefully).

Overall, my experiences as a newcomer have been very positive. But they've also been a bit vague and come with some searching and exploration. I'm still not sure what I "can" or "should" do with some of my ideas. I know all positive suggestions are welcome, and I enjoy that, but the specifics are elusive. For example, I've got something I volunteered for on various mailing lists - updating the CafePress KDE merchandise store - but I don't know how to proceed. I think it should be a Phabricator task, but again, I don't know if I'm allowed to or supposed to create those on my own.

In the mean time, I'm learning the KDE infrastructure and community, happily writing my own code again, even blogging about my experiences (www.bundito.com) with KDE and other et ceteras.

Thanks for this. I hope I can help.

What do you think about a dedicated stack for web development? As there is C++, Qt/QML for the desktop, like wise a dedicated/preferred programming language(s) and framework can be chosen for the web. This would help in onboarding contributors to the web side of things and potentially make management of the web stack easier for both the developers and the system admins.

What do you think about a dedicated stack for web development? As there is C++, Qt/QML for the desktop, like wise a dedicated/preferred programming language(s) and framework can be chosen for the web. This would help in onboarding contributors to the web side of things and potentially make management of the web stack easier for both the developers and the system admins.

In fact, that's already being done with the migration to Wordpress. Of course, we can do a better job of making this information widely known once the migration is complete.

Under the subheading of "Documentation", can we discuss code commenting? And ideally - encourage it? The API docs are good and generally well-maintained, but inside the applications themselves, the quantity and quality of code comments varies quite a bit. Some of our products are quite large and use many of the Frameworks together. It can be daunting at times to decipher how a block of code works and why it was constructed in the way it was.

I'm not suggesting the near-impossible task of documenting old code, but I'd like to see a line or two of comments added when something is patched or a new feature is implemented. These could be nice, brief pointers to help newcomers understand the how and the why.

ervin removed a subscriber: ervin.Apr 21 2018, 3:05 PM
dkardarakos updated the task description. (Show Details)May 2 2018, 8:09 PM
dkardarakos added a subscriber: ervin.

Here are my 2 cents.

I have been using KDE for a decade and I wrote some applications for KDE in the past. For the past 5? years I have mainly contributed with bug reports. Only recently I started to contribute with some patches to existing projects (okular, kdevelop, kio, kile).

As reported by others, the main wall to get started is setting up the development environment. I found the information quite scattered usually. I report here my experience to try to highlight some problems I think we should try to address.

Examples:

Patch to the "Open with" dialog.

To patch the "Open with" dialog, which is part of KIO I figured out I also had to build dolphin to open it.

Was this necessary? Could have I only built KIO and called the Open with dialog from the system installed dolphin? Not clear to me.

So I decided to create a new virtual machine and build dolphin, kio and their dependencies with kdesrc-build following this guide:
To build dolphin it was necessary to build a tons of other KDE component and I had to install a tons of development packages. The previous guide links to this other guide, which is outdated.
So with a painful trial and error, which involved a lot of google searches, I managed to install all the required packages.

In this case I haven't even tried to use KDevelop because of all the dependencies, so I just used kate to edit and konsole to compile. Not very practical for debugging.

I think I did not have too much problems with the environment variables in this case.

Bottom line: Building all those packages with kdesrc-build was a lot of work and I had to use a separate development environment (virtual machine) because I did not want to install all the dependencies in my own system. I still have the doubt if buliding dolphin and all its dependencies was really necessary. Debugging was difficult.

Patches to Okular

In this case I worked on my own system (solution I prefer), without creating a separate development environment. I downloaded okular via git and started to configure KDevelop.
I looked up for the git repository via the Okular website, and only in a second moment I discovered a feature of KDevelop that allows to clone a KDE project directly from within KDevelop. This feature should be advertised more I think Bug 392550

I would loved to have a guide that describe how to configure KDevelop to build okular. In particular:

  • How to create a run configuration

Should I pick one of the Okular targets or should I choose compiled binary? I did not have idea about what are the "targets" when I started. I used CMake in the past but I still deal with it like it is black magic (slowly I am trying to understand it more and more).
Is it enough to just build and execute? Or do I need to build install, and execute?

  • How to configure the environment variables

Setting up the environment variables is very important and usually is a mess. For example Okular is a KPart and if the env variables are not set, when I launch the compiled Okular it uses the system part so I could not see my modifications.
How to setup the env variables is explained in Okular website (rare case where this is). Another problem is that in KDevelop I cannot reuse a defined variable in another variable, so for example the variables as given in this guide won't work. Maybe this is a KDevelop bug (I should notify this).

Moreover in the KDevelop manual the "Environment" feature to setup multiple sets of environment variables is not even explained.

  • How to debug in KDevelop

By default KDevelop was using LLDB as debugger, so I had first to figure out what LLDB is, which led me to LLVM, so I had to figure out what LLVM is. Then given that is was giving me just problems I figured out I could use GDB by installing okteta that was not a dependency of KDevelop (maybe a problem of my distribution). Not really a KDE problem, but part of my odissey.

Debug messages

Another problem I often encounter when I want to debug an application to submit a bug report is how to enable the debug messages. I know I need to edit an *.ini file to enable the logging categories or export an env variable, but every time I forget. A guide on this topic would be useful.

Summary

To sum up I think that each project should have a page that contains the documentation for the developers with:

  • Guide to configure KDevelop to compile and run the project, which includes
    • How to clone it
    • How to configure a KDevelop launch (or more than one if there are multiple for different purposes)
    • How to setup the environment variables within KDevelop

If instead the project is not viable to be built within KDevelop, for example because it requires a lot of other dependencies, it should have:

  • Link to guide on how to setup kdesrc-build
  • List of environment variables required
  • Guide on how to run the compiled binary

This post is way longer then I expected, sorry for the long story but I hope that explaining in detail my experience can be useful to generate some ideas on how to make it easier to start writing code for the ones that come after me.

BTW, Akademy is nearing, registration is open. Could we send a mail to all (!) contributors who have a KDE commit account? I think this does make a lot of sense to trigger everyone (especially new contributors) to come to this event.

aacid added a comment.EditedMay 14 2018, 6:31 PM

BTW, Akademy is nearing, registration is open. Could we send a mail to all (!) contributors who have a KDE commit account? I think this does make a lot of sense to trigger everyone (especially new contributors) to come to this event.

That list already exists, it's called kde-cvs-announce but please make sure spamming everyone is ok. Also, you're "forgetting" about the lots of people that contribute patches but don't have commit access, those are the ones that really don't know about Akademy or think they are "not worthy", though i'm not sure how easy is to find those people :D

Also spamming people is kind of bad GDPR-wise :/

neofytosk added a comment.EditedMay 15 2018, 8:43 AM

It's great to see so much valuable feedback rolling in to this thread, especially from people that are new to KDE. Many thanks for that!

However, there is now too much feedback here that can easily be missed, so I suggest moving the discussion on specific areas of improvement to their own sub-tasks. As you might have seen I've in the meantime started several sub-tasks according to the main ideas in the initial post and in the comments we've gathered since the task was started. The concept is to organize each objective better so we can start working on them and follow their progress.

Please go through the tasks and make sure to share your thoughts on the ones you care about! And of course, please add new sub-tasks if you feel something should be done in an area we did not yet touch upon.

@richardbowen a dedicated task for Documenting Web Development once Wordpress is around would make sense. Similarly, @sharvey a dedicated task could also be held for Improving Code Documentation. Can you guys start new sub-tasks and gather your ideas and any relevant information there?

@simgunz This is some great feedback, can you please leave this comment on T8484: Making settting up a development environment quick and easy as it seems directly related? The discussions linked there could interest you as well.

Thanks @sandroandrade , I 'll reach out to the author and try to get some first hand insight on mozilla's procedures and experience.

obogdan updated the task description. (Show Details)Aug 1 2018, 5:59 AM
obogdan added a subscriber: obogdan.

I was thinking that a way to facilitate a newcomer getting into a project could be to find him a code buddy, another newcomer working on the same project.

The possible advantages are:

  • the two (or more) code buddies can discuss and find solutions in problems common to the project (e.g. setting up a developing environment T8484, analyzing together portion of codes)
  • the previous point would remove some burden from the mentors (likely the newcomer have more free time to discuss with each other, than the free time the mentor can dedicate to them)
  • they could work on common tasks, so they can cooperate on solving an issue or implementing a new feature (especially non-trivial ones)
  • working with someone (in my opinion) boosts the motivation
  • Others?

Possible limitations:

  • The code base of a single project is quite big, so different newcomers may want to work on disjoint part of the code limiting the possible cooperation
  • Others?

A possible way to pair newcomers can be:

  • Publish name and email of the newcomers grouped per project in a web page (as it happens for the mentors) [For the newcomers that want to participate in this thing]
  • During the on boarding phase, the mentor could mention the "code buddy" thing to the newcomer, show him the web page with the list of code buddies and ask him if he/she wants his name on the list

My experience: I noticed that when I collaborate with another person on software my motivation is higher and my laziness is reduced

What do you think about this idea?

hein added a subscriber: hein.Sep 13 2018, 6:37 PM

Sorry if this was posted before, but looks interesting: https://whatcanidoformozilla.org/

Hello! :)
Indeed it was. You might like what the Plasma Mobile project came up with: https://www.plasma-mobile.org/findyourway/

Hey @neofytosk, you can cross off all of the Bugzilla stuff. We've completed those items in the T6832 subtask.

neofytosk updated the task description. (Show Details)Sep 25 2018, 7:45 PM

Hey @neofytosk, you can cross off all of the Bugzilla stuff. We've completed those items in the T6832 subtask.

Done, thanks for your work @acrouthamel!

jucato added a subscriber: jucato.Apr 1 2019, 7:31 AM
jucato updated the task description. (Show Details)Apr 1 2019, 8:05 AM
neofytosk added a comment.EditedMay 5 2019, 9:07 PM

A new date is set for the Streamlined Onboarding Sprint 2019. It will take place on July 22-23 in Nuremberg, Germany. Please register your interest on https://community.kde.org/Sprints/Onboarding/2019!

meven added a subscriber: meven.Jun 10 2019, 2:21 PM

Something i think we could also improve the first patch review process :

Automating things
Documentation is a necessary step for newcomers, yet we need to limit the amount of it newcomer face.
Some of them will only ever do one contribution to "scratch their own itch" attitude.
Automating steps that can be can get a long way to keep contributors focused on what matters most : their contribution, not the process or organization.
This is a redundant theme of my ideas down here.

How to get/find a reviewer
Unless your patch receive the visit of a casual contributor, new contributors cannot know that Dolphin is reviewed by the Dolphin group reviewer and sometimes VDG or kio by the Frameworks one. A great thing would to populate a reviewer based on code repo or even path concerned automatically.
This would also save some time triaging reviews by some of our reviewers.

Say Hello
Newcomer contributors might appreciate a comment to their review, welcoming them and pointing to eventual relevant documentation (like bors bot does in the rust community).

Making sure the commit / review has the right format
Documentation is great such as https://community.kde.org/Policies/Commit_Policy but automation can save time or point to documentation in the first place. We could put in places hooks to check that, for examples, references to bugs respect the "BUG: " format, commit author must contain a real name, etc....

Making sure the code has the right format
I see too often in reviews especially for newcomers comments like "missing space".
IMO this is wasted time for the submitter and the reviewer : code format should be a no-brainer, a NO-OP. I wish projects would have some automatic code formatting enforced with tools such as astyle and some git pre-commit hook. Even seasoned contributors should appreciate some automation on this.

Making sure the code passes tests
It seems to me to hard to discover where tests are run in CI, when they fail and especially when regressions are introduced.
A great feature to have is to test incoming patch before merging, preventing introducing bugs.
In turns it helps give tests some visibility and make their maintenance a necessity.
I guess if we switch to gitlab, this kind of integration would be easier.

Suggest to sign KDE e.V FLA
The code we commit can live a lot longer that we contribute last, and who knows what will license we will be using down a few years.

In case the community didn't have enough on its plate...
Anyway those are me ideas that would have helped me in my first contributions especially and some feature wish to our contribution process.

+1. Phab can actually do a lot of that already. I know it can do tests and automatically add reviewers. Some repos already do in fact. @sysadmin folks just need to add Herald Rules for this automatically. I've filed a task for a bunch of new rules: {T11060}.

Please note that Sysadmin focus now is on the Gitlab migration rather than Phabricator. Setting up all those rules you've asked for is likely to take more than an hour...

Note that it was a conscious decision to not setup rules on a per repository basis, because each Herald rule introduces an approximately 1ms performance penalty when processing actions that involve that rule.

If we were to have 500 rules for reviews, each comment would be subject to half a second of latency on top of ordinary processing time (so would likely end up being a full second) which would create a terrible user experience.

We therefore only create rules on request, and make sure rules cover as broad a range of repositories as possible to minimise the number of Herald rules we have (and the performance impact of them)

Darn, that sounds bad.

Is there any way people can follow the Gitlab migration? I don't know where to go to see the current status and therefore it feels stalled to me, though I'm sure a lot of working is being dibe on the backend that I'm just not aware of.

At the moment there isn't a public task or anything i'm afraid (the list of outstanding items is only contained in discussions between members of Sysadmin at the moment).

Currently the blockers to the transition include:

  1. Rework how the anongit network works to allow it to continue in the Gitlab world
  2. Introduce a sync service to ensure updates to names and email addresses, as well as changes in group membership propagate automatically from Identity to Gitlab
  3. Wait for Gitlab to transition multiple project boards to the Community Edition of Gitlab. This is currently underway from my understanding and is targeted for delivery in July.

Thanks! It might be helpful if there was a public interface to this information (even if it's just read-only to prevent spam).