Improvements to Bugzilla - Making it easier and simpler
Closed, ResolvedPublic

Description

Description

Our new contributor experience is not as accessible as it could be. Documentation is often out of date, Phabricator can be intimidating, Bugzilla doesn't offer any guidance on how to file good bugs or direct people to put patches on Phabricator.

As a result, users file a lot of un-actionable bugs that waste our time and frustrate them when they are closed; many enthusiastic and competent programmers don't contribute; and patches on Bugzilla frequently languish forever. We should work on guiding new potential contributors.

What it will take

  • Change the attachments page to notify users that patches should be submitted to Phabricator instead, along with a link to a how-to page (see D15719)
  • Close for new bugs and hide all unmaintained and obsolete products.
  • Change various statuses to be more meaningful and obvious to users (see https://bugs.kde.org/show_bug.cgi?id=383753):
    • UNCONFIRMED -> REPORTED
    • WONTFIX -> INTENTIONAL
    • INVALID -> NOT A BUG
  • Add project-specific templates to the description field for new Bugzilla bugs that gives users information they need to solve their own issue or provide the information necessary for us to actually triage the bug (see D15630)

How we know we succeeded

  • Bugzilla bugs won't have patches in them that fall by the wayside
  • The volume of un-actionable bugs will slow dramatically and there will be less back-and-forth between developers and users, freeing up bug-triaging resources and turning them into development resources. Bugs will be more valid, better tracked, and not pile up forever
  • More bugs will be accompanied by patches on Phabricator
  • We'll get more new contributors submitting patches on Phabricator

I am willing to put work into this

  • Nate Graham (@ngraham) (writing and deploying documentation and the new Bugzilla template and statuses)
  • Andrew Crouthamel (@acrouthamel)

I am interested

  • Gregor Mi (@gregormi)
  • Hopefully other people
There are a very large number of changes, so older changes are hidden. Show Older Changes
neofytosk added a subscriber: neofytosk.EditedOct 2 2017, 12:49 PM

I was working in parallel in this proposal I just submitted, my suggestion is about a more overarching effort relating to the onboarding of new contributors:
https://phabricator.kde.org/T7116

I think the two could merge as we go ahead, depending on the outcome of discussions.

lydia added a comment.Oct 2 2017, 6:23 PM

Yeah I think it is a good idea to merge them.

ngraham updated the task description. (Show Details)Oct 2 2017, 7:17 PM
cfeck added a subscriber: cfeck.Oct 2 2017, 8:33 PM

Regarding the 'NEW' status: We used that instead of 'CONFIRMED' earlier, see https://bugs.kde.org/show_bug.cgi?id=195305

'UNRESOLVED' (to match 'RESOLVED') or 'OPEN' is what I would prefer for the merged status.

I mentioned a previous discussion about states in bugzilla, but I didn't link the archives. This is the starting point of the thread:
https://mail.kde.org/pipermail/kde-community/2016q4/003262.html

I don't feel strongly about the word we replace it with, but I do agree that we should combine both statuses into one.

gregormi updated the task description. (Show Details)Oct 3 2017, 9:38 AM
gregormi added a subscriber: gregormi.

I think good and stable bindings to Python (see e.g. https://phabricator.kde.org/D7736) can help to attract new contributors. Python is often used in educational contexts. It can be used for quick prototyping, could be used to write small productivity tools with Python/Qt on top of KFrameworks or extending existing applications like Kate with Python-based plugins. This could serve as a low level entry to get familiar with Qt and KDE related technologies.

Hi! my name is Paco and I've started developing for kde two months ago and I'm just getting started... I had worked in other open source project with a smaller scope and, when starting to work with kde what I've missed is:

  • when working in little projects and know someone (specially someone who has experience with the project) get stuck is less common and, when getting troubles, you allways have someone with whom have a relationship to ask often without feel dumb
  • when working in little projects/hackatons you can easily know more people who is starting as you and you feel you're not the only who is a bit lost in the project...
  • when working in little projects is easy to find something you can do and to organice with others to avoid working in the same thing than others, and also is easy to ask 'ey, I'm interesting in this, can you suggest me some issues/tasks to do?' instead of seeing a lot of projects and issues finding something to do.

For these reasons I would like to suggest a dinamic, just for disscus:

We could make 'beginner teams' formed by 4 or 5 beginner and one veteran developer (some kind of mentor) in order to get things more easy. These teams would work arround the mentor ideas; for example, like a mentor, I would like to improve plasma tasks switchers, so I made a beginner team for this and select some tasks from bugzilla/phabricator for beginner who want to work in this with me. Then, people sign up to the team and I start to guide them to solve these tasks. There would be a chatroom in phabricator for the team in order to discurss about what could do each one, beginner could help each others, and the mentor could get things done faster than if he/she works alone.
When things get done, the group could be closed or continue working in other stuff...

What do you think about it?

jsamuel added a subscriber: jsamuel.Oct 7 2017, 4:07 PM

@navarromorales Mentoring is indeed a very important aspect of improving the contribution process and it can help solving issues like the ones you describe.

I know KDE participates in external initiatives like Google's Summer of Code and Code in, but I don't recall seeing something specifically coming from within. So a suggestion here could be towards KDE becoming more systematic in terms of mentoring and organizing its own events related to it.

So, what do we do with this proposal and https://phabricator.kde.org/T7116 now? They are so similar that it really makes little sense from my perspective to keep them separated.
Can @ngraham and @neofytosk maybe see how to proceed with merging them into one which contains all aspects of both?
Otherwise votes would end up split between the two, effectively reducing the chance for either of them to be selected.

@colomar I strongly support the merge. I just added https://phabricator.kde.org/T7116#114484 that would also apply to this ticket. So what shall I do? Link it? Repost it here aswell? Or wait for the merge? ;)

neofytosk added a comment.EditedOct 20 2017, 1:11 AM

Yup, totally agree with merging as these two obviously overlap. Am just not sure if and how merging can be done through phabricator without all the current comments being lost.

If that is not possible, we could start a new ticket to do this. As T7116 is more general in its spectrum, my suggestion is to import T6832 into T7116 as a subsection specifically on coders and bugzilla. Then add a comment including all the relevant comments from both proposals, of course attributing everything accordingly. It would look like this: https://mypads.framapad.org/mypads/?/mypads/group/kde-0fv8g70p/pad/view/kde-proposal-for-new-contributors-y95n8t786. Do let me know if I missed someone/something!

If @ngraham and the rest here agree, I can go ahead and do that.

@neofytosk Sure, I'm fine with that.

Please do merge. The comments will still be available in the ticket that is marked as a duplicate. And it will be linked from here automatically when you merge it in.

neofytosk renamed this task from Improve contribution process for new users to Improvements to Bugzilla - Making it easier and simpler.Apr 5 2018, 7:20 PM
neofytosk reopened this task as Open.
neofytosk updated the task description. (Show Details)
neofytosk added a comment.EditedApr 5 2018, 7:31 PM

In regard to UNCOFIRMED/CONFIRMED status, there's a relevant discussion here:
https://mail.kde.org/pipermail/kde-community/2018q1/004395.html
And more about bugzilla's usefulness here:
https://mail.kde.org/pipermail/kde-community/2018q1/004274.html

I've replied to those threads hoping to get the discussion started again so we can implement the changes we find are more suited for our developers' workflow.

Feel free to add further sub-tasks for specific topics/suggestions relating to this.

And more about bugzilla's usefulness here:
https://mail.kde.org/pipermail/kde-community/2018q1/004274.html

A main point of the discussion in this thread was that we should clean up Bugzilla through removing old & unmaintained products. I just did some basic searching and found 120+ products which had Unmaintained / Deprecated / Replaced by in their summary. There are probably even more products which are unmaintained since multiple years. I think it would be quite helpful to remove these from the "Browse" list in bugzilla. If wanted, I can sit down & filter out all unmaintained or deprecated applications in the next few days.

I decided to do something remotely useful with my free time and sifted through all applications/libraries present on bugs.kde.org, removing most obsolete products. My guidelines to determine whether a product was deprecated were:

  1. Is the product not marked as "unmaintained"/"deprecated"/"replaced"?
  2. Is the product in the repositories of Ubuntu LTS?
  3. Is the product on cgit.kde.org?
  4. If it is on cgit.kde.org, does it have any activity in the last year?

After applying this criteria to all 737 (!) products available, only 455 remained (so nearly 300 products are obsolete, and should be removed from bugs.kde.org). It is of course entirely possible that I have either missed a deprecated product or have accidentally deleted an active product... Feedback is always welcome!

All products:

Only active ones:

Hm, is there a compelling reason to remove obsolete products from the database? If they are marked as obsolete, no new bugs can be filed against it anyway, and those should not show up in a browsable list for the user. Of course if you have admin rights they will show, but how often does an admin have to browse a product list without a filter?

Just to be sure, @xyquadrat : did you already remove the products? In the future, don't remove first and ask later, but do the opposite. It's a bit too late to ask whether you had accidentally deleted an active product.

Ok, so it seems that you only created a list and you are asking for feedback. Fine then, sorry for the misunderstanding, the wording could be misleading.

lueck added a subscriber: lueck.May 20 2018, 1:45 PM

I decided to do something remotely useful with my free time and sifted through all applications/libraries present on bugs.kde.org, removing most obsolete products. My guidelines to determine whether a product was deprecated were:

  1. Is the product not marked as "unmaintained"/"deprecated"/"replaced"?
  2. Is the product in the repositories of Ubuntu LTS?
  3. Is the product on cgit.kde.org?
  4. If it is on cgit.kde.org, does it have any activity in the last year?

    After applying this criteria to all 737 (!) products available, only 455 remained (so nearly 300 products are obsolete, and should be removed from bugs.kde.org). It is of course entirely possible that I have either missed a deprecated product or have accidentally deleted an active product... Feedback is always welcome!

    All products:

    Only active ones:

Removed from All products:
kcharselect
knetattach
konsolekalendar
korgac
kwikdisk
libalkimia
pimsettingsexporter

but afaik these are maintained and released

Hm, is there a compelling reason to remove obsolete products from the database? If they are marked as obsolete, no new bugs can be filed against it anyway, and those should not show up in a browsable list for the user. Of course if you have admin rights they will show, but how often does an admin have to browse a product list without a filter?

I must admit that I just realized that the list of apps you can report a bug against is not the same as the ones accessible through clicking "Browse"...
But I think we should consider hiding the obsolete products not only when reporting bugs, but also in a) the advanced search & b) in the list from Browse. It'd simplify the search process for triagers a lot. Additionally, there are some products which are obsolete but you can still file bugs against them (e.g kaveau).

Ok, so it seems that you only created a list and you are asking for feedback. Fine then, sorry for the misunderstanding, the wording could be misleading.

I am sorry if my wording was unclear, but I am not even able to do such an action (I'm just a normal bug triager, not a sysadmin).

In T6832#142978, @lueck wrote:

Removed from All products:
kcharselect

Right, I must have accidentally deleted that.

knetattach

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

konsolekalendar

I can find the package in Ubuntu 16.04 (from a very old version), but not a repository

korgac

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

kwikdisk

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

libalkimia

I can find the package in Ubuntu 16.04 (from a very old version), but not a repository

pimsettingsexporter

I found pim-data-exporter, which might be the same thing (?)

lueck added a comment.May 21 2018, 7:36 AM

knetattach

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

plasma-desktop/knetattach/

konsolekalendar

I can find the package in Ubuntu 16.04 (from a very old version), but not a repository

pim/akonadi-calendar-tools/konsolekalendar/

korgac

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

pim/korganizer/korgac/

kwikdisk

Not able to find a repository on cgit.kde.org nor a release in Ubuntu 16.04...

kdeutils/kdf

libalkimia

I can find the package in Ubuntu 16.04 (from a very old version), but not a repository

extragear/office/alkimia/

pimsettingsexporter

I found pim-data-exporter, which might be the same thing (?)

Yes pim/pim-data-exporter/

Thanks @lueck for pointing these applications out; I have now re-added them to my active list.

Updated list:

Do we have any traction on moving forward with removing these products? Or at least closing them out? What's the next steps?

This would probably have to be done by the KDE Sysadmin team, so we should probably forward this list to them. Right now I am a bit concerned that not everybody might be content with the decision to simply remove all of those from Bugzilla... But then again, all products that would be deleted are unmaintained and the big majority does not have a single bug reported against the product.

I have bugzilla admin powers and can make some of these changes. However I'd prefer to take it slowly and conservatively, so we can avoid any issues.

I'm not sure we should actually delete the products, as that would delete the history too. But we should definitely close them to new bugs.

My strategy for unmaintained products has been to mass-close all their bugs as RESOLVED UNMAINTAINED with a gentle and polite message, then mark the product as closed.

If you can provide me with a 100% correct list of products that are both unmaintained and have zero bugs, I can mark them all as closed for new bugs.

Beyond that, it might be a useful change to have products that are closed for new bugs also be invisible on the Advanced Search and Browse lists. This isn't something I can do, and would probably be a sysadmin request. Can you contact sysadmins about that?

Beyond that, it might be a useful change to have products that are closed for new bugs also be invisible on the Advanced Search and Browse lists. This isn't something I can do, and would probably be a sysadmin request. Can you contact sysadmins about that?

I'm not sure about removing them from Advanced Search, because then people could not find old bugs from them if they are looking for them e.g. for hints towards fixing a similar bug.
Removing them from Browse would make sense to me, however.

I'm not sure about removing them from Advanced Search, because then people could not find old bugs from them if they are looking for them e.g. for hints towards fixing a similar bug.

Ok, that makes sense to me.

Ok I'll go through the lists later and get you a definitive one.

acrouthamel updated the task description. (Show Details)Sep 4 2018, 6:30 AM
acrouthamel updated the task description. (Show Details)Sep 4 2018, 6:53 AM
acrouthamel added a comment.EditedSep 4 2018, 8:32 PM

Ok, I've reviewed the prior list. I've done the following for each product:

  • Check Bugzilla for age of any remaining bugs
  • Check Git and SVN for age of commits (not created by automated systems)
  • Check KDE Wikis for information about unmaintained or obsolete status
  • Check Google/DDG for information about unmaintained or obsolete status
  • Closed any open bugs left lingering in an unmaintained, dead product.
  • Used gut instinct on any that may still be used, removing them from the list

I've done my best to ensure unmaintained/dead status.

Attached is my final list for your closure and hiding @ngraham.

Thanks so much, @acrouthamel. That list looks sane to me. Anyone else wanna spot-check it?

Also, it looks like closed-for-bugs products already do get hidden from the Browse view but stick around in the advanced search, so we're all good on that front.

Actually many these are already closed, and the remaining ones are obviously obsolete and unmaintained (no bugs, says "unmaintained", etc). I don't think there's much of a possibility for problems here. Starting the work. It's easy enough to re-open anything that needs re-opening (but I doubt it).

Not sure if you did apply the changes already, but in case you did not, please leave open:

  • atlantik and atlantikdesigner (on the process of being reimported, one is on git already)
  • kgrapheditor is technically part of the released kgraphviewer, even if disabled by default for now iirc, but still "released"

Not sure if you did apply the changes already, but in case you did not, please leave open:

  • atlantik and atlantikdesigner (on the process of being reimported, one is on git already)
  • kgrapheditor is technically part of the released kgraphviewer, even if disabled by default for now iirc, but still "released"

Thanks, will keep those open.

Done now. So what's next? My sense is that we probably should keep UNCONFIRMED/CONFIRMED now that our bugs are more actively triaged, since there's some value to the distinction.

Any interest in examining our RESOLVED sub-statuses? e.g. we could change INVALID to NOTABUG, change WONTFIX to INTENTIONAL or ASDESIGNED, change UPSTREAM & DOWNSTREAM to NOTOURBUG or NOTKDE, etc.

Thoughts?

I would restart from the previous discussion (on kde-community@, and probably elsewhere; for sure it should be discussed on the list).

Personally speaking, I would keep UPSTREAM and DOWNSTREAM, because they makes it clear whether the issue is not just "not ours" but in which direction (a library or service we depends upon, or a consumer of what we produce).
NOTABUG is used on at least another big bugzilla instance.

acrouthamel updated the task description. (Show Details)Sep 4 2018, 10:19 PM

I agree with keeping UNCONFIRMED/CONFIRMED. I was going to make an argument for keeping that as I think it makes sense. Bugs come in, someone looks at it, and reproduces it (or takes other action). So for the triagers I think it makes sense to be able to easily find bugs that have not been reproduced yet.

As for names though we could make it more clear with the following:
UNCONFIRMED -> NEW
CONFIRMED -> REPRODUCED

Again, provided there are volunteer triagers, a bug shouldn't sit in NEW for a long time as previously discussed in the one thread. If it does, then that points to a manpower issue, not a technical one.

As for RESOLVED statuses, I like UPSTREAM/DOWNSTREAM as well. Usually there is a comment explaining where the issue lies, so people shouldn't be too confused as to what up/down means. If you're submitting a bug report, you probably have some set of technical knowledge.

I think softening INVALID and WONTFIX is good though, how about:
INVALID -> NOTABUG
WONTFIX -> ASDESIGNED (I think CANTFIX which I read somewhere as an option, sounds like we're incompetent)

acrouthamel added a subscriber: Bugsquad.
acrouthamel removed a subscriber: Bugsquad.
ngraham updated the task description. (Show Details)Sep 4 2018, 10:45 PM
acrouthamel added a subscriber: sysadmin.EditedSep 5 2018, 12:06 AM

Hi everyone just thought I'd drop a note here as well. I've had @sysadmin recreate the Bugsquad, but in Phabricator for easy collaboration. I'd like to resurrect it, with a set schedule of triaging days. I've asked @sysadmin about adding a calendar to the project, or allowing other Calendar edit access. If not, I'll create a Google Calendar people can subscribe to I suppose.

If you're interested in resurrecting the Bugsquad, please feel free to join it as a member.

I'm sure that there is no need to use Google Calendar after all the effort to move part of release schedule and promo stuff out of it...

I hope not, I'd rather keep it in-house.

FYI for everyone, there is a discussion occurring right now on the kde-community mailing list regarding the status names if you'd like to voice your opinion.

sheedy added a subscriber: sheedy.Sep 12 2018, 2:21 PM
acrouthamel updated the task description. (Show Details)Sep 18 2018, 12:33 PM
ngraham updated the task description. (Show Details)Sep 18 2018, 1:39 PM
acrouthamel moved this task from Backlog to Active on the Bugsquad board.Sep 19 2018, 1:37 AM
acrouthamel triaged this task as High priority.Sep 19 2018, 1:47 AM
acrouthamel updated the task description. (Show Details)Sep 21 2018, 5:07 AM
acrouthamel updated the task description. (Show Details)Sep 21 2018, 8:46 PM
acrouthamel updated the task description. (Show Details)

@sitter - I just wanted to loop you in here since you had offered to script the NEEDSINFO closure bot.

acrouthamel updated the task description. (Show Details)Sep 24 2018, 3:36 AM
acrouthamel updated the task description. (Show Details)Sep 25 2018, 1:24 PM
acrouthamel updated the task description. (Show Details)Sep 25 2018, 1:53 PM
acrouthamel removed a subscriber: sitter.
acrouthamel closed this task as Resolved.Sep 25 2018, 1:57 PM
acrouthamel claimed this task.

I'm going to pull the automated NEEDSINFO handling from here, since this task is specific to new bug reporter experience. I'll work with Harald or whoever else would like to help. For now, I have a manual set of searches that seem to work well enough. I have some other automation ideas as well, so I'll just track that all in a separate task.

Due to the completion of the prior sub-task, the checklist is complete regarding improving Bugzilla for new bug reporters.

I'll mark this resolved.

If anyone wants to help me with scripting/automation of some Bugzilla tasks, please contact me.

Woohoo! Awesome work, Andrew. Keep an eye on http://pointieststick.wordpress.com/ tomorrow. :-)

rempt added a subscriber: rempt.Sep 27 2018, 5:46 PM

hm... Not sure if this is part of this task, but the new template says

"SOFTWARE VERSIONS
(available in About System)
KDE Plasma Version:
KDE Frameworks Version:
Qt Version: "

And apart from the Qt version, that's irrelevant for projects like gcompris, and apart from that and the frameworks version (which practically is determined by me), it's irrelevant for projects like krita. 95% of my users don't use KDE plasma, and 90% don't know what it is or why.

Since we've got the new template, my bug reports have become markedly less useful. See for instance https://bugs.kde.org/show_bug.cgi?id=399150 -- so there might be some room for optimization.