Create Discourse Setup
Closed, ResolvedPublic

Description

Discourse is a lovely new forum/mailing list software which is proving popular elsewhere.

At Akademy 2019 it was said that it would be good to set it up for KDE to move over forum.kde.org and, when projects are ready, mailman lists.

We had some queries which I sent to the devs and which are answered here
https://meta.discourse.org/t/discourse-setup-for-kde/128193/1

I set up a testing instance here which works fine
http://discuss.kde.org.uk/
but it still needs some of the plugins and settings discussed in the post above to be decided.

List of issues we had before Akademy here
https://community.kde.org/Infrastructure/Evaluation/Discourse#TODO

I suspect identity will be the blocker for now.

I'm happy to do the setup if given access to machines to do it on.

Restricted Application added a subscriber: sysadmin. · View Herald TranscriptSep 17 2019, 11:53 AM
sitter added a subscriber: sitter.Sep 18 2019, 12:41 PM

(in lieu of a new oauth-enabled identity, if someone shows me code how to use current identity ldap shebang and/or give me some info needed for that I can try to throw together an auth plugin ... not quite sure how to test that reliably without access to anything though?)

Did anyone test the mail interface yet? That would be very useful to reduce friction: in openSUSE, the - proprietary - forum used has a mail interface and this allows people to ask for and offer support according to their medium of choice.

Yeah, we used Jonathan's test instance basically like a mailing list. It works very well.

Discussed issues with upstream who seem responsive and happy to help with them

+1 on the general idea.

Thanks for confirming Upstream seemed responsive.

Given that the categorisation/grouping issue was the biggest hard showstopper, do we know how easy that one is for them to resolve?
(Without that being resolved or a solution promised in a given timeframe, we really cannot proceed as we won't be able to roll this out at all)

Some Krita contributors (@kamathraghavendra) are hosting their own discourse instance at krita-artist.org now. I think it would be interesting to see how well it work for their needs (less help request on irc, better user to user support, better community engagement, ...) .

https://krita-artists.org/ looks lovely and shows how previews are possible for more arty forums.

As I understand Ben your issue with categorisation was having three level hierarchy. Discourse can only do 2 levels of hierarchy, but on the current forum the top level of hierarchy is just a label for sorting you can't post into it and it doesn't help much when searching for a particular forum, Discourse has better search feature for those looking for a forum.

I'd need to check, but I believe Krita Artists has been modified with custom code to support things that such a forum needs.

In terms of categorisation, some of our applications (Amarok & Krita are two that immediately come to mind) do have categorisation within their forum, which brings it to three levels in some cases. This can possibly be worked around though in some cases I suspect.

Worst case we could just name categories to simulate this "Amarok - General" "Amarok - Special" "Amarok - Devel". I expect actually changing upstream would be non-trivial because infinite nesting does have a huge impact on the UI requirements and is also distinctly unfriendly for mobile browsing. That being said, I would revisit the nesting anyway. A lot of it actually can be addressed by utilizing tags instead and have a flatter category structure in general. Which would also be more conducive to community building; subforums act like walled gardens.
I'd really like to get a technical demo off the ground. That is to say: have a test setup we can test-import into, so we can actually identify specific problems to talk to upstream about.

A technical demo would be inaccessible to everyone except those assessing it I assume?

My main concern at the moment is limiting the number of requests we get for setting things up / please give us access to / etc, as i'd really like to focus on the last bits and pieces so we can deliver repository hosting and code review on Gitlab.

merritt added a subscriber: merritt.Mar 4 2020, 6:46 PM

+1 for the idea of Discourse

The current KDE forums feel a bit clunky by today's standards.

I have noticed personally that for projects that use Discourse, engaging with the community feels easier, less barrier to entry, great search tools, post formatting, subscribe via RSS, etc.

For the Docker problem, LCP from openSUSE wants to package discourse for openSUSE. This would probably make it easier for us to host on a server without Docker.

Also, an alternative to Discourse would be updating our mailman 2 installation to mailman 3 and use the hyperkitty archiver to display and interact with the archives. This creates a modern way (forum like) to interact with mailing lists while still being mail based. See for example how this is done in Fedora: https://lists.fedorahosted.org/archives/.

kamathraghavendra added a comment.EditedJul 26 2020, 4:26 AM

I'd need to check, but I believe Krita Artists has been modified with custom code to support things that such a forum needs.

We use a plugin called "Topic List Previews" to enable masonry layout in the artwork category and tags. It is also used to have a top row of featured artworks. Another plugin is for marking an answer as a solution for support request. Both plugins are well maintained and are open source. We don't change the upstream code and try to use minimal plugin as we can so that it is easy for maintenance during upgrades. The only thing to keep in mind is that discourse development moves very fast and timely upgrades are encouraged so if you have more plugins sometimes it becomes a hassle balancing the compatibility and discourse upgrades.

In my experience and observation, having discourse as a forum has really helped us getting good feedback and interaction with our users. Developers find it easier to interact with useds and provide nightly build packages to get quick feedback from the users. Feature requests, bugs are first discussed among users so it reduces a bit of load from bugzilla (i don't know ho much ). It also has enabled us to create a community of users who will provide community support to other users, slowly taking away the burden from developers.

Yeah hyperkitty is pretty nice too.

ognarb removed a subscriber: ognarb.Aug 14 2021, 11:49 AM
sitter added a comment.Nov 9 2022, 1:03 PM

Can we select a server and start setting up new.forum.kde.org? It's quite clear that things aren't going to change and people keep asking for discourse. It's starting to be a running gag that we can't have nice things, and I'd prefer it not to be.

I'll take care of setup and maintenance and whatnot.

Nate raised a similar query somewhat recently.

In keeping with the tradition of how KDE selects software platforms to use i'd prefer rather than just going with the platform everyone is familiar with that we review what is out there so we don't just choose the most immediate to hand well known option. One particular piece of software (which has a fairly Discourse like UI) that i'd like to be considered as part of that process is Flarum - which would fit very well within our existing systems and appears to be under active development (and is community driven as well to boot).

Thoughts?

This late into the discussion of getting a new forum that seems like a pointless delay. There is overwhelming desire to use discourse in the community already, and it is the go-to solution in comparable large communities. And it's not like reviews save us from going down the wrong route (phabricator ;);))

We had the discussion at Akademy in 2019 and reviewed various alternatives and decided on Discourse which is why this request was filed. It's not hard to do and there's plenty of people willing to do the work. But yeah plenty of people like to joke about it being an example of a solution we all agree on which just doesn't happen.

ngraham added a subscriber: ognarb.Nov 11 2022, 7:56 PM

I agree that there is overwhelming desire to use Discourse.

Do we have documentation of what alternatives were evaluated and how the decision was made in 2019 at that Akademy? That seems important from a process point of view so nobody feels left out of the discussion and uses that as the basis for blocking the decision, which is where we've been for three years.

If we don't have any documentation on that, I would humbly suggest we do it again, online, in public, so that everyone feels like they got a seat at the table. In https://invent.kde.org/sysadmin/task-queue/-/issues/31, @ognarb had some good things to say about Flarum, which Ben brought up here too. If it meets our needs, it's easier to deploy, and it's backed by a community rather than a corporation, that sounds pretty good to me.

Hopefully doing such an evaluation--possibly even of just these two options--should not take much time. If the result of that evaluation is the same, then great, now we have buy-in from everyone. If the result is different, then we avoid making a mistake. Seems like a win-win to me.

Can we all agree on this as an actionable path forward so we're not all still arguing about the same thing again in another year?

I don't see how it's my job to disproof a proposal I am backing. The discourse eval is linked in the description of this ticket. Carl did some evaluation of other software, and from where I am standing discourse comes out ahead there as well. And then the requirement list wasn't even complete - we ultimately want to have mailing lists move into a forum (or somehow get deprecated in favor of it), discourse is the only thing properly supporting that. Krita already uses discourse successfully. So does Ubuntu, Fedora, Gitlab, GNOME... Let's not make this more complicated than it is.

I added a new comment to the flarum evaluation: https://invent.kde.org/sysadmin/task-queue/-/issues/31#note_561132 Personally I think discourse is the best and more mature solution and like Sitter I would also be willing to help with the deployment and maintenance of this new service.

We are now planning to move ahead with the setup of a discourse test instance so we can get a sense of how it would work in practice and what, if any, pain points we will hit. Hopefully starting this weekend.

There is the general question of whether to migrate the data out of the old forum. I've pondered this a bit and concluded that there is very little worthwhile stuff. Large amounts of data would be from years ago and probably no longer be applicable. Then there's the brainstorm which is a bit of a mess and getting ignored anyway. Active threads would need to get manually recreated in the new forum, but it saves us all the legacy noise. Would love some thoughts from others on this.

Talking about, we can probably take the opportunity to restructure the categorization substantially. In particular since discourse supports tagging we can probably get away with way fewer categories and instead rely on tags. Many categories are under utilized anyway. For a starter setup I'd propose categorization of the style: Announcements, Brainstorm [arguably could be dropped, but I think we need a place for users to put ideas?], Software (Subcategories: Development, Plasma, Applications), Community (Subcat: VDG, Eco, Translations). Heck, we could even choose to not have the subcategories and instead rely on tagging because we can force new threads into having a tag from a given tag set, so for example to post a question about kwin you can post into the software category but then you have to select a tag from the set [plasma, applications, development]. Food for thought.

Another open question is what to do with the localized portions of the forums. Having them in the same instance feels a bit iffy because it devalues the thread feeds for everyone and seems to do a disservice to building the local community identity. What might make sense is to host them as completely separate forums fr.forum.kde.org etc., it's my understanding that discourse can do this at very little extra cost. https://meta.discourse.org/t/multisite-configuration-with-docker/14084

Authentication-wise we'll have to do some testing. Enabling Google, Github, Facebook, Twitter Auth, along side invent.kde.org OIDC seems the most obvious (all of these need setting up from the sysadmin side for client token magic stuff). One additional thing I'd like to see is whether we can get group membership from invent synced into discourse such that developers get a developer badge automatically.

For mailing list style use I believe we need a pop3 set up (@jriddell am I right?), to that end we'll probably need a forum account on letterbox to pop3/smtp through.

frdbr added a subscriber: frdbr.Nov 18 2022, 1:42 PM

We are now planning to move ahead with the setup of a discourse test instance so we can get a sense of how it would work in practice and what, if any, pain points we will hit. Hopefully starting this weekend.

Could Kdenlive participate in this test?

There is the general question of whether to migrate the data out of the old forum. I've pondered this a bit and concluded that there is very little worthwhile stuff. Large amounts of data would be from years ago and probably no longer be applicable. Then there's the brainstorm which is a bit of a mess and getting ignored anyway. Active threads would need to get manually recreated in the new forum, but it saves us all the legacy noise. Would love some thoughts from others on this.

The hay days of the forum usage are long gone, although there are still some active posts/users which we can ask people to migrate to discourse. No need to carry the legacy data. (Would be nice to have the forum locked and accessible though at least for some time.)

Talking about, we can probably take the opportunity to restructure the categorization substantially. In particular since discourse supports tagging we can probably get away with way fewer categories and instead rely on tags. Many categories are under utilized anyway. For a starter setup I'd propose categorization of the style: Announcements, Brainstorm [arguably could be dropped, but I think we need a place for users to put ideas?], Software (Subcategories: Development, Plasma, Applications), Community (Subcat: VDG, Eco, Translations). Heck, we could even choose to not have the subcategories and instead rely on tagging because we can force new threads into having a tag from a given tag set, so for example to post a question about kwin you can post into the software category but then you have to select a tag from the set [plasma, applications, development]. Food for thought.

As you said having a category Applications with a Kdenlive subcategory could be an option. Then we could have tags like Support, Rendering, Showcase, Effects and Transitions, etc...

Applications

  • Kdenlive
  • Krita
  • Discover
  • Ghostwriter

Or would it make sense to have a category being Kdenlive with subcategories like Support, Rendering, Showcase, Effects and Transitions, etc... and tags could be stuff like coloring, ffmpeg, profiles, workflow, etc..

Kdenlive

  • Support
  • Rendering
  • Showcase
  • Effects and Transitions

Maybe I missed another option as well.

bcooksley closed this task as Resolved.Dec 29 2022, 7:45 PM
bcooksley claimed this task.

This has essentially been completed, with the remainder relocated to https://discuss.kde.org/t/things-to-get-sorted-before-inviting-the-community-for-testing/18 so we'll track it there now.

frdbr added a comment.Dec 29 2022, 8:10 PM

How can I get an invite?

They'll be available once the instance is open for testing.