I'm not sure what sort of permissions we'd be looking at here - from what I can tell the permissions on the server that generates the documentation, along with the permissions on the server that actually serves docs.kde.org are correct and fine.

Based on the logs for the generation process it appears that it isn't even trying to generate the docs anymore.

Tracing through the logs I see that it broke on 19/12/2019, with the following messages. After this no more attempts were made to build the docs for Krusader trunk anymore, so I assume it was blacklisted somewhere (hopefully Luigi knows more about this)

doc_20191219_021504.log:2019-12-19 02:44:42,023:WARNING:kdedocs.environments:Error running "nice -n 19 xsltproc --stringparam kde.common /trunk5/en/kdoctools5-common/ /home/docs/docs/work/trunk5/customization/kde-chunk-online.xsl /home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook" from "/home/docs/website.next/trunk5/en/extragear-utils/krusader", catalog: /home/docs/docs/work/trunk5/customization/catalog.xml:
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/introduction.docbook:96: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/introduction.docbook:106: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/introduction.docbook:152: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:347: parser error : Failure to process entity introduction
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:347: parser error : Entity 'introduction' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/user-interface.docbook:319: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/user-interface.docbook:352: parser error : Entity 'USB' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/user-interface.docbook:818: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:349: parser error : Failure to process entity user-interface
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:349: parser error : Entity 'user-interface' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/vfs.docbook:27: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/vfs.docbook:38: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/vfs.docbook:104: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/advanced-functions.docbook:22: parser error : Failure to process entity vfs
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/advanced-functions.docbook:22: parser error : Entity 'vfs' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/advanced-functions.docbook:61: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:351: parser error : Failure to process entity advanced-functions
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:351: parser error : Entity 'advanced-functions' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/remote-connections.docbook:185: parser error : Entity 'Debian' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/remote-connections.docbook:185: parser error : Entity 'Debian' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/remote-connections.docbook:282: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/krusader-tools.docbook:29: parser error : Failure to process entity remote-connections
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/krusader-tools.docbook:29: parser error : Entity 'remote-connections' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/krusader-tools.docbook:34: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:355: parser error : Failure to process entity krusader-tools
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:355: parser error : Entity 'krusader-tools' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/faq.docbook:214: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/faq.docbook:214: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/faq.docbook:270: parser error : Entity 'Samba' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/faq.docbook:277: parser error : Entity 'Samba' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/faq.docbook:1038: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:357: parser error : Failure to process entity faq
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:357: parser error : Entity 'faq' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/glossary.docbook:77: parser error : Entity 'Debian' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/glossary.docbook:77: parser error : Entity 'Debian' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/glossary.docbook:348: parser error : Entity 'NFS' not defined
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/glossary.docbook:380: parser error : chunk is not well balanced
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:366: parser error : Failure to process entity glossary
doc_20191219_021504.log:/home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook:366: parser error : Entity 'glossary' not defined
doc_20191219_021504.log:unable to parse /home/docs/docs/work/trunk5/en/extragear-utils/krusader/index.docbook

It's not a permission problem; it's "just" that the environment can't compile kdoctools anymore due to the requirements for a newer Qt, so all documentation which uses the newer entities fails.

It means I need to change the workflow and remove the generation of kdoctools, and rely on something prebuilt - which must be a chroot or a container.

I'm experimenting with a low-tech solution with schroot and neon packages. Containers would be another possibility, but that would mean generating everything inside them and keeping the compilation logic, and nowadays I suspect that removing part of the compilation duties from the generation code is definitely a better idea.

Could a statically compiled version of meinproc be used for this?

Also as a temporary measure, assuming the entities aren't compiled into meinproc, would it be possible to install just the updated resources shipped in kdoctools?

The source code of Krusader has been modified in order to make Krusader build with older software.
With that, maybe some time can be gained.

Anyway, as Yuri considered, with enough permissions to modify the aforementioned places, files can be added manually and so problems can be solved (e.g. people seeing broken links, and unable to find documentation) while other solutions arrive or not.

Due to the automation surrounding docs.kde.org, it isn't possible to just place files there manually i'm afraid.

Luigi, do we have any options regarding a statically built binary of meinproc5?

A statically built binary of meinproc5 won't work. Manually copying the files may work as workaround, but it requires code changes anyway to not overwrite the changes.

Related question: can I upgrade library.k.o to bionic? When I approved D25928 I did not realize that one of the python 3 packages was missing on xenial.

That is unfortunate that a static binary isn't an option.

I see no reason why the system can't be upgraded to Ubuntu 18.04 (Bionic). If that won't resolve the meinproc5 issues though, we should consider looking into our options around splitting the parts of the build up into smaller pieces and getting it performed by the Binary Factory in it's containers.

Splitting the parts does not solve the issue, unless you mean removing the part which currently builds the kdoctools sources, which I agree on - that's why I want to use a chroot or a container and I'm working on it. The rest of the builder does not need changes.
The xenial->bionic upgrade won't solve the issue, but it is an unrelated blocker that needs to be solved first after D25928 (my bad), or I can't test any other change.

I'm also working on a version of docs.kde.org using docker for the development setup: https://invent.kde.org/carlschwan/docs-kde-org. I build it because the SoK student who wanted to work on docs.kde.org wasn't able to set up the quite complex development setup. Maybe some parts could be reused for a binary factory setup.

In T12568#218880, @ognarb wrote:

I'm also working on a version of docs.kde.org using docker for the development setup: https://invent.kde.org/carlschwan/docs-kde-org. I build it because the SoK student who wanted to work on docs.kde.org wasn't able to set up the quite complex development setup. Maybe some parts could be reused for a binary factory setup.

That's a bit of an orthogonal problem, I think. Let's see.

Splitting the parts would make it easier to transition the overall build process into a containerised environment (such as that used for building many of our websites on the Binary Factory already), which would eliminate the need for a special server setup for docs.kde.org and make it easier to use things from the latest version of Qt/KF5 on an as needed basis (because containers for building documentation can be based on something like OpenSUSE Tumbleweed, rather than needing to be an LTS server distribution, because the containers will never be public facing)

In the long term this will make the setup easier to maintain and replicate elsewhere (for those that wish to play with it).

If the requirement is the usage of Binary Factory and remove a VM, the current code can be executed in a separate (neon, at this point, never tested with Tumbleweed) container.
However, the big cost in terms of time and resources is the checkout, which would happen with any possible technology; removing the separate server would make that more painful, IMHO. If the problem is the reproducibility of the setup, I have a draft of ansible code too to address it.

The long term objective is to simplify our systems to remove special setups yes, particularly when it comes to long running cronjobs.

The first thing we've shifted to the new approach has been the statically generated Websites, an approach that has worked quite well so far for us.
In the long term I would like to see the generation of api.kde.org, docs.kde.org and the process behind scripty shift into this as well.

In terms of the checkout, this is why I was referring to splitting things up. At the moment the system is reliant on a monolithic process, designed to be run on a single system overnight.
The process of splitting it up would involve breaking down this monolithic process into small chunks (such as down to the individual application level) that can be executed on an as needed basis, so only the bits that have changed get rebuilt.

In the meantime, the generation is working again (or so it seems), but the PDF for some languages can't be generated.

For example for Ukrainian (and Russian) I see
LaTeX Error: Command \\CYRP unavailable in encoding EU1.

and also other warnings and errors related to fonts.

@yurchor , could you please take a look too?

In T12568#220641, @ltoscano wrote:

In the meantime, the generation is working again (or so it seems), but the PDF for some languages can't be generated.

For example for Ukrainian (and Russian) I see
LaTeX Error: Command \\CYRP unavailable in encoding EU1.

and also other warnings and errors related to fonts.

@yurchor , could you please take a look too?

Must be the famous Debian fracturing of TeX packages. Some packages should be installed (texlive-lang-cyrillic, texlive-fonts-extra, I think). It works fine locally.

It would be helpful to see the full log.

In T12568#220642, @yurchor wrote:

In T12568#220641, @ltoscano wrote:

In the meantime, the generation is working again (or so it seems), but the PDF for some languages can't be generated.

For example for Ukrainian (and Russian) I see
LaTeX Error: Command \\CYRP unavailable in encoding EU1.

and also other warnings and errors related to fonts.

@yurchor , could you please take a look too?

Must be the famous Debian fracturing of TeX packages. Some packages should be installed (texlive-lang-cyrillic, texlive-fonts-extra, I think). It works fine locally.

It would be helpful to see the full log.

That was my first thought too, but they are installed (almost all texlive-lang* and texlive-fonts-extra).

Let me try with texlive-formats-extra.

No luck:

~/docs/work/stable5/ru/kdegames/kpat$ nice -n 19 /home/docs/docs/buildpdf.sh /home/docs/docs/work/stable5/ru/kdegames/kpat/index.docbook


No external file support
XSLT stylesheets DocBook -  LaTeX 2e (devel)
===================================================
index.docbook.rtex -> index.docbook.tex
Compiling index.docbook.tex ...
Link to warning.pdf
latex index.docbook_tmp.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex)
 \write18 enabled.
entering extended mode
*** latex error
(changebar)        Even left: 2.61372pt Even right: 458.1075pt.

\c@lstlisting=\count170

! LaTeX Error: Encoding scheme `TU' unknown.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.292 \begin{document}
Type  I <command> <return>  to replace it with another command,
or  <return>  to continue without it.


! LaTeX Error: Command \CYRR unavailable in encoding T1.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
/tmp/foo2.txt

In T12568#220644, @ltoscano wrote:

No luck:

Ok. I will try to make thorough testing and provide test cases to debug the problem tonight.

In T12568#220645, @yurchor wrote:

In T12568#220644, @ltoscano wrote:

No luck:

Ok. I will try to make thorough testing and provide test cases to debug the problem tonight.

The problem should be fixed now (new misc.xsl is needed). The order of instructions in the preamble is crucial now...

From what I understand this has now been effectively resolved?

It seems it worked, thanks a lot Yuri!