Automatically run pg_upgrade when major PostgreSQL upgrade is detected
ClosedPublic

Authored by dvratil on May 12 2019, 12:56 PM.

Details

Summary

This implements detection of version of the db_data cluster and the currently
installed Postgres version (on Linux for now, only), and upgrading the cluster
when Postgres major version upgrade is detected.

This should make it more comfortable for users to use Akonadi with Postgres
since they no longer have to care about Postgres upgrades. The upgrade
will fail if we fail to find executables for the previous Postgres
server version (the one from which we are upgrading), which may differ
between distributions and usually requires an additional package to be
installed.

The process involves creating a new db cluster using the new version of Postgres,
then basically dumping data from the old cluster (that's why we need executables
from the previous version) and importing them into the new cluster. Finally, the old
and the new clusters are swapped and the old one is removed.

Test Plan

Tested on Fedora, upgrading from Postgres 10 to 11

Diff Detail

Repository
R165 Akonadi
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
dvratil created this revision.May 12 2019, 12:56 PM
Restricted Application added a project: KDE PIM. · View Herald TranscriptMay 12 2019, 12:56 PM
Restricted Application added a subscriber: kde-pim. · View Herald Transcript
dvratil requested review of this revision.May 12 2019, 12:56 PM
dvratil retitled this revision from Automatically run pg_upgrade when major PostgreSQL is detected to Automatically run pg_upgrade when major PostgreSQL upgrade is detected.May 12 2019, 12:57 PM
dvratil edited the summary of this revision. (Show Details)May 12 2019, 12:59 PM
knauss added a subscriber: knauss.May 13 2019, 9:26 PM

overall I like this patch - that is great an brings postgresql more to a usable db interface.

src/server/storage/dbconfigpostgresql.cpp
100

We do another if clause for version 11?

211

For Debian it looks like:

pg_ctl (PostgreSQL) 11.3 (Debian 11.3-1)

I would make the regex more explicit, to match to the correct version:

re(QStringLiteral("\(PostgreSQL\) ([0-9]+).[0-9]+"))

256

for Debian it is just:

/usr/lib/postgresql/%1/bin/

You should use the same logic like in postgresSearchPath from DbConfigPostgresql::init to find the old bin dir. I can't see why the old version should be located differently.

262

please change order to:

  1. search for oldBinPath (line 268)
  2. test if oldBinPath.isEmpty(line 275)
  3. remove old_db_data (l260)

This is more in the sense, that we do not touch anything, if the old Postgres is not installed, as we can't migrate at all.

284

shouldn't we check if newDbData alread exists before runInitDb? because this is properly an error sate, as newDbData should not exist at this state.

dvratil marked 4 inline comments as done.May 14 2019, 11:52 AM
dvratil added inline comments.
src/server/storage/dbconfigpostgresql.cpp
100

Good catch, let me fix this separately (and propperly).

256

Thanks.

Tha main difference between this and the lookup in init() is that here we are looking for a specific version of Postgres, while in init() we are looking for the newest one (without knowing what version it is)

dvratil updated this revision to Diff 58062.May 14 2019, 11:54 AM
dvratil marked an inline comment as done.

Addressing comments from review:

  • Improve parsing version from pg_ctl output
  • Add versioned install paths for Postgres on Debian
  • Refactor the main upgradeCluster() method, split it to smaller functions and reorder them a little
knauss added inline comments.May 14 2019, 10:53 PM
src/server/storage/dbconfigpostgresql.cpp
215

really "\\(" ? this is different in different languages, did you tested your regex?

256

I understand the difference, but in` init() you ONLY check /usr/lib/postgresql/*/bin` and not
/usr/lib64/pgsql/postgresql-*/bin and /usr/lib/pgsql/postgresql-*/bin. I would argue that those dictionaries should be searched in init() too. Because also for other Distributions the new postgres version won't be available in normal search path.Otherwise we will end up in situations, where we fix one place and add a new Distribution style and forget in the other function et visa versa.

dvratil added inline comments.May 19 2019, 9:01 AM
src/server/storage/dbconfigpostgresql.cpp
215

The regular expression is "\(", you have to escape the backslash in the C string.

256

The difference is that on Fedora for instance, the latest version is in /usr/bin, /usr/lib64/pgsql/postgresql-* contains the older versions - in init() we only want the latest version, while here we actively look for an older version.

I will refactor this together with the update to versioned lookup in a next patch.

dvratil updated this revision to Diff 58283.May 19 2019, 9:54 AM

Fix a typo

knauss accepted this revision.May 19 2019, 8:20 PM

It looks fine now. The two open issues will be addressed separately.

This revision is now accepted and ready to land.May 19 2019, 8:20 PM
This revision was automatically updated to reflect the committed changes.