Create Task
Maniphest T3719

workspace sideloading
Open, Needs TriagePublic
Actions

Description

At Akademy we can up with the idea of making dev/unstable better to use for developers. Right now if a dev would quickly need to test something in kio they still need to clone kio and build it from scratch.

Enter sideloading.

While cmake is not relocatable, our workspaces have a static somewhat neat path /workspace/$foo-$version/ on the server. We could put the workspace in a tarball and publish it into an s3 or similar system, then have a client side script which downloads the workspace tarballs and unpacks it for use.

Server

  • tarball workspace
  • xz it
  • source package probably doesn't need any additional work since we publish that via apt archive anyway
  • archive workspace tarball somewhere with good storage capacity (s3?)
  • garbage collect regularly

Client

  • script to prepare a build env
  • needs to make sure system is up-to-date
  • build-dep $foo
  • fetch workspace tarball
  • unpack
  • chdir into it, possibly print message about used cmake flag and where the build dir
  • resides

Open Questions

  • our sources aren't full clones, how could we merge our existing artifacts into a full repo without having to have our source become super huge by preserving .git?
  • zsync or similar binary diff download?

Related Objects

sitter created this task.Sep 13 2016, 9:24 AM
sitter updated the task description. (Show Details)Sep 13 2016, 9:26 AM
sitter added a comment.Nov 21 2016, 8:01 AM

This may actually be slightly-entirely more tricky.

Contrary to believe we do not actually use /workspace/foo-version anymore but a static /workspace/build. Combined with the fact that CMake caches cannot be moved without rebuilding this makes things ever so much more complicated.

Options are:

  • client-side does has some chdir command which temporarily symlinks /workspace/kio into /workspace/build (meh and not reentrent)
  • server side we change the build dir to /workspace/$srcname e.g. /workspace/kio (no version) to kind of retain the original intent of the generic /workspace/build path (fairly static and short) while also allowing easier client side use. This does however come at the price of making the directory not hardcodable in the tooling, meaning this is rather invasive as far as code is concerned
  • we could also do the above in sequence. i.e. symlink as stopgap and only invest in path rework if people find the sideloading useful enough
scarlettclark removed scarlettclark as the assignee of this task.May 12 2017, 8:16 PM
scarlettclark added a subscriber: scarlettclark.
sitter moved this task from Discussing to Backlog on the Neon board.Aug 12 2019, 1:05 AM