Support using embedded Python package
Open, NormalPublic

Description

Krita scripting should support using an embedded Python package.

The plugin will need to set the paths appropriately, depending whether the embedded Python package or the system Python install should be used. The path handling is different.

  1. Before any calls to Py_SetPath, Py_SetProgramName or Py_SetPythonHome, Py_GetPath should return the default search path which can be used.
  2. Py_SetPath *replaces* the search path, so it must include the location of the standard libraries for Python to work properly.
  3. There is no reason to manipulate the PYTHONPATH and PYTHONHOME environment variables directly as Py_SetPath can be used instead and it will cause Python to ignore the environment variables.

Platform-specific notes:

Windows (already implemented):

Pretty much any Windows binary releases should contain an embedded Python. It will be too much to expect the user to install an appropriate Python version, especially since a lot of Windows users are not tech-savvy.

We can include the official embeddable Python, which only takes around 12MB. It includes all the standard library pre-compiled inside a zip archive (Python can load them from the zip directly).

Notes:

  1. I would prefer not to extract the Python package to bin/, but instead place them inside python/ next to bin/ / lib/ / share/. This allows easier packaging with current workflow (the script can copy the whole folder instead of having to filter the files from bin/). It also reduce cluttering of both bin/ and the Python search path.
  2. The plugin needs to load python36.dll from the proper location. Currently, the plugin is dynamically linked to the DLL, which means the loader does the dynamic linking, which means it needs to find the DLL when the plugin is being loaded or the plugin will not be loaded at all (so no code in the plugin itself can solve the issue). Manipulating the PATH is hacky and I don't really like it (main.cc currently manipulates the PATH on Windows but it doesn't have any real use so I might remove it). A possible solution is to copy python36.dll to bin/ but this feels wrong. It also needs python3.dll at a later time for some reason. See: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx#search_order_for_desktop_applications
  3. The Python release 3.6 is built with MSVC 2015 which means it uses the Universal C Runtime (Windows 7/8 needs them installed separately: https://www.microsoft.com/en-us/download/details.aspx?id=48234). We might need to consider including it in the installer.
  4. The embeddable Python package doesn't seem to include the license text, we might have to add it manually...

Linux:

I am not sure if Krita under Linux should use the system-wide Python install or include an embedded Python.

For Appimage, even though a Python install will likely be included, it could still be considered a "system-wide" Python install if it is located on /usr and uses default Python paths.

Testing on KDE Neon with the system Python install, it seems that Py_GetPath does not return the dist-packages paths, therefore using Py_SetPath will cause it to not find SIP and fail, so we might still need to manipulate PYTHONPATH if the system-wide Python install is to be used.

TODO

Mac:

TODO

alvinhochun updated the task description. (Show Details)Jul 19 2017, 7:49 PM
alvinhochun updated the task description. (Show Details)Jul 20 2017, 7:49 AM
alvinhochun updated the task description. (Show Details)Jul 20 2017, 9:22 AM
alvinhochun updated the task description. (Show Details)Jul 29 2017, 7:31 AM
alvinhochun lowered the priority of this task from High to Normal.Aug 1 2017, 3:29 PM

Didn't notice this before, but it is possible to edit the manifest file/resource to have the DLL loader load DLLs from a directory:
https://stackoverflow.com/questions/3832290/altering-dll-search-path-for-static-linked-dll

But making this work means the overall package and install structure on Windows has to be changed...