Skip to main content

Build OpenPYPE from source

Introduction​

To build Pype you currently need (on all platforms):

We use CX_Freeze to freeze the code and all dependencies and Poetry for virtual environment management.

This is outline of build steps. Most of them are done automatically via scripts:

  • Virtual environment is created using Poetry in .venv
  • Necessary python modules outside of .venv are stored to ./vendor/python (like PySide2)
  • Necessary third-party tools (like ffmpeg, OpenImageIO and usd libraries) are downloaded to ./vendor/bin
  • OpenPype code is frozen with cx_freeze to ./build
  • Modules are moved from lib to dependencies to solve some Python 2 / Python 3 clashes
  • On Mac application bundle and dmg image will be created from built code.
  • On Windows, you can create executable installer with ./tools/build_win_installer.ps1

Clone OpenPype repository:​

git clone --recurse-submodules https://github.com/pypeclub/OpenPype.git

Platform specific steps​

Windows​

More tools might be needed for installing some dependencies (for example for OpenTimelineIO) - mostly development tools like CMake and Visual Studio

Run from source​

For development purposes it is possible to run OpenPype directly from the source. We provide a simple launcher script for this. To run the powershell scripts you may have to enable unrestricted execution as administrator:

Set-ExecutionPolicy -ExecutionPolicy unrestricted

To start OpenPype from source you need to

  1. Run .\tools\create_env.ps1 to create virtual environment in .venv
  2. Run .\tools\fetch_thirdparty_libs.ps1 to get PySide2, ffmpeg, oiio and other tools needed.
  3. Run .\tools\run_tray.ps1 if you have all required dependencies on your machine you should be greeted with OpenPype igniter window and once you give it your Mongo URL, with OpenPype icon in the system tray.

Step 1 and 2 needs to be run only once (or when something was changed).

To build OpenPype:​

  1. Run .\tools\create_env.ps1 to create virtual environment in .venv
  2. Run .\tools\fetch_thirdparty_libs.ps1 to get PySide2, ffmpeg, oiio and other tools needed.
  3. .\tools\build.ps1 to build OpenPype to .\build

To create distributable OpenPype versions, run .\tools\create_zip.ps1 - that will create zip file with name pype-vx.x.x.zip parsed from current pype repository and copy it to user data dir. You can specify --path \path\to\zip to force it into a different location. This can be used to prepare new version releases for artists in the studio environment without the need to re-build the whole package

Adding dependencies​

Python modules​

If you are extending OpenPype and you need some new modules not included, you can add them to pyproject.toml to [tool.poetry.dependencies] section.

/pyproject.toml
[tool.poetry.dependencies]
python = "3.9.*"
aiohttp = "^3.7"
aiohttp_json_rpc = "*" # TVPaint server
acre = { git = "https://github.com/pypeclub/acre.git" }
opentimelineio = { version = "0.14.0.dev1", source = "openpype" }
#...

It is useful to add comment to it so others can see why this was added and where it is used. As you can see you can add git repositories or custom wheels (those must be added to [[tool.poetry.source]] section).

To add something only for specific platform, you can use markers like:

Install pywin32 only on Windows
pywin32 = { version = "300", markers = "sys_platform == 'win32'" }

For more information see Poetry documentation.

Python modules as thirdparty​

There are some python modules that can be available only in OpenPype and should not be propagated to any subprocess. Best example is PySide2 which is required to run OpenPype but can be used only in OpenPype and should not be in PYTHONPATH for most of host applications. We've decided to separate these breaking dependencies to be able run OpenPype from code and from build the same way.

warning

PySide2 has handled special cases related to it's build process.

Linux​

  • We're fixing rpath of shared objects on linux which is modified during cx freeze processing.

MacOS​

  • QtSql libraries are removed on MacOS because their dependencies are not available and would require to modify rpath of Postgre library.

Binary dependencies​

To add some binary tool or something that doesn't fit standard Python distribution methods, you can use fetch_thirdparty_libs script. It will take things defined in pyproject.toml under [openpype] section like this:

/pyproject.toml
[openpype]

[openpype.thirdparty.ffmpeg.windows]
url = "https://distribute.openpype.io/thirdparty/ffmpeg-4.4-windows.zip"
hash = "dd51ba29d64ee238e7c4c3c7301b19754c3f0ee2e2a729c20a0e2789e72db925"
# ...

This defines FFMpeg for Windows. It will be downloaded from specified url, its checksum will be validated (it's sha256) and it will be extracted to /vendor/bin/ffmpeg/windows (partly taken from its section name).

Script tools​

(replace extension with the one for your system - ps1 for windows, sh for linux/macos)

build​

This will build OpenPype to build directory. If virtual environment is not created yet, it will install Poetry and using it download and install necessary packages needed for build. It is recommended that you run fetch_thirdparty_libs to download FFMpeg, OpenImageIO and others that are needed by OpenPype and are copied during the build.

Arguments​

--no-submodule-update - to disable updating submodules. This allows to make custom-builds for testing feature changes in submodules.

build_win_installer​

This will take already existing build in build directory and create executable installer using Inno Setup and definitions in ./inno_setup.iss. You need OpenPype build using build script, Inno Setup installed and in PATH before running this script.

note

Windows only

create_env​

Script to create virtual environment for build and running OpenPype from sources. It is using Poetry. All dependencies are defined in pyproject.toml, resolved by Poetry into poetry.lock file and then installed. Running this script without Poetry will download it, install it to .poetry and then install virtual environment from poetry.lock file. If you want to update packages version, just run poetry update or delete lock file.

Arguments​

--verbose - to increase verbosity of Poetry. This can be useful for debugging package conflicts.

create_zip​

Script to create packaged OpenPype version from current sources. This will strip developer stuff and package it into zip that can be used for auto-updates for studio wide distributions, etc. Same as:

poetry run python ./tools/create_zip.py

docker_build.sh [variant]​

Script to build OpenPype on Docker enabled systems - usually Linux and Windows with Docker Desktop and Windows Subsystem for Linux (WSL) installed.

It must be run with administrative privileges - sudo ./docker_build.sh.

It will use latest Debian base image to build OpenPype. If you need to build OpenPype for older systems like Centos 7, use centos7 as argument. This will use another Dockerfile to build OpenPype with Centos 7 as base image.

You'll see your build in ./build folder.

fetch_thirdparty_libs​

This script will download necessary tools for OpenPype defined in pyproject.toml like FFMpeg, OpenImageIO and USD libraries and put them to ./vendor/bin. Those are then included in build. Running it will overwrite everything on their respective paths. Same as:

poetry run python ./tools/fetch_thirdparty_libs.py

make_docs​

Script will run sphinx to build api documentation in html. You should see it then under ./docs/build/html.

run_documentation​

This will start up Docusaurus to display OpenPype user documentation. Useful for offline browsing or editing documentation itself. You will need Node.js and Yarn to run this script. After executing it, you'll see new browser window with current OpenPype documentation. Same as:

cd ./website
yarn start

run_mongo​

Helper script to run local mongoDB server for development and testing. You will need mongoDB server installed in standard location or in PATH (standard location works only on Windows). It will start by default on port 2707 and it will put its db files to ../mongo_db_data relative to OpenPype sources.

run_project_manager​

Helper script to start OpenPype Project Manager tool. Same as:

poetry run python start.py projectmanager

run_settings​

Helper script to open OpenPype Settings UI. Same as:

poetry run python start.py settings --dev

run_tests​

Runs OpenPype test suite.

run_tray​

Helper script to run OpenPype Tray. Same as:

poetry run python start.py tray

update_submodules​

Helper script to update OpenPype git submodules. Same as:

git submodule update --recursive --remote