Skip to main content

Busywork tutorial

This tutorial is designed to introduce you to the Maya Plugin, the Desktop App, and the Workflow API. We'll set up a simple job that runs a dummy command to test a round trip of the system.

Install software​

Maya Plugin​

If you haven't already done so, install the Maya plugin. If you are a developer, it is recommended to follow the instructions to install from GitHub in case you want to make modifications.

How to install the Maya plugin

Make sure you can load the plugin in the Maya Plugin Manager.

Desktop App​

Install the Desktop app. You can try a binary release, or pull the repo and either build your own or run in development mode.

How to install the Desktop app

Make sure you can run the app.

Workflow API​

git pull git@github.com:AtomicConductor/render-services.git
cd render-services/workflow-api
make up

For more info, consult the Workflow API README

If it succeeds, you should see a message like this:

✔ Container workflow-api-mysql-1         Started  10.4s
✔ Container workflow-api-redis-1         Started  10.4s
✔ Container workflow-api-workflow-api-1  Started   0.3s

Now shut it down, since we'll want to map in some volumes.

make down

To map the Maya project folder and the bin folder into the container: Edit the docker-compose.yml file and add the following two volumes to the workflow-api service below the credentials mapping:

volumes:
  - ./credentials.json:/tmp/credentials.json
  - /path-to/cwmaya/cwmaya/projects:/path-to/cwmaya/cwmaya/projects
  - /path-to/cwmaya/cwmaya/bin:/path-to/cwmaya/cwmaya/bin

Now restart the container by running make up again.

You are now ready to submit fake jobs.

Create a job in Maya​

Open Maya and Set Project to cwmaya/projects/dack. It's the only project we have right now. Load the CoreWeave plugin in the Plugin Manager. You should see a new menu item in the main menu bar called CoreWeave. Click on it and choose Storm Window.

Create a template​

The Storm Window will appear, loaded with a template. We'll discard the default template and create a new one. Use the menu item: Tools -> Create Template -> cwChainSubmission. Different templates define different graph topologies. The cwChainSubmission template is designed to run a series of almost identical tasks one after the other. This is handy for local testing since the local workflow API is not yet set up to handle parallel tasks. To find out more about templates, or learn to write your own template.

Set up the template to run the busywork command​

The busywork command is a simple script that simulates a process, such as a render, that creates a file on disk over time while outputting progress to the console. Since no DCCs are installed in the local workflow API, we'll use this command to test the system.

  • Go to the Frames tab and set the Custom frames to 1-7. Turn off Scout frames, since we don't wan the bother of unholding tasks.
  • Go to the Work tab and set the Label to to Chainlink-{start:>02}. This uses the Python f-string syntax to format the label with the number of the start of the chunk, padded to 2 zeros, for that task. All node labels must be unique within the graph and if you don't set them, they will be made unique automatically, with a prefix you may not like.
  • Don't worry about hardware and software settings for now. They are not relevant while using the local workflow API.
  • In the Storm menu bar, chose Tools -> Show spec tokens. A window apopears that shows the available tokens and values. You can use these tokens in several fields in the UI.
  • Set one command: busywork {imagesdir}/img.{start:>02}.exr 1000 10 > {imagesdir}/img.{start:>02}.exr.log. The arguments are the output file, the number of bytes to write, and the number of seconds to take. The output is redirected to a log file.
  • Set the PATH environment variable to include the location of the busywork command. By putting the variable name in square brackets, you are telling the system to append to the PATH variable, rather than replace it. If you replace the PATH variable, the system will not function correctly.
    • key: [PATH]
    • value: {mayaprojdir}/../../bin
  • Set the output path to {imagesdir}. This path serves the same purpose as regular Conductor jobs, to declare the folder writable. However, each task can have a different output path.
  • Go to the Job tab and set Job name and other fields as you wish.

Visualize the graph​

  • Hit the Send to Composer button. This will send the graph to the Desktop app and open the composer view, where you can see the structure and connections between the nodes. You can also see the resolved values of the attributes for all nodes in the attribute panels to the right.
  • You can examine the raw JSON payload from Maya by going to Tools -> Show spec. (Currently, you need to have VS Code installed for this to work.

Submit the job​

  • Hit the Submit button. This will send the job to the local workflow API.

Soon, (but not as of this writing) you'll be able to see the job in the Desktop app, and watch the overall progress of the job in the list, or watch progress for one job sweep through the graph in the monitor view. For now however, you can choose Tools -> List jobs where you'll see the job in the list along with it's percentage. If you hit the refresh button, you'll see the percentage increase as the job progresses. If it looks like something went wrong, press Show nodes in VS Code to see the raw JSON response from the API, which will likely show some failures. This doc will evolve a lot, so keep your eye on the updates message in Basecamp.