Setting up the workflow¶

The starting point for operating the workflow is the Start page. To open an existing workflow, double-click its name in the list on the Start page. To create a new workflow, click the New workflow button on that page.

Understanding the setup controls¶

When you create or open a workflow, the workflow is displayed in the workflow designer. The main elements of the workflow designer are as follows:

The workflow design surface is a diagram showing the blocks and the links between them. Here you can add, modify, or delete blocks and links, as well as connect blocks to workflow inputs and outputs. At the top of the diagram there is a command menu and a toolbar for a number of tasks related to the workflow setup.
The Workflow contents pane is displayed by clicking the button on the toolbar. Here you can review the contents of all composite blocks and select any block or the entire workflow to view or edit their properties.
The Block library pane is displayed by clicking the button on the toolbar. From the Common library area in this pane, you can add new blocks to the workflow. The Workflow library area represents a library of blocks that were added to the workflow.
The block configuration pane is used to view or change basic properties and settings of the block, including its inputs and outputs. This pane appears if any block is selected in the workflow diagram or in the Workflow contents pane. The Composite properties pane for configuring composite blocks differs from the Block properties pane for configuring other blocks. The pane for configuring the workflow is similar to the Composite properties pane, and appears if the root element is selected in the Workflow contents pane, or the [In] or [Out] area is selected in the workflow diagram.
The Run setup pane is used to view or change the workflow input parameters and their values. Here you can also specify which data should be stored in the workflow run results. The Run setup pane appears if no block is selected in the workflow diagram. To unselect all blocks, click in an empty area of the workflow diagram.

Adding and configuring blocks and links¶

To add blocks to the workflow, open the Block library pane by clicking the button. Blocks are added by dragging them from this pane onto the workflow design surface. You can also add a block by double-clicking its name in the Block library pane.

Select (click) a block in the workflow diagram to view or change its basic settings in the block configuration pane. If the pane does not show up, click the button on the toolbar. Double-click the block to open a dialog where you can view or change all of the block's configuration settings. For their description, refer to the documentation for this block.

Hover over the block in the workflow diagram to display the link handle . To create a link between two blocks, drag the link handle from one block to another. A dialog will appear prompting you to specify which ports you want to link.

The links between blocks in the workflow diagram are indicated by arrows. Hover over the arrow to view the links it represents. Double-click the arrow to open a dialog where you can view and modify those links.

Managing composite blocks¶

There are two ways to add a composite block to the workflow:

Group existing blocks and their links by placing them in a new composite block, and then proceed to editing the workflow within that block. To create a composite block in this way, select one or more blocks in the workflow diagram and click the button on the toolbar.
Create an empty composite block and then proceed to composing a workflow within that block. To create an empty composite block, click the button on the toolbar.

To go inside a composite block, hover over it and click the icon that appears. The workflow inside a composite block is built in the same way as the entire workflow: you can add blocks and links to it, configure inputs and outputs, etc. To exit a composite block, hover over the [In] or [Out] area in its workflow diagram and click the icon that appears.

Configuring inputs and outputs of a composite block¶

To configure the inputs and outputs of a composite block, use the Composite properties pane. In this pane, you can add inputs and outputs, as well as manage their settings, including:

Enable batch mode

If batch mode is enabled, the block expects to receive a certain array of values for processing on this port. The computations will be performed for each element of the array; concurrent processing of multiple elements is possible (see Parallelization settings). The block output will contain an array of computation results. If batch mode is enabled on multiple ports, the workflow must ensure that the input array size is the same on all those ports.
Uplink

Connects this port to the input or output of the parent block. For details, see Uplinks.
Add this port to workflow parameters or make it a personal parameter

If the port is a parameter, it appears under Input parameters in the Run setup pane. Parameter values there are editable while preparing a workflow run in configuration mode. If you share your workflow to other users with read-only access, they cannot edit port values in the workflow, however they can create new runs and set parameter values in Run setup. Values of personal parameters are protected: they are saved in a user's personal storage, and others cannot see them even if they have access to workflows or runs created by that user. See Configuring parameter input and results output for details.
Add to the workflow results

If a port is added to the workflow results, the results pane displays the value of that port after the workflow run is completed. For details, see Running workflow and viewing results.

You can add input and output ports for a composite block from its Composite properties configuration pane. However, the ports added in this way by default have no connection with the workflow enclosed within the block. In this case, additional configuration is required by creating uplinks.

Uplinks¶

Like any block, a composite block has inputs and outputs. The difference is that within a composite block, its inputs and outputs can be linked to inputs and outputs of its child blocks. Such links are called uplinks as they propagate child inputs and outputs up the block hierarchy.

Uplinks enable you to connect blocks that are outside of a composite block with its child blocks. For example, suppose you want to connect the output of block A, which is outside of the composite block, to the input of block B, which is inside the composite block. This requires the following steps:

In the composite block, add an input port and create an uplink from that input port to the input port of block B.
Connect the output port of block A to the input port of the composite block that you prepared in the previous step.

There are two ways to uplink a certain port within a composite block:

Depending upon the port direction, add an input or output in the Composite properties pane, and then create a link by dragging the link handle to the [In] area (in case of input port) or to the [Out] area (in case output port).

or
Select the block whose port you want to uplink, and then, in the block configuration pane, click the uplink button next to the name of that port. Depending upon the port direction, this will add a new input or output to the composite block and link it with that port.

From a configuration perspective, the entire workflow is similar to a composite block. As with configuring a composite block, you can link certain ports of the child blocks of the workflow to its inputs and outputs. However, unlike the inputs and outputs of a composite block, the outputs of a workflow serve to issue its results, and the inputs are used to pass parameters to the workflow.

Configuring parameter input and results output¶

Usually there are certain ports in the workflow, which require values initially, so that the workflow can run - certain input parameters of simulation or other design process. Such ports are essentially workflow inputs, and users of the workflow usually change their values in each run - that is, they vary the underlying process parameters.

It is possible to assign values to ports directly, in the Block properties pane while editing the workflow; however, there are a number of disadvantages to this:

Usually, workflow inputs are various ports of different blocks. Users will have to navigate the workflow and find those blocks to set the input values.
Editing ports requires write access to the workflow. If you share your workflow to many users, you will have to give them all full access, which means they will also be able to edit the workflow, change settings in blocks and so on.
Port values are saved in the workflow. In shared workflows, users who have access will see and edit each other's values.
Port values are stored unencrypted in workflow files.

To avoid those issues, you can add ports to workflow parameters. Compare:

All parameters are listed in the Run setup pane for convenient editing.
Workflow users can change the parameter values even if they have read-only access. To do this, they create a new run in configuration mode and edit parameters there.
Parameter values set in run configuration mode are saved only in that run. This is as well useful if you have full access to the workflow - you can assign certain values to ports of workflow blocks,
The parameter values set for a run in configuration mode are saved only in that run. As a side note, this makes it possible to implement parameters with defaults. To create such defaults, first set the parameter values when you edit the workflow, so there are certain values stored in the workflow itself. When your workflow user creates a new run, the initial values of parameters in the run settings are copied from the workflow - thus also saving the users from having to specify all parameters even if they vary only a few of them. Then, users can change some or all of the run parameters, and their changes are saved only in the run settings, without changing the workflow. The next run created will get the same initial values of parameters, which it will copy from the workflow.
Personal parameters give you an option to protect user values. These are special parameters, values of which are secured in the user's personal storage. When the user launches a workflow run, pSeven Enterprise reads user's personal values and passes them to blocks in the workflow - while other users cannot ever access those values.

Upon completion of the workflow execution, it is normally required to save the values of certain ports for further analysis. These ports need to be added to the workflow results. The port values will be saved and, after the workflow execution is completed, they will remain available in the run directory.

Managing parameters and their presets¶

The workflow parameters are listed under Input parameters in the Run setup pane. For a port to be listed, it must have the (parameter) or (personal parameter) setting enabled in the block configuration pane for the respective block. As the Input parameters area also lists all blocks of the workflow, you can add ports to the parameter list as follows:

Hover over the name of the block and click the icon that appears at the end of the respective entry in the list.
Select the desired ports in the pop-up box that appears.

To edit a parameter value in the Run setup pane, click it first to display the edit box; after that, you can:

Input simple values (numbers, strings) directly in the edit box.
Click the "..." button on the right to open the Edit value dialog - for editing values such as tables or matrices.
Click the chevron button on the right and select a value from the drop-down list. This button appears only if you edit a personal parameter, allowing you to select from your saved personal values.

Values of parameters (except the personal ones) are saved in the workflow or in run settings, based on where you edit them. Personal parameters are saved in users' personal storages, workflow and run settings contain only references to them. If you edit a personal parameter in a run in configuration mode, you will get an additional dialog with these options:

Save - save the new value in your personal storage, replacing the value of this personal parameter saved earlier, if it exists. No undo: if you save, you cannot recover the previous value unless you input it again.
Use once - set the parameter value for the current run only, don't save or update it in your personal storage. If you have already saved the value of this parameter earlier, the saved value will not change. If there is no saved value, the new value will not be saved either.

To manage your saved values of personal parameters, open your user menu on the top right in the navigation bar, and select Personal parameters. In the Personal parameters dialog that opens, you can view, edit, or remove the saved values. This is your personal value storage, which also tracks where they are used: you can note that if you've set the same value in several workflows, that value occurs in the list more than once. While editing any of such occurrences in the Personal parameters dialog, you will have the option to update all those occurrences at once.

Each set of parameter values specified in the Run setup pane is called a preset. Multiple presets can be created. Before starting a workflow run, it is possible to select the desired preset, and the workflow run starts with the parameter values that are specified in the selected preset.

The name of the currently selected preset is displayed in the Preset field at the top of the Input parameters area. Initially, there is only one preset named default. To manage presets, click the icon at the right end of the Preset field and use the commands from the menu that appears. For example, to create a new preset, click Add preset.

A new preset is created by copying the current one, and has the same parameter values. In the new preset, you can change parameter values to meet your requirements. If there are several presets, the desired one can be selected from the drop-down list in the Preset field. All parameters will be assigned values from the selected preset.

Managing results output¶

The ports included in the workflow results are listed under Results in the Run setup pane. By default, the list includes all outputs of the workflow. For other ports to be included in the workflow results, they must have the setting enabled in the block configuration pane for the respective block. As the Results area also lists all composite blocks of the workflow, you can add ports to the results list as follows:

Hover over the name of the block and click the icon that appears at the end of the respective entry in the list.
Select the desired ports in the pop-up box that appears.

Only ports of composite blocks can be included directly into workflow results. As for ports of other blocks, uplinks must be leveraged to store port values in the workflow results. For example, you could uplink the port to an output of a composite block, and then add that output to the results list.

Managing block memory limits¶

pSeven Enterprise can monitor block memory usage and forcefully stop blocks with high memory consumption, if there is not enough memory to run other blocks. This behavior depends on the resource management settings, which only administrators can change.

When pSeven Enterprise forces a block to stop, the workflow run fails with a message like:

No memory limit specified for the block, and the hostname node where it runs needs more free memory.

The message normally includes the hostname of the Kubernetes cluster node where the error occurred, which might be useful for administrators if they need to investigate an issue.

To avoid that error, specify memory limits to your blocks:

Open the workflow and select the block that was stopped by pSeven Enterprise.
In the Block properties pane, expand Settings at the bottom and set a memory limit.

To find out how much memory your block needs, you can review its memory usage statistics in the previous runs:

Open a completed run and open the Workflow contents pane on the run tab. The pane will display the peak memory usage for each block.
Find the block that was forced to stop and note its memory usage.
If you have more runs of the same workflow, check them too. Find the maximum value of the block's memory usage across all runs.
Set the block's memory limit according to the value you have found. For greater stability, add 10-20% on top of that maximum.

When you set a memory limit for a block, pSeven Enterprise will start the block only if that amount of free memory is available, and will reserve this memory for the block while it runs. The block, in turn, must respect the limit: if its memory consumption exceeds the limit, pSeven Enterprise will force it to stop after a short grace period. In this case, your workflow run will fail with the following message:

The block has exceeded the memory limit specified in its settings.

Any block with a memory limit set has priority over all blocks without limits. If there is a shortage of memory, pSeven Enterprise will begin to stop the blocks without limits, while the memory-limited blocks will continue running, as long as they respect their memory limits.

How to check if resource management is enabled

Resource management, with its memory usage monitoring, is a system-wide feature, which may be enabled or disabled by pSeven Enterprise administrators. To find out if it is enabled in your deployment, run an empty workflow and check the first message in its run log.

If you see the following message, then resource management is disabled:

Resource management is disabled. pSeven Enterprise disregards block resource requirements and does not limit block resource usage.

In this case, you can change the block resource settings and save them, but they will apply only if your workflow is moved to another pSeven Enterprise deployment, where resource management is enabled.
If you see the following message, then resource management is enabled:

Total run requirements: memory: 512 MB, cpu: 0.01 cores, jobs: 1

In this case, the block memory limit settings apply as described in this section.

Updating blocks to a newer version¶

After the release of a new version of a block, the prototype of that block is first updated in the common library. The pSeven Enterprise administrator performs this update during the deployment of the new release. For reference, you can view the version number of the block in the Block library pane: when you hover over a block in that pane, the version number is displayed in a pop-up box.

Even after the deployment of a new release is complete and the blocks are updated in the common library, the blocks are not updated automatically in the existing workflows. Each workflow has its own block library, which you can view under Workflow library in the Block library pane. Updating blocks in the workflow is performed from this library.

After updating blocks in the common library, some of the blocks in the workflow may have a lower version number than in the common library. In the Workflow library area, an update notification appears next to the name of each block that has a lower version number. Click this notification to update all occurrences of the block in the workflow. You can roll back the update using the Undo command if, for some reason, the newer version of the block is not suitable for this workflow.