Administration¶

This guide covers the tasks involved in administering pSeven Enterprise and extension nodes after they have been deployed.

Related guides

Version upgrade: installing updates.
pSeven Enterprise deployment and Extension node deployment: initial deployment.

User management¶

To manage user accounts, sign in to the pSeven Enterprise admin interface, and open the Users page.

Users page URL: {admin URL}/app_auth/user/

The Users page lists the pSeven Enterprise user accounts along with some of the current settings for each account. To view or manage account settings, click the account ID or username in the list. On the Change user page that appears, you can view or change user permissions assigned to the given account, as well as view some of its settings and usage info. The means for creating new user accounts, changing user credentials, such as email or password, as well as other account settings, are provided on the Users page of the authentication service interface. To open that page, click Manage users in the pSeven Enterprise admin interface page header.

Understanding accounts¶

Accounts are used to sign in to pSeven Enterprise and gain access to its resources. In the sign-in form, the user enters the username or email of their account along with the password. The user can request a password reset by clicking the Forgot password link. The password reset link is sent to the email specified in the user's account.

Account configuration involves:

Adding accounts. The account is created through the authentication service. The first sign-in using the newly created account passes it to the authorization service, after which you can view or change the user permissions for that account.
Managing user account data. For existing accounts, you can view or change user credentials, such as email or password, as well as other settings, such as the user first name and last name. You may need to reset a user's password by hand if their credentials do not specify a valid email address to send a password reset link to.
User permissions. These determine the actions allowed to the account. By setting permissions, an account can be restricted to certain tasks - for example, it can only allow its holder to use pSeven Enterprise apps published to AppsHub.
End-user account configuration. End-user accounts provide access to pSeven Enterprise apps, workflows, and user block development features.
Admin account configuration. Admin accounts are used to perform administrative tasks, including user management.
Applying user storage quotas. Quotas set the limits for the file storage space occupied by user files.
Disabling an account. To sign in, the account must be enabled (active). When created, every account is enabled by default. Accounts can be disabled if necessary, and re-enabled at any time. Disabling is the recommended alternative to deleting accounts.
Changing the username. To allow users to change their username, certain changes must be made to pSeven Enterprise authentication service settings. By default, changing the username is not allowed.

The pSeven Enterprise license imposes a restriction on the number of active users. The total number of enabled accounts with user permissions must not exceed the licensed number of users. Otherwise, only admins will be allowed to sign in. If the number of active user accounts exceeds the license limit, an admin has the option to disable some user accounts to comply with the terms of the license.

The total number of accounts within a pSeven Enterprise deployment is limited by the maxUsers parameter in the deployment configuration file values.yaml. If more accounts are created, some of them may be unusable. The maxUsers parameter value can be increased only by re-deploying pSeven Enterprise (see pSeven Enterprise deployment), or when upgrading pSeven Enterprise to a new version (see Upgrading pSeven Enterprise).

When you create an account, it is assigned a unique ID, which the account retains for its entire lifetime. The ID of the account cannot be changed, nor can it be re-used to identify another account. This ensures that the permissions of each account are secured, since they are verified against the account ID.

The ID of each account always remains unique, it is never re-used within the same pSeven Enterprise deployment. If an admin deletes a user's account and then creates a new account for that user, the new account is assigned a new ID that is different from the ID of any other account within the given deployment. The permissions of the new account get associated with the new ID, whereas the permissions of the deleted account are dropped along with the ID of that account.

So, each account requires a unique ID that has never been used by any account in the given pSeven Enterprise deployment. This causes the value of the account ID to increment for each newly added account. If the account ID exceeds the maxUsers parameter value, the account gets unusable. For this reason, when upgrading pSeven Enterprise to a newer version, you must not decrease the maxUsers parameter value. Otherwise, some of the existing accounts may become unusable after the upgrade because their IDs exceed the new maxUsers value.

Since deleted account IDs cannot be reused, it is advisable that accounts be disabled (see Disabling an account) rather than deleted when necessary to revoke access to pSeven Enterprise or reduce the number of accounts to meet license requirements. Disabled accounts cannot be used to sign in, nor are they accounted for by the license. A disabled account can be enabled again at any time, since it keeps user data and settings.

Every enabled account that is assigned any of the user permissions (see User permissions) is counted as a licensed user; the license limits the number of such accounts. The license does not limit the number of disabled accounts and the number of admin accounts that do not possess user permissions. However, the total number of accounts, including deleted ones, must not exceed the value of the maxUsers parameter.

Deleting accounts should be avoided as IDs of deleted accounts are irretrievably lost. This also applies to admin accounts. To revoke admin rights for a particular account, clear the check boxes related to the admin settings (see Admin account configuration), or disable that account (see Disabling an account).

Important

Never revoke admin rights from all accounts or disable all admin accounts. Otherwise, admin access will be blocked as nobody will be able to sign in for admin tasks or restore access.

Adding accounts¶

New user accounts are first created in the authentication service. Upon its first sign-in, the newly created account is passed to the authorization service to assign it user permissions. Then, you can view or change the user permissions for that account.

Thus, to add a new user account, you first need to create it in the authentication service:

Sign in to the pSeven Enterprise admin interface.
Click Manage users in the page header to go to the authentication service interface.
On the Users page that appears, click the Add user button.
Fill in the Create user form that appears. You must enter at least a username. Other fields are optional, you can fill them later (see Managing user account data). By default, changing the username is not allowed (see Changing the username).

This form does not allow you to set a password. Ways to set a password for a user account are as follows:
- Having created the account, set its password on the Credentials tab of the account edit page as described in Managing user account data.
  
  - OR -
- Let the user set a password: enter a valid email in the Create user form so that the user can receive a password reset link by clicking Forgot password in the sign-in form.
When you are done, click Create in the Create user form to add the new account to the authentication service.

To manage user permissions and other properties of a newly created account, you must first sign in with it to pSeven Enterprise. Upon the first sign-in, the account is activated and passed to the authorization service where it is assigned default user permissions (see User permissions). Then, you can manage the account using the instructions that follow.

Account pool limits

The size of the pSeven Enterprise account pool (the number of available account IDs) is limited by the maxUsers parameter in the pSeven Enterprise deployment configuration file values.yaml and it cannot be changed without re-deployment. For security reasons, account IDs are never reused: each newly added user account takes up a certain unique ID, but removing an account does not return its ID to the pool. You can only add accounts if any unused IDs are available in the pool. Avoid adding temporary or test accounts as they are going to consume account IDs, which are single-use. For more information, see Understanding accounts.

Why pSeven Enterprise never reuses account IDs

An account ID is a single-use unique identifier, which is used to generate unique user IDs for various pSeven Enterprise services. Reusing an account ID is a security risk: if Bob is a new user who gets an account ID that previously belonged to Alice (a deleted account), then Bob may gain ownership of Alice's workflows, apps she had published to AppsHub, get access to files other users had shared with Alice, and so on.

Managing user account data¶

The user's credentials and other account data are controlled by the authentication service. To administer this data, go to the Users page of the authentication service interface:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the Users page, in the Username column, click the name of the user account to open its edit page.

On the Details tab of the account edit page, you can set or change the email and other settings, such as the user first name and last name. On the Credentials tab, you can set or reset the password for this user. When setting the password, note that the Temporary switch is on by default, which forces the user to change the password at next sign-in.

User permissions¶

User permissions for an account are controlled by the authorization service. They are administered on the Change user page for the given user account. To open the Change user page:

Sign in to the pSeven Enterprise admin interface, and navigate to the Users page at {admin URL}/app_auth/user/.
In the list on the Users page, click the ID or username of the desired account.

When adding a new user account, it is assigned the default permissions of a regular end-user account:

app_auth | user | Can access AppsHub. Allows users to sign in to AppsHub and run apps according to their workspace permissions.
app_auth | user | Can access Studio and AppsHub. Allows users to edit and run workflows in Studio, sign in to AppsHub and run apps according to their workspace permissions. Required to enable publishing apps - the user needs Studio access and the "Publish apps" workspace permission.
app_auth | user | Can develop blocks. Required to develop user blocks. A user with this permission can download block prototypes from the common block library and upload custom blocks to the workflow (user) block library.

The pSeven Enterprise license includes two separate counters for users who can access AppsHub only and users who have both Studio and AppsHub access. Often you can optimize user permissions for efficient license usage - for example, remove the AppsHub and Studio access permissions from admin accounts so those accounts are not counted by the license, or remove the Studio permission for users who run only apps from AppsHub.

End user account configuration¶

Typical end user account configurations are: an AppsHub-only user, a Studio user, or a block developer. Set their permissions as described below.

An AppsHub-only user signs in only to AppsHub and runs only apps published by other users. This is a simplified account type, which does not need Studio access because the user is not going to edit or run workflows. For an AppsHub-only user, set the following permissions:
- Keep or add the app_auth | user | Can access AppsHub permission.
- Remove the app_auth | user | Can access Studio and AppsHub permission to avoid counting the user as a Studio user in the license.
- Remove the app_auth | user | Can develop blocks permission. A user without Studio access cannot download and upload blocks anyway, though it is advisable to remove that permission to avoid confusion.
A Studio user composes workflows from blocks available in the common block library, runs workflows and analyzes results, and may publish workflows to AppsHub. This is the common account type, with permissions set as follows:
- Keep the app_auth | user | Can access AppsHub permission if it exists, otherwise skip. A Studio user gets this permission by default, after you save account settings. All Studio users can access AppsHub, although they do not count as AppsHub users in the license.
- Keep or add the app_auth | user | Can access Studio and AppsHub permission.
- Remove the app_auth | user | Can develop blocks permission. A Studio user with that permission can download blocks from the block library and see the block code, which may be unwanted for security reasons.
A block developer works on extending the block library with custom user blocks and needs Studio access to download, upload and test the block code. For block developers, set the following permissions:
- Keep the app_auth | user | Can access AppsHub permission if it exists, otherwise skip - same as for Studio users.
- Keep or add the app_auth | user | Can access Studio and AppsHub permission - required for testing blocks.
- Keep or add the app_auth | user | Can develop blocks permission - essential for a block developer, required to download and upload blocks.

Admin account configuration¶

To configure an account as admin, set the following permissions for that account:

Select the "Staff status" check box. This permission is required to sign in to the pSeven Enterprise admin interface but does not grant access to admin functions.
Select the "Superuser status" check box. This status grants all administrative permissions without the need to add them explicitly.

Optionally, you can select "Staff status" only and then select only specific administrative permissions from the list in User permissions. This way you can create sub-admin accounts with limited rights, if those are required by your organization.
Remove the following default user permissions to avoid counting the admin account as a licensed AppsHub or Studio user:
- Remove the app_auth | user | Can access AppsHub permission.
- Remove the app_auth | user | Can access Studio and AppsHub permission.
- Remove the app_auth | user | Can develop blocks permission.

An admin account set up as described above has no access to the end user interface (AppsHub and Studio) and can only sign in to the admin interface, so it does not count as a licensed AppsHub or Studio user. You can create a number of admin accounts without any license concerns - but keep in mind that each admin account still consumes a certain account ID (see Understanding accounts).

To allow the admin to create or change user accounts, the account must be further configured on the authentication service:

Click Manage users in the pSeven Enterprise admin interface page header.
On the Users page that appears, in the Username column, click the name of the admin account.
On the User details page that appears, go to the Role mapping tab.
On the Role mapping tab, assign the admin role to the account.

Applying user storage quotas¶

pSeven Enterprise allows user file storage quota to be set, provided quota is enabled in the user data storage as described in Enabling storage quota (see also Data storage directories). Users who have reached their quota limit will only be able to launch runs, if they have access to some workspace (where the workspace quotas apply instead of user's), and have the permissions to run apps in that workspace. Other launch options, as well as file upload, will be unavailable to such users, until they remove some content from their personal storage - for example, delete some run results they no longer need.

With storage quota enabled in the user data storage, admins can set a default quota that applies to all users as well as individual quotas for specific users. The individual quota setting overrides the default one for the respective user.

To set the default quota for all users:

Sign in to the pSeven Enterprise admin interface and open the Settings page at {admin URL}/app_pseven/settings/.
Enter quota (in megabytes) in the "Default user storage quota" field.
Click the Update users quota button to apply the quota setting, and then click Save.

To set a quota for a specific user:

Sign in to the pSeven Enterprise admin interface and open the Users page at {admin URL}/app_auth/user/.
Click the user's name, and then, in the Advanced area at the bottom of the settings page that appears, enter quota (in megabytes) in the "Storage quota" field.
Click Save.

Disabling an account¶

In pSeven Enterprise, disabling an account is used to revoke its access (deactivate the account) instead of deleting account information and data. Disabled accounts are not counted by the license, so there is no need to delete accounts if you hit a license limit. Also, as explained in Understanding accounts, you cannot free up account IDs by deleting accounts, so there is practically no reason to delete accounts.

Why pSeven Enterprise never reuses account IDs

An account ID is a single-use unique identifier, which is used to generate unique user IDs for various pSeven Enterprise services. Reusing an account ID is a security risk: if Bob is a new user who gets an account ID that previously belonged to Alice (a deleted account), then Bob may gain ownership of Alice's workflows, apps she had published to AppsHub, get access to files other users had shared with Alice, and so on.

To disable an account, open the Change user page for that account and clear the "Active" check box in the Permissions area. There is no need to remove user permissions when disabling an account, as disabled accounts cannot sign in to pSeven Enterprise regardless of the account type. Disabling an account does not remove user files, workflow, apps published by that user and other user data.

If later required, you can re-enable a disabled account by selecting the "Active" check box.

Changing the username¶

A username is assigned when creating the user account in the pSeven Enterprise authentication service (see Adding accounts). Once a user account has been created, changing its username is disabled by default, as this can lead to undesirable effects, such as losing permissions to use an extension node (see Configuring user authorization on extension nodes). However, since your organization may need to enable changing the username, you have the following options to do so:

Change the pSeven Enterprise authentication service setting that prevents the username from being changed. As a result, each pSeven Enterprise user will be able to change their username, and the pSeven Enterprise deployment admin will be able to change the username for any pSeven Enterprise user.
Set up federation with an external user identity storage (see Users from Active Directory or LDAP server). In this case, only a person with the necessary permissions in the external identity storage can change the username.

If you decide to configure the pSeven Enterprise authentication service to allow users to change their usernames, complete the following steps:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the left sidebar, under Configure, click Realm settings, and then go to the Login tab on the PSeven realm settings page that appears.
On the Login tab, turn the Edit username switch On.

As a result of these steps, each user can change their username by using the Manage profile command from the pSeven Enterprise user menu. pSeven Enterprise administrators can also change the username for any user on the Details tab of the account edit page (see Managing user account data). To cancel the user ability to change their username, set the Edit username switch to Off. When this switch is set to Off, neither users nor administrators can change the username in pSeven Enterprise.

Workspaces¶

Workspaces are a way to divide the resources available to pSeven Enterprise between multiple user teams. The concept is similar to Kubernetes namespaces but works at the pSeven Enterprise application level rather than the cluster level. You can create isolated workspaces for users from different departments or even organizations - this way a single deployment can provide each user team with a private working area, containing the team's workflows, apps, and files.

Every workspace provides:

A team-shared resource pool to host the team's workflow and app runs.
A collection of AppsHub apps.
A shared storage for the team's workflows, app, their run results, and other files.

All contents and resources of a given workspace are only available to members of that workspace. Other (non-member) users won't even know that such a workspace exists. This way, a workspace team can collaborate on workflows, share data and apps without interfering with other teams, which use their own workspaces.

Any user can optionally be a member in one or many workspaces (that is, teams). The level of workspace access for a user is defined individually in each workspace by setting the user's permissions in that particular workspace.

A workspace typically has resource quotas in place, which set the limits for its storage space and the number of run slots. The members of that workspace share its resource pool: they can upload files to the workspace storage, or host workflow and app runs, which occupy the shared run slots provided by the workspace. The members' personal resource quotas do not apply in such cases - that is, the users who have the permission to host workflow or app runs in some workspace can do so even if they have already used up their personal run quotas.

Workspace members can also host their runs personally by selecting that option when they launch a workflow or an app from the workspace. Such runs do not consume any workspace resources: run directories are created in the user storage, runs occupy user's personal run slots, and so on. Naturally, the personal resource quotas do apply to such runs.

Configuration basics¶

To create and configure workspaces, sign in to the pSeven Enterprise admin interface and open the Workspaces page, or go to {admin URL}/app_pseven/workspace/. The general steps to set up a new workspace are:

Open the Add workspace page using a button on the Workspaces page, or go to {admin URL}/app_pseven/workspace/add.
Name the workspace. The name identifies the workspace to users - for example, when they open a workspace in Studio or view app details in AppsHub.
Set the workspace resource quotas.
Set the default permissions. These will only apply to the new user accounts, created later than this workspace. Existing users get no access to newly created workspaces, even if you set the default permissions.
- To create a private workspace, leave all default permissions unset. All users, existing and new, will gain access to the workspace only after you set their individual permissions specific to this workspace.
- To create a workspace available to all users, set at least one default permission. The safest ones are "View apps" or "View files", which give the read-only level of workspace access. New users will be able to access this workspace right after their first sign-in. Existing users will not gain access by default - you will have to set their individual permissions in this workspace.
Open access to the workspace for existing users as needed. To allow an existing user into a workspace, set any individual permission for that user in the workspace. All such permissions apply to a specific user in a specific workspace - for example, to allow a user into two workspaces, you will have to set user's permissions in both.
- If you want workspace members to be able to add apps into their workspace, at least one member must have the "Publish apps" permission.

You can also change workspace settings and user permissions later, after creating the workspace: click its name on the Workspaces page to open the page with those settings.

Maximum number of workspaces

The maximum number of workspaces is defined by the maxWorkspaces parameter in the deployment configuration file values.yaml. That parameter sets the number of available workspace IDs. In contrast with the user account IDs, workspace IDs are reusable - so maxWorkspaces is simply the number of workspaces, which can exist at the same time. Even if you reach that maximum, you can delete a workspace to free up the place for a new one.

Resource quotas¶

Workspaces support storage quotas and run limits, which define the workspace's resource pool. Users who run workflows or apps in a workspace consume its resources, and their personal resource quotas do not apply to such runs.

Workspace storage is consumed when users create workflows there, host workflow or app runs, publish apps, store results and so on.
The run limit is the number of run slots available in a workspace. Every workflow or app run hosted in a workspace occupies one of its run slots.

Workspace storage quotas require the disk quota support enabled on the workspace data storage NFS server - see Enabling storage quota.

You can set default quotas, which apply to all workspaces, as well as workspace-specific quotas. The quotas set for a specific workspace override the default ones for that workspace.

To set the default quotas for all workspaces:

Sign in to the admin interface and open the Settings page, or go to {admin URL}/app_pseven/settings/.
Enter the storage quota (in megabytes) in the "Default workspace storage quota (MB)" field.
Enter the number of app run slots in the "Max active runs per workspace" field.
Click Save at the bottom of the page.

To set quotas for a specific workspace:

Sign in to the admin interface and open the Workspaces page, or go to {admin URL}/app_pseven/workspace/.
Click the name of the workspace to open the Change workspace page. There, enter quota (in megabytes) in the "Storage quota" field; enter the number of run slots in the "Max active runs" field.
Click Save at the bottom of the page.

Workspace permissions¶

User access to a workspace is determined by their permissions in that specific workspace. Every user actually has independent permission settings in every workspace. The default workspace permissions are simply configuration shortcuts, which apply certain permission settings to a user only once, when you create the user account. If you change the default permissions of a workspace, the permission settings of existing users and workspace members will not change - new defaults will only apply to new users you add afterwards.

graph LR
  u1[User Alice]
  u2[User Bob]
  perm_u1w1[Permissions<br>Alice has in W1]
  perm_u1w2[Permissions<br>Alice has in W2]
  perm_u2w2[Permissions<br>Bob has in W2]
  subgraph w1 [Workspace W1]
    apps1[W1 apps]
    resources1[W1 resource<br>quotas]
  end
  subgraph w2 [Workspace W2]
    apps2[W2 apps]
    resources2[W2 resource<br>quotas]
  end

  u1 --- perm_u1w1
  perm_u1w1 --- w1
    u1 --- perm_u1w2
  perm_u1w2 --- w2
  u2 --- perm_u2w2
  perm_u2w2 --- w2

For a new workspace, you have the option to set permissions while you create it. For existing workspaces, you can change permissions in its settings:

Sign in to the admin interface and open the Workspaces page, or go to {admin URL}/app_pseven/workspace/.
Click the name of the workspace to open the Change workspace page.
Add or remove users as needed, set their permissions.
Click Save at the bottom of the page.

The permission set used in workspaces is the following:

View apps. Allows viewing and opening this workspace's apps in AppsHub. Does not permit using the resources of this workspace to run apps, yet users can run apps from this workspace by hosting their runs personally (user's personal quotas apply).
Run apps. Allows users to host their app runs in this workspace, consuming the workspace resources. Also allows users to rename or delete their own runs, but not others'.
Publish apps. Allows users to add apps to this workspace by publishing their workflows. Also allows users to delete the apps they have published, but not others.
Manage existing apps. Allows deleting and renaming any run and any app in this workspace, including those created by other users.
Read files. Allows viewing this workspace in Studio. Does not permit using the resources of this workspace, yet users can copy or run workflows from it by hosting their runs runs personally (user's personal quotas apply).
Run and manage workflows. Allows creating, editing, and running workflows in this workspace. Such runs consume the workspace resources. Also allows deleting the workspace files and workflows.

There are certain dependencies - some higher level permissions require lower level ones and enable them automatically. For example, if you enable "Run apps" or "Publish apps" for a user, they also get the "View apps" permission as a prerequisite.

Public workspace¶

While workspaces are designed to be private, it is also possible to create public workspaces, which are open to all users by default. This is achieved by setting the default permissions while creating a workspace, which must be done before creating any user accounts - since the workspace's default permission settings never apply to the already existing user accounts.

For example, you can set up a public repository of apps in AppsHub by creating a workspace with the "Publish apps" default permission ("View apps" will be auto-enabled). Users will be able to publish apps, run any app, and manage the apps and data they own - update, rename, delete and so on. All app runs will consume personal (user) resources, because users do not have the permission to host their runs in the workspace. Also, you or selected users should have the "Manage existing apps" permission in this workspace - they will be moderators, who are able to clean up unneeded apps or runs, for example.

Another kind of such setup is to enable only the "View apps" default workspace permission. Users will be able to open apps and run them using personal resources, but will be unable to publish apps to this workspace. The Examples workspace, created during the pSeven Enterprise installation, actually uses this configuration.

Users from Active Directory or LDAP server¶

The pSeven Enterprise deployment includes a user identity and authentication service, which stores pSeven Enterprise user accounts in its database. The user is authenticated against their account each time they enter their username and password to sign in to pSeven Enterprise. To create user accounts, a pSeven Enterprise administrator can use the admin interface (see Adding accounts). User accounts created in this way are not connected to user identity storages that already exist in the organization, and user authentication is conducted by pSeven Enterprise's own service.

Organizations using pSeven Enterprise normally have their own user identity and authentication services. A typical example is AD DS (Active Directory Domain Services). It will most likely not be possible to migrate your organization's user accounts to the database of the pSeven Enterprise authentication service, so account federation is provided to allow users to sign in to pSeven Enterprise with their credentials from the organization's user identity storage. pSeven Enterprise provides user account federation for AD DS as well as other flavors of LDAP server.

This section describes how to set up user account federation, which is appropriate for most organizations that use AD DS; federation with a generic LDAP server is also briefly discussed. By federating with AD DS, users in your organization can sign in to pSeven Enterprise using the same username and password they use to sign in to their work computers. It is important to note that user authentication in this case is performed by AD DS, the pSeven Enterprise authentication service does not have any information about the user's password.

Federation uses accounts that originate in AD DS and have a partial copy in the database of the pSeven Enterprise authentication service. Changes to user account attributes only go in one direction: if an attribute is changed in AD DS, the new attribute value is copied to the corresponding account of the pSeven Enterprise user. It is possible to make changes directly to the pSeven Enterprise user account, but the data in that account is never copied to AD DS and can be automatically superseded with the user account data from AD DS.

To set up user federation, use the pSeven Enterprise authentication service web interface:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the left sidebar, under Configure, click User federation, and then click the Add Ldap providers card on the User federation page that appears. If the card is missing, select the "LDAP" item from the Add new provider drop-down on that page.
Use the page that appears to specify the provider settings.

Provider setup¶

To configure the provider, you need to prepare the following information about AD DS used in your organization:

The Fully Qualified Domain Name (FQDN) of the computer running the AD DS domain controller on the network where your pSeven Enterprise deployment is located. If it turns out that your user identity storage provider cannot connect to AD DS by computer name, you will need the computer's IP address.
The Distinguished Name (DN) of the Organization Unit (OU) that holds the user accounts to sign in to pSeven Enterprise. Example: OU=p7users,DC=prod,DC=company,DC=corp. The user accounts for the provider configuration in question should be direct descendants of this OU object in the AD DS tree. Example of such a user DN: CN=John Smith,OU=p7users,DC=prod,DC=company,DC=corp.

The above restrictions simplify provider configuration as they match its default settings. By changing the search depth and using the LDAP filter in the provider settings, you can choose exactly those accounts in the given OU or container that will be used to sign in to pSeven Enterprise.
The Distinguished Name (DN) and password of the AD DS user account that has sufficient rights to search for and read user accounts in the OU specified above. Example: CN=p7provider,CN=users,DC=prod,DC=company,DC=corp.

In the pSeven Enterprise authentication service web interface, set the provider settings listed below, leave the rest of the settings default.

UI display name
The name of this provider to display in the list on the User federation web interface page - for example, Active Directory.
Vendor
Choose "Active Directory" from the drop-down list. This way you choose the configuration that matches the provider for AD DS.
Connection URL
In this field, enter the Fully Qualified Domain Name (FQDN) of the computer running the AD DS domain controller. Precede the name with the ldap:// prefix, such as ldap://dc-office.prod.company.corp. Depending on your network configuration, you may need to add the LDAP port number, such as ldap://dc-office.prod.company.corp:389, or specify the computer's IP address instead of the computer name.

Click the Test connection button to see if the provider can connect to the computer specified.
Bind DN
In this field, enter the Distinguished Name (DN) of the AD DS user account that has sufficient rights to search for and read user accounts in the OU specified by the Users DN setting. Example: CN=p7provider,CN=users,DC=prod,DC=company,DC=corp.
Bind credentials
In this field, enter the password of the user specified by the Bind DN setting.

Click the Test authentication button to test the connection to AD DS on behalf of the specified user. If the test fails, verify that you entered the correct user DN and password, or contact your AD DS administrator for advice.
Edit mode
Choose "UNSYNCED" from the drop-down list. With this choice, it is possible to make changes directly to pSeven Enterprise user accounts, but the data in those accounts is never copied to AD DS and can be automatically superseded with data from AD DS.
Users DN
In this field, enter the Distinguished Name (DN) of the Organization Unit (OU) that holds the user accounts to sign in to pSeven Enterprise, such as OU=p7users,DC=prod,DC=company,DC=corp.
Username LDAP attribute
In this field, enter userPrincipalName - this attribute holds the AD DS user logon name. As a result, when signing in to pSeven Enterprise, the user will have to enter their name they normally use to sign in to their work computer.
RDN LDAP attribute
In this field, enter name. This attribute specifies the relative distinguished name (RDN) of the user account in the AD DS storage.

When you are done with the settings listed above, click the Save button at the bottom of the web interface page. The name of the provider you have created appears on the User federation page. Click the name of the provider to open its settings page. Click the Mappers tab on that page to set up copying user account attributes from the AD DS storage to the pSeven Enterprise authentication service database (see Attribute mappers).

Using secure LDAP¶

If secure LDAP is required to connect to your AD DS domain controllers, configure the provider's Connection URL as follows:

Enter the ldaps:// prefix and the LDAPS port number (636 by default), such as ldaps://dc1.prod.corp:636.

Connection using LDAPS requires registering the appropriate certificate in pSeven Enterprise, otherwise the Connection URL setting with the ldaps:// prefix causes a connection error. The certificate registration is performed when deploying pSeven Enterprise - for instructions, see LDAPS setup in the deployment guide.

Attribute mappers¶

The user identity storage provider uses so-called mappers to copy certain attributes from the AD DS user account to the corresponding pSeven Enterprise user account. All the required attribute mappers are created as part of the provider, you can view them on the Mappers tab of the provider settings page. Each mapper watches for changes to a particular attribute in AD DS and ensures that the data in that attribute is copied to the pSeven Enterprise user account.

The username mapper ensures that the pSeven Enterprise username is the same as the AD DS user logon name. Configure this mapper as follows:

In the list on the Mappers tab, click "username".
On the page that appears, make sure that the LDAP Attribute field specifies userPrincipalName - this attribute holds the AD DS user logon name.

The mappers are also used to copy the following data from AD DS:

Mapper	Data
first name	Given name of the user
last name	Surname of the user
email	The user's email address
MSAD account controls	Account enabled or disabled

You do not need to change the default settings for these mappers.

Starting up federation¶

Once you have set up the provider and its mappers, you can start up federation of user accounts:

At the top of the provider settings page, click the Action button and select the Sync all users action.

For each AD DS user account that matches the provider settings, this will create the associated account of the pSeven Enterprise user.

It is recommended to start up federation provided that pSeven Enterprise does not yet have accounts for users from AD DS to be federated. It may not be possible to create an associated account if the user already has an account in pSeven Enterprise, which was created, for example, following the Adding accounts instructions. If you encounter this issue, delete the previously created pSeven Enterprise user account; alternatively, you could perform account matching as described in the pSeven Enterprise upgrade notice in the v2022.09 changelog.

In the future, to copy changes from AD DS user accounts to their associated pSeven Enterprise accounts, use the Sync changed users item from the Action button's drop-down on the provider settings page. Copying changes will also create the associated account for each AD DS user account that matches the provider settings but does not have an associated account.

Instead of using the Action button, you can configure the provider to copy changes automatically on a scheduled basis:

In the Synchronization settings area on the provider settings page, turn the Periodic changed users sync switch On.

The user identity storage provider allows making changes to associated accounts. As a result, the data in the associated account may not match the data from AD DS. To restore the data match, use the Synch all users item from the Action button's drop-down on the provider settings page, or turn the Periodic full sync switch On in the Synchronization settings area on the provider settings page.

Other LDAP servers¶

pSeven Enterprise provides federation of user accounts not only with AD DS, but also with many other flavors of LDAP server, including:

Active Directory Lightweight Directory Services (AD LDS)
Red Hat Directory Server
IBM Tivoli Directory Server
Novell eDirectory Directory Services
Opensource LDAP server implementations such as OpenLDAP, ApacheDS, and OpenDJ

Configuring federation with these flavors of LDAP server is similar to that described in the previous sections for Active Directory Domain Services (AD DS). You start by setting up the user identity storage provider, choose the "UNSYNCED" mode from the Edit Mode list, and then select the Vendor list item that matches your LDAP server (select "Other" if your LDAP server is not listed). The fields on the provider settings page are filled with default values based on the selected Vendor setting. Consult your LDAP server documentation to verify that the following field values are correct, as the default values may not match the actual attribute schema of your LDAP server:

Username LDAP attribute
The username of the pSeven Enterprise user will be copied from this attribute. If you specify an attribute other than the default, it must also be specified in the username mapper (see Attribute mappers).
RDN LDAP attribute
This attribute specifies the relative distinguished name (RDN) of the user account in the object tree of the LDAP server.
UUID LDAP attribute
This attribute specifies the unique identifier for the object in the LDAP server storage. Many LDAP servers store the unique identifier in the object's entryUUID attribute. However, some LDAP servers may use a different attribute, such as objectGUID.
User object classes
The object class names that are listed in the LDAP server user account's objectClass attribute. Federation only applies to LDAP server objects whose objectClass contains all the object classes specified in this field.

Connection to the LDAP server and search for LDAP objects that represent user accounts for federation are configured similarly to the AD DS provider described earlier (see Provider setup). To configure connection to the LDAP server, use the Connection URL and Bind type fields, as well as the Bind DN and Bind credentials fields for non-anonymous connection. To configure search, specify the search base in the Users DN field, select the desired depth of the search from the Search scope drop-down and, if necessary, specify an additional LDAP filter in the User LDAP filter field. To use the LDAPS protocol, the connection string in the Connection URL field must contain the prefix ldaps:// and the approrpiate port number (see Using secure LDAP).

Having completed the provider setup, check the setup of its mappers (see Attribute mappers). Specifically, check the username mapper: the LDAP attribute that is mapped to the pSeven Enterprise username must match the attribute specified in the provider's Username LDAP attribute setting. You do not usually need to change the default mapper settings.

Now you can copy data from the LDAP server, creating pSeven Enterprise user accounts as described in Starting up federation.

Since the pSeven Enterprise authentication service is based on the Keycloak platform, more information on configuring and using federation with Active Directory or LDAP server can be found in the Using external storage section of the Keycloak Server Administration guide.

Setting up single sign-on (SSO)¶

This section describes how to set up single sign-on (SSO), enabling users to sign in to pSeven Enterprise with their credentials from an external identity provider. It is assumed that your organization has a Keycloak server deployed to serve as an identity provider for signing in to various applications, including pSeven Enterprise. As a result of the single sign-on setup described below, users registered with your organization's Keycloak server will be able to sign in to pSeven Enterprise in the same way that they normally sign in to other applications used in your organization.

To integrate pSeven Enterprise with an external identity provider, a series of steps must be taken both on your organization's Keycloak server and in the pSeven Enterprise authentication service settings:

The administrator of your organization's Keycloak server is required to register pSeven Enterprise as an OpenID Connect protocol-based client. You provide your organization's Keycloak server administrator with certain parameters of your pSeven Enterprise deployment and request them to create a client with the appropriate settings.
Once the client is created, you can use its parameters to register your organization's Keycloak server as the identity provider for your pSeven Enterprise deployment. In the pSeven Enterprise authentication service, you create an identity provider that uses the client parameters to communicate with the Keycloak server when performing single sign-on.

Once you have set up the identity provider, the pSeven Enterprise sign-in form displays a button to authenticate the user with that provider. The button label is the display name assigned to the identity provider in the pSeven Enterprise authentication service. Users click the button and sign in to pSeven Enterprise with their credentials located on your organization's Keycloak server.

The rest of this section describes how to prepare the client for pSeven Enterprise on the Keycloak server and how to set up the identity provider in the pSeven Enterprise authentication service.

Registering pSeven Enterprise as an OpenID client¶

To register pSeven Enterprise, the Keycloak server administrator needs the following information about the desired client settings:

Client type
pSeven Enterprise client type is "OpenID Connect".
Client ID
Names and identifies pSeven Enterprise as a client. It is advisable to specify a meaningful name such as pSevenEnterprise or pSevenKeycloak.
Valid redirect URIs
This client setting should allow any resource under the pSeven Enterprise sign-in URL, for example: https://pseven.company.com/*. Depending on your pSeven Enterprise deployment configuration, a port number may be required in the sign-in URL, such as https://pseven.company.com:30080/*.
Web origins
Specify + to permit all origins of valid redirect URIs.
Client authentication
In the client details, this switch on the Settings tab must be On, and the "Client Id and Secret" option must be selected in the Client Authenticator field on Credentials tab.

Prepare the information listed above and send it to the Keycloak server administrator, who will create a client for pSeven Enterprise. Note that the pSeven Enterprise client must be created in the Keycloak realm that contains the users that will sign in to pSeven Enterprise via SSO. For client setup instructions, see Managing OpenID Connect clients in the Keycloak Server Administration guide.

To add your organization's Keycloak server as an identity provider to pSeven Enterprise, you are going to need a few client parameters from Keycloak. Request the following information from the Keycloak server administrator:

Client ID
The ID assigned to the pSeven Enterprise client on the Keycloak server.
Client Secret
The contents of the Client Secret field located on the Credentials tab of the client details page in the Keycloak admin interface.
OpenID Endpoint Configuration
To get this configuration, the Keycloak administrator can open the settings page for the Keycloak realm where the pSeven Enterprise client is created, click the OpenID Endpoint Configuration link, and then copy and save the entire contents of the page that appears to a text file. You will import that file when setting up the identity provider in pSeven Enterprise.

Provider setup¶

To set up the identity provider, use the pSeven Enterprise authentication service web interface:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the left sidebar, under Configure, click Identity providers, and then click the Keycloak OpenID Connect card on the Identity providers page that appears. If the card is missing, select the "Keycloak OpenID Connect" item from the Add provider drop-down on that page.
Use the page that appears to specify the provider settings.

On the provider settings page, specify the settings listed below. Leave all other settings on that page default.

Alias
Names and identifies the identity provider. It is advisable to specify a meaningful name indicating the protocol and server of this provider, such as oidc-keycloak.
Display name
The name to display on the pSeven Enterprise sign-in button corresponding to this identity provider.
Use discovery endpoint
Turn this switch Off.
Import config from file
In this field, upload the OpenID Endpoint Configuration file that was collected when configuring the client on the Keycloak server.

As a result, the client endpoint URLs from the imported file are substituted into the appropriate provider setting fields.
Client authentication
Make sure that "Client secret sent as post" is selected from the drop-down in this field.
Client ID
In this field, enter the ID assigned to the pSeven Enterprise client on the Keycloak server (for example, pSevenKeycloak).
Client Secret
In this field, enter the string of characters representing the client secret that was collected when configuring the pSeven Enterprise client on the Keycloak server.

When you are done with the settings listed above, click the Add button at the bottom of the provider settings page to add the provider to the pSeven Enterprise authentication service.

Having added the provider, click Identity providers on the left sidebar, and then, in the list on the page that appears, click the provider's name to open its settings, and change them as follows on the Settings tab:

Backchannel logout
Turn this switch On. To access this switch, click Advanced in the OpenID Connect settings area.
Trust Email
Turn this switch to On. Otherwise, the identity provider might ask users to verify their email upon the first sign-in. This switch is in the Advanced settings area.

When you are done, click Save at the bottom of the provider settings page to apply your changes.

Once you have configured the identity provider, you must restrict user profile management in pSeven Enterprise; otherwise, users will be able to change their account details or passwords, which causes issues when synchronizing pSeven Enterprise user accounts with the Keycloak server:

In the pSeven Enterprise authentication service web interface, on the left sidebar, under Configure, click Clients.
On the Clients page, click "account" in the Client ID column.
On the Account page that appears, turn the Enabled switch Off.
On the left sidebar, under Configure, click Realm settings, and then go to the Login tab on the PSeven realm settings page that appears.
On the Login tab, turn the Edit username and Forgot password switches Off.

To test your setup, open the pSeven Enterprise user sign-in page. The sign-in form should display an additional sign-in button labeled with the Display Name you have specified in the identity provider settings. Clicking that button redirects a user to your organization's Keycloak sign-in page, where the user has to sign in once to get access to pSeven Enterprise. Subsequent sign-ins to pSeven Enterprise with the identity provider do not require users to enter their credentials as long as they are signed in to the organization account.

Upon that first sign-in, normally a user who has signed in at the organization's Keycloak sign-in page is immediately redirected to the pSeven Enterprise end-user interface. If you sign in as a user and get additional dialogs asking you to update account information, verify your identity provider settings with the Keycloak server administrator. In particular, check that:

The User Info URL field in the provider's OpenID Connect config settings area contains a correct URL.
The Trust Email switch in the provider's Advanced settings area is On.

User session expiration settings¶

Immediately after a successful sign-in to pSeven Enterprise, the user session begins. During this session, the user accesses pSeven Enterprise services and resources without re-authentication. Before the session expires, the user can even close all their browser windows and then reopen the pSeven Enterprise web interface without entering their password.

The session ends when the user signs out of pSeven Enterprise with the Sign out command. In addition, the authentication service itself forces the session to end after a certain expiration time. Once the user session has ended, the user will have to re-enter their name and password to sign in to pSeven Enterprise.

The user session expiration settings are controlled by the authentication service. For all users, the same settings apply that you can view or change in the authentication service web interface:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the left sidebar, under Configure, click Realm settings, and then go to the Sessions tab on the page that appears.

The pSeven Enterprise user session expiration is determined by the following settings on the Sessions tab:

Offline Session Idle
The period of time after which the session is forced to end when there is no user activity. For example, if set to 1 day, this setting causes the session to end provided that the user has not been active on pSeven Enterprise for 24 hours.
Offline Session Max Limited
Enables the Offline Session Max setting.
Offline Session Max
The period of time after which the session is forced to end regardless of user activity. For example, if set to 15 days, this setting ensures that the session ends no later than 15 days after the user signs in to pSeven Enterprise.

It is highly advisable to manage pSeven Enterprise user session expiration using only the settings listed above, without changing any other settings on the Sessions tab.

Auditing user and admin activity¶

pSeven Enterprise logs user and admin activity as event records, which you can view in the admin web interface:

Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
On the page that appears, in the left sidebar menu, click Events.

The User events tab displays user-related events such as successfully signing in to pSeven Enterprise, entering an incorrect password, or changing a user account. The Admin events tab displays events triggered by administrator actions. The Search field on these tabs helps you find the events you are interested in.

To view or change event settings, click Realm settings in the left sidebar menu, and then, on the page that appears, click the Events tab. The events triggered by administrator actions are saved by default, since Save events switch is On in the Admin events settings area on that tab. To save user-related events, turn the Save events switch On in the User events settings area as well.

Since pSeven Enterprise user and admin activity logging is based on the Keycloak platform, more information on configuring and using events for auditing can be found in the Configuring auditing to track events section of the Keycloak Server Administration guide.

Configuring user authorization on extension nodes¶

By default, an extension node is available to all pSeven Enterprise users, so any user can run tasks on the node. To restrict user access, you can set the DA__P7__ALLOWED_USERS environment variable on the node.

The extension node installer does not set the DA__P7__ALLOWED_USERS variable, so by default that variable does not exist, and all users are allowed to run tasks on the node.

When setting DA__P7__ALLOWED_USERS, add it to system variables, not user variables for the current user. If you set the DA__P7__ALLOWED_USERS environment variable on a node, the value should be a regular expression that matches names of users who are authorized to run tasks on that specific node.

If DA__P7__ALLOWED_USERS is not set, the node is available to all users without restrictions.
If DA__P7__ALLOWED_USERS is set, a user can run tasks on the node only if the regular expression in DA__P7__ALLOWED_USERS matches the username.

Username change

If any user specified by DA__P7__ALLOWED_USERS changes the username (see Changing the username), you will have to change the regular expression in DA__P7__ALLOWED_USERS to match the new username. Otherwise, the renamed user will lose access to the extension node.

Note that users who are not authorized to use a node cannot configure their blocks to run on the node. They also cannot run a workflow that contains such blocks, even if this workflow is shared to them by a user who is authorized on the node.

Engineering software on extension nodes¶

Engineering software installers often create environment variables, required for the launch and normal operation of that software. A typical example is a variable that contains a path to application DLLs or to a license file. However, workflow blocks executed on an extension node run in a safe environment that includes only required system environment variables by default. If the block library of your deployment is supposed to contain blocks that run certain CAD/CAE software packages on extension nodes, all environment variables required by those packages must be added to the block runtime on the extension node. Otherwise, an application launched by such a block will stop with an error, because its required environment variables are unset in the block runtime by default - so the block will not be able to run the software packages it needs.

To add environment variables to the block runtime, the extension node installation provides the block_environment.ini file located in its release (installation) directory. You can find the path to that directory in the DA__P7__RELEASE_DIR environment variable.

Each additional environment variable is added to the block_environment.ini file as follows:

BLOCK_ENVIRONMENT = $(BLOCK_ENVIRONMENT) YOUR_VARIABLE_NAME='variable value'

Each variable should be added on a separate line. Enclose values of variables in single quotes '. If a variable's value itself contains a single quote, escape it by adding another single quote next to it: VARIABLE_NAME='variable''s value'.

After making changes to the block_environment.ini file, reboot the extension node for your changes to take effect.

Maintaining block runtime environment¶

Each block in user workflows requires a specific version of the block runtime environment (runenv). A runenv is a Docker image that must be available in your Registry. User workflows often contain different block versions, and each block version requires its own specific runenv version. Your Registry must provide all the runenv images required by blocks; otherwise, some user workflows are going to fail.

Why do you need to keep runenv images from previous versions

When you upgrade pSeven Enterprise, existing user workflows and blocks are not updated automatically. Keeping their existing versions avoids changes in workflow behavior that may arise from updates, thus ensuring stability. Users have the option to upgrade blocks in their workflows but usually opt out. Consequently, workflows created in earlier versions of pSeven Enterprise retain their requirements for previous runenv versions. For this reason, you should keep the runenv images that already exist in your Registry, even after you upgrade pSeven Enterprise, as those images remain necessary for running previously created workflows.

The most reliable approach to maintaining runenv images in your Registry is to add and keep all their versions available from the pSeven SAS Registry. By doing so, you can be sure that your deployment is backwards compatible with the workflows created in earlier versions of pSeven Enterprise.

Alternatively, you can store only those runenv versions specific to your deployment history - for example, if you installed v2024.10 and then upgraded to v2025.02, add those two runenv versions but skip others in between. This is only good as a starting point; be aware that users may upload workflows created on other pSeven Enterprise deployments - for example, a workflow created and edited in v2025.01 then uploaded to your v2025.02 will require its corresponding runenv version from v2025.01, which you may not have. In such cases, you will need to add that missing runenv version as described in this section, or negotiate with users to update blocks in their uploaded workflows, thus switching those blocks to the latest runenv version.

If a required image is missing from your Registry, you can add it from the pSeven SAS Registry at any time without stopping your deployment. First, check which runenv versions your deployment requires:

Connect to the Pod running a pSeven Enterprise appserver instance.
There, enter the management command: pseven-manage list_runenv.

The list_runenv command scans the pSeven Enterprise file storages for blocks, prints their runenv requirements, and finishes with a summary like the following:

Collecting the block runtime requirements from your deployment.
Scanning /Users...
...

Finished searching.
Your deployment requires these runenv versions:
  python3_10:2024.10.27-1730021829.874ab762-master-release
  python3_10:2024.11.30-1731435553.2f175856-master-release
  inhouse_gen2:latest
Check that your Registry contains the corresponding runenv images.
Add the missing images in order to avoid errors in user workflows.

In the above example, the python3_10:{version} environments are versions of the stock runenv; you can pull their images from the pSeven SAS Registry. Then, inhouse_gen2:latest is a custom runenv, presumably required by some custom-developed user block. To get or update the image of the latter, you will need to contact the developers of that block.

For each version of the stock runenv (python3_10:{version}), there are two corresponding images, named pseven-runenv-python3_10:{version} (the main one) and pseven-extnode:{version} (the image for extension nodes). Note that {version} is the full version number with a tag, and your Registry must contain exactly those image versions that match the runenv versions output by pseven-manage list_runenv.

To update your Registry with the images of a specific runenv version (such as python3_10:{version}):

docker login registry.pseven.io  # Log in to the pSeven SAS Registry
docker login {your registry address}  # Log in to your local Registry
# Add the main image.
docker pull registry.pseven.io/pseven-runenv-python3_10:{version}
docker tag registry.pseven.io/pseven-runenv-python3_10:{version} {your registry}/pseven/pseven-runenv-python3_10:{version}
docker push {your registry}/pseven/pseven-runenv-python3_10:{version}
# Add the extension node image (if you don't use extension nodes, skip this).
docker pull registry.pseven.io/pseven-extnode:{version}
docker tag registry.pseven.io/pseven-extnode:{version} {your registry}/pseven/pseven-extnode:{version}
docker push {your registry}/pseven/pseven-extnode:{version}

To check which runenv versions are available in the pSeven SAS Registry, you can list the runenv image tags. Send this API request (python3_10 runenv as an example), using your pSeven SAS Registry access credentials ({user}:{password}):

curl -X GET -u {user}:{password} -L https://registry.pseven.io/v2/pseven-runenv-python3_10/tags/list | jq '.tags | .[]'

After you have added all the required images to your Registry, generate the extension node setup packages. For each of the runenv versions in your Registry ({version}), run:

docker run --rm -v .:/out --env DA__P7__RELEASE_TAG={version} registry.pseven.io/pseven-extnode:{version}

The above command creates an archive named pSevenRunEnv-{version}.zip. Deliver it to each of your extension nodes, unpack it, and run bootstrap-runenv.bat to install. Repeat that for each runenv version used in your deployment.

Installing additional Python modules¶

User block developers and users who run Python script blocks often need Python modules that are not included with pSeven Enterprise by default. This section provides instructions for installing additional Python modules into the pSeven Enterprise block runtime environment on the Kubernetes cluster, as well as on Windows extension nodes. Modules added in this way are available to all workflow blocks running within the given pSeven Enterprise deployment.

Installing modules into the pSeven Enterprise deployment¶

When executed on the Kubernetes cluster, Python script blocks as well as other blocks leverage the block runtime environment created during pSeven Enterprise deployment. You can get a list of Python modules that are available in that environment by test running the script described in the Python script block guide section Listing modules. The list of modules appears on the Log tab in the Python script block configuration dialog that you use to test run the script. If you need a module that is not in the resulting list, you must install that module package into the block runtime environment for your pSeven Enterprise deployment.

Installing additional Python modules may create issues for users

Installing additional Python modules into the pSeven Enterprise deployment affects the block execution service, and might break the operation of existing workflows. This approach should only be used when absolutely necessary. Note that users can install additional modules into their workflows (see the Python script block guide section Adding modules) or configure their blocks to run on a Windows extension node that has required modules installed (see Installing modules on an extension node).

If your users experience workflow run issues after you install additional Python modules, uninstall pSeven Enterprise, and then reinstall it without the additional modules.

Installing additional Python modules into the pSeven Enterprise deployment requires that you have a local Docker image registry. The installation procedure in general terms is as follows:

Make sure your local registry contains the images of your deployed pSeven Enterprise version. You can get those images from the pSeven SAS Registry at registry.pseven.io - for details, see Image registry setup in the pSeven Enterprise deployment guide.
Build a custom Docker image for the pseven-htcondorexecute service with additional Python modules, giving it a unique name; then, upload it to your local image registry.
Make changes to your pSeven Enterprise deployment configuration file values.yaml:
- Make sure that the registry.url, registry.username, and registry.password parameters specify the URL, username and password to access your local registry.
- Make sure that the image.* parameters match the location of the pSeven Enterprise images in your local registry.
- In the image.htcondorexecute parameter, specify the URL to match the custom Docker image you have built for the pseven-htcondorexecute service with additional Python modules.
Uninstall your existing pSeven Enterprise release, and then reinstall it using the file values.yaml you have prepared.

Let's take as an example the installation of Python modules from the emoji package into a pSeven Enterprise deployment. The pSeven Enterprise images are assumed to be in the local image registry at registry.company.net/pseven/.

First, you have to build an updated image for the pseven-htcondorexecute service, and upload it to your local registry:

Log in to your image registry:
```
docker login registry.company.net
```
Create a new local directory where you will build the image and change to that directory:
```
mkdir pseven-htcondorexecute-build && cd pseven-htcondorexecute-build
```

In the local directory, create a Dockerfile with the following contents:

# Make sure that the version of the image being pulled from the registry exactly matches your deployed pSeven Enterprise version (v2022.08.01-1659381555.fe6143b8 in this example).
FROM registry.company.net/pseven/pseven-htcondorexecute:v2022.08.01-1659381555.fe6143b8

# Installing Python modules from the emoji package.
RUN : "Install Python emoji." \
    && source ${DA__P7__RUNENV__PYTHON3__ACTIVATE} \
    && pip install emoji

Use the docker build command to build your custom image. Specify an image name composed of the original image name and the -custom suffix:
```
docker build . -t registry.company.net/pseven/pseven-htcondorexecute:v2022.08.01-1659381555.fe6143b8-custom
```

Use the docker push command to upload the newly built image to your registry:

docker push registry.company.net/pseven/pseven-htcondorexecute:v2022.08.01-1659381555.fe6143b8-custom

Next, make changes to your pSeven Enterprise deployment configuration file values.yaml:

Make sure that the registry.url parameter specifies the URL of your local registry (registry.company.net).
Make sure that the registry.username and registry.password parameters specify the username and password to access your local registry.
Make sure that the image.* parameters specify the URLs where pSeven Enterprise images are located in your local registry. For example:
```
image:
    app: registry.company.net/pseven/pseven-app:v2022.08.01-1659381555.fe6143b8
```

In the image.htcondorexecute parameter, specify the URL of your custom pseven-htcondorexecute image:

image:
    htcondorexecute: registry.company.net/pseven/pseven-htcondorexecute:v2022.08.01-1659381555.fe6143b8-custom

Reinstall pSeven Enterprise with the new values.yaml file:

Uninstall your existing pSeven Enterprise release. For instructions, see Uninstalling pSeven Enterprise.
Reinstall pSeven Enterprise with the new values.yaml file and the previously used release name.
Wait for the deployment finalization message to appear.

The procedure for building a custom pseven-htcondorexecute image described above must be repeated every time when upgrading pSeven Enterprise to a new version. For the FROM directive in the Dockerfile, be sure to specify the pseven-htcondorexecute image of the exact pSeven Enterprise version that will be deployed. If you do not rebuild your custom image or do not update the version of the original pseven-htcondorexecute image in the Dockerfile, the new pSeven Enterprise version deployment will not work. When upgrading, you must also give a new name to the newly built custom image, matching the new pSeven Enterprise version, and update the custom image URL in values.yaml. As a new name, you could use the original image name followed by the -custom suffix.

A summary of the upgrade steps in this case is as follows:

Make sure your local registry contains the images of the new pSeven Enterprise version, to which you are going to upgrade.
Based on the original image of the new version, build a custom Docker image for the pseven-htcondorexecute service with additional Python modules, giving it a unique name. Upload the updated custom image to your registry.
In your deployment configuration file values.yaml, update the image.htcondorexecute parameter to match the name of the updated custom pseven-htcondorexecute image.
Follow the instructions in section Upgrading pSeven Enterprise to deploy the new pSeven Enterprise version from your local registry.

Installing modules on an extension node¶

When executed on a Windows extension node, Python script blocks as well as user blocks leverage the Python runtime environment created during node deployment. You can get a list of Python modules that are available in that environment by using the script described in the Python script block guide section Listing modules: configure a Python script block with that script to run on the extension node (select the name of the node from the Run on: dropdown in the Block properties pane), execute the block within a workflow run, and then view the resulting list in the run log. If you need a module that is not in the resulting list, you must install that module package into the Python runtime environment on the extension node.

Before making changes, back up the Python runtime folder:

Sign in to the computer running your extension node and activate the Python runtime. To do this, open an elevated Command Prompt window and enter the following command in that window:
```
call %DA__P7__RUNENV__PYTHON3__ACTIVATE%
```
Note the path that appears in parentheses to the left of the command prompt. This is the path to the folder that holds the Python runtime environment for Python script blocks.
Create a backup copy of that folder in another location not related to the extension node installation. This copy can be used to restore the environment if the package installation below fails.

Having created the backup of the Python runtime folder, you can add modules using pip at the command prompt in the window where you have activated the Python runtime. For example, to install the latest version of some_package from Python Package Index, enter the following command:

pip install some_package

If you are not satisfied with the changes made when installing new packages, you can roll back them by restoring the Python runtime folder from the backup.

The procedure for installing additional modules must be repeated every time after the extension node is upgraded to a new pSeven Enterprise version. For this reason, it is advisable to maintain and store a list of additional modules installed on the extension node. After upgrading the extension node, you can use this list to find out which additional modules to install.

Backing up and restoring user data¶

To ensure that pSeven Enterprise user data can be recovered in the event of a system failure or data loss, the following backups are required:

A backup copy of the values.yaml configuration file for your pSeven Enterprise deployment, which you can obtain with the following command:
```
helm get values pseven-rl -n pseven-ns -o yaml > values.yaml
```
Here pseven-rl and pseven-ns stand for the pSeven Enterprise release name and namespace. Those are example names used in this guide and other pSeven Enterprise administration guides. If you are using another names, replace pseven-rl and pseven-ns in example commands with the names that you actually use.
Backups of the file storages specified by the storage.* parameters in values.yaml.

File values.yaml provides the storage.userdata.*, storage.workspacedata.*, and storage.shareddata.* parameters to specify the NFS server address and file storage paths. Use your NFS server backup tools to back up each of these file storages on a regular basis.
A backup of the database specified by the postgres.dbname parameter in values.yaml.

File values.yaml provides the postgres.* parameters to specify the PostgreSQL server and database name used in this pSeven Enterprise deployment. Use your PostgreSQL server backup tools to back up the pSeven Enterprise database on a regular basis. For database backup instructions, see Backup and Restore in the PostgreSQL server documentation.

Steps to restore pSeven Enterprise user data from backups:

Uninstall your existing pSeven Enterprise release. For instructions, see Uninstalling pSeven Enterprise.
From the backups, restore the file storages specified by the storage.* parameters in values.yaml. To do this, use your NFS server backup management tools.
From the backup, restore the database specified by the postgres.dbname parameter in values.yaml. For instructions on restoring a database from its backup, see Backup and Restore in the PostgreSQL server documentation.
Use the instructions that follow to reinstall your pSeven Enterprise release.

To reinstall the pSeven Enterprise release:

Run the command to install your pSeven Enterprise chart with the backup copy of your configuration file values.yaml:
```
helm install pseven-rl pseven-{YYYY.MM.DD}.tgz -f values.yaml -n pseven-ns --set fixStorages=true --timeout 180m --wait --debug | tee pseven-{YYYY.MM.DD}.log
```
Since you have restored data from backups, you must enable the file storage check by adding the --set fixStorages=true option in helm install. This option is required to restore user access in the file storages.

Installation timeout

Note the --timeout option in the examples of installation commands. That option is required because during installation Helm downloads several GB of Docker images from the Registry, so the default Helm's operation timeout (5 minutes) is too short to complete the required downloads. If you enable the storage check, you should set the timeout even longer, because the check requires additional time. You may also want to additionally increase the timeout values depending on your network and deployment configuration.
Wait for the installation completion message to appear. Verify that the file storage check has run - the completion message should contain the following note:

Fixing storages enabled: deployment was run with the "--set fixStorages=true" option, fixing user permissions in data storages.

Information security incidents and operations¶

For pSeven Enterprise, information security incidents are events which may indicate that user data or documents (workflows, workflow run results) have been compromised, or that the deployment configuration has been tampered with. Some examples of such events:

Unauthorized access to the NFS server hosting the pSeven Enterprise data storage.
Unauthorized access to the pSeven Enterprise database on the PostgreSQL server.
Unauthorized intervention in the normal execution of the ongoing workflow runs.
Unauthorized changes in workflows.

Typical incidents¶

Information security incidents in pSeven Enterprise are usually the result of accidental or intentional violation of protective measures in relation to such system components as:

The pSeven Enterprise Studio and AppsHub tools for composing, editing, and publishing workflows - an incident may result in the disclosure of sensitive data or in workflow malfunction.
The NFS server that stores pSeven Enterprise user and application data - an incident creates a threat of disclosure of confidential information, including data that may be an intellectual property or a trade secret (for example, training data for digital models).
The PostgreSQL server database used, in particular, to cache workflow run logs - an incident may cause the disclosure of confidential design results and parameters.

Typical information security incidents related to pSeven Enterprise can be categorized as follows:

Unauthorized access

This category of incident involves malicious use of certain features and/or vulnerabilities to gain access to protected system components. Examples of such incidents are vulnerability exploits aimed at obtaining additional access rights beyond those legally granted to a particular user or admin. A sign of unauthorized access attempts can also be incidents that indicate the creation of accounts violating your organization's user management policies.

This type of incident may involve physical access to storage devices that held sensitive data such as password files or encryption keys. Such incidents can also be caused by uncontrolled changes in the system and/or its incorrect functioning, as a result of which the organization's personnel or third-party personnel gain access to security-sensitive information without legal permission. Preventing such incidents requires the careful monitoring of changes in the settings of both the pSeven Enterprise deployment and its IT infrastructure environment.
Privilege escalation

Typically, privilege escalation occurs when the user account is granted more permissions in pSeven Enterprise than the user needs to do their jobs. Thus, a user account with the app_auth | user | Can access Studio and AppsHub permission is authorized to edit, run, and publish pSeven Enterprise workflows, as well as sign in to AppsHub and run apps. An account with so broad authority could be used, for example, to maliciously disclose or change confidential design workflows.

Privilege escalation in its consequences is in many ways similar to unauthorized access. However, this is a separate category of incidents, since in this case the intruder exercises legal access to security-sensitive resources and data. Preventing such incidents requires implementation and tight control of the account creation and configuration policies.
Account compromise

This type of incident involves the malicious use of someone else's account to access protected resources and security-sensitive data in pSeven Enterprise. This also includes using a pSeven Enterprise admin account to create and configure user accounts in violation of the organization's user management policies. Note that admin rights can be granted to any pSeven Enterprise user account, thereby giving it full access to other users' permissions controls.

Compromising a privileged user or admin account on pSeven Enterprise will eventually result in the same consequences as privilege escalation. In fact, an incident falls into this category if, as a result of its investigation, it turns out that the true owner of the compromised account was not able to use their credentials to commit malicious actions. To prevent such incidents, the use of monitoring tools is required to detect suspicious activity of pSeven Enterprise admins and users.

Emergency protective actions are usually taken as an immediate response to an incident once it has been detected and confirmed. In the event of unauthorized access attempts, such an emergency action can be to disable the attacked service or system component, followed by identifying and neutralizing the source of the attack. If account privilege escalation or compromise is detected, the respective account is disabled and its settings are revised.

After immediate response actions, additional steps may be required to restore the normal operation of the affected pSeven Enterprise services and its IT environment components, such as restoring data from backups, installing updates (patches) to address known vulnerabilities, or disabling compromised items. As a last resort, you may need to redeploy pSeven Enterprise and restore its database from a backup on the PostgreSQL server. User files and application data are restored from a backup on the NFS server. For details, see Backing up and restoring user data.

Another incident response action might be to reinforce the monitoring of pSeven Enterprise user and admin activity. This action should follow the procedures adopted by the organization's information security incident management system. It may be appropriate to introduce additional monitoring steps to assist in the detection of unusual or suspicious events that may indicate an information security incident. Monitoring leverages user and admin activity logs provided by the pSeven Enterprise authentication service (see Auditing user and admin activity). Activity logs help investigate the incident in depth, and are instrumental in identifying the system elements that have been compromised.

Be aware that incident investigations often reveal deficiencies in the configuration and maintenance of the current pSeven Enterprise deployment and its IT infrastructure environment. Planning and implementing remedial actions to address such deficiencies is a priority aspect of incident response as it helps to prevent the incident from reoccurring. An important aspect of responding to an incident is also the implementation of additional protective measures, such as improved policies for managing user and admin accounts and their rights to access protected data, settings, and system components.

Operations¶

Operations related to handling security incidents can be categorized as follows:

Detect

It is necessary to detect the presence of intruders in the system, who in most cases are interested in remaining hidden, which allows them to freely achieve their goals. This may take the form of responding to suspicious activity alerts or proactively searching for anomalous events in pSeven Enterprise user and admin activity logs.
Respond

When potentially malicious activity is detected, an investigation should be conducted to determine whether it is an attack or a false alarm, and then to determine the scope and purpose of the attacker's actions.
Recover

The ultimate goal is to preserve or restore the confidentiality, integrity, and availability of information assets and services during and after an attack.

The most significant security threat comes from attacks performed by humans in real time. The threat from automated/repetitive attacks can be significantly reduced by using various anti-malware programs. While human attacks can be more difficult to counter due to their greater agility compared to automated attacks, the agility of the security operator is usually the same as or greater than that of a human attacker, which in this case contributes to the successful detection and response to security incidents.

The main security control plane in pSeven Enterprise is the management of user and admin accounts, with a pivotal role belonging to the Keycloak server that provides authentication of users and admins, as well as the auditing of their activity. The Keycloak server, in conjunction with a separate authorization service, creates and maintains accounts that control access to all resources, including user access to their data and workflows, and access to the administrator control panels. pSeven Enterprise provides the following account flavors:

Local accounts

Local accounts are created on the authentication service built into the pSeven Enterprise deployment (see Adding accounts). The user access level assigned to the account is controlled by the authorization service (see User permissions) and by an access check on extension nodes (see Configuring user authorization on extension nodes). With the authorization service, the account can also be granted administrator rights (see Admin account configuration). The built-in Keycloak server tracks the activity of local accounts, and logs the respective events (see Auditing user and admin activity).
Federated accounts

Local accounts in pSeven Enterprise can be associated with user accounts from Active Directory Domain Services (see Users from Active Directory or LDAP server). These so-called federated accounts enable users to sign in to pSeven Enterprise with the same username and password that they use to sign in to their work computers, while receiving the level of access granted to the corresponding local accounts in pSeven Enterprise. In this case, the user is authenticated on the Active Directory domain controller, whereas their access rights and activity are controlled and tracked by pSeven Enterprise services in relation to the user's local account.
Single sign-on (SSO)

With single sign-on, local accounts in pSeven Enterprise can obtain identities from an external Keycloak server (see Setting up single sign-on (SSO)). Single sign-on enables the user to sign in to pSeven Enterprise without entering a username and password, provided they are already signed in to some other application on the corporate network. In this case, the user is authenticated on the corporate Keycloak server, whereas their access rights and activity are controlled and tracked by pSeven Enterprise services in relation to the user's local account in the same way as in the case of federated accounts.

When establishing information security incident related operations in a pSeven Enterprise environment, it is advisable to:

Regularly audit the use of accounts and passwords, as well as authentication methods, to counter the most common security attacks.
Implement a strategy to continuously monitor and alert on activities that might indicate a security threat.

Below are guidelines for such operations in the following areas:

User accounts - account creation and modification.
Admin accounts - permissions to perform administrative tasks.
Configuration data - access to the pSeven Enterprise database on the PostgreSQL server.

User accounts

Keeping track of anomalous events related to user accounts is one of the most important aspects of protecting your organization and data while running pSeven Enterprise. To discover anomalous behavior, you first need to determine what is considered normal and expected. Once you have determined the expected behavior, you can audit user and admin activity to check for events that fall outside the tolerances you define. The suggestions below can help you determine what is normal when creating and managing pSeven Enterprise user accounts.

For creating user accounts, consider the following:

Strategy and principles for using account creation tools. For example, if there are any standard user account attributes and prescribed attribute value formats.

Particular attention should be paid to accounts that are created for a short period of time and then quickly deleted, accounts that do not follow naming policy rules, and accounts created outside of standardized management processes.
Approved sources for account creation. For example, only federated accounts that originate from Active Directory Domain Services might be allowed.
Monitoring and alert strategy for user accounts created outside of approved sources. For example, if there is a list of approved admins eligible to create local user accounts by hand.
Monitoring and alert strategy for user accounts created by an admin account that is not intended for that task. For example, if there is a list of approved admins eligible to create user accounts.
Monitoring and alert strategy for user accounts missing standard attributes, such as employee ID or not following corporate naming conventions.

For managing existing user accounts, consider the following:

Monitoring and alert strategy for user accounts modified, disabled or deleted by an admin account that is not intended for account management. For example, if there is a list of approved admins eligible to manage existing user accounts.
Strategy, principles, and process for account deletion and retention. For example, whether disabled accounts should be retained and whether account deletion is allowed.

In the case of federated user accounts, the following must also be considered:

The domains and organizational units (OUs) that hold user accounts for federation, that is, the synchronization scope of user accounts in Active Directory Domain Services. For example, if there is a list of approved admins eligible to change the synchronization scope, and how often the synchronization scope is checked.
The process for creating federated accounts and controlling their synchronization. For example, whether accounts are synchronized automatically on a scheduled basis or only by hand.

The source of event records you use for investigation and monitoring is the pSeven Enterprise user and admin activity log (see Auditing user and admin activity).

Admin accounts

Admin account is a local account that has the pSeven Enterprise administrator flag set (see Admin account configuration). Such accounts normally do not have access to user data and workflows, and are used only for administrative tasks. However, admin accounts have full access to the user account management interfaces and service settings that control account creation. Therefore, the safety and integrity of sensitive data and pSeven Enterprise configuration settings largely depend upon how well admin accounts are protected.

Privileged rights are permanently assigned to admin accounts and can be exercised at any time. Because of this, if an admin account is compromised, it can have a strongly negative effect. It is highly recommended that you monitor events related to the use and any changes to admin accounts as a matter of priority. It is also a good idea to check the admin accounts and make sure they are required.

When investigating and monitoring security events associated with administrator accounts, the following points deserve special attention:

Built-in admin account

The pSeven Enterprise deployment provides a built-in account for performing administrative tasks when there are no other admin accounts (see Admin access). This account is intended only for conditions where other admin accounts cannot be used. Since the built-in admin account is not used under normal conditions, monitoring should focus on alerting each time it is used or changed.
Admin account creation

Monitor the emergence of new admin accounts. As with user accounts, particular attention should be paid to accounts that do not follow naming policy rules and accounts created outside of standardized management processes. Note that any existing local account can be made an admin account (see Admin account configuration). Changes to local accounts that result in assigning them administrator rights should have a high priority in your security monitoring, alerting, and response processes.
Admin account sign-in

Monitor all admin account sign-in activity by analyzing event entries in the respective log. Specifically, monitoring should focus on sign-in failure and bad password threshold, changes to admin passwords, and admin accounts that do not follow naming policy rules. It is also important to establish the appropriate normal usage pattern for each admin account and to alert on all unplanned admin sign-ins.
Changes made by admin accounts

Monitor all completed and attempted changes by any admin account. This data enables you to establish what is normal activity for each admin account and helps to discover activity that deviates from the expected. Any events indicating unplanned changes made by admin accounts should be alerted on immediately. Logs of such events should be retained to facilitate any subsequent investigations.
Changes made to admin accounts

Investigate changes to admin accounts, especially in cases where the change provides additional permissions. Normally, admin accounts should not have permissions allowing them to access pSeven Enterprise user interfaces. Granting such access to an admin account may indicate an attempt to compromise sensitive user data and workflows.

As with user accounts, the source of event records to monitor is the pSeven Enterprise user and admin activity log (see Auditing user and admin activity).

Configuration data

pSeven Enterprise stores its configuration data in the database on the PostgreSQL server. Since configuration data includes security-sensitive deployment settings, security operations should involve the auditing of activities in that database.

Audit logging of database activities in PostgreSQL can be done using PostgreSQL Auditing Extension (pgAudit), which provides detailed session and/or object audit logging. Session audit logging emits detailed logs of executed statements. Object audit logging represents audit scoped to specific relations. You can choose to set up one or both types of logging.

Configure pgAudit parameters to start logging. See the pgAudit documentation for the definition of each parameter. Test the parameters first and confirm that you are getting the expected behavior.

To quickly get started, set the pgaudit.log parameter to WRITE, and review your PostgreSQL server log. Each audit entry is indicated by the AUDIT: prefix near the beginning of the log line. For a description of the format of audit entries, refer to the pgAudit documentation.

Notes¶

This section discusses some important considerations to be aware of when administering pSeven Enterprise.

Understanding run logs storage¶

The pSeven Enterprise logging system employs two data stores - the user file storage on the NFS server and the PostgreSQL database. The NFS server is used as the persistent log storage, while the PostgreSQL database provides a short-term log cache for faster access to logs. The key points to understand how pSeven Enterprise processes run logs are:

Workflow runs only write their logs to files in the user data storage.
The Run log pane in pSeven Enterprise Studio only reads cached logs from the PostgreSQL database.
pSeven Enterprise caches a run log from the log file to the database and updates the cached log on demand only, when users view that log in the Run log pane.
pSeven Enterprise sets a size cap for a cached log in the database. Neither users nor administrators can change that cap. Note though that users can change or even remove the log file size limit in their workflow runs.
The Run log pane cannot ever display more logs than allowed by the cached log size cap.
Unused cached logs are deleted from the database by timeout. Log files saved to the user data storage are never deleted automatically.

While a workflow run executes, it writes the run log to a file located within the run directory, in a hidden subdirectory intended for pSeven Enterprise system files. Log writes do not involve the log cache in any way so as to avoid putting the related additional load on the PostgreSQL database server. Thus writing a run log increases the total size of that run directory, using the file storage space and spending the storage quota of the user who has started that run, but does not use any storage space on the database server.

Database access occurs while a user is viewing a log in the Run log pane in pSeven Enterprise Studio. For each run log that is displayed to a user, pSeven Enterprise creates a database cache holding messages uploaded from that log file to the database (the cached log). When displaying a run log to a user, pSeven Enterprise checks whether the cached log is up to date with the log file. If the file contains log messages that are not cached yet, those messages are added to the cached log. In this event, no more than 100000 latest messages are read from the log file. The messages read from the log file are added to the cache in batches, and the maximum size of a batch is 10000 messages. The size of a cached log, in turn, has a threshold of 110000 messages. The cached log size may briefly exceed that threshold, but as soon as pSeven Enterprise detects that, it truncates the cached log, keeping only the latest 100000 messages and removing the rest from the cache. For example:

If the cached log contains 85000 messages, and 28000 new messages are found in the log file, then 28000 messages are read from the file and added to the cache in 3 batches. With every next batch, the cached log grows to 95000 messages, then 105000 messages, then 113000 messages. Once the third batch has been added, the cached log size exceeds the 110000 messages threshold, so 13000 oldest messages are removed from the cache to bring the size to 100000 messages.
If the cached log contains 30000 messages, and 150000 new messages are found in the log file, then 100000 messages (the maximum allowed number) are read from the file and added to the cache in 10 batches. At a certain point, the cached log size reaches the 110000 messages threshold but does not exceed it yet, so the cached log is not yet truncated. Adding the next message batch brings the cached log size to 120000 messages, and at that time the cached log is truncated to 100000 messages. This is the worst case scenario considering the cached log size, so 120000 messages is effectively the size cap.
If there is no cached log (no user has viewed the log yet), and the log file contains 175000 messages, then 100000 messages (in 10 batches) are read from the file and cached, once a user opens the Run log pane in the workflow run, which has produced that log. The remaining 75000 oldest messages will not be displayed to the user in the Run log pane; to read them, the user will have to download the log file.

The Run log pane in pSeven Enterprise Studio always reads the log from the database cache and never accesses the log files directly. Moreover, for each log file, there is only a single cached log in the database, regardless of the number of users who open that same log. That is, if many users view the log of the same (shared) workflow run at the same time, that log is cached only once, and all users in fact view the same single cached log.

The cached log size threshold explained above effectively limits the length of the log that the Run log pane will display to 100000-110000 recent log messages. The older messages do not exist in the cached log, from which the Run log pane gets data, or exist only for a short period during the cache update, after which those messages are deleted from the cache. Users who want to view a larger log have the option to download the log file from the Run log pane menu. By default, the log file size is not limited, so the log files are not truncated - in this case, the file size is only capped by the user's storage quota. However users can set a log file size limit upon launching a workflow run - in which case pSeven Enterprise tracks the log file size and truncates the file to roughly half (removes the beginning) whenever the full file size exceeds the user-defined limit. Note that setting a log file size limit (for example, 100 MB) does not actually secure 100 MB of the run log - it ensures that the log file size cannot exceed 100 MB although secures only 50 MB, from the user's point of view; the final log size after the workflow run completes may appear anywhere between 50 MB and 100 MB.

The log file size limit set by user does not change the cached log size threshold. Nevertheless the file size limit can be useful to reduce the database load: if that limit is low enough to make the number of messages saved in the log file less than 100000, then it indirectly applies a cached log size limit, which is lower than the threshold set by pSeven Enterprise.

Reading the log messages from a file, which takes place during the cached log updates, is considerably slower than reading from the database cache, which provides the log messages displayed to users in the Run log pane. In common usage scenarios this is not perceived by users: for example, if a user opens the Run log pane and starts monitoring the log, pSeven Enterprise continuously updates the cached log while the pane is open, so the amount of data read from the file on each update is not much, and there is only a negligible delay before a new message, which appears in the log file, is displayed in the user interface. In worst case scenarios there is going to be a noticeable delay when a user opens the Run log pane and before the pane displays the log messages. For example, if a user opens the log of a workflow run that has never been viewed by anyone yet (so there is no cached log of that run), all messages in the log file are considered new, so up to 100000 messages will be read and loaded to the cache at once. The Run log pane will display the loading animation (no messages) until pSeven Enterprise finishes loading all the new messages to the cache. From the user's point of view, it means that there will be a delay in displaying the log, which quite possibly reaches a few tens of seconds, and might be a few minutes in the case of a lengthy log containing large messages. However once a run log is finally cached, further it can be accessed fast by any user who opens that same log; the loading delay occurs only the first time when opening a log that has not been viewed recently (in a day), or has never been opened before.

The lifetime of a cached log in the database is no less than a day, and gets prolonged on user access. pSeven Enterprise performs the database cleanup several times a day. During the cleanup, cached logs that have not been accessed by any user in the past 24 hours are removed from the database entirely. Cached logs created or accessed no more than 24 hours ago remain in the database after the cleanup.

Therefore maintaining the logging system requires a certain volume of database storage, which generally depends on user activity related to viewing run logs and the possible size of a log message sent by a block in a user workflow. That dependency makes the requirement hard to estimate beforehand, even knowing the number of active users and the average number of daily workflow runs. Instead, it is advisable to do as follows:

Set the initial database storage size with a safety margin. It is recommended to begin with at least 100 GB storage.
Decide on a certain threshold (for example, 60% usage) and monitor the database storage usage over time. If you find out that the actual usage is near the threshold, increase the storage volume.
If you find out that the storage usage stays consistently in a certain range (for example, over a few weeks), consider the peak usage from that range as an empirical estimate. Set the storage size based on that estimate with an additional safety margin - for example, 20% higher than the observed peak usage. Continue monitoring the storage usage and increase the storage size as required to keep the actual usage below 80% of the available storage.

Administration¶

User management¶

Understanding accounts¶

Adding accounts¶

Managing user account data¶

User permissions¶

End user account configuration¶

Admin account configuration¶

Applying user storage quotas¶

Disabling an account¶

Changing the username¶

Workspaces¶

Configuration basics¶

Resource quotas¶

Workspace permissions¶

Public workspace¶

Users from Active Directory or LDAP server¶

Provider setup¶

Using secure LDAP¶

Attribute mappers¶

Starting up federation¶

Other LDAP servers¶

Setting up single sign-on (SSO)¶

Registering pSeven Enterprise as an OpenID client¶

Provider setup¶

Sign-in check¶

User session expiration settings¶

Auditing user and admin activity¶

Configuring user authorization on extension nodes¶

Engineering software on extension nodes¶

Maintaining block runtime environment¶

Installing additional Python modules¶

Installing modules into the pSeven Enterprise deployment¶

Installing modules on an extension node¶

Backing up and restoring user data¶

Information security incidents and operations¶

Typical incidents¶

Operations¶

Notes¶

Understanding run logs storage¶