Uncertainty quantification¶

This guide provides an overview and brief instructions on using Uncertainty quantification blocks to estimate design results under uncertainty in design parameters.

Uncertainty quantification is a beta block

The Uncertainty quantification block is in beta. Future pSeven Enterprise updates will keep compatibility with the current version of the block, but this beta version may still have some features missing or not yet working as intended.

Getting started¶

The Uncertainty quantification block is intended for exploring the response of a numerical model to random deviations of its input parameters. For example, if a model represents a certain design, the Uncertainty quantification block enables you to assess the impact of uncertainty of design parameters, such as material properties or operating conditions, on its technical and operational characteristics. The exploration is based on feeding the model with a certain number of design points, represented by pseudo-random variable values, with subsequent analysis of the received response values. Upon completion of its execution, the block issues analysis results, a HTML report, and collections of all the explored variable and response values.

flowchart TB
  uq(Uncertainty quantification)
  model(Model)
  results([Results])
  uq -->|Variables| model
  model -->|Responses| uq
  uq --> results

The block operation depends upon exploration mode:

Probabilistic modeling

This mode is used to find out the probability distribution of the deviation of the model's output parameters, considering the uncertainties of the input parameters.

The block generates pseudo-random input data for the model being explored, and analyzes its responses, estimating their statistical characteristics. When the block finishes, it writes a report with the exploration results, showing which probability distributions best match the responses received from the model.
Reliability analysis

This mode enables you to find out to what extent the model complies with the constraints on its outputs under uncertainty of input parameters.

The block generates variable values and feeds them into the model being explored, in the same way as in Probabilistic modeling mode, and then based on response data, it calculates a reliability estimate, which is the probability that the response values are within the specified range. The block allows you to set the lower and upper bounds of the value range for each response on a per-component basis. When the block finishes, it outputs the reliability estimate value and writes a report with full results.

To feed model inputs, the block employs variables defined in its configuration. Variable values are generated by sampling random variables with configurable probability distributions. You can select the desired sampling technique in the block configuration for each exploration mode: Monte Carlo (Monte Carlo sampling) or Latin Hypercube (Latin hypercube based sampling). It is also possible to specify correlation between components of the variables.

Variables and responses in the Uncertainty quantification block settings describe the model inputs and outputs, whereas the outputs are calculated by a separate block, the so-called "blackbox". When receiving variable values, the blackbox returns the model outputs as response values. The Uncertainty quantification block accumulates and eventually outputs the entire collection of data it exchanged with the blackbox, including model inputs and outputs, that is, all the explored variable and response values.

flowchart TB
  request[\ /]
  blackbox(Blackbox)
  response[\ /]
  request -->|Variable values| blackbox
  blackbox -->|Response values| response

When executed as part of a workflow, the Uncertainty quantification block outputs the variable values to the connected blackbox and receives the response values. The blackbox can be a Composite block containing some computational chain, or a set of blocks, each of which calculates certain model outputs.

To incorporate the Uncertainty quantification block in the workflow:

In the block configuration dialog, add variables and responses, configure them according to your uncertainty exploration task.

For each variable, a request port is automatically created, through which the block issues the values of this variable. For each response, a response port is automatically created on which the block waits to receive the values of that response. The request and response ports jointly provide a blackbox communication loop:
```
flowchart TB
  uq(Uncertainty quantification)
  blackbox(Blackbox)
  uq -->|From request ports| blackbox
  blackbox -->|To response ports| uq
```
Create links from the request ports to the corresponding blackbox inputs.
Create links from the blackbox outputs to the corresponding response ports.
In the block configuration dialog, use the "Mode" setting to select desired exploration mode.

Note that the block has the "Smart selection" toggle turned on by default, which automatically sets exploration mode depending on the properties of the variables and responses. To change exploration mode by hand, you must first turn off the "Smart selection" toggle.

You can also set the exploration mode via the Exploration mode input port.
Optionally, in the "Budget" field of the block configuration dialog, enter the exploration budget - the limit on the number of calls to the blackbox for response evaluations. By default, the appropriate limit is selected automatically based on other block settings.

This setting determines the number of design points generated during block execution. By default, the number of points is equal to the number of blackbox calls. When batch mode is enabled (see Batch mode), the number of points is equal to the number of blackbox calls multiplied by the Batch size value (see Advanced settings).

You can also change this setting via the Exploration budget input port.

Configuration basics¶

The Uncertainty quantification block operation is based on variables and responses. Variable values are passed to the inputs of the blackbox that represents the model being explored, and then responses receive their values from the blackbox outputs. To conduct an exploration using this block, you must configure at least one variable and at least one response.

Variables¶

The block configuration dialog provides the Variables pane for managing variables. In this pane, you can add or remove variables, as well as view or change their properties.

When adding a variable, a request port is automatically created to pass the values of that variable to the blackbox for response evaluation. The name of the request port is the same as the name of the variable. In addition, special input ports are created for each variable, through which you can change the properties of the variable. The name of such a port is composed of the variable's name followed by the property name.

Each variable corresponds to an ordered list row in the Variables pane. The order of the variables in the list determines the order in which their components appear in the result tables the block outputs. Variable properties are represented by list row fields to view or edit, including:

Name

The name identifies the variable, and is part of the names of its ports. The name is assigned to the variable when it is created, you can change the name in the Variables pane. When renaming a variable, the name of each of its ports changes accordingly.

Each variable must have a unique name among all variables and responses within the block configuration, and it must contain only allowed characters. The following characters are not allowed in variable names: . < > : " / \ | ? *.
Dimension

The dimension of the variable is the number of its components, each of which is some numeric value. The dimension of a scalar variable is equal to one, a vector variable has a dimension of 2 or higher. A vector variable can be used, for example, instead of multiple scalar variables to reduce the number of ports passing data to the blackbox.

A vector variable can be thought of as a composition of several scalar variables grouped under a single name. The components of such a multidimensional variable are scalar random variables with the same probability distribution.

The dimension of the variable affects:
- The type of variable values on the its request port - a numeric scalar if the dimension is 1, or a numeric vector if the dimension is greater than one. In the latter case, the number of vector components on the request port is equal to the dimension of the variable. If batch mode is enabled (see Batch mode), then, depending upon the dimension of the variable, its request port issues a one-dimensional array of scalar or vector values, with the number of array elements specified by the Batch size setting.
- The number of columns representing the values of the variable components in the result tables - the length of the table row section containing the respective component values is equal to the dimension of the variable.
You can view or change the dimension of the variable in the "Dimension" field in the Variables pane. This property can also be set through the variable's Dimension input port, which enables you, for example, to change the dimension of the variable using a workflow parameter.

Distribution

This property specifies the type and parameters of the probability distribution of the random variable used to generate the variable values. In the case of a vector variable, all its components have the same probability distribution type, as well as, by default, the same values of the distribution parameters. For each parameter, you can set an individual parameter value for each component of a vector variable using a vector of parameter values; the number of components of such a vector must be equal to the dimension of the variable, each of its components sets the parameter value for the respective component of the variable. All components of the variables are statistically independent by default, it is possible to set up correlation on a per-component basis.

You can view or change the probability distribution setting for the variable in the "Distribution" field in the Variables pane. This setting can also be modified through the variable's Distribution input port, which enables you, for example, to change the probability distribution of the variable using a workflow parameter.

All distribution types and parameters available in the block configuration are listed in the table that follows.

Distribution	Parameters
Arcsine	a - lower bound $a$ b - upper bound $b>a$
Beta	alpha - shape parameter $\alpha>0$ beta - shape parameter $\beta>0$ a - lower bound $a$ b - upper bound $b>a$
Burr	c - shape parameter $c>0$ k - shape parameter $k>0$
Chi	nu - degrees of freedom $\nu>0$
Chi-square	nu - degrees of freedom $\nu>0$
Exponential	lambda - rate parameter $\lambda>0$ gamma - location parameter $\gamma$
Frechet	alpha - shape parameter $\alpha>0$ beta - scale parameter $\beta>0$ gamma - location parameter $\gamma$
Gamma	k - shape parameter $k>0$ lambda - rate parameter $\lambda>0$ gamma - location parameter $\gamma$
Generalized Pareto	sigma - scale parameter $\sigma>0$ xi - shape parameter $\xi$ mu - location parameter $\mu$
Gumbel	beta - scale parameter $\beta>0$ gamma - location parameter $\gamma$
Inverse normal	mu - location parameter $\mu>0$ lambda - shape parameter $\lambda>0$
Laplace	mu - location parameter $\mu$ lambda - rate parameter $\lambda>0$
Logistic	mu - location parameter $\mu$ beta - scale parameter $\beta>0$
LogNormal	muLog - location parameter $\mu_\ell$ sigmaLog - scale parameter $\sigma_\ell>0$ gamma - location parameter $\gamma$
LogUniform	aLog - lower bound $a_\ell$ bLog - upper bound $b_\ell>a_\ell$
Normal	mu - location parameter (mean) $\mu$ sigma - scale parameter (standard deviation) $\sigma>0$
Rayleigh	beta - scale parameter $\beta>0$ gamma - location parameter $\gamma$
Rice	beta - scale parameter $\beta>0$ nu - location parameter $\nu\geq0$
Student	nu - generalized number degrees of freedom $\nu>0$
Triangular	a - lower bound $a$ m - mode $a \leq m \leq b$ b - upper bound $b>a$
Uniform	a - lower bound $a$ b - upper bound $b>a$

Correlation setup¶

The block provides for setting statistical correlation between individual components of the variables. You can:

Enable or disable correlation.

When correlation is disabled, all components of the variables are statistically independent. To apply correlation settings, it must be enabled; when correlation is disabled, all existing settings are preserved, but have no effect. To enable correlation, select the "Correlation" check box in the block configuration dialog, or set a value of True on the Advanced.Correlated variables input port. To disable correlation, clear that check box or set a value of False on that port.
Specify the pairs of correlated components and correlation coefficients.

When setting up a pair of correlated components, you specify a pair of variables, the index of the desired component, and the value of the correlation coefficient. The correlation setting is a table, each row of which has the following fields:
- First variable name, Component index (first) - Variable name and index of the first correlated component.
- Second variable name, Component index (second) - Variable name and index of the second correlated component.
- Coefficient - The value of the correlation coefficient for the given pair of components.
The component index is zero-based, that is, the index of the first component is 0, the index of the second is 1, and so on. The type of correlation coefficient is the same for all pairs, it can be Pearson, Kendall, or Spearman coefficient.

To manage the table of correlated components, a correlation settings dialog is provided that opens when you click the button next to the "Correlation" field in the block configuration dialog. You can also manage the table of correlated components through the Advanced.Correlated pairs input port. The desired type of correlation coefficient is specified in the correlation settings dialog or via the Advanced.Correlation type input port.

Responses¶

The block configuration dialog provides the Responses pane for managing responses. In this pane, you can add or remove responses, as well as view or change their properties.

When adding a response, a response port is automatically created to receive the values of that response from the blackbox used for response evaluation. The name of the response port is the same as the name of the response. In addition, special input ports are created for each response, through which you can change the properties of the response. The name of such a port is composed of the response's name followed by the property name.

Each response corresponds to an ordered list row in the Responses pane. The order of the responses in the list determines the order in which their components appear in the result tables the block outputs. Response properties are represented by list row fields to view or edit, including:

Name

The name identifies the response, and is part of the names of its ports. The name is assigned to the response when it is created, you can change the name in the Responses pane. When renaming a response, the name of each of its ports changes accordingly.

Each response must have a unique name among all variables and responses within the block configuration, and it must contain only allowed characters. The following characters are not allowed in response names: . < > : " / \ | ? *.
Dimension

The dimension of the response is the number of its components, each of which is some numeric value. The dimension of a scalar response is equal to one, a vector response has a dimension of 2 or higher. A vector response can be used, for example, instead of multiple scalar responses to reduce the number of ports receiving data from the blackbox.

A vector response can be thought of as composition of several scalar responses grouped under a single name. For each component of such a vector response, you can set an individual upper bound and lower bound.

The dimension of the response affects:
- The expected type of response values on the respective port - a numeric scalar if the dimension is 1, or a numeric vector if the dimension is greater than one. In the latter case, the number of vector components on the response port must be equal to the dimension of the response. If batch mode is enabled (see Batch mode), then, depending upon the dimension of the response, its port expects to receive a batch of scalar or vector values, with the number of values in the batch matching the Batch size setting.
- The number of columns representing the values of the response components in the result tables - the length of the table row section containing the respective component values is equal to the dimension of the response.
You can view or change the dimension of the response in the "Dimension" field in the Responses pane. This property can also be set through the response's Dimension input port, which enables you, for example, to change the dimension of the response using a workflow parameter.
Lower bound, Upper bound

The bounds of the response determine the range of its acceptable values in reliability analysis. The main task of such an analysis is to estimate the probability that the response value falls within a given range. For this reason, when in Reliability analysis mode, at least one of the responses must have some bound specified. Responses that have no values specified for both bounds are excluded from the reliability analysis. When in Probabilistic modeling mode, response bounds are not used and can be omitted.

The lower and upper bound settings define a range of values as follows:
- If only the lower bound or only the upper bound is specified, the range contains all values above or below that bound respectively.
- If both the lower bound and upper bound are specified, the range contains all values between those bounds.
The response bound setting can be a number of the Real type or a value of the Vector type:
- Use a number to set the same bound for all components of the response value.
  
  This setting is normally used for scalar (single-component) responses, but it also works for vector responses (multiple components, each with the same bound).
- Use a vector to set an individual bound for each component of the response value.
  
  The bounds vector in this case must have the same number of components as the dimension of the response, with each component of the bounds vector representing the bound for the respective component of the response. To specify no bound for a particular component of the response, leave empty the corresponding component of the bounds vector.
You can view or change the bounds of the response in the "Lower bound" and "Upper bound" fields in the Responses pane. The bounds can also be set through the response's Lower bound and Upper bound input ports, which enables you, for example, to change the bounds of the response by using workflow parameters.

Input ports¶

The block's input ports serve to manage general exploration parameters and advanced settings; you can also use input ports to administer the properties of variables and responses:

The Exploration budget and Exploration mode ports enable you to change the exploration parameters represented by the "Budget" and "Mode" fields in the block configuration dialog. For more information on these parameters, see Getting started.
The <variable-name>.Dimension and <variable-name>.Distribution ports enable you to change the variable properties described in Variables.
The <response-name>.Dimension, <response-name>.Lower bound, and <response-name>.Upper bound ports enable you to change the response properties described in Responses.
The Advanced.Correlated pairs, Advanced.Correlated variables, and Advanced.Correlation type ports enable you to set up the correlation between variable components, as described in Correlation setup.
The Probabilistic modeling.Sampling technique and Reliability analysis.Sampling technique ports, as well as other input ports of the Advanced category enable you to manage the advanced settings.

Driver ports¶

A request port is automatically created for each variable, and a response port is automatically created for each response. These ports are identified by their variable and response names. When running within a workflow, the block outputs variable values to the request ports and waits for response values on the response ports. See Getting started for further details.

The request port output values match the variable's dimension: numeric scalars or vectors, each of which is the value of the variable in a certain design point. The number of vector components on the request port is equal to the dimension of the variable. If batch mode is enabled (see Batch mode), then, depending upon the dimension of the variable, its request port issues a one-dimensional array of scalar or vector values, with the number of array elements specified by the Batch size setting.

The type of input values expected on the response port depends upon the response's dimension - these must be numeric scalars or vectors, each of which represents the value of the response at the corresponding design point. The number of vector components sent to the response port must be equal to the dimension of the response. If batch mode is enabled (see Batch mode), then, depending upon the dimension of the response, its port expects to receive a batch of scalar or vector values, with the number of values in the batch matching the Batch size setting.

Results¶

The Uncertainty quantification block outputs its main results to ports. In its working directory, it also creates a subfolder to store the result files - the HTML report and tables with data it collected. That results folder has the same name as the block; if such a folder already exists, the folder name is prepended with the name of the parent Composite block.

Result ports¶

Results output to ports are the reliability estimate and related confidence metric (in reliability analysis mode), and collected designs (in all modes).

Reliability estimate - the main reliability analysis result, the estimated probability that model responses satisfy constraints.
Reliability RSD - confidence metric, relative standard deviation of the estimate (also known as the coefficient of variation). The closer this value is to 0, the more accurate the estimate. Commonly, values <0.05 are considered acceptable, and values >0.05 indicate indicate that the exploration budget was too low to get a good estimate of reliability.
Designs - outputs a table containing all values of the variables and responses collected during the exploration. Within a table row, first go the values of the variables, followed by the corresponding response values.
Response designs - outputs a table containing all collected values of the responses.
Variable designs - outputs a table containing all values of the variables.

Each row in the output table corresponds to one of the explored design points. Within a row, the variable and response values go in the same order as the variables and responses are listed in the block configuration dialog. For each variable or response, its component values occupy as many adjacent columns as the dimension of that variable or response.

HTML report¶

The main result file is the HTML report saved as Report.html to the results folder.

A Probabilistic modeling report contains the following results, for each component sample of the response values received from the blackbox:

Statistical parameters of the sample, such as the min and max values, mean, median, variance, etc.
Indicators of goodness-of-fit criteria for this sample with different parametric probability distributions. For each presented probability distribution, its parameters are displayed, estimated by analyzing the sample, as well as the indicators of the Lilliefors fitting test and Bayesian information criterion (BIC). The distributions with the best fit are marked in green.
Plots of the smoothed sample probability distribution function versus the best fit parametric probability distribution function.

A Reliability analysis report contains the following results:

The reliability estimate, its RSD, and the approximate bounds of its confidence intervals for a certain set of confidence levels.
Plots showing the convergence of the bounds of confidence intervals to the reliability estimate as the number of response evaluations increases. The plots represent confidence intervals for the same set of confidence levels as above.

Data tables¶

In addition to the HTML report, the block also saves the variable and response values collected during the exploration to table files in the results folder:

Designs.csv, Designs.p7table - all values, variables and responses together.
Variable designs.csv, Variable designs.p7table - values of the variables only.
Response designs.csv, Response designs.p7table - response values only.

Each row in the output table corresponds to one of the explored design points. Within a row, the variable and response values go in the same order as the variables and responses are listed in the block configuration dialog. For each variable or response, its component values occupy as many adjacent columns as the dimension of that variable or response.

Advanced settings¶

You can view or change advanced settings in the Settings pane; to open the pane, click in the block configuration dialog. Advanced settings can also be administered through block input ports, each of which has the same name as the respective setting in the Settings pane.

The Uncertainty quantification block has the following advanced settings:

Probabilistic modeling.Sampling technique, Reliability analysis.Sampling technique

The block generates its variable values by sampling random variables with configurable probability distributions. Use the Sampling technique setting to specify the desired kind of sampling for each exploration mode separately: Monte Carlo (Monte Carlo sampling) or Latin Hypercube (Latin hypercube based sampling).
Advanced.Batch mode

Enable or disable batch output of variable values for parallel evaluation of responses. See Batch mode for details.
Advanced.Batch size

Set the batch size of variable values for parallel evaluation of responses. See Batch size for details.
Advanced.Random seed

Specify the seed of the pseudo-random sequence for generating variable values. See Random seed for details.
Advanced.Use fixed random seed

Ensure the reproducibility of results by repeating the pseudo-random variable values in different runs of the block.

Batch mode¶

A flag to enable or disable batch output of variable values for parallel evaluation of responses.

Valid setting: True (enable), False (disable)
Default setting: False

When requesting response values, the Uncertainty quantification block can output a batch of variable values for response evaluation. This batch mode is normally used when the blocks that evaluate responses - blackboxes - can process multiple input values in parallel.

If batch mode is enabled, then for each variable, a batch of multiple values of that variable is output on its request port. The number of values in the batch is determined by the Batch size setting, and the type of values depends upon the dimension of the variable. If batch mode is disabled, the block issues variable values one at a time, regardless of the Batch size setting.

Batch size¶

This setting determines the number of values of each variable passed in a batch for parallel evaluation of responses.

Valid setting: none or an integer greater than 1
Default setting: none

If Batch mode is set to True, the Uncertainty quantification block is allowed to output a batch of variable values in each request to evaluate responses. The Batch size setting determines the number of values held in the batch.

The batch of values is output as one-dimensional arrays to the request ports associated with the variables. Each element of such an array represents a certain value of the respective variable. The array has as many elements as specified by the Batch size setting, and the type of elements depends upon the dimension of the variable.

It is important to note that the exploration budget setting in batch mode refers to the number of batches, not the number of design points. The block generates no more batches than the exploration budget; hence, the maximum number of the explored design points is equal to the exploration budget multiplied by the Batch size setting.

Random seed¶

This setting specifies the seed of the pseudo-random sequence for generating variable values.

Valid setting: 0 or a positive integer
Default setting: 0

This setting has an effect if Use fixed random seed is set to True. In this case, the seed specified by the Random seed setting determines the same pseudo-random sequence for different runs of the block, allowing block execution results to be reproduced. The Random seed setting has no effect if Use fixed random seed is set to False.

Use fixed random seed¶

This setting is used to ensure reproducibility of results by repeating the pseudo-random variable values in different runs of the block.

Valid setting: True (ensure reproducibility), False (no reproducibility)
Default setting: True

The Uncertainty quantification block generates variable values using a pseudo-random sequence of numbers, which may differ in different runs of the block. Such non-deterministic generation of variable values causes different results of repeated runs of the block with the same setup. If you need to ensure reproducible run results, you can use this setting to enable deterministic variable value generation.

The pseudo-random sequence used in the Uncertainty quantification block is determined by the seed. The Use fixed random seed setting enforces the use of the same fixed seed in different runs, thereby ensuring reproducible run results. This setting can be one of the following:

True - The same seed, specified by the Random seed setting, is applied at the beginning of each run of the block. This ensures reproducibility of the block execution results for repeated block runs within the workflow run, as well as for different runs of the workflow.
False - A seed is arbitrarily selected at the beginning of each block run, the Random seed setting has no effect. Different runs of the block with the same setup can produce different results even within the same workflow run.