Tensor Approximations (TA) and incomplete Tensor Approximations (iTA) in pSeven Core

• This DoE is a Cartesian product of two sets - a one-dimensional one and a two-dimensional one. We call these two sets factors.
• The factors themselves have no regular structure - in this example, hey just consist of random points in 1D (the first factor) or in 2D (the second factor).

More generally, for any collection of variables  \(S=\{x_1, x_2, \dots, x_N\}\), we may:

1. Partition \(S\) into some number of disjoint subsets \(S_1\sqcup S_2\sqcup\dots\sqcup S_n\) (not necessarily in the order of the variables \(x_1, x_2, \dots, x_N\));
2. Define a DoE  \(Q_k\) for each subset \(S_k\);
3. Form the full DoE as the Cartesian product \(Q=Q_1\times Q_2\times\dots\times Q_n\).

Why does this construction deserve our attention? Two reasons:

• This partially factored type of DoE occurs naturally in a number of contexts. Imagine, for example, that you perform experiments with some device and measure some physical quantity like temperature or pressure using sensors placed at fixed positions. You perform experiments with different device settings. Then, your DoE is naturally factored: one factor corresponds to the position of sensors and the other to the device settings. Note that the two factors need not have any particular structure: in particular, we do not require the positions of sensors or the device settings to form a regular grid (which would be necessary if we were to use a full-factorial DoE).
• The factored structure of DoE can be quite useful in constructing approximation models for the response functions evaluated on this DoE - it allows us to accelerate approximation modeling and/or make the models more accurate. More on that in the next section.

Mathematically, most conventional methods of constructing approximations (linear regression, splines, kriging, neural networks) boil down to representing them as linear combinations of some basic functions. Tensor approximations can then be roughly described as those in which:

• We associate with each factor an "elementary" approximation method (linear regression, splines, etc.) and choose an appropriate basis;
• We obtain the whole approximation as a linear combination of tensor products of those elementary basis functions.

Accordingly, tensor approximations "inherit" to some extent the properties of the elementary approximation methods chosen for their factors. pSeven Core GT Approx automatically chooses a reasonable elementary method for each factor, but the user can also set them manually if needed.

Why can a tensor approximations be better than a usual approximation?

Factorization of DoE is very often accompanied by DoE's anisotropy that can have many forms:

• DoEs in different factors can be very different in size;
• the factors can have very different dimensions;
• the response function can very differently depend on variables from different factors.

Standard approximation methods are usually not flexible enough to handle such issues and the resulting approximation models turn out to be very inaccurate.

In 1D, splines are a very efficient, fast and accurate method that can handle millions of training points, but splines do not directly extend to multi-dimensional scattered data. On the other hand, linear regression can be efficiently used in any dimension with any data, but it will be very inaccurate for nonlinear functions. An alternative, nonlinear methods such as kriging or neural networks are accurate for nonlinear data, but they are much slower and usually limited to smaller sets due to memory limitations.

Tensor Approximations can efficiently combine the strengths of different methods by exploiting the factored structure of the data set and choosing (or letting the user specify) an appropriate modeling method for each individual factor.

Even if the data is not anisotropic, Tensor Approximations substantially augment the modeling options -- for example, the tensor product of splines, if applicable, is often at the same time faster, more accurate, and has smaller memory footprint than alternative approximations.

The engine model in question could be mathematically represented as a function \(y=f(x_1,x_2,x_3,x_4,x_5,x_6,x_7)\).

• The pairs of variables \((x_1,x_2)\) described a point along the engine boundary: \(x_1\) was position along the engine axis and  \(x_2\) – the radial position. There were a large number (∼700) of such points and they formed a complex curve.

• The tuples \((x_3,x_4,x_5,x_6)\) described external simulation conditions ("flying points"). There were about 150 such flying points.
• The variable \(x_7\) was a simulation parameter (heat transfer coefficient) that could take 5 different values independently of the other variables.
• The response \(y\) was a vector containing several physical quantities of interest simulated at the engine boundary.

The goal of the project was to create an approximation model for \(f\), as simulations tended to be lengthy and expensive. The function \(f\) was highly anisotropic: on the one hand, it had a highly nontrivial dependence on the spatial position:

(here, different curves correspond to different tuples \((x_3,x_4,x_5,x_6,x_7)\)). On the other hand, an examination had shown that\(f\) was close to linear with respect to the parameters \((x_3,x_4,x_5,x_6)\). 

In this problem, we constructed a tensor approximation with 3 factors:

Factor 1:

• Variables: \(S_1=\{x_1, x_2\}\) (the spatial position). This factor is formally 2D, but as the values lie on a 1D curve, the two variables could be replaced with a single variable\(x_1'\) by a simple transformation.  

• Elementary approximation method: splines (pSeven Core SPLT technique), because of 1D and the large amount of points.

Factor 2:

• Variables: \(S_2=\{x_3,x_4,x_5,x_6\}\) (the external flight conditions). 

• Elementary approximation method: linear regression, because of the almost linear dependence of the response on these variables. We could also use any other multi-dimensional method, for example, pSeven Core GP (Gaussian Processes), but the improvement in accuracy was not big, while training time would grow.

Factor 3:

• Variables: \(S_3=\{x_7\}\) (heat transfer coefficient). 

• Elementary approximation method: splines, as this factor is 1D.

For the training set, factor 2 was reduced to just 10 randomly chosen tuples (out of the 150 available in the database). All the remaining data was used as a test set, and the approximation was found to be very accurate, as seen on this scatter plot:

Sometimes, the DoE has an incomplete partially factorial structure. This means a factored DoE with some points missing, like in this figure:

For such problems, pSeven Core GT Approx offers the incomplete Tensor Approximation (iTA) method. Mathematically, it is also based on tensor products of factorial base functions, but the algorithm is more involved since we deal now with not-so-well-structured and less abundant training data. iTA is usually quite fast and, if there are not too many omissions, it tends to produce accurate approximations.

Currently, pSeven Core iTA can only work with 1D factors and tensor products of splines.

This time we get the following result:

The difference is small but clearly present - the approximation is smoother than the original field (which is not necessarily a drawback).

Now let us push the loss of date to the limit and consider the 99% loss rate -- that leaves just 58 values out of the original 5800! Note that there are now big areas on the wing's surface without any pressure values.

Here is the result of iTA:

The approximation has suffered a noticeable loss of accuracy, but still correctly reflects the general trends.

We remark that in all these cases the training time for the approximation was no longer than a few seconds.