Comprehensive Method for Obtaining Multi-Fidelity Surrogate Models for Design Space Approximation: Application to Multi-Dimensional Simulations of Condensation Due to Mixing Streams

Abstract

In engineering problems, design space approximation using accurate computational models may require conducting a simulation for each explored working point, which is often not feasible in computational terms. For problems with numerous parameters and computationally demanding simulations, the possibility of resorting to multi-fidelity surrogates arises as a means to alleviate the effort by employing a reduced number of high-fidelity and expensive simulations and predicting a much cheaper low-fidelity model. A multi-fidelity approach for design space approximation is therefore proposed, requiring two different designs of experiments to assess the best combination of surrogate models and an intermediate meta-modeled variable. The strategy is applied to the prediction of condensation that occurs when two humid air streams are mixed in a three-way junction, which occurs when using low-pressure exhaust gas recirculation to reduce piston engine emissions. In this particular case, most of the assessed combinations of surrogate and intermediate variables provide a good agreement between observed and predicted values, resulting in the lowest normalized mean absolute error (3.4%) by constructing a polynomial response surface using a multi-fidelity additive scaling variable that calculates the difference between the low-fidelity and high-fidelity predictions of the condensation mass flow rate.

1. Introduction

Typical engineering problems are based on meaningful simplifications of the behnts to those avior of complex systems. Ideally, one would like to be able to predict the performance of the system based on a combination of inputs that define its characteristics and particular operating conditions. To do so, a set of (training) data is generated using experimental measurements or numerical simulations, and a so-called surrogate model of the system is built. Such experiments and simulations are usually expensive [1], so the number of evaluations should be kept as low as possible. Surrogate models (also known as black-box models or metamodels), therefore, aim at predicting the outcome of the system with adequate accuracy at different points from those employed to construct the model. In the framework of Industry 4.0 and the paradigm shift to the Digital Twin concept [2], surrogate models are believed to play a vital role [3,4,5].

Surrogate models can be used for many different purposes [1], including sensitivity analyses [6,7,8], optimization [9,10,11] and design space approximation. The latter term is applied to cases in which the surrogate is required to represent the global behavior of a complex system over the whole set of potential operating conditions. For instance, surrogates can be used to create turbocompressor [12,13,14] and turbine [15] characteristic maps, which provide the performance of the devices in terms of pressure ratio and isentropic efficiency. Heat transfer in heat exchangers [16] and in other applications [17], aero thermoelastic performance of hypersonic vehicles [18], piston engine volumetric efficiency [19] and microstructure reconstruction of porous media [20] are problems of design space exploration that have been addressed using surrogate modeling.

There is a wide variety of surrogate models able to satisfy different objectives. Yondo et al. [21] conducted a comprehensive review of different surrogate-based methods regarding aircraft aerodynamic studies. The most common types of surrogates found in the literature are the Radial Basis Function (RBF) model, the Polynomial Response Surface (PRS) model, and the Kriging model. Reihani et al. [22] used PRS to assess the sensitivity of low-pressure exhaust gas recirculation characteristics on turbocharger compressor performance. Simpson et al. [23] found that kriging surrogates were more accurate than PRS ones in the scope of aerospike nozzle design. In contrast, Fang et al. [24] concluded that RBF models presented a higher quality to predict non-linear effects than PRS models in several test problems. Nevertheless, the accuracy of surrogate models is case-dependent [25], so methods that seek to obtain adequate surrogate models for a particular application should compare between different alternatives [26].

For problems with numerous parameters and computationally or these casdemanding simulations, engineers often cannot succeed in conducting an affordable numerical campaign to develop an accurate surrogate of a problem design space due to the required dense sampling. For these cases, the possibility of resorting to the so-called multi-fidelity surrogates arises to alleviate the computational effort. Multifidelity (MF) surrogate models integrate information from high-fidelity (HF) and low-fidelity (LF) simulations, obtaining an adequate trade-off between the high computational requirements of HF models and the lack of precision of LF- models [1]. The design space approximation allows building the MF-surrogate model by map** the HF model’s parameter space through the LF’s space, despite having different dimensions.

Surrogate models involve some potential drawbacks that should be addressed. First, their use should be limited to high-fidelity simulations requiring high computational performance. The use of low-fidelity models usually leads to a reduction in the accuracy of the analyses, with the sole purpose of reducing computational time. On the other hand, using surrogate models for multi-objective problems requires specific algorithms and suitable sampling techniques capable of solving real-world engineering design problems.

Research in develo** surrogate models has experienced a remarkable increase in the last decade, especially for exploring the design space and supporting optimal designs. Recent research has applied heuristic optimization algorithms to solve the optimization problems associated with surrogate models [27,28]. In addition, another way of building surrogate models using metaheuristic algorithms has been proposed [29,30]. In line with previous limitations of surrogate models, new algorithms focused on Surrogate-Based Optimization (SBO) have been proposed [21]. Other novel research approaches are oriented towards evolutionary optimization, including multi-objective approaches.

In this way, the surrogate employs not only the outcome from the expensive high-fidelity simulations but also involves the prediction of a much cheaper low-fidelity model, which could even be an analytical approximation [31]. The existing literature reviews [21,32,33] show that multi-fidelity surrogates have been applied essentially to optimization [10,34,35] and uncertainty quantification [36,37], with a few works in which there is some degree of application to design space exploration [38,39]. Unfortunately, useful surrogate methods for optimization may not be the right choices for design space approximation [25]. Furthermore, the performance of such surrogates is often assessed using synthetic and benchmark problems, so it is yet to be determined how to adequately employ multi-fidelity surrogates for original, authentic scientific, and engineering problems.

The main contribution of this paper is the proposal of a systematic method using multi-fidelity surrogate models for space design approximation. The proposed methodology is applied in a problem of condensation mass flow in internal combustion engines, as an example of application in a complex engineering environment.

Therefore, an original procedure is developed to assess different multi-fidelity surrogate models systematically using well-established error metrics. The developed method is presented using a step-by-step procedure employing thorough mathematical notation, so that the interested researchers can obtain a multi-fidelity surrogate model useful for their problem. Different designs of experiments are employed to obtain multi-purpose numerical campaigns that only consider the relevant parameters according to the objective of each campaign. The computational effort of the multi-fidelity strategy proposed in this work has been reduced. Indeed, the novel introduction of the metamodeled variable in the scope of space design exploration allows increasing the accuracy of the surrogate at the same cost. In this way, the method can cope with situations in which it is not affordable to explore many parameters of a space design using expensive simulations, such as those conducted using 3D CFD.

The strategy is applied to the prediction of condensation when two humid air streams are mixed on a three-way junction with a twofold objective: to prove the method’s validity in a real problem (in contrast with the synthetic examples employed in most of the works dealing with surrogates) and to set an example of how to apply the method. In this way, the results obtained from the numerical campaigns described in this work are available in the web version of this article.

Determining the condensation mass flow rate generated by mixing two streams of moist air is relevant for air conditioning [40] and internal combustion engines [41]. Such condensation can be roughly estimated using low-fidelity models (0D models, as described in Section 4.2 or more accurately predicted using high-fidelity models (3D CFD simulations, as described in Section 4.3) following [41,42]. In this work, the former will be used as a low-fidelity model of condensation mass flow rate and the latter as its high-fidelity counterpart, showing in this way how to apply a multi-fidelity surrogate strategy to predict condensation in a three-way junction.

The paper is organized as follows. The proposed method for obtaining multi-fidelity surrogate models for design space approximation is presented in Section 2. The statistical methodology is covered in Section 3, describing the selected designs of experiments, the characteristics of the screening stage, the surrogate models employed, and the error metrics considered in the framework of the developed strategy for creating accurate multi-fidelity surrogate models. Section 4 briefly overviews the low and high fidelity models employed to predict condensation due to mixing streams. The application of the proposed multi-fidelity method to this particular case is presented in Section 5. Finally, some concluding remarks are provided in Section 6.

2. Proposed Multi-Fidelity Method

Table 1. List of method steps and actions to be conducted depending on the fidelity level.

Stage and Included Steps	High-Fidelity Model 1	Multi-Fidelity Framework
1. Conceptual modeling
1.1 Definition of N model parameters and l number of levels	$X_N=\left\{x_i^l\right\},i=1,\cdots ,N;\ell =1,\cdots ,L$	-
1.2 Definition of target and intermediate variables	$z_t$ = $y\left(x\right)$$z_m\left(y,x\right):y=f_m\left(z_m,x\right)$	$z_m^{MF}\left(y^{HF},y^{LF}\right):y^{HF}=f_m^{MF}(z_m^{MF},y^{LF})$
2. Identification of significant parameters
2.1 Determination of screening subset	$X_{scr}\subseteq X_N$	-
2.2 Running the models at screening points	$Y_{scr}=y\left(X_{scr}\right)$$Z_{m,scr}=z_m\left(Y_{scr},X_{scr}\right)$	$Y_{scr}^{LF}=y^{LF}\left(X_{scr}\right)$$Z_{m,scr}^{MF}=z_m^{MF}\left(Y_{scr}^{HF},Y_{scr}^{LF}\right)$
2.3 Identification of $N^{\prime }$ significant parameters ($x_j$)	$\forall j,\exists z:F_N\left(x_j,z\right)>F_{(L-1),residual}\left(x_j,z\right)$
2.4 Removal of $N-N^{\prime }$ non-significant parameters	$X_{N^{\prime }}=\left\{x_j^l\right\},j=1,\cdots ,N^{\prime };\ell =1,\cdots ,L$$N^{\prime }⩽N$	-
3. Surrogate modeling (for space design exploration)
3.1 Selection of surrogate candidates	$g_s:{\widehat{z}}_s=g_s\left(x\right)$
3.2 Determination of training subset	$X_{tra}\subseteq X_N^{\prime }$	-
3.3 Running the models at training points	$Y_{tra}=y\left(X_{tra}\right)$$Z_{m,tra}=z_m\left(Y_{tra},X_{tra}\right)$	$Y_{tra}^{LF}=y^{LF}\left(X_{tra}\right) Z_{m,tra}^{MF}=z_m^{MF}\left(Y_{tra}^{HF},Y_{tra}^{LF}\right)$
3.4 Construction of surrogate models	$g_{s,t}:min{\epsilon }_{s,t}\left({\widehat{Y}}_{tra},Y_{tra}\right) g_{s,m}:min{\epsilon }_{s,m}\left({\widehat{Z}}_{m,tra},Z_{m,tra}\right)$	$g_{s,m}^{MF}:min{\epsilon }_{s,m}^{MF}\left({\widehat{Z}}_{m,tra}^{MF},Z_{m,tra}^{MF}\right)$
3.5 Determination of verification subset	$X_{ver}\equiv X_{scr}$	-
3.6 Prediction of target variable at verification subset	${\widehat{Y}}_{s,t,ver}=g_{s,t}\left(X_{ver}\right)$${\widehat{Y}}_{s,m,ver}=f_m\left(g_{s,m}\left(X_{ver}\right),X_{ver}\right)$	${\widehat{Y}}_{s,m,ver}^{MF}=f_m^{MF}\left(g_{s,m}^{MF}\left(X_{ver}\right),y^{LF}\left(X_{ver}\right)\right)$
3.7 Calculation of error metrics and selection of best surrogate variable and model	$z,s:min\epsilon \left({\widehat{Y}}_{s,z,ver},Y_{ver}\right)$
3.8 Usage of surrogate for space design exploration	$Y_{exp}\simeq g_{s,t}\left(X_{exp}\right)$$Y_{exp}\simeq f_m^{HF}\left(g_{s,m}^{HF}\left(X_{exp}\right),X_{ver}\right)$	$Y_{exp}\simeq f_m^{MF}\left(g_{s,m}^{MF}\left(X_{exp}\right),y^{LF}\left(X_{exp}\right)\right)$

¹ For the sake of simplicity, HF superscript is omitted from the notation in this column, as it affects all variables and functions.

enumerates all the steps included in the comprehensive method designed in this work, which provides multi-fidelity surrogate models for design space approximation. The table distinguishes between the actions required in the HF and MF frameworks. The mathematical notation featured in Table 1 avoids particularities of the problem used as an example (condensation in mixing streams) and, thus, allows researchers from other fields to use the proposed method.

The first stage of Table 1 is the Conceptual modeling, in which one defines the complete dataset $X_N=\left\{x_i^l\right\}$ by identifying the model parameters i and determining the particular values for each parameter at level l. Section 5.1.1 shows an example of such steps. The conceptual modeling ends by selecting a set of intermediate variables $\left(z_m\right)$ that will be assessed for their use as surrogates, together with the target variable, which is the one that is required to be mapped $\left(z_t\right)$. For the studied application, these variables are defined in Section 5.1.2.

The next section of Table 1 is the Identification of significant parameters, which aims to reduce the computational effort of the surrogate construction. In this way, a subset of $X_{scr}$ points can be determined using a certain design of experiments (a fractional factorial design is suggested in this work; see Section 3.1.1) and then run High-Fidelity models $Y_{scr}^{HF}$ and Low-Fidelity models $Y_{scr}^{LF}$ at these working points to obtain a multi-fidelity evaluation. After the screening analysis over all metamodeled variables z (using, for instance, ANOVA, as described in Section 3.2), the non-significant parameters can be removed, leaving only $N^{\prime }$ significant parameters for the subsequent steps.

The last stage of the proposed method (see Table 1) is the Surrogate modeling (for space design exploration) devoted to the surrogate modeling itself. Firstly, s surrogate candidates are selected (see Section 3.3), and a set of training points $X_{tra}$ is determined using another design of experiments. Here, the Box-Behnken design is employed, as justified in Section 3.1.2. Next, High-Fidelity $\left(g_{s,t},g_{s,m}^{HF}\right)$ and Multi-Fidelity $\left(g_{s,m}^{MF}\right)$ surrogate models are constructed using the evaluation of High-Fidelity $Y_{tra}^{HF}$ and Low-Fidelity models $Y_{tra}^{HF}$ at the training points. These surrogate models are then used to predict values of the target variable at the verification points ${\widehat{Y}}_{s,t,ver}$, ${\widehat{Y}}_{s,m,ver}^{HF}$ and ${\widehat{Y}}_{s,m,ver}^{MF}$. Notice that these works consider the screening subset $X_{scr}$ to be used as verification subset $X_{ver}$ to reduce the computational cost. Since the exact results are already available from stage 2.2 of Table 1, the error metrics (as defined in Section 3.4) are evaluated, allowing the different combinations of HF or MF surrogate models with the target or intermediate variables to be classified as adequate or not in terms of accuracy. Finally, the best couple of surrogate s and variable z is selected to be used for space design exploration at any working point within the assessed range $X_{exp}$.

3. Statistical Methodology

3.1. Design of Experiments

The proposed multi-fidelity method requires several sets of working points for screening, training, and verification purposes. Different designs of experiments are considered, depending on the required characteristics of each set of points. In this way, a fractional factorial design obtains the screening points, and a Box-Behnken design is used to provide the training points. For the sake of computational effort, the same set of points employed for screening will also be used to verify the accuracy of the surrogate models.

3.1.1. Fractional Factorial Design

The proposed method requires a subset of points $X_{scr}$ to conduct the screening stage. The full factorial design would be a candidate for this purpose, as it allows exploration a wide variety of combinations between the parameters at different levels, entailing a high level of resolution in the estimation effects. However, it may produce a verification subset with a prohibitive computational cost, depending on the number of parameters in the model and the cost of each simulation. Furthermore, presenting great accuracy in estimating high-level interaction terms at the screening stage is unnecessary.

Thus, to save a little computational effort, it was decided to perform a fractional factorial design instead. The objective of the factorial fractions is to identify the characteristic parameters with the greatest effects on the high and multi-fidelity variables. One-half fraction designs reduce the number of simulations by half. Still, researchers can further reduce the number of simulations by selecting a quarter, an eighth, or even a sixteenth fraction. Increasing the fraction’s denominator reduces the number of simulations required and the resolution needed to estimate higher-order interaction effects. Factor fractions of order p for factors with two levels and N characteristic parameters, with the possible inclusion of an intermediate level for center points, are represented by $2^{N-p}$.

The resolution of the fraction design influences the number of high-order interaction effects that are confounded to estimate main effects and low-order interactions based on a reduced number of simulations. Since the characteristics of this fraction design meet the requirements of the screening stage as well as the verification stage, the verification subset will coincide with the screening subset ($X_{ver}\equiv X_{scr}$) so that the effort of this extensive numerical campaign is exploited twice.

3.1.2. Box-Behnken Design

When fitting a polynomial response surface (PRS) model, quadratic surfaces are estimated to represent the curved shape of continuous factors. Box-Benkhen designs [43] are incomplete second-order designs based on three levels, making them appropriate for obtaining training subsets $X_{tra}$ to construct surrogates. The number of simulations this design requires is $2N(N-1)$. Center points can be added to this set of simulations to improve the statistical power of the analysis. This design is more efficient than other response surface designs, such as composite centers or three-level full factorial designs [44]. In addition, this design does not include combinations of levels where all characteristic parameters are simultaneously at high or low levels. Therefore, it only forces the execution of simulations under extreme conditions that may provide meaningful results in design space exploration.

3.2. Screening Stage

The proposed method (see Table 1) conducts a screening phase to identify the characteristic parameters that generate significant differences in either high-fidelity or multi-fidelity variables. In this analysis stage, the focus is on direct and linear effects rather than possible quadratic or nonlinear effects. Analysis of variance (ANOVA) is a versatile method for this screening phase as it allows the exclusion of nonlinear terms from the model, thus increasing its power. Thus, non-significant terms that usually correspond to interaction effects can be specifically eliminated to improve the significance of the model by including only the main results. ANOVA proves to be a valuable technique to screen input parameters and thus reduce the extension of subsequent numerical [45] or experimental campaigns [46] for optimization.

This analysis of variance (ANOVA) is based on multiple comparisons of the means of the high and multi-fidelity variables at different levels of the characteristic input parameters. In the ANOVA model, the F-statistic and its corresponding p-value are critical in identifying the characteristic parameters that cause significant differences in the high-fidelity and multi-fidelity variables. This F-statistic is calculated for the main effect of each characteristic parameter as well as for the residual variability of the model. The F-statistic follows a Fisher-Snedecor distribution, assuming the null hypothesis ($H_0$) that different levels of the characteristic parameters do not produce significant differences in the high and multi-fidelity variables. The F statistic represents a ratio between two measures of variability weighted by degrees of freedom (mean squares): the variability explained by each characteristic parameter relative to the residual variability of the model. Thus, the F statistic represents the variability explained by each characteristic parameter relative to the variability not explained (residual) by the model.

The variability explained by different levels of characteristic parameters can be calculated as the Explained Sum of Squares (ESS):

$$ESS=\sum n_i\frac{{({\overline{Y}}_i-\overline{Y})}^2}{l-1}$$

(1)

where ${\overline{Y}}_i$ represents the sample mean of the high or multi-fidelity variable at each level of the characteristic parameter, $\overline{Y}$ represents the overall mean of this variable, and l denotes the number of levels of the parameter.

The residual (intra-level) variability (Residual Sum of Squares) is calculated as follows:

$$RSS=\sum _{ij}\frac{{(Y_{ij}-{\overline{Y}}_i)}^2}{N-l}$$

(2)

where $Y_{ij}$ represents the j-th simulation at each of the i-th level of the high or multi-fidelity variable, and $\overline{Y}$ represents the mean value of that variable at each level of the characteristic parameter, N represents the total sample size, and l represents the number of levels.

Using both terms, the F statistic of each parameter characteristic can be computed as the ratio between the variability explained by different levels and the (unexplained) residual variability, within levels of characteristic parameters.

$$F=\frac{ESS}{RSS}$$

(3)

Along with this F-statistic, p-values can be obtained for identifying significant effects ($p\text{-}\mathrm{value}<\alpha$). These p-values are based on the comparison between calculated F-statistics and critical values of the Fisher - Snedecor distribution. The degrees of freedom for the numerator and denominator of this distribution are calculated by combining the number of levels and the screening sample size. A 95% confidence level is usually considered for assessing the significance of p-values, although more restrictive approaches (99% confidence level). In this screening stage, some low-level interactions (double or even triple) may show significant effects. The retention of these interaction terms is at the discretion of the researcher, although in most cases it is not interesting for research.

3.3. Surrogate Models

A key stage in the developed method (see Table 1) is the construction of surrogate models. In this work, three widely employed surrogate models have been considered: Polynomial Response Surface (PRS), Radial Basis Function (RBF), and Ordinary Kriging. PRS models are based on classical regression models and analysis of variance to analyze the relationship between a response variable and a set of predictor variables. RBF models are based on a collection of functions modeling a measure of distance (usually Euclidean) between the point to be measured and a fixed point (usually the origin or an established center), as described in Section 3.3.2. Furthermore, ordinary kriging models are built from a kernel function considering the deviation between the metamodel variable of interest and its prediction, as shown in Section 3.3.3. Thus, RBF and Ordinary kriging models differ from the PRS model in the use of collection functions to approximate the variable of interest, whereas PRS models apply classical modelization techniques to estimate the parameters of the model. All of them will be used to obtain metamodels of every variable z assessed, based on the training subset $X_{tra}$. Notice that the MF surrogates developed in this work are based on scaling functions $z_m^{MF}$ [1].

3.3.1. Polynomial Response Surface Models (PRS)

Polynomial response surface models (PRS) have been extensively used for data fitting and regression analysis to examine the relationship between variables and outputs [47,48,49]. A second-order polynomial surrogate can be written as:

$$\begin{array}{c}\hfill g_s={\beta }_0+\sum _{i=1}^{N^{{}^{\prime }}}{\beta }_i{x}_i+\sum _{i=1}^{N^{{}^{\prime }}}\sum _{j=1}^{N^{{}^{\prime }}}{\beta }_{ij}{x}_i{x}_j\end{array}$$

(4)

The calibration of the coefficients $\beta$ of Equation (4) for the polynomial response surface has been set using the R software library [50].

3.3.2. Radial Basis Function Models (RBF)

A so-called radial function employs the Euclidean distance between the input parameter and a fixed point, known as the center. A Radial Basis Function (RBF) model is constructed using a linear superposition of different radial basis functions, which is presented in Equation (5):

$$\begin{array}{c}\hfill g_s=\sum _{t=1}^{N_{tra}}{w}_t\psi \left(∥x-x_t∥\right)\end{array}$$

(5)

To use the combination of $N_{tra}$ radial functions as a RBF surrogate, each point $x_t$ corresponding to the training subset ($X_{tra}=\left\{x_t\right\}$) is employed as the center, $w_t$ are weights associated with each center $x_t$ and $\psi$ is a basis function that acts over the aforementioned euclidean distance. In this work, the library PySMO developed by the Institute for the Design of Advanced Energy Systems based on the work of Forrester and Rippa et al. [10,51] is employed on Python to construct RBF surrogates. Depending on the variable to be metamodeled, the best function $\psi$ amongst those available in the library (linear, cubic, thin-plane spline, gaussian, and multi-quadratic) is selected to obtain the greatest fitting.

3.3.3. Ordinary Kriging Model

A surrogate model combining a known function $f\left(x\right)$ (regression) with a function $Z\left(x\right)$ as Equation (6) shows:

$$\begin{array}{c}\hfill g_s=\sum _{t=1}^{N_{tra}}{\beta }_{t}f\left(x\right)+Z\left(x\right)\end{array}$$

(6)

is referred to as a Kriging model due to its first use in the field of geostatistics [52]. The so-called kernel $Z\left(x\right)$ considers the deviation between the metamodeled variable and the prediction of the deterministic function $f\left(x\right)$ (here considered a constant) using a stochastic Gaussian process, so Kriging models are also known as Gaussian process regressions. In Equation (6), ${\beta }_t$ are the regression coefficients, $N_{tra}$ the number of training points, and $Z\left(x\right)$ presents zero mean and a co-variance of ${\sigma }^2\psi$, where the correlation function $\psi$ employed in this work is the squared exponential (see [53,54]).

For this work, Kriging surrogates have been built using a Toolbox on Python environment developed by Bouhlel et al. [53]. According to **ao et al. [55], Kriging models better describe the local features of the data to be metamodeled than PRS. Comparing Equation (6) with Equation (5), the functions $\psi$ play a similar role as either radial basis functions in the RBF model or kernels in the Kriging model [56]. In this way, Chandrashekarappa and Duvigneau [57] noted that the normal distribution considered in the Kriging kernel functions is more adequate for experimental measurements than for computational simulations, which is the application example considered in this work (Section 4 and Section 5).

3.4. Error Metrics

Several steps of the proposed strategy to develop multi-fidelity surrogates (see Section 2) require a quantitative assessment of the obtained models, to evaluate and compare their performance to select the best combination of metamodeled variables and surrogate types. Generally, these metrics check the error committed between the observed values and the predicted values, but the roles of the observed and predicted data at the corresponding stage will determine which is the error indicator best suited for the task.

The first indicator employed in this work is the coefficient of determination $R^2$. It quantifies the proportion of the total variability of the data that the model can explain. To do so, this error metric employs the co-variance of the predicted and observed cases as well as the variance of each case, as can be seen in Equation (7):

$$R^2=\frac{{\sigma }_{z\widehat{z}}^2}{{\sigma }_z^2{\sigma }_{\widehat{z}}^2}=\frac{{\left(\frac{{\sum }_{i=1}^n\left(z_i-\overline{z}\right)\left({\widehat{z}}_i-\overline{\widehat{z}}\right)}{n}\right)}^2}{\frac{{\sum }_{i=1}^n{\left(z_i-\overline{z}\right)}^2}{n}\frac{{\sum }_{i=1}^n{\left({\widehat{z}}_i-\overline{\widehat{z}}\right)}^2}{n}}$$

(7)

Notice that $R^2$ in this work is only employed to assess the fitting capability of each model outcome ($y_i$) to the training data ($x_i$). Indeed, the coefficient is known to be inadequate when attempting to evaluate the predicting capability of the models in the verification stage [58]. For the latter purpose, the Mean Absolute Error (MAE) is preferred [58,59]. MAE directly calculates the average magnitude of the error between the values predicted by a surrogate (${\widehat{y}}_i$) and the values observed at the verification stage ($y_i$), as Equation (8) shows.

$$MAE=\frac{{\sum }_{i=1}^n\left|y_i-{\widehat{y}}_i\right|}{n}$$

(8)

Following the work performed by Roy et al. [60], MAE can be applied as a means of qualitative assessment to determine whether a surrogate model can be considered to provide a good prediction or not. To do so, the 5% of data with the highest residual values is first removed, and then MAE is normalized considering the training dataset range $R_{tra}$ and the dispersion of the errors. This elimination of a small part (5%) of the values with the highest residuals in the test set is performed to attenuate the influence of possible outliers that increase prediction errors. This procedure aims to evaluate the quality of the prediction using 95% of the data present in the test set, thus increasing the robustness of the estimate. Equations (9) and (10) show the conditions required to grant a good evaluation to the surrogate. When the two conditions are fulfilled at the same time, a good agreement is obtained. If only one of these conditions is satisfied, a moderate agreement is set. If no condition is met, a bad agreement is therefore considered.

$$\widetilde{MAE}=\frac{MAE(95\%)}{R_{tra}}\le 0.1$$

(9)

$${\widetilde{MAE}}_{\sigma }=\frac{MAE(95\%)+3·\sigma }{R_{tra}}\le 0.2$$

(10)

4. Description of the Application Problem and Employed Multi-Fidelity Models

4.1. Background of Volume Condensation Due to Mixing

The application of this method is devoted to characterizing a problem that appears in the automotive sector when a vehicle features an internal combustion engine such as the one depicted in Figure 1. For such turbocharged engines, the use of the so-called Low-Pressure Exhaust Gas Recirculation (LP-EGR) is being widely researched to reduce ${\mathrm{CO}}_2$, ${\mathrm{NO}}_x$, and particulate matter emissions [61,62]. EGR consists of mixing a fraction of exhaust gases (with high humidity content) with fresh air coming from the ambient. As Figure 1 shows, LP-EGR performs this mixture upstream of the compressor, so condensation can happen at certain cold conditions, either at the LP-EGR cooler [63,64] or at the three-way junction [65,66]. This possibility of generating condensation upstream of the compressor constitutes one of the main drawbacks of LP-EGR. Indeed, water droplets reaching the impeller will erode its blades and thus reduce its lifetime and performance [41,67]. In this way, the ability to predict the condensation mass flow rate for the whole engine map would allow automotive engineers to maximize the LP-EGR rate for each working point (and thus minimize ${\mathrm{CO}}_2$, ${\mathrm{NO}}_x$ and particulate matter emissions) without exceeding the damage threshold of the impeller, to keep its durability.

Focusing on the mixing process at three-way junctions, the explanation of how two sub saturated streams can produce condensates is given by the psychrometric diagram displayed in Figure 2. Following the LP-EGR junction highlighted in Figure 1, a humid, warm stream of EGR can be blended with a flow of cold ambient air. In these cases, Figure 2 confirms that their mixture can be located in the supersaturation region so that the excess of water until the point “mix” lies within the saturation line should be condensed into the form of liquid water.

4.2. Low Fidelity: 0D (Perfect Mixing) Condensation Model

In this work, a 0D mixing condensation model is considered a low-fidelity model. Figure 3 shows the parameters of this model. It can be regarded as an evolution of the one developed by Serrano et al. [41], in which a total (perfect) mixing between the inlet streams (air and EGR) is considered in an isobaric and isenthalpic way, as depicted in Figure 2. After calculating the mixing condensation, the 0D model employed in this work also evaluates the temperature decrease due to the area reduction (labeled “nozzle” in Figure 3) usually existing in LP-EGR junctions as a potential source of additional condensation. More details about the psychrometric calculations to obtain the condensation considering perfect mixing can be found in the work of Serrano et al. [41].

In the perfect mixing process, both streams are completely mixed in a homogeneous flow, obtaining the maximum amount of water condensed in perfect conditions.

4.3. High Fidelity: 3D CFD Condensation Model

This work employs a 3D CFD model of the three-way junction embedded with a condensation subroutine as a high-fidelity model. The approach used is the one described by Galindo et al. [68] and has been validated in several ways using a series of different experimental techniques: secondary flows measured with laser particle image velocimetry [68], qualitative condensation distributions observed using planar laser-induced visualization [68], temperature fields [69] and condensation mass fluxes [69]. Indeed, this numerical tool has been able to predict which junction configuration provides the least amount of condensation mass flow rate, which in turn provides the lowest impeller wear in agreement with the level observed in experimental durability tests [70].

The geometry of the study is an LP-EGR thee-way junction where fresh inlet air and EGR are mixed before the compressor. As shown in Figure 4, there are two inlet branches (a 50 mm diameter duct for air and a 22.4 mm diameter pipe for EGR) and one outlet duct, whose diameter is 34 mm. The nozzle section after the EGR insertion represents the tapered inlet of the compressor. To reduce the impact of boundary locations on the region of interest, both inlet ducts are extruded three additional diameters upstream, and the outlet pipe is extended two diameters downstream.

The mesh depicted by Figure 4 comprises polyhedral cells with a base size of 1.5 mm and eight prism layers next to the walls to improve the boundary layer resolution. A mesh independence study has selected this grid of 3M cells. Employing a coarser grid of 300 thousand cells results in a difference of 10.4% in condensation at the outlet section. Refining the mesh instead until it features 11M cells provides a difference in condensation of only 3%. Considering the computational effort of the numerical campaign (more than 180 simulations), the mesh of 3M cells was therefore selected, which achieves $y^{+}$ to be below 1.5 for 99% of the wall cells.

The segregated solver of Simcenter STAR-CCM+ is selected for the simulations. Regarding the turbulence formulation, Reynolds-Average Navier-Stokes (RANS) was selected with the $SST$ k-$\omega$ turbulence model [71]. The cases are initially run steadily, and their condensation mass flow rate is monitored. For those working points with an oscillation in this variable above $2\%$, the temporal treatment was switched to unsteady RANS (URANS), employing a second-order implicit transient solver with a time-step size of $\Delta t={10}^{-4}$ s. As boundary condition types, the mass flow rate, specific humidity, stagnation temperature, and turbulence intensity are specified at both inlets, whereas the static pressure is set at the outlet. The particular values for the numerical campaign are indicated in Section 5.1.1, which correspond to representative conditions for a C-segment passenger car engine.

A condensation subroutine [42] is embedded into the 3D CFD model. This condensation submodel reads the psychrometric state of each cell and compares it to the corresponding saturated equilibrium state. When supersaturation is detected, the model instantaneously condenses the required amount of water, interacting with the transport equations solved by the 3D CFD code using the corresponding source terms. For more details, please refer to Serrano et al. [42].

4.4. Comparison between Low-Fidelity (LF) and High-Fidelity (HF) Condensation Prediction

To assess the difference in performance between the 0D model (LF) and the 3D CFD model (HF), two cases are selected from the numerical campaign that will be described in Section 5.2.1: one delivering low condensation (Figure 5) and another providing more significant condensation (Figure 6). The results predicted by these models are included in

Table 2. Condensation mass flow rates predicted by the LF (0D) and HF (3D) model for two working points.

Working Point	${\dot{\mathit{m}}}_{0\mathit{D}}$ [kg/h]	${\dot{\mathit{m}}}_{3\mathit{D}}$ [kg/h]
A (Low condensation)	1.182	0.2406
B (High condensation)	2.673	1.651

.

The working points of Table 2 present the same air mass flow rate, but point B features a much greater EGR (humid stream) mass flow rate than point A, which is the main reason for the increase in the 0D condensation mass flow rate by 130%. The high-fidelity model also predicts an increase in condensation mass flow rate when shifting from case A to case B, but a much greater one (590%; see Table 2). Whereas the LF 0D model only considers the global psychrometric information, the HF 3D CFD model also considers the local flow features.

The different predictions of the LF and HF models can therefore be explained based on the flow patterns depicted in Figure 5 and Figure 6. Figure 5 presents a low EGR mass flow rate, which results in a wall-jet flow configuration (see Kimura et al. [72]), while Figure 6, with a greater EGR rate, displays a flow pattern in the threshold between a deflecting-jet and an im**ing-jet [72]. The 0D model provides an upper bound for the condensation mass flow rate, as it considers a perfect mixture between the streams. The different deviation between the 3D and the 0D models depends on the flow mixing, as the 3D CFD model predicted, as condensation is strongly correlated with mixing [66]. The mixing intensity (and thus condensation) is much greater for the high condensation case (Figure 6) due to the high penetration of the EGR stream than for the low condensation case (Figure 5). In the framework of multi-fidelity modeling [73], the 0D case could be useful to provide the psychrometric information of the operating conditions, being a LF model with little computational effort. However, a comprehensive set of expensive 3D CFD simulations will still be required. The examples analyzed in this section show how the deviation between the LF and HF models can differ when shifting the working point.

5. Application of the Proposed Method to the Problem of Volume Condensation Due to Mixing: Results and Discussion

5.1. Conceptual Modeling

5.1.1. Operating Conditions: Definition of Model Parameters and Their Range

The developed method starts the conceptual modeling stage by identifying the model parameters, as stated in Table 1. In this case, one only needs to identify the boundary conditions employed in the 3D CFD model ($x_{HF}$), as described in Section 4.3, and the inputs of the 0D model ($x_{LF}$), as depicted by Figure 3. By doing so,

Table 3. HF model parameters and levels considered for the different designs of experiments.

Parameters	Variables	Units	Low, Medium and High Values
Inlet air mass flow	${\dot{m}}_{air}$	[kg/h]	50, 175, 300
Inlet temperature	$T_{air}$	[${}^{\circ }$C]	−10, 0, 10
Inlet relative humidity	${RH}_{air}$	[%]	50, 75, 100
Inlet turbulent intensity	$I_{air}$	[%]	1, 5.5, 10
EGR temperature	$T_{egr}$	[${}^{\circ }$C]	40, 50, 60
EGR rate	${EGR}_{rate}$	[%]	10, 25, 40
EGR specific humidity	$w_{EGR}$	[g/kg]	30, 50, 70
EGR turbulent intensity	$I_{EGR}$	[%]	1, 5.5, 10
Outlet pressure	$P_{out}$	[bar]	0.8, 0.9, 1

is obtained, providing $N=9$ parameters. Notice that $N_{LF}=7$, as the parameters for the LF model are the same as the HF one save for the turbulent intensities at both inlet and EGR ducts. Furthermore, three levels are selected for each parameter, with values covering a typical range of working points in a piston engine featuring LP-EGR that may produce condensation. The rightmost column of Table 3 includes the matrix of eligible parameter values $x_i^{\ell }$ that will be employed for conducting the different designs of experiments. Notice that, in this case, the geometry of the junction has been kept constant so that the exploration is performed in terms of operating conditions.

5.1.2. Definition of Target and Intermediate Variables

The method developed in this work is applied to obtain a surrogate for the condensation mass flow rate that 3D CFD simulations (HF model) would predict. Therefore, this is the target variable $z_t$ sought:

$$z_t={\dot{m}}_{cond.,3D}$$

(11)

Furthermore, the method suggests (see Table 1) that additional intermediate variables could be modeled through surrogates. The underlying idea is that the distribution of such variables across the design space could be more adequate to build surrogate models than that of the target variable.

Table 4. Target and intermediate variables studied.

Name	Variable	Type
3D condensation	${\dot{m}}_{cond.,3D}$	HF Target var.
Condensation humidity	$w_{cond.,3D}$	HF Intermediate var.
Condensation ratio	$\phi$	MF Multiplicative scaling var.
Condensation difference	$\Delta$	MF Additive scaling var.
Junction efficiency	$\eta$	MF Hybrid scaling var.

presents the target and intermediate variables metamodeled in this work.

An intermediate variable can be defined within the domain of the HF model ($z_m$), as long as it involves the target variable and other input parameters. This is the case of the condensation humidity, which normalizes the target condensation mass flow rate by the inlet mass flow rate of ambient dry air:

$$\begin{array}{c}\hfill \left(z_{m=w_{cond.,3D}}\right):w_{cond.,3D}=\frac{{\dot{m}}_{cond.,3D}·\left(1+w_{air}\right)}{{\dot{m}}_{air}}·1000\end{array}$$

(12)

$$\begin{array}{c}\hfill \left(f_{m=w_{cond.,3D}}\right):{\dot{m}}_{cond.,3D}=\frac{{\dot{m}}_{air}}{1+w_{air}}·\frac{w_{cond.,3D}}{1000}\end{array}$$

(13)

Notice that the equation for the function $f_m$ is also included before, as it will be required to obtain later ${\dot{m}}_{cond.,3D}$ on the basis of the surrogate model for $w_{cond.,3D}$ and the parameters $w_{air}$ and ${\dot{m}}_{air}$ corresponding to the working point studied.

But aside from the HF intermediate variables, this work aims to show how to build and assess multi-fidelity surrogate models for space design exploration. In this way, Jiang et al. [1] identified three types of variables for MF models based on scaling functions: multiplicative, additive, and hybrid variables. For instance, a multiplicative variable can be defined as just the ratio between the HF and LF predictions for condensation mass flow rate:

$$\begin{array}{c}\hfill \left(z_{m=\phi }^{MF}\right):\phi =\frac{{\dot{m}}_{cond.,3D}}{{\dot{m}}_{cond.,0D}}\end{array}$$

(14)

$$\begin{array}{c}\hfill \left(f_{m=\phi }^{MF}\right):{\dot{m}}_{cond.,3D}={\dot{m}}_{cond.,0D}·\phi \end{array}$$

(15)

An MF additive variable is also built by calculating the difference between the condensation mass flow rates obtained by the 0D (LF) and 3D CFD (HF) models:

$$\begin{array}{c}\hfill \left(z_{m=\Delta }^{MF}\right):\Delta ={\dot{m}}_{cond.,0D}-{\dot{m}}_{cond.,3D}\end{array}$$

(16)

$$\begin{array}{c}\hfill \left(f_{m=\Delta }^{MF}\right):{\dot{m}}_{cond.,3D}={\dot{m}}_{cond.,0D}-\Delta \end{array}$$

(17)

Finally, a hybrid variable may combine multiplications and additions of the LF and HF variables in its definition. In this work, a so-called junction efficiency $\eta$ is defined:

$$\begin{array}{c}\hfill \left(z_{m=\eta }^{MF}\right):\eta =\frac{{\dot{m}}_{cond.,0D}-{\dot{m}}_{cond.,3D}}{{\dot{m}}_{cond.,0D}}·100\end{array}$$

(18)

$$\begin{array}{c}\hfill \left(f_{m=\eta }^{MF}\right):{\dot{m}}_{cond.,3D}={\dot{m}}_{cond.,0D}·(1-\eta /100),\end{array}$$

(19)

So that it is useful not only as an MF intermediate variable for design space exploration but also as a figure of merit to assess the junction design [74,75]. Indeed, since ${\dot{m}}_{cond.,0D}$ is an upper bound for ${\dot{m}}_{cond.,3D}$, the junction efficiency defined in 18 ranges from 0% (maximum condensation) to 100% (avoids all condensation).

To conclude this section, the working points already discussed in Section 4.4 are again applied to obtain the values of the intermediate variables stated in Table 4, based on the LF and HF results featured in Table 2. In this way,

Table 5. Values of target and intermediate variables for two working points.

Working Point	${\dot{\mathit{m}}}_{\mathit{cond}.,3\mathit{D}}$ [kg/h]	${\mathit{w}}_{\mathit{cond}.,3\mathit{D}}$ [g/kg]	$\mathit{\eta }$ [%]	$\mathsf{\Delta }$ [kg/h]	$\mathit{\phi }$ [-]
A (Low condensation)	0.2406	0.8023	79.65	0.9417	0.2035
B (High condensation)	1.651	5.505	38.24	1.022	0.6176

shows an example of the values that these different variables may take. For instance, the lower condensation case compared with the high condensation case (see Figure 5 and Figure 6) results in a greater junction efficiency in terms of avoiding condensation (79.65% vs. 38.24%), following Table 5.

5.2. Determination of Significant Parameters

5.2.1. Selection of Screening Points and Results of the HF and LF

The objective of the screening stage is to identify the input parameters whose variation generates significant effects on the variable of interest. To achieve this goal, it is advisable to work with $2^{N-p}$ factorial fractions, where k represents the number of factors ($k=9$ in this paper) and p the number of partitions ($p=2$ in this paper) to the total set of simulations using the full factorial design ($2^N=2^9=512$). The results obtained from a set of simulations designed by fractional factorial design always have a lower resolution than those obtained from a full factorial design. When the number of factors is high, the decrease in resolution is usually compensated by a remarkable reduction in the number of simulations. Using this method, 128 simulations obtained as a quarter of the full factorial design ($2^{9-2}$) were generated. Simulations were executed for the 3D Model and the 0D Model. Target and intermediate variables were computed on the basis of these simulated values. See the Supplementary Materials.

High and low levels of characteristic parameters were complemented with several center points to improve the accuracy of the standard error of estimation. When condensation phenomena were not physically feasible due to the influence of local irregular flows or negligible amounts of condensed water, values of condensation mass flow were considered missing. Some other experiments were slightly modified to ensure condensation was produced for a more profitable data analysis. Based on this screening design, the variability in the five metamodeled variables was assessed for different combinations in the initial set of characteristic parameters.

5.2.2. Anova and Removal of Non-Significant Parameters

The analysis of variance described in Section 3.2 compares the variability of each metamodeled variable z regarding the main effects produced by each model parameter.

Table 6. Identification of significant main effects according to ANOVA F-statistics for different HF model parameters and metamodeled variables.

Parameter	$\mathit{F}\left({\dot{\mathit{m}}}_{\mathit{cond}.,3\mathit{D}}\right)$	$\mathit{F}\left({\mathit{w}}_{\mathit{cond}.,3\mathit{D}}\right)$	$\mathit{F}\left(\mathit{\phi }\right)$	$\mathit{F}\left(\mathsf{\Delta }\right)$	$\mathit{F}\left(\mathit{\eta }\right)$
Inlet air mass flow	59.6 **	31.1 **	38.8 **	87.9 **	38.7 **
Inlet temperature	4.4 *	16.1 **	35.8 **	0.2	52.5 **
Inlet relative humidity	0.5	0.9	27.2 **	0.5	15.4 **
Inlet turbulent intensity	0.1	0.0	0.0	0.1	0.0
EGR temperature	11.8 **	27.1 **	24.7 **	18.7 **	25.2 **
EGR rate	53.4 **	103.1 **	143.1 **	22.0 **	128.9 **
EGR specific humidity	38.8 **	85.8 **	61.3 **	53.0 **	77.7 **
EGR turbulent intensity	0.2	0.4	0.0	0.3	0.1
Outlet pressure	0.03	0.3	1.00	0.6	1.8

Significant effects with 99.0% (**) or 95.0% (*) confidence level.

presents these values in terms of the F-statistic defined in Equation (3).

Comparing the results shown in Table 6 with the corresponding critical value, three non-significant parameters (inlet turbulent intensity, EGR turbulent intensity, and outlet pressure) were identified and excluded from subsequent analysis. It is worth noting the model parameters that were deemed significant (usually with the highest confidence level of $99\%$) for every variable z, namely, inlet mass flow, EGR temperature, EGR humidity, and EGR rate. The remaining parameters (inlet temperature and relative humidity are significant or not depending on the variable considered, so they will not be dismissed, as the best variable to construct a surrogate is not known yet.

5.3. Surrogate Modeling for Space Design Exploration

5.3.1. Selection of Surrogate Candidates

All surrogate models defined in Section 3.3 are assessed for this problem, so $s=PRS,$ $RBF,Krig$.

5.3.2. Selection of Training Points and Subsequent Obtention of HF and LF Results

To calibrate the surrogate models corresponding to each of the five metamodeled variables, a numerical campaign of 54 simulations was produced using a Box–Behnken design (see Section 3.1.2), considering the remaining six significant parameters. Two replicates of this design were added with center points to deal with some missing data (certain combinations of operating conditions do not produce condensation, as Figure 2 shows) and also to improve the accuracy of the standard error of estimation. These replicates were obtained by including random modifications to previously discarded (non-significant) factors. Thanks to these enhancements, the percentage of missing data was reduced to $13\%$ due to the sample size increase. The collection of previous HF and LF simulations (54 working points) was subsequently used as the training data set to obtain the surrogate models.

5.3.3. Construction of Surrogate Models and Prediction of Target Variable at Verification Points

This work has selected three surrogate models: Polynomial Response Surface (PRS), Radial Basis Function (RBF), and Kriging model. These have already been described in Section 3.3. These three different surrogate models s were constructed for each of the variables defined in Section 5.1.2, seeking to minimize the error between the metamodeled variable ${\widehat{Z}}_{m,tra}=g_{s,m}\left(X_{tra}\right)$ and the actual values $Z_{m,tra}$ at the training dataset $X_{tra}$.

An example is presented applying the surrogate Polynomial Response Surface model ($s=PRS$) for the multi-fidelity intermediate variable “condensation difference” ($m=\Delta$) This model determines the coefficients $\beta$ of Equation (4) that minimize the error of comparing $\widehat{\Delta }$ (the metamodel of $\Delta$) against $\Delta$ itself at the set of training points. A $\widehat{\Delta }$ value of 0.89 kg/h is predicted for case 24 among the training data when applying the surrogate model. Comparing this value with the actual $\Delta$ for such a working point (0.96 kg/h), an error of 7% appears between the observed and predicted values.

5.3.4. Calculation of Error Metrics and Assessment of Surrogate Models

The error metrics described in Section 3.4 were finally calculated for the 15 combinations of surrogated models and variables above, as shown in

Table 7. Evaluation of the surrogate models to predict water condensation for different metamodeled variables.

Surrogate Models		Training		Verification				Evaluation
		${\mathit{R}}^2\left({\mathit{z}}_{\mathit{m}}\right)$	${\mathit{R}}^2\left({\mathit{z}}_{\mathit{t}}\right)$	MAE	MAE 95%	$\widetilde{\mathit{M}\mathit{A}\mathit{E}}$	${\widetilde{\mathit{M}\mathit{A}\mathit{E}}}_{\mathit{\sigma }}$
PRS	${\dot{m}}_{cond.,3D}$	0.94		0.1596	0.139	0.0601	0.1851	Good
	$w_{cond.,3D}$	0.95	0.98	0.1376	0.096	0.0415	0.1425	Good
	$\phi$	0.56	0.39	0.4587	0.2428	0.1044	0.4605	Bad
	$\Delta$	0.95	0.98	0.0969	0.0793	0.0341	0.1001	Good
	$\eta$	0.97	1.0	0.1274	0.0829	0.0356	0.1726	Good
RBF	${\dot{m}}_{cond.,3D}$ (Gauss.) *	1.0		0.1824	0.1553	0.0688	0.1988	Good
	$w_{cond.,3D}$ (Gauss.) *	1.0	1.0	0.1078	0.0826	0.0355	0.1252	Good
	$\phi$ (Multi-quad.) *	1.0	0.74	0.1413	0.1061	0.0457	0.1867	Good
	$\Delta$ (Cubic) *	1.0	1.0	0.2348	0.2052	0.0883	0.2566	Moderate
	$\eta$ (Multi-quad.) *	1.0	1.0	0.1601	0.0925	0.0398	0.2313	Moderate
Kriging	${\dot{m}}_{cond.,3D}$	1.0		0.2391	0.2176	0.0936	0.1964	Good
	$w_{cond.,3D}$	1.0	1.0	0.1409	0.1128	0.0473	0.1603	Good
	$\phi$	1.0	0.85	0.1467	0.1066	0.0459	0.1916	Good
	$\Delta$	1.0	1.0	0.1616	0.1409	0.0606	0.1486	Good
	$\eta$	1.0	1.0	0.1171	0.0681	0.0293	0.1767	Good

* On each case is selected the best transformation type to obtain the greatest $R^2$ evaluating the study variable.

. Coefficients of determination $R^2$ (see Equation (7)) were obtained to assess the fitting of each surrogate model to the corresponding variable in the training dataset. This adjustment is also evaluated for intermediate variables when the corresponding surrogate represents the target variable $z_t$.

Generally speaking, good values in terms of $R^2$ were obtained for most combinations, as Table 7 displays. The analysis showed perfect goodness of fit for the estimations employing radial basis function models and kriging models. However, when such surrogates employ the ratio of condensations $\phi$ between HF and LF models, their fitting to the training dataset deteriorates when the results are translated into values of HF condensation mass flow rate ${\dot{m}}_{cond.,3D}$, which is the target variable. Polynomial Response Surface models also provide excellent (but not perfect) calibrations, except for the ratio of condensations $\phi$.

The ultimate test for the surrogate models is their ability to predict results at points not used for their training, i.e., the verification stage. To do so, several metrics based on the Mean Square Error (MAE) of Equation (8) were identified in Section 3.4, such as the MAE retaining 95% of data with the smallest residuals and the normalized $\widetilde{MAE}$ (Equation (9)) and ${\widetilde{MAE}}_{\sigma }$ (Equation (10)). Finally, Table 7 includes a qualitative assessment of the predictive capability of each metamodel based on the last two indicators, in accordance with the work of Roy et al. [60].

Table 7 shows that 12 combinations of surrogate models and variables provide good predictive capability; 2 obtain a moderate only one presents a lousy performance. The kriging model is the only surrogate that provides evaluation deemed as “good” for all metamodeled variables. Interestingly, at least one MF variable performs better than the target variable for each surrogate model. Still, not all MF variables necessarily provide better agreement than the HF target variable (such as $\phi$ for PRS and $\Delta$ and $\eta$ for Kriging model). This idea was already noted by Fernández-Godino et al. [76], who identify that MF models “may be less accurate than a surrogate built using only HF training data points”.

The results included in Table 7 support the current work of assessing not only MF but also HF intermediate variables. Indeed, the HF intermediate variable $w_{cond.,3D}$ performs better than the HF target variable ${\dot{m}}_{cond.,3D}$ for all surrogate models when applied to the problem of volume condensation due to mixing. Furthermore, $w_{cond.,3D}$ is the best intermediate variable to be surrogated using a Radial Basis Function since it presents the lowest $\widetilde{MAE}$ and ${\widetilde{MAE}}_{\sigma }$ amongst all assessed variables, including the MF ones. For the Kriging model, there is not an MF variable that presents lower values of both $\widetilde{MAE}$ and ${\widetilde{MAE}}_{\sigma }$ than those of $w_{cond.,3D}$, so the best intermediate variable to be metamodeled using a Kriging model will depend on the weighting of such error metrics. Finally, the best overall combination of surrogate and variable is the Polynomial Response Surface of the MF variable $\Delta$, by the values of $\widetilde{MAE}$ and ${\widetilde{MAE}}_{\sigma }$ shown in Table 7.

To visualize the results underlying Table 7 and exemplify the implications of the multi-fidelity proposal, an analysis of the surrogate model based on a polynomial response surface acting over condensation difference (Equation (16)) is conducted since this combination is the one providing the lowest error, as aforementioned. Figure 7 shows the calibration plots for the observed and predicted (metamodeled) values of the intermediate variable $\Delta$, and the equivalent plot in terms of the target variable ${\dot{m}}_{cond.,3D}$. This latter calibration plot has been obtained on the basis of the surrogate of $\Delta$ and the LF model ${\dot{m}}_{cond.,0D}$ by means of Equation (17). Considering the training dataset (green triangles), the surrogate is constructed for the intermediate variable (see Figure 7) with a $R^2\left(z_m\right)$ = 0.95, increasing the correlation up to $R^2\left(z_t\right)$ = 0.98 when representing the target variable. For the verification dataset (blue circles), the error for the difference is much greater than that of the target variable, in which a few points lie outside the depicted 10% dispersion band. A close-up of Figure 7 shows that the performance of the surrogate is less adequate close to the origin. However, this region corresponds to low mass flow rates, so their contribution to the absolute error is low (notice that the normalized $\widetilde{MAE}$ for this combination is below $3.5\%$). Indeed, when applying this MF surrogate model, it is worth reminding that greater impeller damage is produced with greater condensation mass flow rates [70].

Finally, a graphical representation of the relation between the MF variables and the six significant input parameters regarded by ANOVA (see Section 5.2.2) is provided. Figure 8, Figure 9 and Figure 10 show such 3D plots by considering pairs of parameters and leaving the omitted ones at their medium level (see Table 3). These figures represent low-fidelity, intermediate-variable, and high-fidelity features. Due to the negligible computational effort of the LF model employed in this work (see Section 4.2), the surface representation of 0D representations is obtained using an ad-hoc 100 × 100 simulation campaign followed by linear interpolation. Intermediate figures showing differences are the graphical depictions of the corresponding polynomial response surface. Furthermore, Equation (17) allows us to obtain the metamodel of ${\dot{m}}_{cond.,3D}$ using the proposed MF strategy, represented in the predicted condensation. To further assess the method, the actual condensation mass flow rate calculated by the HF 3D CFD simulation is depicted as a small blue sphere in the rightmost subplots of Figure 8, Figure 9 and Figure 10. Comparing this value with the one predicted using the MF PRS surrogate based on $\Delta$, a relative error of $0.23\%$ is obtained for this specific point.

Considering these figures, the similarities between the LF prediction and the prediction of the target variable are likely to be one of the root causes of this combination providing the overall best goodness-of-fit. Indeed, Fernández-Godino et al. [76] considered that a strong correlation between LF and HF data indicates a good performance of the subsequent MF surrogate model. In this case, the metamodeled MF intermediate variable $\Delta$ moderates this relationship by correcting the differences in magnitude.

6. Concluding Remarks

The present work provides a methodological proposal for obtaining and assessing surrogate models for design space approximation. Apart from specific metamodels of the target variable, the usage of intermediate HF variables and MF scaling functions to provide surrogates is addressed systematically and quantitatively for the first time, to the author’s knowledge. Considering the steadily increasing production of new metamodels in digital twins and machine learning frameworks, the proposed method could be useful to discern which combinations of surrogate models and intermediate variables should provide the best results for design space exploration. The developed method is founded upon a screening, training, and verification strategy, reusing the screening dataset for the verification stage to reduce the computational effort. After describing the comprehensive method, it is applied to the complex engineering problem of condensation prediction at three-way junctions. This original example validates the proposed methodology for the case of “real-life” problems compared with the synthetic and benchmark problems often found in the literature.

In this case study, the best reduction of the normalized MAEs has been found by constructing a polynomial response surface of a multi-fidelity additive scaling variable that computes the difference between the low-fidelity and high-fidelity predictions of condensation mass flow rate. These results confirm the utility of low-fidelity models in combination with selected campaigns of HF simulations to evaluate the performance of systems with otherwise highly computationally demanding models. Despite the overall performance of a PRS combination, the radial-basis function, and Kriging models obtained the best agreement for the training datasets, with the latter model providing “good” surrogates for all variables assessed in terms of normalized MAEs. Furthermore, HF intermediate variables (derived from the combination of target variables and model parameters) improve the accuracy of the surrogates built directly from the target variable, which is a benefit without resorting to MF models.

However, these findings are case-specific and cannot be generalized for application to other problems. Indeed, this work proposes a strategy to explore all combinations of surrogate models and intermediate variables (both HF and MF), evaluating them regarding the metamodeling error after the verification stage. For instance, in this work, there are two explored variables ($\frac{\eta }{100}$ and $\phi$) whose summation yields one, yet they behave in a very different way. Therefore, we recommend assessing this double-entry matrix for each studied problem, as the computational effort of conducting simulation campaigns for complex engineering problems is much greater than that of constructing and evaluating surrogates. In future research, the specific algorithms to build the surrogates for intermediate variables in the training datasets could be modified to minimize not only the error of this intermediate variable but also the error in terms of the target variable, which may further improve their results in the verification stage.

Regarding the application of the developed method to the characterization of condensation produced in an LP-EGR junction, the proposed strategy has provided different findings. Both inlet and EGR turbulence intensities as well as outlet pressure were found to have a non-significant effect on condensation for all metamodeled variables assessed. Considering the best combination of surrogate (PRS) and multi-fidelity variable (condensation difference) found in this work, there are even less significant parameters (effect of inlet temperature and relative humidity can be neglected). Therefore, the campaign of 54 HF simulations required to build the surrogate and thus characterize the condensation produced by a certain junction geometry could be further reduced. With such affordable computational effort, several challenges could now be addressed in the framework of internal combustion engine cold start, such as determining the adequate moment to switch between HP and LP EGR modes [62,77] or introducing devices to limit the production or the impact of condensation, which could be a cooler bypass [78] and a water separator [79].