Dependencies

For neural network training:

• numpy
• torch
• dropblock
• optuna
• scipy
• copy
• time
• warnings

For plots/visualization tools:

• matplotlib
• torchsummary
• tabulate

Dataset generation

• To generate datasets for fluid flows using mriFilter and feMorph, visit the README located in ./dataGeneration/.

Pytorch training

In the main directory, there are many files that can be used across 1-D, 2-D, and 3-D datasets.

Networks

Class definitions are found in networks_nd.py. The networks were originally designed to be used in a multitask fashion; therefore, each network has two outputs: segmentation (to which a segmentation activation function is applied) + regression (to which a regression activation function is applied). Note that both of these activation functions are specified by the user, so these outputs do not necessarily need to represent segmentation or regression; the functionality simply allows the user to specify different activation functions for each and different loss functions for each (if desired, the same activation and same loss function could be specified for both). During network evaluation, these two outputs are returned as a tuple (seg_output, reg_output). If using the networks in a single-task fashion, you can simply ignore one of these tuples. e.g. always return zero for the segmenation loss and only consider the second element of the returned tuple for final predictions.

• UpDownNet: base class from which all the other networks inherit. This defines how an encoder block and decoder block is built.
• Encoder (Encoder diagram): basic structure of an encoder network is multiple encoder blocks (each composed of two convolutions, max pooling). The number of these blocks is specified by depth. One encoder block is denoted by green rectangle in image.
• Decoder (Decoder diagram): basic structure of a decoder network is multiple decoder blocks (each composed of two convolutions, upsampling). The number of these blocks is specified by depth. One decoder block is denoted by violet rectangle in image.
• UNet (UNet diagram): similar structure to Ronneberger et al. This is built by an encoder followed by a decoder.
• ONet (ONet diagram): This is essentially built as two separate UNet's.
• XNet (XNet diagram): This is two separate UNet's with an additional skip connection connecting the two UNet's.
• WNet (WNet diagram): One UNet followed by another UNet, where output from first UNet also feeds as input to second UNet.
• To see further details of each network, utilize the show_summary function.
• To create an instance of a network, use the get_model function. This asserts that input parameters are constrained to only that which is implemented by the network classes.

The function get_model accepts a dictionary for the model arguments; the keys for this dictionary include:

• net_type: e.g. options for the networks above are 'encoder', 'decoder', 'unet', 'onet', 'xnet', and 'wnet' respectively.
• depth: the total number of encoder blocks or the total number of decoder blocks. for encoder-decoder networks, the depth is the same for both the encoder and decoder.
• skip_type: concatenation, additive, or no skip connection (as defined as SKIP_CAT, SKIP_ADD, and SKIP_NONE in base_constants_nd.py respectively). Note that encoder or decoder only networks only accept no skip connection.
• num_pixels: size of input will be (*, input_channels,) + (num_pixels, )*dim, e.g. dim equals 2 for the 2d case. input_channels is also a model argument. The size at the first index is determined by the batch size, but this is not needed for network instantiation.
• inner_activation: activation function that is used after every hidden layer. A no-argument function that returns an instance of torch.nn.Module.
• kernel_size: every convolutional layer will create a kernel of size kernel_size^dim
• use_dropblock: dictionary to specify dropblock arguments. e.g. {"mode": True, "ichan": True, "nr_steps": 100, "stop_value": 0.1, "block_size": 3}.
  • mode: where to place dropblock layers. True includes a dropblock layer after every convolutional layer.
  • nr_steps: number of steps in dropblock linear scheduler.
  • stop_value: final drop probability after nr_steps steps.
  • block_size:
  • ichan: True for dropblock masks independent across channels or "shared" for the same mask on all channels.
  • A list of dictionaries can also be utilized to include dropblock only after some layers.

Multifidelity networks

Class definitions are found in multifidelity_networks_nd.py

• MultifidelityNet: base class from which all the other networks inherit. This ultimately inherits from UpDownNet, so creation of encoder/decoder components remain the same for these networks.

• UNet_Implicit: UNet with implicit feedback for low-fidelity outputs (i.e. low-fidelity outputs are not fed back into the network).

• UNet_Explicit: UNet with explicit feedback for low-fidelity outputs (i.e. low-fidelity outputs are fed back into the network).

• Decoder_Implicit: Decoder with implicit feedback for low-fidelity outputs.

• Decoder_Explicit: Decoder with explicit feedback for low-fidelity outputs.

• A forward pass (i.e. evaluation) of these networks will return (seg_output, mf_outs) where mf_outs is a list of the multifidelity regression outputs containing [HF, LF1, LF2, ...] where low fidelity approximants are ordered from highest-resolution to lowest-resolution for both unet and decoder networks.

• To create an instance of a network, use the get_model function. The parameter 'fidelity' determines the feedback functionality, with options 'implicit' or 'explicit'.

• Further network details can be found in [1], [2].

Training

To train a model, the easiest option is to use the wrapper train_model located in model_training_nd.py. Arguments for this function are specified by the Training class, since train_model creates an object of the class Training and calls its train function. For more specific functionality not defined by this class, options include explicitly creating the object and calling whichever methods that are relevant to your application. Otherwise, you could also inherit the Training class to change sections of its functionality.

Visualization

The file visualize_nd.py contains functions which can be used to visualize different components for any of 1-D, 2-D, or 3-D. For example, some of this functionality includes:

• given an input, get the model output with get_model_output
• plotting loss profiles with plot_loss
• calculate accuracy scores with get_iou_score (for segmentation tasks), calculate_rsquared (for regression tasks) and to calculate both with get_accuracy

Miscellaneous utils

The file utils_nd.py has various functions which may be useful for easily running tests. This includes

• waiting until all processes in a list have completed with wait
• converting various data types to a string that can be used in a filename with to_str
• converting the dataset of a torch dataloader to a torch tensor with loader_to_array
• finding previously saved models which match a current test but was completed with fewer epochs in search_lower_epochs_runs (useful for resuming training)
• performing a hyperparameter grid search with grid_search. Multiprocess grid search option is found in grid_search_multicpu or grid_search_multicpu_nosave_nd to complete it without saving any of the final models (may be useful if a user has limited storage). Example usage of these functions can be found in ./twoDnets/01_tests/run_tests.py

Optimization

To go beyond the hyperparameter grid search found in the utils, one can use a more comprehensive search found in optimize_nd.py. Example usage is found in ./twoDnets/01_tests/optimize_2d.py

Documentation specific to 1-D, 2-D, and 3-D

• For 1-D flow dataset training with pytorch, visit the README located in ./oneDnets/.
• For 2-D flow dataset training with pytorch, visit the README located in ./twoDnets/.
• For 3-D flow dataset training with pytorch, visit the README located in ./threeDnets/.

References

[1] Partin, L., Rushdi, A.A., Schiavazzi, D.E., 2022. Multifidelity data fusion in convolutional encoder/decoder assembly networks for computational fluid dynamics, in: AIAA SCITECH 2022 Forum, p. 0803.

[2] Partin, L., Geraci, G., Rushdi, A., Eldred, M., Schiavazzi, D., 2021. Multifidelity data fusion in convolutional encoder/decoder assembly networks for computational fluid dynamics applications, in: Smith, J.D., Galvan, E. (Eds.), Computer Science Research Institute Summer Proceedings 2021, The Computer Science Research Institute at Sandia National Laboratories. pp. 102–119.