fusilli.fusionmodels.tabularimagefusion.concat_img_maps_tabular_data

Concatenating the input data of the first tabular modality and the feature maps of the image modality.

Classes

ConcatImageMapsTabularData(prediction_task, ...)

Concatenating the input data of the first tabular modalities and the feature maps of the image modality.

class ConcatImageMapsTabularData(prediction_task, data_dims, multiclass_dimensions)

Bases: ParentFusionModel, Module

Concatenating the input data of the first tabular modalities and the feature maps of the image modality.

prediction_task

Type of prediction to be performed.

Type:

str

img_layers

Dictionary containing the layers of the image data.

Type:

nn.ModuleDict

fused_layers

Sequential layer containing the fused layers. Calculated in the calc_fused_layers() method.

Type:

nn.Sequential

final_prediction

Sequential layer containing the final prediction layers. The final prediction layers take in the number of features of the fused layers as input. Calculated in the calc_fused_layers() method.

Type:

nn.Sequential

fused_dim

Number of features of the fused layers. Calculated in the calc_fused_layers() method.

Type:

int

__init__(prediction_task, data_dims, multiclass_dimensions)

Parameters:

• prediction_task (str) – Type of prediction to be performed.

• data_dims (list) – List containing the dimensions of the data.

• multiclass_dimensions (int) – Number of classes in the multiclass classification task.

calc_fused_layers()

Calculate the fused layers.

Return type:

None

forward(x1, x2)

Forward pass of the model.

Parameters:

• x1 (torch.Tensor) – Input tensor for the first tabular modality.

• x2 (torch.Tensor) – Input tensor for the image modality.

Returns:

out_pred – Tensor containing the predicted values.

Return type:

torch.Tensor

fusion_type = 'operation'

Type of fusion.

Type:

str

get_fused_dim()

Get the number of features of the fused layers.

Return type:

None

method_name = 'Concatenating tabular data with image feature maps'

Name of the method.

Type:

str

modality_type = 'tabular_image'

Type of modality.

Type:

str