.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/customising_behaviour/plot_modify_layer_sizes.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_customising_behaviour_plot_modify_layer_sizes.py: How to modify fusion model architecture ################################################ This tutorial will show you how to modify the architectures of fusion models. More guidance on what can be modified in each fusion model can be found in the :ref:`modifying-models` section. .. warning:: Some of the fusion models have been designed to work with specific architectures and there are some restrictions on how they can be modified. For example, the channel-wise attention model requires the two modalities to have the same number of layers. Please read the notes section of the fusion model you are interested in to see if there are any restrictions. .. GENERATED FROM PYTHON SOURCE LINES 18-25 Setting up the experiment ------------------------- First, we will set up the experiment by importing the necessary packages, specifying MNIST data paths, and setting the parameters for the experiment. For a more detailed explanation of this process, please see the example tutorials. .. GENERATED FROM PYTHON SOURCE LINES 25-61 .. code-block:: Python # sphinx_gallery_thumbnail_path = '_static/modify_thumbnail.png' import matplotlib.pyplot as plt import os import torch.nn as nn from torch_geometric.nn import GCNConv, ChebConv from fusilli.data import prepare_fusion_data from fusilli.train import train_and_save_models from fusilli.fusionmodels.tabularfusion.attention_weighted_GNN import AttentionWeightedGNN prediction_task = "regression" output_paths = { "checkpoints": "checkpoints", "losses": "loss_logs/modify_layers", "figures": "loss_figures", } for dir in output_paths.values(): os.makedirs(dir, exist_ok=True) # empty the loss log directory (only needed for this tutorial) for dir in os.listdir(output_paths["losses"]): for file in os.listdir(os.path.join(output_paths["losses"], dir)): os.remove(os.path.join(output_paths["losses"], dir, file)) # remove dir os.rmdir(os.path.join(output_paths["losses"], dir)) data_paths = { "tabular1": "../../_static/mnist_data/mnist1_regression.csv", "tabular2": "../../_static/mnist_data/mnist2_regression.csv", "image": "", } .. GENERATED FROM PYTHON SOURCE LINES 62-107 Specifying the model modifications ---------------------------------- Now, we will specify the modifications we want to make to the model. We are using the :class:`~fusilli.fusionmodels.tabularfusion.attention_weighted_GNN.AttentionWeightedGNN` model for this example. This is a graph-based model which has a pretrained MLP (multi-layer perceptron) to get attention weights, and a graph neural network that uses the attention weights to perform the fusion. The following modifications can be made to the method that makes the graph structure: :class:`~fusilli.fusionmodels.tabularfusion.attention_weighted_GNN.AttentionWeightedGraphMaker`: .. list-table:: :widths: 40 60 :header-rows: 1 :stub-columns: 0 * - Attribute - Guidance * - :attr:`~.AttentionWeightedGraphMaker.early_stop_callback` - ``EarlyStopping`` object from ``from lightning.pytorch.callbacks import EarlyStopping`` * - :attr:`~.AttentionWeightedGraphMaker.edge_probability_threshold` - Integer between 0 and 100. * - :attr:`~.AttentionWeightedGraphMaker.attention_MLP_test_size` - Float between 0 and 1. * - :attr:`~.AttentionWeightedGraphMaker.AttentionWeightingMLPInstance.weighting_layers` - ``nn.ModuleDict``: final layer output size must be the same as the input layer input size. * - :attr:`~.AttentionWeightedGraphMaker.AttentionWeightingMLPInstance.fused_layers` - ``nn.Sequential`` The following modifications can be made to the **fusion** model :class:`~fusilli.fusionmodels.tabularfusion.attention_weighted_GNN.AttentionWeightedGNN`: .. list-table:: :widths: 40 60 :header-rows: 1 :stub-columns: 0 * - Attribute - Guidance * - :attr:`~.AttentionWeightedGNN.graph_conv_layers` - ``nn.Sequential`` of ``torch_geometric.nn`` Layers. * - :attr:`~.AttentionWeightedGNN.dropout_prob` - Float between (not including) 0 and 1. Let's modify the model! More info about how to do this can be found in :ref:`modifying-models`. .. GENERATED FROM PYTHON SOURCE LINES 107-141 .. code-block:: Python layer_mods = { "AttentionWeightedGNN": { "graph_conv_layers": nn.Sequential( ChebConv(392, 50, K=3), ChebConv(50, 100, K=3), ChebConv(100, 130, K=3), ), "dropout_prob": 0.4, }, "AttentionWeightedGraphMaker": { "edge_probability_threshold": 80, "attention_MLP_test_size": 0.3, "AttentionWeightingMLPInstance.weighting_layers": nn.ModuleDict( { "Layer 1": nn.Sequential( nn.Linear(784, 500), nn.ReLU()), "Layer 2": nn.Sequential( nn.Linear(500, 128), nn.ReLU()), "Layer 3": nn.Sequential( nn.Linear(128, 128), nn.ReLU()), "Layer 4": nn.Sequential( nn.Linear(128, 500), nn.ReLU()), "Layer 5": nn.Sequential( nn.Linear(500, 784), nn.ReLU()), } )}, } .. GENERATED FROM PYTHON SOURCE LINES 142-144 Loading the data and training the model --------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 144-162 .. code-block:: Python # load data datamodule = prepare_fusion_data(prediction_task=prediction_task, fusion_model=AttentionWeightedGNN, data_paths=data_paths, output_paths=output_paths, layer_mods=layer_mods, max_epochs=5) # train trained_model_list = train_and_save_models( data_module=datamodule, fusion_model=AttentionWeightedGNN, layer_mods=layer_mods, max_epochs=5, ) .. rst-class:: sphx-glr-script-out .. code-block:: none Changed edge_probability_threshold in AttentionWeightedGraphMaker Changed attention_MLP_test_size in AttentionWeightedGraphMaker Changed weighting_layers in AttentionWeightedGraphMaker Checked params in AttentionWeightedGraphMaker Reset fused layers in AttentionWeightedGraphMaker Checked params in AttentionWeightedGraphMaker Training: | | 0/? [00:00` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_modify_layer_sizes.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_modify_layer_sizes.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_