Note

Go to the end to download the full example code.

How to modify fusion model architecture

This tutorial will show you how to modify the architectures of fusion models.

More guidance on what can be modified in each fusion model can be found in the Modify the Fusion Models section.

Warning

Some of the fusion models have been designed to work with specific architectures and there are some restrictions on how they can be modified.

For example, the channel-wise attention model requires the two modalities to have the same number of layers. Please read the notes section of the fusion model you are interested in to see if there are any restrictions.

Setting up the experiment

First, we will set up the experiment by importing the necessary packages, specifying MNIST data paths, and setting the parameters for the experiment.

For a more detailed explanation of this process, please see the example tutorials.

# sphinx_gallery_thumbnail_path = '_static/modify_thumbnail.png'
import matplotlib.pyplot as plt
import os
import torch.nn as nn
from torch_geometric.nn import GCNConv, ChebConv

from fusilli.data import prepare_fusion_data
from fusilli.train import train_and_save_models

from fusilli.fusionmodels.tabularfusion.attention_weighted_GNN import AttentionWeightedGNN

prediction_task = "regression"

output_paths = {
    "checkpoints": "checkpoints",
    "losses": "loss_logs/modify_layers",
    "figures": "loss_figures",
}

for dir in output_paths.values():
    os.makedirs(dir, exist_ok=True)

# empty the loss log directory (only needed for this tutorial)
for dir in os.listdir(output_paths["losses"]):
    for file in os.listdir(os.path.join(output_paths["losses"], dir)):
        os.remove(os.path.join(output_paths["losses"], dir, file))
    # remove dir
    os.rmdir(os.path.join(output_paths["losses"], dir))

data_paths = {
    "tabular1": "../../_static/mnist_data/mnist1_regression.csv",
    "tabular2": "../../_static/mnist_data/mnist2_regression.csv",
    "image": "",
}

Specifying the model modifications

Now, we will specify the modifications we want to make to the model.

We are using the AttentionWeightedGNN model for this example. This is a graph-based model which has a pretrained MLP (multi-layer perceptron) to get attention weights, and a graph neural network that uses the attention weights to perform the fusion.

The following modifications can be made to the method that makes the graph structure: AttentionWeightedGraphMaker:

Attribute	Guidance
`early_stop_callback`	`EarlyStopping` object from `from lightning.pytorch.callbacks import EarlyStopping`
`edge_probability_threshold`	Integer between 0 and 100.
`attention_MLP_test_size`	Float between 0 and 1.
`weighting_layers`	`nn.ModuleDict`: final layer output size must be the same as the input layer input size.
`fused_layers`	`nn.Sequential`

The following modifications can be made to the fusion model AttentionWeightedGNN:

Attribute	Guidance
`graph_conv_layers`	`nn.Sequential` of `torch_geometric.nn` Layers.
`dropout_prob`	Float between (not including) 0 and 1.

Let’s modify the model! More info about how to do this can be found in Modify the Fusion Models.

layer_mods = {
    "AttentionWeightedGNN": {
        "graph_conv_layers": nn.Sequential(
            ChebConv(392, 50, K=3),
            ChebConv(50, 100, K=3),
            ChebConv(100, 130, K=3),
        ),
        "dropout_prob": 0.4,
    },
    "AttentionWeightedGraphMaker": {
        "edge_probability_threshold": 80,
        "attention_MLP_test_size": 0.3,
        "AttentionWeightingMLPInstance.weighting_layers": nn.ModuleDict(
            {
                "Layer 1": nn.Sequential(
                    nn.Linear(784, 500),
                    nn.ReLU()),
                "Layer 2": nn.Sequential(
                    nn.Linear(500, 128),
                    nn.ReLU()),
                "Layer 3": nn.Sequential(
                    nn.Linear(128, 128),
                    nn.ReLU()),
                "Layer 4": nn.Sequential(
                    nn.Linear(128, 500),
                    nn.ReLU()),
                "Layer 5": nn.Sequential(
                    nn.Linear(500, 784),
                    nn.ReLU()),
            }
        )},
}

Loading the data and training the model

# load data
datamodule = prepare_fusion_data(prediction_task=prediction_task,
                                 fusion_model=AttentionWeightedGNN,
                                 data_paths=data_paths,
                                 output_paths=output_paths,
                                 layer_mods=layer_mods,
                                 max_epochs=5)

# train
trained_model_list = train_and_save_models(
    data_module=datamodule,
    fusion_model=AttentionWeightedGNN,
    layer_mods=layer_mods,
    max_epochs=5,
)

Changed edge_probability_threshold in AttentionWeightedGraphMaker
Changed attention_MLP_test_size in AttentionWeightedGraphMaker
Changed weighting_layers in AttentionWeightedGraphMaker
Checked params in AttentionWeightedGraphMaker
Reset fused layers in AttentionWeightedGraphMaker
Checked params in AttentionWeightedGraphMaker

Training: |          | 0/? [00:00<?, ?it/s]
Training:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 0:   9%|▉         | 1/11 [00:00<00:00, 33.21it/s]
Epoch 0:   9%|▉         | 1/11 [00:00<00:00, 32.79it/s]
Epoch 0:  18%|█▊        | 2/11 [00:00<00:00, 43.14it/s]
Epoch 0:  18%|█▊        | 2/11 [00:00<00:00, 42.81it/s]
Epoch 0:  27%|██▋       | 3/11 [00:00<00:00, 48.13it/s]
Epoch 0:  27%|██▋       | 3/11 [00:00<00:00, 47.86it/s]
Epoch 0:  36%|███▋      | 4/11 [00:00<00:00, 51.53it/s]
Epoch 0:  36%|███▋      | 4/11 [00:00<00:00, 51.30it/s]
Epoch 0:  45%|████▌     | 5/11 [00:00<00:00, 52.49it/s]
Epoch 0:  45%|████▌     | 5/11 [00:00<00:00, 52.28it/s]
Epoch 0:  55%|█████▍    | 6/11 [00:00<00:00, 54.68it/s]
Epoch 0:  55%|█████▍    | 6/11 [00:00<00:00, 54.52it/s]
Epoch 0:  64%|██████▎   | 7/11 [00:00<00:00, 55.04it/s]
Epoch 0:  64%|██████▎   | 7/11 [00:00<00:00, 54.86it/s]
Epoch 0:  73%|███████▎  | 8/11 [00:00<00:00, 56.20it/s]
Epoch 0:  73%|███████▎  | 8/11 [00:00<00:00, 56.06it/s]
Epoch 0:  82%|████████▏ | 9/11 [00:00<00:00, 56.27it/s]
Epoch 0:  82%|████████▏ | 9/11 [00:00<00:00, 56.15it/s]
Epoch 0:  91%|█████████ | 10/11 [00:00<00:00, 57.00it/s]
Epoch 0:  91%|█████████ | 10/11 [00:00<00:00, 56.89it/s]
Epoch 0: 100%|██████████| 11/11 [00:00<00:00, 57.85it/s]
Epoch 0: 100%|██████████| 11/11 [00:00<00:00, 57.74it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 271.79it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 295.12it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 310.60it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 319.72it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 330.57it/s]


Epoch 0: 100%|██████████| 11/11 [00:00<00:00, 52.42it/s]
Epoch 0: 100%|██████████| 11/11 [00:00<00:00, 52.30it/s]
Epoch 0:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 1:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 1:   9%|▉         | 1/11 [00:00<00:00, 59.12it/s]
Epoch 1:   9%|▉         | 1/11 [00:00<00:00, 58.02it/s]
Epoch 1:  18%|█▊        | 2/11 [00:00<00:00, 63.76it/s]
Epoch 1:  18%|█▊        | 2/11 [00:00<00:00, 63.12it/s]
Epoch 1:  27%|██▋       | 3/11 [00:00<00:00, 65.97it/s]
Epoch 1:  27%|██▋       | 3/11 [00:00<00:00, 65.51it/s]
Epoch 1:  36%|███▋      | 4/11 [00:00<00:00, 66.90it/s]
Epoch 1:  36%|███▋      | 4/11 [00:00<00:00, 66.53it/s]
Epoch 1:  45%|████▌     | 5/11 [00:00<00:00, 67.43it/s]
Epoch 1:  45%|████▌     | 5/11 [00:00<00:00, 67.13it/s]
Epoch 1:  55%|█████▍    | 6/11 [00:00<00:00, 67.67it/s]
Epoch 1:  55%|█████▍    | 6/11 [00:00<00:00, 67.41it/s]
Epoch 1:  64%|██████▎   | 7/11 [00:00<00:00, 67.72it/s]
Epoch 1:  64%|██████▎   | 7/11 [00:00<00:00, 67.49it/s]
Epoch 1:  73%|███████▎  | 8/11 [00:00<00:00, 67.51it/s]
Epoch 1:  73%|███████▎  | 8/11 [00:00<00:00, 67.31it/s]
Epoch 1:  82%|████████▏ | 9/11 [00:00<00:00, 67.36it/s]
Epoch 1:  82%|████████▏ | 9/11 [00:00<00:00, 67.18it/s]
Epoch 1:  91%|█████████ | 10/11 [00:00<00:00, 67.28it/s]
Epoch 1:  91%|█████████ | 10/11 [00:00<00:00, 67.13it/s]
Epoch 1: 100%|██████████| 11/11 [00:00<00:00, 67.27it/s]
Epoch 1: 100%|██████████| 11/11 [00:00<00:00, 67.12it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 360.71it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 340.57it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 342.55it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.48it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.26it/s]


Epoch 1: 100%|██████████| 11/11 [00:00<00:00, 60.57it/s]
Epoch 1: 100%|██████████| 11/11 [00:00<00:00, 60.42it/s]
Epoch 1:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 2:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 2:   9%|▉         | 1/11 [00:00<00:00, 68.22it/s]
Epoch 2:   9%|▉         | 1/11 [00:00<00:00, 66.69it/s]
Epoch 2:  18%|█▊        | 2/11 [00:00<00:00, 68.13it/s]
Epoch 2:  18%|█▊        | 2/11 [00:00<00:00, 67.37it/s]
Epoch 2:  27%|██▋       | 3/11 [00:00<00:00, 68.12it/s]
Epoch 2:  27%|██▋       | 3/11 [00:00<00:00, 67.59it/s]
Epoch 2:  36%|███▋      | 4/11 [00:00<00:00, 67.56it/s]
Epoch 2:  36%|███▋      | 4/11 [00:00<00:00, 67.17it/s]
Epoch 2:  45%|████▌     | 5/11 [00:00<00:00, 67.52it/s]
Epoch 2:  45%|████▌     | 5/11 [00:00<00:00, 67.20it/s]
Epoch 2:  55%|█████▍    | 6/11 [00:00<00:00, 67.66it/s]
Epoch 2:  55%|█████▍    | 6/11 [00:00<00:00, 67.41it/s]
Epoch 2:  64%|██████▎   | 7/11 [00:00<00:00, 68.01it/s]
Epoch 2:  64%|██████▎   | 7/11 [00:00<00:00, 67.80it/s]
Epoch 2:  73%|███████▎  | 8/11 [00:00<00:00, 68.39it/s]
Epoch 2:  73%|███████▎  | 8/11 [00:00<00:00, 68.20it/s]
Epoch 2:  82%|████████▏ | 9/11 [00:00<00:00, 68.59it/s]
Epoch 2:  82%|████████▏ | 9/11 [00:00<00:00, 68.41it/s]
Epoch 2:  91%|█████████ | 10/11 [00:00<00:00, 68.53it/s]
Epoch 2:  91%|█████████ | 10/11 [00:00<00:00, 68.37it/s]
Epoch 2: 100%|██████████| 11/11 [00:00<00:00, 68.19it/s]
Epoch 2: 100%|██████████| 11/11 [00:00<00:00, 68.05it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 364.69it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 345.21it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 346.77it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 348.39it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 355.96it/s]


Epoch 2: 100%|██████████| 11/11 [00:00<00:00, 61.32it/s]
Epoch 2: 100%|██████████| 11/11 [00:00<00:00, 61.16it/s]
Epoch 2:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 3:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 3:   9%|▉         | 1/11 [00:00<00:00, 63.33it/s]
Epoch 3:   9%|▉         | 1/11 [00:00<00:00, 62.09it/s]
Epoch 3:  18%|█▊        | 2/11 [00:00<00:00, 64.25it/s]
Epoch 3:  18%|█▊        | 2/11 [00:00<00:00, 63.55it/s]
Epoch 3:  27%|██▋       | 3/11 [00:00<00:00, 64.34it/s]
Epoch 3:  27%|██▋       | 3/11 [00:00<00:00, 63.88it/s]
Epoch 3:  36%|███▋      | 4/11 [00:00<00:00, 64.92it/s]
Epoch 3:  36%|███▋      | 4/11 [00:00<00:00, 64.56it/s]
Epoch 3:  45%|████▌     | 5/11 [00:00<00:00, 65.40it/s]
Epoch 3:  45%|████▌     | 5/11 [00:00<00:00, 65.11it/s]
Epoch 3:  55%|█████▍    | 6/11 [00:00<00:00, 65.65it/s]
Epoch 3:  55%|█████▍    | 6/11 [00:00<00:00, 65.40it/s]
Epoch 3:  64%|██████▎   | 7/11 [00:00<00:00, 65.64it/s]
Epoch 3:  64%|██████▎   | 7/11 [00:00<00:00, 65.41it/s]
Epoch 3:  73%|███████▎  | 8/11 [00:00<00:00, 65.48it/s]
Epoch 3:  73%|███████▎  | 8/11 [00:00<00:00, 65.28it/s]
Epoch 3:  82%|████████▏ | 9/11 [00:00<00:00, 65.33it/s]
Epoch 3:  82%|████████▏ | 9/11 [00:00<00:00, 65.15it/s]
Epoch 3:  91%|█████████ | 10/11 [00:00<00:00, 65.21it/s]
Epoch 3:  91%|█████████ | 10/11 [00:00<00:00, 65.04it/s]
Epoch 3: 100%|██████████| 11/11 [00:00<00:00, 65.17it/s]
Epoch 3: 100%|██████████| 11/11 [00:00<00:00, 65.03it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 353.77it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 336.50it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 340.88it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.25it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 350.86it/s]


Epoch 3: 100%|██████████| 11/11 [00:00<00:00, 58.86it/s]
Epoch 3: 100%|██████████| 11/11 [00:00<00:00, 58.71it/s]
Epoch 3:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 4:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 4:   9%|▉         | 1/11 [00:00<00:00, 70.16it/s]
Epoch 4:   9%|▉         | 1/11 [00:00<00:00, 68.65it/s]
Epoch 4:  18%|█▊        | 2/11 [00:00<00:00, 67.24it/s]
Epoch 4:  18%|█▊        | 2/11 [00:00<00:00, 66.47it/s]
Epoch 4:  27%|██▋       | 3/11 [00:00<00:00, 65.90it/s]
Epoch 4:  27%|██▋       | 3/11 [00:00<00:00, 65.31it/s]
Epoch 4:  36%|███▋      | 4/11 [00:00<00:00, 61.41it/s]
Epoch 4:  36%|███▋      | 4/11 [00:00<00:00, 61.06it/s]
Epoch 4:  45%|████▌     | 5/11 [00:00<00:00, 61.43it/s]
Epoch 4:  45%|████▌     | 5/11 [00:00<00:00, 61.12it/s]
Epoch 4:  55%|█████▍    | 6/11 [00:00<00:00, 60.02it/s]
Epoch 4:  55%|█████▍    | 6/11 [00:00<00:00, 59.81it/s]
Epoch 4:  64%|██████▎   | 7/11 [00:00<00:00, 60.80it/s]
Epoch 4:  64%|██████▎   | 7/11 [00:00<00:00, 60.62it/s]
Epoch 4:  73%|███████▎  | 8/11 [00:00<00:00, 60.83it/s]
Epoch 4:  73%|███████▎  | 8/11 [00:00<00:00, 60.67it/s]
Epoch 4:  82%|████████▏ | 9/11 [00:00<00:00, 61.49it/s]
Epoch 4:  82%|████████▏ | 9/11 [00:00<00:00, 61.34it/s]
Epoch 4:  91%|█████████ | 10/11 [00:00<00:00, 61.72it/s]
Epoch 4:  91%|█████████ | 10/11 [00:00<00:00, 61.58it/s]
Epoch 4: 100%|██████████| 11/11 [00:00<00:00, 61.90it/s]
Epoch 4: 100%|██████████| 11/11 [00:00<00:00, 61.76it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 357.81it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.44it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 329.62it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 335.98it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 345.58it/s]


Epoch 4: 100%|██████████| 11/11 [00:00<00:00, 56.12it/s]
Epoch 4: 100%|██████████| 11/11 [00:00<00:00, 55.99it/s]
Epoch 4:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 5:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 5:   9%|▉         | 1/11 [00:00<00:00, 66.47it/s]
Epoch 5:   9%|▉         | 1/11 [00:00<00:00, 64.96it/s]
Epoch 5:  18%|█▊        | 2/11 [00:00<00:00, 66.05it/s]
Epoch 5:  18%|█▊        | 2/11 [00:00<00:00, 65.33it/s]
Epoch 5:  27%|██▋       | 3/11 [00:00<00:00, 66.06it/s]
Epoch 5:  27%|██▋       | 3/11 [00:00<00:00, 65.54it/s]
Epoch 5:  36%|███▋      | 4/11 [00:00<00:00, 66.31it/s]
Epoch 5:  36%|███▋      | 4/11 [00:00<00:00, 65.95it/s]
Epoch 5:  45%|████▌     | 5/11 [00:00<00:00, 66.41it/s]
Epoch 5:  45%|████▌     | 5/11 [00:00<00:00, 66.10it/s]
Epoch 5:  55%|█████▍    | 6/11 [00:00<00:00, 66.27it/s]
Epoch 5:  55%|█████▍    | 6/11 [00:00<00:00, 66.01it/s]
Epoch 5:  64%|██████▎   | 7/11 [00:00<00:00, 65.94it/s]
Epoch 5:  64%|██████▎   | 7/11 [00:00<00:00, 65.72it/s]
Epoch 5:  73%|███████▎  | 8/11 [00:00<00:00, 65.65it/s]
Epoch 5:  73%|███████▎  | 8/11 [00:00<00:00, 65.45it/s]
Epoch 5:  82%|████████▏ | 9/11 [00:00<00:00, 65.46it/s]
Epoch 5:  82%|████████▏ | 9/11 [00:00<00:00, 65.28it/s]
Epoch 5:  91%|█████████ | 10/11 [00:00<00:00, 65.39it/s]
Epoch 5:  91%|█████████ | 10/11 [00:00<00:00, 65.23it/s]
Epoch 5: 100%|██████████| 11/11 [00:00<00:00, 65.38it/s]
Epoch 5: 100%|██████████| 11/11 [00:00<00:00, 65.24it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 358.98it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 338.33it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 339.20it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 340.21it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 348.82it/s]


Epoch 5: 100%|██████████| 11/11 [00:00<00:00, 59.01it/s]
Epoch 5: 100%|██████████| 11/11 [00:00<00:00, 58.86it/s]
Epoch 5:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 6:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 6:   9%|▉         | 1/11 [00:00<00:00, 68.22it/s]
Epoch 6:   9%|▉         | 1/11 [00:00<00:00, 66.56it/s]
Epoch 6:  18%|█▊        | 2/11 [00:00<00:00, 65.30it/s]
Epoch 6:  18%|█▊        | 2/11 [00:00<00:00, 64.50it/s]
Epoch 6:  27%|██▋       | 3/11 [00:00<00:00, 64.65it/s]
Epoch 6:  27%|██▋       | 3/11 [00:00<00:00, 64.15it/s]
Epoch 6:  36%|███▋      | 4/11 [00:00<00:00, 64.50it/s]
Epoch 6:  36%|███▋      | 4/11 [00:00<00:00, 64.12it/s]
Epoch 6:  45%|████▌     | 5/11 [00:00<00:00, 64.09it/s]
Epoch 6:  45%|████▌     | 5/11 [00:00<00:00, 63.71it/s]
Epoch 6:  55%|█████▍    | 6/11 [00:00<00:00, 63.57it/s]
Epoch 6:  55%|█████▍    | 6/11 [00:00<00:00, 63.32it/s]
Epoch 6:  64%|██████▎   | 7/11 [00:00<00:00, 63.24it/s]
Epoch 6:  64%|██████▎   | 7/11 [00:00<00:00, 63.02it/s]
Epoch 6:  73%|███████▎  | 8/11 [00:00<00:00, 63.10it/s]
Epoch 6:  73%|███████▎  | 8/11 [00:00<00:00, 62.88it/s]
Epoch 6:  82%|████████▏ | 9/11 [00:00<00:00, 63.43it/s]
Epoch 6:  82%|████████▏ | 9/11 [00:00<00:00, 63.28it/s]
Epoch 6:  91%|█████████ | 10/11 [00:00<00:00, 64.03it/s]
Epoch 6:  91%|█████████ | 10/11 [00:00<00:00, 63.90it/s]
Epoch 6: 100%|██████████| 11/11 [00:00<00:00, 64.49it/s]
Epoch 6: 100%|██████████| 11/11 [00:00<00:00, 64.37it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 389.41it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 359.52it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 348.44it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 335.66it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 341.63it/s]


Epoch 6: 100%|██████████| 11/11 [00:00<00:00, 58.27it/s]
Epoch 6: 100%|██████████| 11/11 [00:00<00:00, 58.12it/s]
Epoch 6:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 7:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 7:   9%|▉         | 1/11 [00:00<00:00, 66.00it/s]
Epoch 7:   9%|▉         | 1/11 [00:00<00:00, 64.46it/s]
Epoch 7:  18%|█▊        | 2/11 [00:00<00:00, 62.99it/s]
Epoch 7:  18%|█▊        | 2/11 [00:00<00:00, 62.28it/s]
Epoch 7:  27%|██▋       | 3/11 [00:00<00:00, 63.86it/s]
Epoch 7:  27%|██▋       | 3/11 [00:00<00:00, 63.39it/s]
Epoch 7:  36%|███▋      | 4/11 [00:00<00:00, 61.84it/s]
Epoch 7:  36%|███▋      | 4/11 [00:00<00:00, 61.48it/s]
Epoch 7:  45%|████▌     | 5/11 [00:00<00:00, 61.64it/s]
Epoch 7:  45%|████▌     | 5/11 [00:00<00:00, 61.34it/s]
Epoch 7:  55%|█████▍    | 6/11 [00:00<00:00, 61.93it/s]
Epoch 7:  55%|█████▍    | 6/11 [00:00<00:00, 61.69it/s]
Epoch 7:  64%|██████▎   | 7/11 [00:00<00:00, 60.81it/s]
Epoch 7:  64%|██████▎   | 7/11 [00:00<00:00, 60.61it/s]
Epoch 7:  73%|███████▎  | 8/11 [00:00<00:00, 61.43it/s]
Epoch 7:  73%|███████▎  | 8/11 [00:00<00:00, 61.27it/s]
Epoch 7:  82%|████████▏ | 9/11 [00:00<00:00, 61.92it/s]
Epoch 7:  82%|████████▏ | 9/11 [00:00<00:00, 61.78it/s]
Epoch 7:  91%|█████████ | 10/11 [00:00<00:00, 62.35it/s]
Epoch 7:  91%|█████████ | 10/11 [00:00<00:00, 62.22it/s]
Epoch 7: 100%|██████████| 11/11 [00:00<00:00, 62.69it/s]
Epoch 7: 100%|██████████| 11/11 [00:00<00:00, 62.51it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 352.40it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 317.23it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 322.75it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 329.02it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 334.58it/s]


Epoch 7: 100%|██████████| 11/11 [00:00<00:00, 56.58it/s]
Epoch 7: 100%|██████████| 11/11 [00:00<00:00, 56.45it/s]
Epoch 7:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 8:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 8:   9%|▉         | 1/11 [00:00<00:00, 66.27it/s]
Epoch 8:   9%|▉         | 1/11 [00:00<00:00, 64.85it/s]
Epoch 8:  18%|█▊        | 2/11 [00:00<00:00, 64.71it/s]
Epoch 8:  18%|█▊        | 2/11 [00:00<00:00, 64.01it/s]
Epoch 8:  27%|██▋       | 3/11 [00:00<00:00, 66.01it/s]
Epoch 8:  27%|██▋       | 3/11 [00:00<00:00, 65.55it/s]
Epoch 8:  36%|███▋      | 4/11 [00:00<00:00, 64.59it/s]
Epoch 8:  36%|███▋      | 4/11 [00:00<00:00, 64.26it/s]
Epoch 8:  45%|████▌     | 5/11 [00:00<00:00, 64.87it/s]
Epoch 8:  45%|████▌     | 5/11 [00:00<00:00, 64.59it/s]
Epoch 8:  55%|█████▍    | 6/11 [00:00<00:00, 64.97it/s]
Epoch 8:  55%|█████▍    | 6/11 [00:00<00:00, 64.71it/s]
Epoch 8:  64%|██████▎   | 7/11 [00:00<00:00, 64.95it/s]
Epoch 8:  64%|██████▎   | 7/11 [00:00<00:00, 64.73it/s]
Epoch 8:  73%|███████▎  | 8/11 [00:00<00:00, 65.33it/s]
Epoch 8:  73%|███████▎  | 8/11 [00:00<00:00, 65.15it/s]
Epoch 8:  82%|████████▏ | 9/11 [00:00<00:00, 64.59it/s]
Epoch 8:  82%|████████▏ | 9/11 [00:00<00:00, 64.43it/s]
Epoch 8:  91%|█████████ | 10/11 [00:00<00:00, 65.13it/s]
Epoch 8:  91%|█████████ | 10/11 [00:00<00:00, 64.99it/s]
Epoch 8: 100%|██████████| 11/11 [00:00<00:00, 65.55it/s]
Epoch 8: 100%|██████████| 11/11 [00:00<00:00, 65.42it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 385.40it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 361.67it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 357.36it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 355.05it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 361.37it/s]


Epoch 8: 100%|██████████| 11/11 [00:00<00:00, 59.37it/s]
Epoch 8: 100%|██████████| 11/11 [00:00<00:00, 59.22it/s]
Epoch 8:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 9:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 9:   9%|▉         | 1/11 [00:00<00:00, 68.51it/s]
Epoch 9:   9%|▉         | 1/11 [00:00<00:00, 66.90it/s]
Epoch 9:  18%|█▊        | 2/11 [00:00<00:00, 67.73it/s]
Epoch 9:  18%|█▊        | 2/11 [00:00<00:00, 66.96it/s]
Epoch 9:  27%|██▋       | 3/11 [00:00<00:00, 67.97it/s]
Epoch 9:  27%|██▋       | 3/11 [00:00<00:00, 67.46it/s]
Epoch 9:  36%|███▋      | 4/11 [00:00<00:00, 67.70it/s]
Epoch 9:  36%|███▋      | 4/11 [00:00<00:00, 67.32it/s]
Epoch 9:  45%|████▌     | 5/11 [00:00<00:00, 68.17it/s]
Epoch 9:  45%|████▌     | 5/11 [00:00<00:00, 67.87it/s]
Epoch 9:  55%|█████▍    | 6/11 [00:00<00:00, 67.94it/s]
Epoch 9:  55%|█████▍    | 6/11 [00:00<00:00, 67.66it/s]
Epoch 9:  64%|██████▎   | 7/11 [00:00<00:00, 67.43it/s]
Epoch 9:  64%|██████▎   | 7/11 [00:00<00:00, 67.20it/s]
Epoch 9:  73%|███████▎  | 8/11 [00:00<00:00, 67.51it/s]
Epoch 9:  73%|███████▎  | 8/11 [00:00<00:00, 67.32it/s]
Epoch 9:  82%|████████▏ | 9/11 [00:00<00:00, 67.84it/s]
Epoch 9:  82%|████████▏ | 9/11 [00:00<00:00, 67.68it/s]
Epoch 9:  91%|█████████ | 10/11 [00:00<00:00, 68.02it/s]
Epoch 9:  91%|█████████ | 10/11 [00:00<00:00, 67.87it/s]
Epoch 9: 100%|██████████| 11/11 [00:00<00:00, 68.17it/s]
Epoch 9: 100%|██████████| 11/11 [00:00<00:00, 68.03it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 349.79it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 336.58it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 340.57it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.08it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.62it/s]


Epoch 9: 100%|██████████| 11/11 [00:00<00:00, 61.32it/s]
Epoch 9: 100%|██████████| 11/11 [00:00<00:00, 61.17it/s]
Epoch 9:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 10:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 10:   9%|▉         | 1/11 [00:00<00:00, 68.49it/s]
Epoch 10:   9%|▉         | 1/11 [00:00<00:00, 66.89it/s]
Epoch 10:  18%|█▊        | 2/11 [00:00<00:00, 67.20it/s]
Epoch 10:  18%|█▊        | 2/11 [00:00<00:00, 66.30it/s]
Epoch 10:  27%|██▋       | 3/11 [00:00<00:00, 66.19it/s]
Epoch 10:  27%|██▋       | 3/11 [00:00<00:00, 65.64it/s]
Epoch 10:  36%|███▋      | 4/11 [00:00<00:00, 64.83it/s]
Epoch 10:  36%|███▋      | 4/11 [00:00<00:00, 64.40it/s]
Epoch 10:  45%|████▌     | 5/11 [00:00<00:00, 65.02it/s]
Epoch 10:  45%|████▌     | 5/11 [00:00<00:00, 64.72it/s]
Epoch 10:  55%|█████▍    | 6/11 [00:00<00:00, 65.17it/s]
Epoch 10:  55%|█████▍    | 6/11 [00:00<00:00, 64.91it/s]
Epoch 10:  64%|██████▎   | 7/11 [00:00<00:00, 64.86it/s]
Epoch 10:  64%|██████▎   | 7/11 [00:00<00:00, 64.64it/s]
Epoch 10:  73%|███████▎  | 8/11 [00:00<00:00, 64.67it/s]
Epoch 10:  73%|███████▎  | 8/11 [00:00<00:00, 64.48it/s]
Epoch 10:  82%|████████▏ | 9/11 [00:00<00:00, 64.52it/s]
Epoch 10:  82%|████████▏ | 9/11 [00:00<00:00, 64.35it/s]
Epoch 10:  91%|█████████ | 10/11 [00:00<00:00, 64.64it/s]
Epoch 10:  91%|█████████ | 10/11 [00:00<00:00, 64.49it/s]
Epoch 10: 100%|██████████| 11/11 [00:00<00:00, 64.75it/s]
Epoch 10: 100%|██████████| 11/11 [00:00<00:00, 64.61it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 360.89it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 334.25it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 333.87it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 337.79it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 346.57it/s]


Epoch 10: 100%|██████████| 11/11 [00:00<00:00, 58.48it/s]
Epoch 10: 100%|██████████| 11/11 [00:00<00:00, 58.34it/s]
Epoch 10:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 11:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 11:   9%|▉         | 1/11 [00:00<00:00, 68.86it/s]
Epoch 11:   9%|▉         | 1/11 [00:00<00:00, 67.30it/s]
Epoch 11:  18%|█▊        | 2/11 [00:00<00:00, 62.61it/s]
Epoch 11:  18%|█▊        | 2/11 [00:00<00:00, 61.89it/s]
Epoch 11:  27%|██▋       | 3/11 [00:00<00:00, 63.62it/s]
Epoch 11:  27%|██▋       | 3/11 [00:00<00:00, 63.13it/s]
Epoch 11:  36%|███▋      | 4/11 [00:00<00:00, 63.84it/s]
Epoch 11:  36%|███▋      | 4/11 [00:00<00:00, 63.46it/s]
Epoch 11:  45%|████▌     | 5/11 [00:00<00:00, 64.05it/s]
Epoch 11:  45%|████▌     | 5/11 [00:00<00:00, 63.76it/s]
Epoch 11:  55%|█████▍    | 6/11 [00:00<00:00, 63.80it/s]
Epoch 11:  55%|█████▍    | 6/11 [00:00<00:00, 63.55it/s]
Epoch 11:  64%|██████▎   | 7/11 [00:00<00:00, 63.97it/s]
Epoch 11:  64%|██████▎   | 7/11 [00:00<00:00, 63.76it/s]
Epoch 11:  73%|███████▎  | 8/11 [00:00<00:00, 62.96it/s]
Epoch 11:  73%|███████▎  | 8/11 [00:00<00:00, 62.77it/s]
Epoch 11:  82%|████████▏ | 9/11 [00:00<00:00, 62.90it/s]
Epoch 11:  82%|████████▏ | 9/11 [00:00<00:00, 62.74it/s]
Epoch 11:  91%|█████████ | 10/11 [00:00<00:00, 63.18it/s]
Epoch 11:  91%|█████████ | 10/11 [00:00<00:00, 63.03it/s]
Epoch 11: 100%|██████████| 11/11 [00:00<00:00, 62.78it/s]
Epoch 11: 100%|██████████| 11/11 [00:00<00:00, 62.65it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 361.80it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 345.81it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 345.13it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 345.83it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 355.07it/s]


Epoch 11: 100%|██████████| 11/11 [00:00<00:00, 56.95it/s]
Epoch 11: 100%|██████████| 11/11 [00:00<00:00, 56.81it/s]
Epoch 11:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 12:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 12:   9%|▉         | 1/11 [00:00<00:00, 68.77it/s]
Epoch 12:   9%|▉         | 1/11 [00:00<00:00, 66.96it/s]
Epoch 12:  18%|█▊        | 2/11 [00:00<00:00, 65.33it/s]
Epoch 12:  18%|█▊        | 2/11 [00:00<00:00, 64.51it/s]
Epoch 12:  27%|██▋       | 3/11 [00:00<00:00, 65.55it/s]
Epoch 12:  27%|██▋       | 3/11 [00:00<00:00, 65.02it/s]
Epoch 12:  36%|███▋      | 4/11 [00:00<00:00, 65.17it/s]
Epoch 12:  36%|███▋      | 4/11 [00:00<00:00, 64.78it/s]
Epoch 12:  45%|████▌     | 5/11 [00:00<00:00, 65.29it/s]
Epoch 12:  45%|████▌     | 5/11 [00:00<00:00, 64.98it/s]
Epoch 12:  55%|█████▍    | 6/11 [00:00<00:00, 65.63it/s]
Epoch 12:  55%|█████▍    | 6/11 [00:00<00:00, 65.39it/s]
Epoch 12:  64%|██████▎   | 7/11 [00:00<00:00, 66.04it/s]
Epoch 12:  64%|██████▎   | 7/11 [00:00<00:00, 65.82it/s]
Epoch 12:  73%|███████▎  | 8/11 [00:00<00:00, 65.98it/s]
Epoch 12:  73%|███████▎  | 8/11 [00:00<00:00, 65.79it/s]
Epoch 12:  82%|████████▏ | 9/11 [00:00<00:00, 66.09it/s]
Epoch 12:  82%|████████▏ | 9/11 [00:00<00:00, 65.92it/s]
Epoch 12:  91%|█████████ | 10/11 [00:00<00:00, 66.00it/s]
Epoch 12:  91%|█████████ | 10/11 [00:00<00:00, 65.83it/s]
Epoch 12: 100%|██████████| 11/11 [00:00<00:00, 65.78it/s]
Epoch 12: 100%|██████████| 11/11 [00:00<00:00, 65.64it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 354.94it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 337.05it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 336.96it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 341.80it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 350.49it/s]


Epoch 12: 100%|██████████| 11/11 [00:00<00:00, 59.36it/s]
Epoch 12: 100%|██████████| 11/11 [00:00<00:00, 59.22it/s]
Epoch 12:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 13:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 13:   9%|▉         | 1/11 [00:00<00:00, 69.24it/s]
Epoch 13:   9%|▉         | 1/11 [00:00<00:00, 67.72it/s]
Epoch 13:  18%|█▊        | 2/11 [00:00<00:00, 68.49it/s]
Epoch 13:  18%|█▊        | 2/11 [00:00<00:00, 67.72it/s]
Epoch 13:  27%|██▋       | 3/11 [00:00<00:00, 68.47it/s]
Epoch 13:  27%|██▋       | 3/11 [00:00<00:00, 67.95it/s]
Epoch 13:  36%|███▋      | 4/11 [00:00<00:00, 68.27it/s]
Epoch 13:  36%|███▋      | 4/11 [00:00<00:00, 67.86it/s]
Epoch 13:  45%|████▌     | 5/11 [00:00<00:00, 67.93it/s]
Epoch 13:  45%|████▌     | 5/11 [00:00<00:00, 67.61it/s]
Epoch 13:  55%|█████▍    | 6/11 [00:00<00:00, 67.68it/s]
Epoch 13:  55%|█████▍    | 6/11 [00:00<00:00, 67.40it/s]
Epoch 13:  64%|██████▎   | 7/11 [00:00<00:00, 66.93it/s]
Epoch 13:  64%|██████▎   | 7/11 [00:00<00:00, 66.70it/s]
Epoch 13:  73%|███████▎  | 8/11 [00:00<00:00, 66.78it/s]
Epoch 13:  73%|███████▎  | 8/11 [00:00<00:00, 66.58it/s]
Epoch 13:  82%|████████▏ | 9/11 [00:00<00:00, 66.80it/s]
Epoch 13:  82%|████████▏ | 9/11 [00:00<00:00, 66.63it/s]
Epoch 13:  91%|█████████ | 10/11 [00:00<00:00, 66.61it/s]
Epoch 13:  91%|█████████ | 10/11 [00:00<00:00, 66.45it/s]
Epoch 13: 100%|██████████| 11/11 [00:00<00:00, 65.53it/s]
Epoch 13: 100%|██████████| 11/11 [00:00<00:00, 65.31it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 351.05it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 333.77it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 336.18it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 339.16it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 347.91it/s]


Epoch 13: 100%|██████████| 11/11 [00:00<00:00, 59.02it/s]
Epoch 13: 100%|██████████| 11/11 [00:00<00:00, 58.87it/s]
Epoch 13:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 14:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 14:   9%|▉         | 1/11 [00:00<00:00, 69.32it/s]
Epoch 14:   9%|▉         | 1/11 [00:00<00:00, 67.82it/s]
Epoch 14:  18%|█▊        | 2/11 [00:00<00:00, 66.78it/s]
Epoch 14:  18%|█▊        | 2/11 [00:00<00:00, 66.02it/s]
Epoch 14:  27%|██▋       | 3/11 [00:00<00:00, 65.33it/s]
Epoch 14:  27%|██▋       | 3/11 [00:00<00:00, 64.83it/s]
Epoch 14:  36%|███▋      | 4/11 [00:00<00:00, 65.41it/s]
Epoch 14:  36%|███▋      | 4/11 [00:00<00:00, 64.98it/s]
Epoch 14:  45%|████▌     | 5/11 [00:00<00:00, 65.47it/s]
Epoch 14:  45%|████▌     | 5/11 [00:00<00:00, 65.17it/s]
Epoch 14:  55%|█████▍    | 6/11 [00:00<00:00, 63.18it/s]
Epoch 14:  55%|█████▍    | 6/11 [00:00<00:00, 62.93it/s]
Epoch 14:  64%|██████▎   | 7/11 [00:00<00:00, 63.20it/s]
Epoch 14:  64%|██████▎   | 7/11 [00:00<00:00, 62.98it/s]
Epoch 14:  73%|███████▎  | 8/11 [00:00<00:00, 61.69it/s]
Epoch 14:  73%|███████▎  | 8/11 [00:00<00:00, 61.50it/s]
Epoch 14:  82%|████████▏ | 9/11 [00:00<00:00, 60.42it/s]
Epoch 14:  82%|████████▏ | 9/11 [00:00<00:00, 60.24it/s]
Epoch 14:  91%|█████████ | 10/11 [00:00<00:00, 60.35it/s]
Epoch 14:  91%|█████████ | 10/11 [00:00<00:00, 60.21it/s]
Epoch 14: 100%|██████████| 11/11 [00:00<00:00, 60.14it/s]
Epoch 14: 100%|██████████| 11/11 [00:00<00:00, 60.02it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 309.34it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 310.52it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 314.53it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 318.42it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 327.83it/s]


Epoch 14: 100%|██████████| 11/11 [00:00<00:00, 54.36it/s]
Epoch 14: 100%|██████████| 11/11 [00:00<00:00, 54.23it/s]
Epoch 14:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 15:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 15:   9%|▉         | 1/11 [00:00<00:00, 61.09it/s]
Epoch 15:   9%|▉         | 1/11 [00:00<00:00, 59.79it/s]
Epoch 15:  18%|█▊        | 2/11 [00:00<00:00, 61.52it/s]
Epoch 15:  18%|█▊        | 2/11 [00:00<00:00, 60.83it/s]
Epoch 15:  27%|██▋       | 3/11 [00:00<00:00, 61.30it/s]
Epoch 15:  27%|██▋       | 3/11 [00:00<00:00, 60.84it/s]
Epoch 15:  36%|███▋      | 4/11 [00:00<00:00, 62.02it/s]
Epoch 15:  36%|███▋      | 4/11 [00:00<00:00, 61.67it/s]
Epoch 15:  45%|████▌     | 5/11 [00:00<00:00, 62.40it/s]
Epoch 15:  45%|████▌     | 5/11 [00:00<00:00, 62.12it/s]
Epoch 15:  55%|█████▍    | 6/11 [00:00<00:00, 62.65it/s]
Epoch 15:  55%|█████▍    | 6/11 [00:00<00:00, 62.35it/s]
Epoch 15:  64%|██████▎   | 7/11 [00:00<00:00, 62.33it/s]
Epoch 15:  64%|██████▎   | 7/11 [00:00<00:00, 62.14it/s]
Epoch 15:  73%|███████▎  | 8/11 [00:00<00:00, 62.49it/s]
Epoch 15:  73%|███████▎  | 8/11 [00:00<00:00, 62.32it/s]
Epoch 15:  82%|████████▏ | 9/11 [00:00<00:00, 62.76it/s]
Epoch 15:  82%|████████▏ | 9/11 [00:00<00:00, 62.60it/s]
Epoch 15:  91%|█████████ | 10/11 [00:00<00:00, 63.06it/s]
Epoch 15:  91%|█████████ | 10/11 [00:00<00:00, 62.92it/s]
Epoch 15: 100%|██████████| 11/11 [00:00<00:00, 63.22it/s]
Epoch 15: 100%|██████████| 11/11 [00:00<00:00, 63.10it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 364.56it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 347.12it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 347.91it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 348.40it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 355.58it/s]


Epoch 15: 100%|██████████| 11/11 [00:00<00:00, 57.27it/s]
Epoch 15: 100%|██████████| 11/11 [00:00<00:00, 57.13it/s]
Epoch 15:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 16:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 16:   9%|▉         | 1/11 [00:00<00:00, 61.23it/s]
Epoch 16:   9%|▉         | 1/11 [00:00<00:00, 59.98it/s]
Epoch 16:  18%|█▊        | 2/11 [00:00<00:00, 61.88it/s]
Epoch 16:  18%|█▊        | 2/11 [00:00<00:00, 61.24it/s]
Epoch 16:  27%|██▋       | 3/11 [00:00<00:00, 63.40it/s]
Epoch 16:  27%|██▋       | 3/11 [00:00<00:00, 62.92it/s]
Epoch 16:  36%|███▋      | 4/11 [00:00<00:00, 64.02it/s]
Epoch 16:  36%|███▋      | 4/11 [00:00<00:00, 63.66it/s]
Epoch 16:  45%|████▌     | 5/11 [00:00<00:00, 64.10it/s]
Epoch 16:  45%|████▌     | 5/11 [00:00<00:00, 63.81it/s]
Epoch 16:  55%|█████▍    | 6/11 [00:00<00:00, 64.35it/s]
Epoch 16:  55%|█████▍    | 6/11 [00:00<00:00, 64.10it/s]
Epoch 16:  64%|██████▎   | 7/11 [00:00<00:00, 64.73it/s]
Epoch 16:  64%|██████▎   | 7/11 [00:00<00:00, 64.53it/s]
Epoch 16:  73%|███████▎  | 8/11 [00:00<00:00, 65.10it/s]
Epoch 16:  73%|███████▎  | 8/11 [00:00<00:00, 64.93it/s]
Epoch 16:  82%|████████▏ | 9/11 [00:00<00:00, 65.43it/s]
Epoch 16:  82%|████████▏ | 9/11 [00:00<00:00, 65.27it/s]
Epoch 16:  91%|█████████ | 10/11 [00:00<00:00, 65.76it/s]
Epoch 16:  91%|█████████ | 10/11 [00:00<00:00, 65.61it/s]
Epoch 16: 100%|██████████| 11/11 [00:00<00:00, 65.84it/s]
Epoch 16: 100%|██████████| 11/11 [00:00<00:00, 65.70it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 379.78it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 340.39it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 340.82it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.25it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 346.68it/s]


Epoch 16: 100%|██████████| 11/11 [00:00<00:00, 59.39it/s]
Epoch 16: 100%|██████████| 11/11 [00:00<00:00, 59.24it/s]
Epoch 16:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 17:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 17:   9%|▉         | 1/11 [00:00<00:00, 67.18it/s]
Epoch 17:   9%|▉         | 1/11 [00:00<00:00, 65.81it/s]
Epoch 17:  18%|█▊        | 2/11 [00:00<00:00, 64.50it/s]
Epoch 17:  18%|█▊        | 2/11 [00:00<00:00, 63.76it/s]
Epoch 17:  27%|██▋       | 3/11 [00:00<00:00, 64.79it/s]
Epoch 17:  27%|██▋       | 3/11 [00:00<00:00, 64.30it/s]
Epoch 17:  36%|███▋      | 4/11 [00:00<00:00, 63.52it/s]
Epoch 17:  36%|███▋      | 4/11 [00:00<00:00, 63.17it/s]
Epoch 17:  45%|████▌     | 5/11 [00:00<00:00, 63.57it/s]
Epoch 17:  45%|████▌     | 5/11 [00:00<00:00, 63.22it/s]
Epoch 17:  55%|█████▍    | 6/11 [00:00<00:00, 63.72it/s]
Epoch 17:  55%|█████▍    | 6/11 [00:00<00:00, 63.48it/s]
Epoch 17:  64%|██████▎   | 7/11 [00:00<00:00, 63.42it/s]
Epoch 17:  64%|██████▎   | 7/11 [00:00<00:00, 63.21it/s]
Epoch 17:  73%|███████▎  | 8/11 [00:00<00:00, 62.88it/s]
Epoch 17:  73%|███████▎  | 8/11 [00:00<00:00, 62.70it/s]
Epoch 17:  82%|████████▏ | 9/11 [00:00<00:00, 62.89it/s]
Epoch 17:  82%|████████▏ | 9/11 [00:00<00:00, 62.73it/s]
Epoch 17:  91%|█████████ | 10/11 [00:00<00:00, 62.57it/s]
Epoch 17:  91%|█████████ | 10/11 [00:00<00:00, 62.43it/s]
Epoch 17: 100%|██████████| 11/11 [00:00<00:00, 62.78it/s]
Epoch 17: 100%|██████████| 11/11 [00:00<00:00, 62.66it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 370.16it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 348.97it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 351.07it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 352.61it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 359.90it/s]


Epoch 17: 100%|██████████| 11/11 [00:00<00:00, 57.05it/s]
Epoch 17: 100%|██████████| 11/11 [00:00<00:00, 56.92it/s]
Epoch 17:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 18:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 18:   9%|▉         | 1/11 [00:00<00:00, 70.30it/s]
Epoch 18:   9%|▉         | 1/11 [00:00<00:00, 68.68it/s]
Epoch 18:  18%|█▊        | 2/11 [00:00<00:00, 68.32it/s]
Epoch 18:  18%|█▊        | 2/11 [00:00<00:00, 67.49it/s]
Epoch 18:  27%|██▋       | 3/11 [00:00<00:00, 67.07it/s]
Epoch 18:  27%|██▋       | 3/11 [00:00<00:00, 66.46it/s]
Epoch 18:  36%|███▋      | 4/11 [00:00<00:00, 65.83it/s]
Epoch 18:  36%|███▋      | 4/11 [00:00<00:00, 65.40it/s]
Epoch 18:  45%|████▌     | 5/11 [00:00<00:00, 64.94it/s]
Epoch 18:  45%|████▌     | 5/11 [00:00<00:00, 64.58it/s]
Epoch 18:  55%|█████▍    | 6/11 [00:00<00:00, 63.55it/s]
Epoch 18:  55%|█████▍    | 6/11 [00:00<00:00, 63.29it/s]
Epoch 18:  64%|██████▎   | 7/11 [00:00<00:00, 62.91it/s]
Epoch 18:  64%|██████▎   | 7/11 [00:00<00:00, 62.70it/s]
Epoch 18:  73%|███████▎  | 8/11 [00:00<00:00, 61.36it/s]
Epoch 18:  73%|███████▎  | 8/11 [00:00<00:00, 61.18it/s]
Epoch 18:  82%|████████▏ | 9/11 [00:00<00:00, 61.64it/s]
Epoch 18:  82%|████████▏ | 9/11 [00:00<00:00, 61.48it/s]
Epoch 18:  91%|█████████ | 10/11 [00:00<00:00, 60.70it/s]
Epoch 18:  91%|█████████ | 10/11 [00:00<00:00, 60.55it/s]
Epoch 18: 100%|██████████| 11/11 [00:00<00:00, 60.99it/s]
Epoch 18: 100%|██████████| 11/11 [00:00<00:00, 60.86it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 349.64it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 331.19it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 332.63it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 333.99it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 341.69it/s]


Epoch 18: 100%|██████████| 11/11 [00:00<00:00, 55.23it/s]
Epoch 18: 100%|██████████| 11/11 [00:00<00:00, 55.10it/s]
Epoch 18:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 19:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 19:   9%|▉         | 1/11 [00:00<00:00, 64.45it/s]
Epoch 19:   9%|▉         | 1/11 [00:00<00:00, 62.98it/s]
Epoch 19:  18%|█▊        | 2/11 [00:00<00:00, 62.64it/s]
Epoch 19:  18%|█▊        | 2/11 [00:00<00:00, 61.91it/s]
Epoch 19:  27%|██▋       | 3/11 [00:00<00:00, 63.52it/s]
Epoch 19:  27%|██▋       | 3/11 [00:00<00:00, 63.04it/s]
Epoch 19:  36%|███▋      | 4/11 [00:00<00:00, 64.28it/s]
Epoch 19:  36%|███▋      | 4/11 [00:00<00:00, 63.91it/s]
Epoch 19:  45%|████▌     | 5/11 [00:00<00:00, 64.72it/s]
Epoch 19:  45%|████▌     | 5/11 [00:00<00:00, 64.42it/s]
Epoch 19:  55%|█████▍    | 6/11 [00:00<00:00, 65.04it/s]
Epoch 19:  55%|█████▍    | 6/11 [00:00<00:00, 64.78it/s]
Epoch 19:  64%|██████▎   | 7/11 [00:00<00:00, 65.00it/s]
Epoch 19:  64%|██████▎   | 7/11 [00:00<00:00, 64.78it/s]
Epoch 19:  73%|███████▎  | 8/11 [00:00<00:00, 64.85it/s]
Epoch 19:  73%|███████▎  | 8/11 [00:00<00:00, 64.66it/s]
Epoch 19:  82%|████████▏ | 9/11 [00:00<00:00, 64.80it/s]
Epoch 19:  82%|████████▏ | 9/11 [00:00<00:00, 64.63it/s]
Epoch 19:  91%|█████████ | 10/11 [00:00<00:00, 64.56it/s]
Epoch 19:  91%|█████████ | 10/11 [00:00<00:00, 64.40it/s]
Epoch 19: 100%|██████████| 11/11 [00:00<00:00, 64.57it/s]
Epoch 19: 100%|██████████| 11/11 [00:00<00:00, 64.43it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 357.91it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 336.28it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 335.89it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 338.00it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 347.66it/s]


Epoch 19: 100%|██████████| 11/11 [00:00<00:00, 58.30it/s]
Epoch 19: 100%|██████████| 11/11 [00:00<00:00, 58.14it/s]
Epoch 19:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 20:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 20:   9%|▉         | 1/11 [00:00<00:00, 68.03it/s]
Epoch 20:   9%|▉         | 1/11 [00:00<00:00, 66.41it/s]
Epoch 20:  18%|█▊        | 2/11 [00:00<00:00, 66.61it/s]
Epoch 20:  18%|█▊        | 2/11 [00:00<00:00, 65.81it/s]
Epoch 20:  27%|██▋       | 3/11 [00:00<00:00, 65.06it/s]
Epoch 20:  27%|██▋       | 3/11 [00:00<00:00, 64.54it/s]
Epoch 20:  36%|███▋      | 4/11 [00:00<00:00, 65.12it/s]
Epoch 20:  36%|███▋      | 4/11 [00:00<00:00, 64.75it/s]
Epoch 20:  45%|████▌     | 5/11 [00:00<00:00, 65.19it/s]
Epoch 20:  45%|████▌     | 5/11 [00:00<00:00, 64.88it/s]
Epoch 20:  55%|█████▍    | 6/11 [00:00<00:00, 65.16it/s]
Epoch 20:  55%|█████▍    | 6/11 [00:00<00:00, 64.90it/s]
Epoch 20:  64%|██████▎   | 7/11 [00:00<00:00, 65.23it/s]
Epoch 20:  64%|██████▎   | 7/11 [00:00<00:00, 64.97it/s]
Epoch 20:  73%|███████▎  | 8/11 [00:00<00:00, 65.15it/s]
Epoch 20:  73%|███████▎  | 8/11 [00:00<00:00, 64.96it/s]
Epoch 20:  82%|████████▏ | 9/11 [00:00<00:00, 63.41it/s]
Epoch 20:  82%|████████▏ | 9/11 [00:00<00:00, 63.25it/s]
Epoch 20:  91%|█████████ | 10/11 [00:00<00:00, 63.61it/s]
Epoch 20:  91%|█████████ | 10/11 [00:00<00:00, 63.46it/s]
Epoch 20: 100%|██████████| 11/11 [00:00<00:00, 62.83it/s]
Epoch 20: 100%|██████████| 11/11 [00:00<00:00, 62.68it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 345.89it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 331.00it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 322.17it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 329.24it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 339.03it/s]


Epoch 20: 100%|██████████| 11/11 [00:00<00:00, 56.77it/s]
Epoch 20: 100%|██████████| 11/11 [00:00<00:00, 56.63it/s]
Epoch 20:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 21:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 21:   9%|▉         | 1/11 [00:00<00:00, 67.82it/s]
Epoch 21:   9%|▉         | 1/11 [00:00<00:00, 66.19it/s]
Epoch 21:  18%|█▊        | 2/11 [00:00<00:00, 66.91it/s]
Epoch 21:  18%|█▊        | 2/11 [00:00<00:00, 66.13it/s]
Epoch 21:  27%|██▋       | 3/11 [00:00<00:00, 66.48it/s]
Epoch 21:  27%|██▋       | 3/11 [00:00<00:00, 65.94it/s]
Epoch 21:  36%|███▋      | 4/11 [00:00<00:00, 66.04it/s]
Epoch 21:  36%|███▋      | 4/11 [00:00<00:00, 65.59it/s]
Epoch 21:  45%|████▌     | 5/11 [00:00<00:00, 65.87it/s]
Epoch 21:  45%|████▌     | 5/11 [00:00<00:00, 65.54it/s]
Epoch 21:  55%|█████▍    | 6/11 [00:00<00:00, 65.60it/s]
Epoch 21:  55%|█████▍    | 6/11 [00:00<00:00, 65.28it/s]
Epoch 21:  64%|██████▎   | 7/11 [00:00<00:00, 65.18it/s]
Epoch 21:  64%|██████▎   | 7/11 [00:00<00:00, 64.95it/s]
Epoch 21:  73%|███████▎  | 8/11 [00:00<00:00, 64.87it/s]
Epoch 21:  73%|███████▎  | 8/11 [00:00<00:00, 64.67it/s]
Epoch 21:  82%|████████▏ | 9/11 [00:00<00:00, 65.04it/s]
Epoch 21:  82%|████████▏ | 9/11 [00:00<00:00, 64.87it/s]
Epoch 21:  91%|█████████ | 10/11 [00:00<00:00, 65.09it/s]
Epoch 21:  91%|█████████ | 10/11 [00:00<00:00, 64.93it/s]
Epoch 21: 100%|██████████| 11/11 [00:00<00:00, 65.24it/s]
Epoch 21: 100%|██████████| 11/11 [00:00<00:00, 65.08it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 358.61it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.40it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 342.02it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 344.04it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.82it/s]


Epoch 21: 100%|██████████| 11/11 [00:00<00:00, 58.94it/s]
Epoch 21: 100%|██████████| 11/11 [00:00<00:00, 58.79it/s]
Epoch 21:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 22:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 22:   9%|▉         | 1/11 [00:00<00:00, 69.56it/s]
Epoch 22:   9%|▉         | 1/11 [00:00<00:00, 68.08it/s]
Epoch 22:  18%|█▊        | 2/11 [00:00<00:00, 67.70it/s]
Epoch 22:  18%|█▊        | 2/11 [00:00<00:00, 66.95it/s]
Epoch 22:  27%|██▋       | 3/11 [00:00<00:00, 65.25it/s]
Epoch 22:  27%|██▋       | 3/11 [00:00<00:00, 64.73it/s]
Epoch 22:  36%|███▋      | 4/11 [00:00<00:00, 64.53it/s]
Epoch 22:  36%|███▋      | 4/11 [00:00<00:00, 64.16it/s]
Epoch 22:  45%|████▌     | 5/11 [00:00<00:00, 64.81it/s]
Epoch 22:  45%|████▌     | 5/11 [00:00<00:00, 64.51it/s]
Epoch 22:  55%|█████▍    | 6/11 [00:00<00:00, 65.22it/s]
Epoch 22:  55%|█████▍    | 6/11 [00:00<00:00, 64.97it/s]
Epoch 22:  64%|██████▎   | 7/11 [00:00<00:00, 65.31it/s]
Epoch 22:  64%|██████▎   | 7/11 [00:00<00:00, 65.08it/s]
Epoch 22:  73%|███████▎  | 8/11 [00:00<00:00, 65.31it/s]
Epoch 22:  73%|███████▎  | 8/11 [00:00<00:00, 65.12it/s]
Epoch 22:  82%|████████▏ | 9/11 [00:00<00:00, 65.22it/s]
Epoch 22:  82%|████████▏ | 9/11 [00:00<00:00, 65.05it/s]
Epoch 22:  91%|█████████ | 10/11 [00:00<00:00, 65.26it/s]
Epoch 22:  91%|█████████ | 10/11 [00:00<00:00, 65.11it/s]
Epoch 22: 100%|██████████| 11/11 [00:00<00:00, 65.45it/s]
Epoch 22: 100%|██████████| 11/11 [00:00<00:00, 65.31it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 358.92it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.70it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 336.50it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 338.82it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 347.05it/s]


Epoch 22: 100%|██████████| 11/11 [00:00<00:00, 59.04it/s]
Epoch 22: 100%|██████████| 11/11 [00:00<00:00, 58.90it/s]
Epoch 22:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 23:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 23:   9%|▉         | 1/11 [00:00<00:00, 67.63it/s]
Epoch 23:   9%|▉         | 1/11 [00:00<00:00, 65.96it/s]
Epoch 23:  18%|█▊        | 2/11 [00:00<00:00, 65.93it/s]
Epoch 23:  18%|█▊        | 2/11 [00:00<00:00, 65.14it/s]
Epoch 23:  27%|██▋       | 3/11 [00:00<00:00, 65.81it/s]
Epoch 23:  27%|██▋       | 3/11 [00:00<00:00, 65.23it/s]
Epoch 23:  36%|███▋      | 4/11 [00:00<00:00, 65.82it/s]
Epoch 23:  36%|███▋      | 4/11 [00:00<00:00, 65.45it/s]
Epoch 23:  45%|████▌     | 5/11 [00:00<00:00, 66.21it/s]
Epoch 23:  45%|████▌     | 5/11 [00:00<00:00, 65.90it/s]
Epoch 23:  55%|█████▍    | 6/11 [00:00<00:00, 66.22it/s]
Epoch 23:  55%|█████▍    | 6/11 [00:00<00:00, 65.96it/s]
Epoch 23:  64%|██████▎   | 7/11 [00:00<00:00, 66.16it/s]
Epoch 23:  64%|██████▎   | 7/11 [00:00<00:00, 65.90it/s]
Epoch 23:  73%|███████▎  | 8/11 [00:00<00:00, 65.96it/s]
Epoch 23:  73%|███████▎  | 8/11 [00:00<00:00, 65.75it/s]
Epoch 23:  82%|████████▏ | 9/11 [00:00<00:00, 65.29it/s]
Epoch 23:  82%|████████▏ | 9/11 [00:00<00:00, 65.09it/s]
Epoch 23:  91%|█████████ | 10/11 [00:00<00:00, 65.23it/s]
Epoch 23:  91%|█████████ | 10/11 [00:00<00:00, 65.07it/s]
Epoch 23: 100%|██████████| 11/11 [00:00<00:00, 65.22it/s]
Epoch 23: 100%|██████████| 11/11 [00:00<00:00, 65.08it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 359.13it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 341.68it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 339.64it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 340.59it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 347.88it/s]


Epoch 23: 100%|██████████| 11/11 [00:00<00:00, 58.86it/s]
Epoch 23: 100%|██████████| 11/11 [00:00<00:00, 58.72it/s]
Epoch 23:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 24:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 24:   9%|▉         | 1/11 [00:00<00:00, 60.78it/s]
Epoch 24:   9%|▉         | 1/11 [00:00<00:00, 59.46it/s]
Epoch 24:  18%|█▊        | 2/11 [00:00<00:00, 60.78it/s]
Epoch 24:  18%|█▊        | 2/11 [00:00<00:00, 60.10it/s]
Epoch 24:  27%|██▋       | 3/11 [00:00<00:00, 61.08it/s]
Epoch 24:  27%|██▋       | 3/11 [00:00<00:00, 60.44it/s]
Epoch 24:  36%|███▋      | 4/11 [00:00<00:00, 61.71it/s]
Epoch 24:  36%|███▋      | 4/11 [00:00<00:00, 61.38it/s]
Epoch 24:  45%|████▌     | 5/11 [00:00<00:00, 62.73it/s]
Epoch 24:  45%|████▌     | 5/11 [00:00<00:00, 62.46it/s]
Epoch 24:  55%|█████▍    | 6/11 [00:00<00:00, 63.25it/s]
Epoch 24:  55%|█████▍    | 6/11 [00:00<00:00, 63.01it/s]
Epoch 24:  64%|██████▎   | 7/11 [00:00<00:00, 63.67it/s]
Epoch 24:  64%|██████▎   | 7/11 [00:00<00:00, 63.43it/s]
Epoch 24:  73%|███████▎  | 8/11 [00:00<00:00, 63.92it/s]
Epoch 24:  73%|███████▎  | 8/11 [00:00<00:00, 63.75it/s]
Epoch 24:  82%|████████▏ | 9/11 [00:00<00:00, 64.06it/s]
Epoch 24:  82%|████████▏ | 9/11 [00:00<00:00, 63.89it/s]
Epoch 24:  91%|█████████ | 10/11 [00:00<00:00, 64.16it/s]
Epoch 24:  91%|█████████ | 10/11 [00:00<00:00, 64.01it/s]
Epoch 24: 100%|██████████| 11/11 [00:00<00:00, 64.24it/s]
Epoch 24: 100%|██████████| 11/11 [00:00<00:00, 64.10it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 357.14it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.55it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 339.08it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.64it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 349.05it/s]


Epoch 24: 100%|██████████| 11/11 [00:00<00:00, 58.08it/s]
Epoch 24: 100%|██████████| 11/11 [00:00<00:00, 57.94it/s]
Epoch 24:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 25:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 25:   9%|▉         | 1/11 [00:00<00:00, 69.16it/s]
Epoch 25:   9%|▉         | 1/11 [00:00<00:00, 67.54it/s]
Epoch 25:  18%|█▊        | 2/11 [00:00<00:00, 67.31it/s]
Epoch 25:  18%|█▊        | 2/11 [00:00<00:00, 66.53it/s]
Epoch 25:  27%|██▋       | 3/11 [00:00<00:00, 64.92it/s]
Epoch 25:  27%|██▋       | 3/11 [00:00<00:00, 64.42it/s]
Epoch 25:  36%|███▋      | 4/11 [00:00<00:00, 64.61it/s]
Epoch 25:  36%|███▋      | 4/11 [00:00<00:00, 64.24it/s]
Epoch 25:  45%|████▌     | 5/11 [00:00<00:00, 64.40it/s]
Epoch 25:  45%|████▌     | 5/11 [00:00<00:00, 64.09it/s]
Epoch 25:  55%|█████▍    | 6/11 [00:00<00:00, 64.28it/s]
Epoch 25:  55%|█████▍    | 6/11 [00:00<00:00, 64.03it/s]
Epoch 25:  64%|██████▎   | 7/11 [00:00<00:00, 64.51it/s]
Epoch 25:  64%|██████▎   | 7/11 [00:00<00:00, 64.30it/s]
Epoch 25:  73%|███████▎  | 8/11 [00:00<00:00, 64.88it/s]
Epoch 25:  73%|███████▎  | 8/11 [00:00<00:00, 64.69it/s]
Epoch 25:  82%|████████▏ | 9/11 [00:00<00:00, 65.04it/s]
Epoch 25:  82%|████████▏ | 9/11 [00:00<00:00, 64.87it/s]
Epoch 25:  91%|█████████ | 10/11 [00:00<00:00, 65.25it/s]
Epoch 25:  91%|█████████ | 10/11 [00:00<00:00, 65.10it/s]
Epoch 25: 100%|██████████| 11/11 [00:00<00:00, 64.60it/s]
Epoch 25: 100%|██████████| 11/11 [00:00<00:00, 64.47it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 367.47it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 342.21it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 339.10it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 341.97it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.98it/s]


Epoch 25: 100%|██████████| 11/11 [00:00<00:00, 58.27it/s]
Epoch 25: 100%|██████████| 11/11 [00:00<00:00, 58.13it/s]
Epoch 25:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 26:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 26:   9%|▉         | 1/11 [00:00<00:00, 67.83it/s]
Epoch 26:   9%|▉         | 1/11 [00:00<00:00, 66.28it/s]
Epoch 26:  18%|█▊        | 2/11 [00:00<00:00, 65.58it/s]
Epoch 26:  18%|█▊        | 2/11 [00:00<00:00, 64.83it/s]
Epoch 26:  27%|██▋       | 3/11 [00:00<00:00, 65.52it/s]
Epoch 26:  27%|██▋       | 3/11 [00:00<00:00, 64.92it/s]
Epoch 26:  36%|███▋      | 4/11 [00:00<00:00, 65.58it/s]
Epoch 26:  36%|███▋      | 4/11 [00:00<00:00, 65.19it/s]
Epoch 26:  45%|████▌     | 5/11 [00:00<00:00, 65.59it/s]
Epoch 26:  45%|████▌     | 5/11 [00:00<00:00, 65.28it/s]
Epoch 26:  55%|█████▍    | 6/11 [00:00<00:00, 65.53it/s]
Epoch 26:  55%|█████▍    | 6/11 [00:00<00:00, 65.27it/s]
Epoch 26:  64%|██████▎   | 7/11 [00:00<00:00, 65.40it/s]
Epoch 26:  64%|██████▎   | 7/11 [00:00<00:00, 65.18it/s]
Epoch 26:  73%|███████▎  | 8/11 [00:00<00:00, 65.50it/s]
Epoch 26:  73%|███████▎  | 8/11 [00:00<00:00, 65.30it/s]
Epoch 26:  82%|████████▏ | 9/11 [00:00<00:00, 65.68it/s]
Epoch 26:  82%|████████▏ | 9/11 [00:00<00:00, 65.51it/s]
Epoch 26:  91%|█████████ | 10/11 [00:00<00:00, 65.95it/s]
Epoch 26:  91%|█████████ | 10/11 [00:00<00:00, 65.80it/s]
Epoch 26: 100%|██████████| 11/11 [00:00<00:00, 66.08it/s]
Epoch 26: 100%|██████████| 11/11 [00:00<00:00, 65.94it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 368.96it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 347.77it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 348.64it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 347.24it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 354.29it/s]


Epoch 26: 100%|██████████| 11/11 [00:00<00:00, 59.69it/s]
Epoch 26: 100%|██████████| 11/11 [00:00<00:00, 59.54it/s]
Epoch 26:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 27:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 27:   9%|▉         | 1/11 [00:00<00:00, 68.94it/s]
Epoch 27:   9%|▉         | 1/11 [00:00<00:00, 67.31it/s]
Epoch 27:  18%|█▊        | 2/11 [00:00<00:00, 68.02it/s]
Epoch 27:  18%|█▊        | 2/11 [00:00<00:00, 67.24it/s]
Epoch 27:  27%|██▋       | 3/11 [00:00<00:00, 67.36it/s]
Epoch 27:  27%|██▋       | 3/11 [00:00<00:00, 66.81it/s]
Epoch 27:  36%|███▋      | 4/11 [00:00<00:00, 66.62it/s]
Epoch 27:  36%|███▋      | 4/11 [00:00<00:00, 66.23it/s]
Epoch 27:  45%|████▌     | 5/11 [00:00<00:00, 64.85it/s]
Epoch 27:  45%|████▌     | 5/11 [00:00<00:00, 64.54it/s]
Epoch 27:  55%|█████▍    | 6/11 [00:00<00:00, 64.39it/s]
Epoch 27:  55%|█████▍    | 6/11 [00:00<00:00, 64.14it/s]
Epoch 27:  64%|██████▎   | 7/11 [00:00<00:00, 64.34it/s]
Epoch 27:  64%|██████▎   | 7/11 [00:00<00:00, 64.10it/s]
Epoch 27:  73%|███████▎  | 8/11 [00:00<00:00, 64.35it/s]
Epoch 27:  73%|███████▎  | 8/11 [00:00<00:00, 64.17it/s]
Epoch 27:  82%|████████▏ | 9/11 [00:00<00:00, 64.48it/s]
Epoch 27:  82%|████████▏ | 9/11 [00:00<00:00, 64.32it/s]
Epoch 27:  91%|█████████ | 10/11 [00:00<00:00, 64.74it/s]
Epoch 27:  91%|█████████ | 10/11 [00:00<00:00, 64.59it/s]
Epoch 27: 100%|██████████| 11/11 [00:00<00:00, 64.85it/s]
Epoch 27: 100%|██████████| 11/11 [00:00<00:00, 64.71it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 356.84it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 322.27it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 323.76it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 328.25it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 336.84it/s]


Epoch 27: 100%|██████████| 11/11 [00:00<00:00, 58.35it/s]
Epoch 27: 100%|██████████| 11/11 [00:00<00:00, 58.19it/s]
Epoch 27:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 28:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 28:   9%|▉         | 1/11 [00:00<00:00, 64.55it/s]
Epoch 28:   9%|▉         | 1/11 [00:00<00:00, 63.17it/s]
Epoch 28:  18%|█▊        | 2/11 [00:00<00:00, 63.46it/s]
Epoch 28:  18%|█▊        | 2/11 [00:00<00:00, 62.74it/s]
Epoch 28:  27%|██▋       | 3/11 [00:00<00:00, 63.12it/s]
Epoch 28:  27%|██▋       | 3/11 [00:00<00:00, 62.65it/s]
Epoch 28:  36%|███▋      | 4/11 [00:00<00:00, 61.51it/s]
Epoch 28:  36%|███▋      | 4/11 [00:00<00:00, 61.16it/s]
Epoch 28:  45%|████▌     | 5/11 [00:00<00:00, 62.10it/s]
Epoch 28:  45%|████▌     | 5/11 [00:00<00:00, 61.81it/s]
Epoch 28:  55%|█████▍    | 6/11 [00:00<00:00, 62.49it/s]
Epoch 28:  55%|█████▍    | 6/11 [00:00<00:00, 62.26it/s]
Epoch 28:  64%|██████▎   | 7/11 [00:00<00:00, 62.89it/s]
Epoch 28:  64%|██████▎   | 7/11 [00:00<00:00, 62.66it/s]
Epoch 28:  73%|███████▎  | 8/11 [00:00<00:00, 62.88it/s]
Epoch 28:  73%|███████▎  | 8/11 [00:00<00:00, 62.68it/s]
Epoch 28:  82%|████████▏ | 9/11 [00:00<00:00, 63.21it/s]
Epoch 28:  82%|████████▏ | 9/11 [00:00<00:00, 63.05it/s]
Epoch 28:  91%|█████████ | 10/11 [00:00<00:00, 62.30it/s]
Epoch 28:  91%|█████████ | 10/11 [00:00<00:00, 62.16it/s]
Epoch 28: 100%|██████████| 11/11 [00:00<00:00, 62.53it/s]
Epoch 28: 100%|██████████| 11/11 [00:00<00:00, 62.40it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 370.46it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 333.30it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 335.76it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 338.84it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 343.33it/s]


Epoch 28: 100%|██████████| 11/11 [00:00<00:00, 56.64it/s]
Epoch 28: 100%|██████████| 11/11 [00:00<00:00, 56.50it/s]
Epoch 28:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 29:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 29:   9%|▉         | 1/11 [00:00<00:00, 66.16it/s]
Epoch 29:   9%|▉         | 1/11 [00:00<00:00, 64.65it/s]
Epoch 29:  18%|█▊        | 2/11 [00:00<00:00, 65.05it/s]
Epoch 29:  18%|█▊        | 2/11 [00:00<00:00, 64.29it/s]
Epoch 29:  27%|██▋       | 3/11 [00:00<00:00, 65.18it/s]
Epoch 29:  27%|██▋       | 3/11 [00:00<00:00, 64.67it/s]
Epoch 29:  36%|███▋      | 4/11 [00:00<00:00, 65.57it/s]
Epoch 29:  36%|███▋      | 4/11 [00:00<00:00, 65.18it/s]
Epoch 29:  45%|████▌     | 5/11 [00:00<00:00, 65.66it/s]
Epoch 29:  45%|████▌     | 5/11 [00:00<00:00, 65.29it/s]
Epoch 29:  55%|█████▍    | 6/11 [00:00<00:00, 65.73it/s]
Epoch 29:  55%|█████▍    | 6/11 [00:00<00:00, 65.48it/s]
Epoch 29:  64%|██████▎   | 7/11 [00:00<00:00, 65.94it/s]
Epoch 29:  64%|██████▎   | 7/11 [00:00<00:00, 65.72it/s]
Epoch 29:  73%|███████▎  | 8/11 [00:00<00:00, 66.14it/s]
Epoch 29:  73%|███████▎  | 8/11 [00:00<00:00, 65.94it/s]
Epoch 29:  82%|████████▏ | 9/11 [00:00<00:00, 66.31it/s]
Epoch 29:  82%|████████▏ | 9/11 [00:00<00:00, 66.14it/s]
Epoch 29:  91%|█████████ | 10/11 [00:00<00:00, 66.27it/s]
Epoch 29:  91%|█████████ | 10/11 [00:00<00:00, 66.11it/s]
Epoch 29: 100%|██████████| 11/11 [00:00<00:00, 66.25it/s]
Epoch 29: 100%|██████████| 11/11 [00:00<00:00, 66.11it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 368.67it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 342.11it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 341.94it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.76it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.68it/s]


Epoch 29: 100%|██████████| 11/11 [00:00<00:00, 59.75it/s]
Epoch 29: 100%|██████████| 11/11 [00:00<00:00, 59.59it/s]
Epoch 29:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 30:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 30:   9%|▉         | 1/11 [00:00<00:00, 68.32it/s]
Epoch 30:   9%|▉         | 1/11 [00:00<00:00, 66.67it/s]
Epoch 30:  18%|█▊        | 2/11 [00:00<00:00, 67.48it/s]
Epoch 30:  18%|█▊        | 2/11 [00:00<00:00, 66.72it/s]
Epoch 30:  27%|██▋       | 3/11 [00:00<00:00, 66.98it/s]
Epoch 30:  27%|██▋       | 3/11 [00:00<00:00, 66.45it/s]
Epoch 30:  36%|███▋      | 4/11 [00:00<00:00, 66.74it/s]
Epoch 30:  36%|███▋      | 4/11 [00:00<00:00, 66.34it/s]
Epoch 30:  45%|████▌     | 5/11 [00:00<00:00, 66.21it/s]
Epoch 30:  45%|████▌     | 5/11 [00:00<00:00, 65.90it/s]
Epoch 30:  55%|█████▍    | 6/11 [00:00<00:00, 66.03it/s]
Epoch 30:  55%|█████▍    | 6/11 [00:00<00:00, 65.73it/s]
Epoch 30:  64%|██████▎   | 7/11 [00:00<00:00, 65.67it/s]
Epoch 30:  64%|██████▎   | 7/11 [00:00<00:00, 65.44it/s]
Epoch 30:  73%|███████▎  | 8/11 [00:00<00:00, 65.50it/s]
Epoch 30:  73%|███████▎  | 8/11 [00:00<00:00, 65.30it/s]
Epoch 30:  82%|████████▏ | 9/11 [00:00<00:00, 65.40it/s]
Epoch 30:  82%|████████▏ | 9/11 [00:00<00:00, 65.23it/s]
Epoch 30:  91%|█████████ | 10/11 [00:00<00:00, 65.16it/s]
Epoch 30:  91%|█████████ | 10/11 [00:00<00:00, 65.00it/s]
Epoch 30: 100%|██████████| 11/11 [00:00<00:00, 65.06it/s]
Epoch 30: 100%|██████████| 11/11 [00:00<00:00, 64.92it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 364.15it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.54it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 338.64it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 340.63it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 350.62it/s]


Epoch 30: 100%|██████████| 11/11 [00:00<00:00, 58.77it/s]
Epoch 30: 100%|██████████| 11/11 [00:00<00:00, 58.62it/s]
Epoch 30:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 31:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 31:   9%|▉         | 1/11 [00:00<00:00, 68.94it/s]
Epoch 31:   9%|▉         | 1/11 [00:00<00:00, 67.32it/s]
Epoch 31:  18%|█▊        | 2/11 [00:00<00:00, 67.52it/s]
Epoch 31:  18%|█▊        | 2/11 [00:00<00:00, 66.71it/s]
Epoch 31:  27%|██▋       | 3/11 [00:00<00:00, 66.60it/s]
Epoch 31:  27%|██▋       | 3/11 [00:00<00:00, 66.06it/s]
Epoch 31:  36%|███▋      | 4/11 [00:00<00:00, 65.11it/s]
Epoch 31:  36%|███▋      | 4/11 [00:00<00:00, 64.72it/s]
Epoch 31:  45%|████▌     | 5/11 [00:00<00:00, 64.44it/s]
Epoch 31:  45%|████▌     | 5/11 [00:00<00:00, 64.13it/s]
Epoch 31:  55%|█████▍    | 6/11 [00:00<00:00, 62.80it/s]
Epoch 31:  55%|█████▍    | 6/11 [00:00<00:00, 62.56it/s]
Epoch 31:  64%|██████▎   | 7/11 [00:00<00:00, 63.02it/s]
Epoch 31:  64%|██████▎   | 7/11 [00:00<00:00, 62.73it/s]
Epoch 31:  73%|███████▎  | 8/11 [00:00<00:00, 63.20it/s]
Epoch 31:  73%|███████▎  | 8/11 [00:00<00:00, 63.01it/s]
Epoch 31:  82%|████████▏ | 9/11 [00:00<00:00, 63.34it/s]
Epoch 31:  82%|████████▏ | 9/11 [00:00<00:00, 63.18it/s]
Epoch 31:  91%|█████████ | 10/11 [00:00<00:00, 63.08it/s]
Epoch 31:  91%|█████████ | 10/11 [00:00<00:00, 62.94it/s]
Epoch 31: 100%|██████████| 11/11 [00:00<00:00, 63.24it/s]
Epoch 31: 100%|██████████| 11/11 [00:00<00:00, 63.11it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 343.51it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 325.06it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 325.11it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 328.27it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 336.46it/s]


Epoch 31: 100%|██████████| 11/11 [00:00<00:00, 57.09it/s]
Epoch 31: 100%|██████████| 11/11 [00:00<00:00, 56.95it/s]
Epoch 31:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 32:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 32:   9%|▉         | 1/11 [00:00<00:00, 59.89it/s]
Epoch 32:   9%|▉         | 1/11 [00:00<00:00, 58.57it/s]
Epoch 32:  18%|█▊        | 2/11 [00:00<00:00, 60.40it/s]
Epoch 32:  18%|█▊        | 2/11 [00:00<00:00, 59.74it/s]
Epoch 32:  27%|██▋       | 3/11 [00:00<00:00, 61.51it/s]
Epoch 32:  27%|██▋       | 3/11 [00:00<00:00, 61.08it/s]
Epoch 32:  36%|███▋      | 4/11 [00:00<00:00, 62.90it/s]
Epoch 32:  36%|███▋      | 4/11 [00:00<00:00, 62.55it/s]
Epoch 32:  45%|████▌     | 5/11 [00:00<00:00, 63.80it/s]
Epoch 32:  45%|████▌     | 5/11 [00:00<00:00, 63.52it/s]
Epoch 32:  55%|█████▍    | 6/11 [00:00<00:00, 64.58it/s]
Epoch 32:  55%|█████▍    | 6/11 [00:00<00:00, 64.35it/s]
Epoch 32:  64%|██████▎   | 7/11 [00:00<00:00, 65.06it/s]
Epoch 32:  64%|██████▎   | 7/11 [00:00<00:00, 64.85it/s]
Epoch 32:  73%|███████▎  | 8/11 [00:00<00:00, 65.25it/s]
Epoch 32:  73%|███████▎  | 8/11 [00:00<00:00, 65.06it/s]
Epoch 32:  82%|████████▏ | 9/11 [00:00<00:00, 65.42it/s]
Epoch 32:  82%|████████▏ | 9/11 [00:00<00:00, 65.24it/s]
Epoch 32:  91%|█████████ | 10/11 [00:00<00:00, 65.51it/s]
Epoch 32:  91%|█████████ | 10/11 [00:00<00:00, 65.36it/s]
Epoch 32: 100%|██████████| 11/11 [00:00<00:00, 65.52it/s]
Epoch 32: 100%|██████████| 11/11 [00:00<00:00, 65.38it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 367.86it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 349.29it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 348.85it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 350.74it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 359.86it/s]


Epoch 32: 100%|██████████| 11/11 [00:00<00:00, 59.27it/s]
Epoch 32: 100%|██████████| 11/11 [00:00<00:00, 59.12it/s]
Epoch 32:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 33:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 33:   9%|▉         | 1/11 [00:00<00:00, 70.89it/s]
Epoch 33:   9%|▉         | 1/11 [00:00<00:00, 69.30it/s]
Epoch 33:  18%|█▊        | 2/11 [00:00<00:00, 69.34it/s]
Epoch 33:  18%|█▊        | 2/11 [00:00<00:00, 68.52it/s]
Epoch 33:  27%|██▋       | 3/11 [00:00<00:00, 68.70it/s]
Epoch 33:  27%|██▋       | 3/11 [00:00<00:00, 68.16it/s]
Epoch 33:  36%|███▋      | 4/11 [00:00<00:00, 68.47it/s]
Epoch 33:  36%|███▋      | 4/11 [00:00<00:00, 68.06it/s]
Epoch 33:  45%|████▌     | 5/11 [00:00<00:00, 68.14it/s]
Epoch 33:  45%|████▌     | 5/11 [00:00<00:00, 67.80it/s]
Epoch 33:  55%|█████▍    | 6/11 [00:00<00:00, 67.63it/s]
Epoch 33:  55%|█████▍    | 6/11 [00:00<00:00, 67.36it/s]
Epoch 33:  64%|██████▎   | 7/11 [00:00<00:00, 67.08it/s]
Epoch 33:  64%|██████▎   | 7/11 [00:00<00:00, 66.84it/s]
Epoch 33:  73%|███████▎  | 8/11 [00:00<00:00, 66.61it/s]
Epoch 33:  73%|███████▎  | 8/11 [00:00<00:00, 66.41it/s]
Epoch 33:  82%|████████▏ | 9/11 [00:00<00:00, 66.30it/s]
Epoch 33:  82%|████████▏ | 9/11 [00:00<00:00, 66.12it/s]
Epoch 33:  91%|█████████ | 10/11 [00:00<00:00, 66.23it/s]
Epoch 33:  91%|█████████ | 10/11 [00:00<00:00, 66.05it/s]
Epoch 33: 100%|██████████| 11/11 [00:00<00:00, 66.07it/s]
Epoch 33: 100%|██████████| 11/11 [00:00<00:00, 65.92it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 350.87it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 331.47it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 334.03it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 335.78it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 342.59it/s]


Epoch 33: 100%|██████████| 11/11 [00:00<00:00, 59.47it/s]
Epoch 33: 100%|██████████| 11/11 [00:00<00:00, 59.32it/s]
Epoch 33:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 34:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 34:   9%|▉         | 1/11 [00:00<00:00, 67.43it/s]
Epoch 34:   9%|▉         | 1/11 [00:00<00:00, 65.78it/s]
Epoch 34:  18%|█▊        | 2/11 [00:00<00:00, 65.49it/s]
Epoch 34:  18%|█▊        | 2/11 [00:00<00:00, 64.69it/s]
Epoch 34:  27%|██▋       | 3/11 [00:00<00:00, 65.06it/s]
Epoch 34:  27%|██▋       | 3/11 [00:00<00:00, 64.54it/s]
Epoch 34:  36%|███▋      | 4/11 [00:00<00:00, 64.52it/s]
Epoch 34:  36%|███▋      | 4/11 [00:00<00:00, 64.13it/s]
Epoch 34:  45%|████▌     | 5/11 [00:00<00:00, 64.46it/s]
Epoch 34:  45%|████▌     | 5/11 [00:00<00:00, 64.16it/s]
Epoch 34:  55%|█████▍    | 6/11 [00:00<00:00, 64.44it/s]
Epoch 34:  55%|█████▍    | 6/11 [00:00<00:00, 64.14it/s]
Epoch 34:  64%|██████▎   | 7/11 [00:00<00:00, 64.21it/s]
Epoch 34:  64%|██████▎   | 7/11 [00:00<00:00, 63.96it/s]
Epoch 34:  73%|███████▎  | 8/11 [00:00<00:00, 64.11it/s]
Epoch 34:  73%|███████▎  | 8/11 [00:00<00:00, 63.90it/s]
Epoch 34:  82%|████████▏ | 9/11 [00:00<00:00, 63.61it/s]
Epoch 34:  82%|████████▏ | 9/11 [00:00<00:00, 63.44it/s]
Epoch 34:  91%|█████████ | 10/11 [00:00<00:00, 62.48it/s]
Epoch 34:  91%|█████████ | 10/11 [00:00<00:00, 62.31it/s]
Epoch 34: 100%|██████████| 11/11 [00:00<00:00, 62.28it/s]
Epoch 34: 100%|██████████| 11/11 [00:00<00:00, 62.07it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 340.12it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 326.01it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 327.77it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 328.83it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 336.57it/s]


Epoch 34: 100%|██████████| 11/11 [00:00<00:00, 56.24it/s]
Epoch 34: 100%|██████████| 11/11 [00:00<00:00, 56.11it/s]
Epoch 34:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 35:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 35:   9%|▉         | 1/11 [00:00<00:00, 67.08it/s]
Epoch 35:   9%|▉         | 1/11 [00:00<00:00, 65.46it/s]
Epoch 35:  18%|█▊        | 2/11 [00:00<00:00, 64.80it/s]
Epoch 35:  18%|█▊        | 2/11 [00:00<00:00, 64.02it/s]
Epoch 35:  27%|██▋       | 3/11 [00:00<00:00, 63.43it/s]
Epoch 35:  27%|██▋       | 3/11 [00:00<00:00, 62.93it/s]
Epoch 35:  36%|███▋      | 4/11 [00:00<00:00, 63.55it/s]
Epoch 35:  36%|███▋      | 4/11 [00:00<00:00, 63.18it/s]
Epoch 35:  45%|████▌     | 5/11 [00:00<00:00, 63.73it/s]
Epoch 35:  45%|████▌     | 5/11 [00:00<00:00, 63.43it/s]
Epoch 35:  55%|█████▍    | 6/11 [00:00<00:00, 63.82it/s]
Epoch 35:  55%|█████▍    | 6/11 [00:00<00:00, 63.57it/s]
Epoch 35:  64%|██████▎   | 7/11 [00:00<00:00, 63.92it/s]
Epoch 35:  64%|██████▎   | 7/11 [00:00<00:00, 63.70it/s]
Epoch 35:  73%|███████▎  | 8/11 [00:00<00:00, 63.90it/s]
Epoch 35:  73%|███████▎  | 8/11 [00:00<00:00, 63.71it/s]
Epoch 35:  82%|████████▏ | 9/11 [00:00<00:00, 63.47it/s]
Epoch 35:  82%|████████▏ | 9/11 [00:00<00:00, 63.30it/s]
Epoch 35:  91%|█████████ | 10/11 [00:00<00:00, 62.32it/s]
Epoch 35:  91%|█████████ | 10/11 [00:00<00:00, 62.13it/s]
Epoch 35: 100%|██████████| 11/11 [00:00<00:00, 62.27it/s]
Epoch 35: 100%|██████████| 11/11 [00:00<00:00, 62.13it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 357.57it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 316.40it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 320.93it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 326.03it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 331.02it/s]


Epoch 35: 100%|██████████| 11/11 [00:00<00:00, 56.29it/s]
Epoch 35: 100%|██████████| 11/11 [00:00<00:00, 56.16it/s]
Epoch 35:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 36:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 36:   9%|▉         | 1/11 [00:00<00:00, 67.37it/s]
Epoch 36:   9%|▉         | 1/11 [00:00<00:00, 65.68it/s]
Epoch 36:  18%|█▊        | 2/11 [00:00<00:00, 65.04it/s]
Epoch 36:  18%|█▊        | 2/11 [00:00<00:00, 64.25it/s]
Epoch 36:  27%|██▋       | 3/11 [00:00<00:00, 63.42it/s]
Epoch 36:  27%|██▋       | 3/11 [00:00<00:00, 62.93it/s]
Epoch 36:  36%|███▋      | 4/11 [00:00<00:00, 63.32it/s]
Epoch 36:  36%|███▋      | 4/11 [00:00<00:00, 62.95it/s]
Epoch 36:  45%|████▌     | 5/11 [00:00<00:00, 63.04it/s]
Epoch 36:  45%|████▌     | 5/11 [00:00<00:00, 62.75it/s]
Epoch 36:  55%|█████▍    | 6/11 [00:00<00:00, 62.75it/s]
Epoch 36:  55%|█████▍    | 6/11 [00:00<00:00, 62.50it/s]
Epoch 36:  64%|██████▎   | 7/11 [00:00<00:00, 60.86it/s]
Epoch 36:  64%|██████▎   | 7/11 [00:00<00:00, 60.65it/s]
Epoch 36:  73%|███████▎  | 8/11 [00:00<00:00, 60.68it/s]
Epoch 36:  73%|███████▎  | 8/11 [00:00<00:00, 60.50it/s]
Epoch 36:  82%|████████▏ | 9/11 [00:00<00:00, 60.75it/s]
Epoch 36:  82%|████████▏ | 9/11 [00:00<00:00, 60.58it/s]
Epoch 36:  91%|█████████ | 10/11 [00:00<00:00, 59.92it/s]
Epoch 36:  91%|█████████ | 10/11 [00:00<00:00, 59.78it/s]
Epoch 36: 100%|██████████| 11/11 [00:00<00:00, 60.17it/s]
Epoch 36: 100%|██████████| 11/11 [00:00<00:00, 60.05it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 336.97it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 324.81it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 324.85it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 327.07it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 334.39it/s]


Epoch 36: 100%|██████████| 11/11 [00:00<00:00, 54.50it/s]
Epoch 36: 100%|██████████| 11/11 [00:00<00:00, 54.33it/s]
Epoch 36:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 37:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 37:   9%|▉         | 1/11 [00:00<00:00, 66.03it/s]
Epoch 37:   9%|▉         | 1/11 [00:00<00:00, 64.41it/s]
Epoch 37:  18%|█▊        | 2/11 [00:00<00:00, 64.06it/s]
Epoch 37:  18%|█▊        | 2/11 [00:00<00:00, 63.26it/s]
Epoch 37:  27%|██▋       | 3/11 [00:00<00:00, 63.60it/s]
Epoch 37:  27%|██▋       | 3/11 [00:00<00:00, 63.10it/s]
Epoch 37:  36%|███▋      | 4/11 [00:00<00:00, 63.63it/s]
Epoch 37:  36%|███▋      | 4/11 [00:00<00:00, 63.25it/s]
Epoch 37:  45%|████▌     | 5/11 [00:00<00:00, 63.45it/s]
Epoch 37:  45%|████▌     | 5/11 [00:00<00:00, 63.15it/s]
Epoch 37:  55%|█████▍    | 6/11 [00:00<00:00, 63.34it/s]
Epoch 37:  55%|█████▍    | 6/11 [00:00<00:00, 63.09it/s]
Epoch 37:  64%|██████▎   | 7/11 [00:00<00:00, 63.30it/s]
Epoch 37:  64%|██████▎   | 7/11 [00:00<00:00, 63.09it/s]
Epoch 37:  73%|███████▎  | 8/11 [00:00<00:00, 63.32it/s]
Epoch 37:  73%|███████▎  | 8/11 [00:00<00:00, 63.14it/s]
Epoch 37:  82%|████████▏ | 9/11 [00:00<00:00, 63.63it/s]
Epoch 37:  82%|████████▏ | 9/11 [00:00<00:00, 63.45it/s]
Epoch 37:  91%|█████████ | 10/11 [00:00<00:00, 63.46it/s]
Epoch 37:  91%|█████████ | 10/11 [00:00<00:00, 63.30it/s]
Epoch 37: 100%|██████████| 11/11 [00:00<00:00, 63.27it/s]
Epoch 37: 100%|██████████| 11/11 [00:00<00:00, 63.14it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 341.31it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 323.31it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 311.17it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 315.44it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 326.19it/s]


Epoch 37: 100%|██████████| 11/11 [00:00<00:00, 56.84it/s]
Epoch 37: 100%|██████████| 11/11 [00:00<00:00, 56.66it/s]
Epoch 37:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 38:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 38:   9%|▉         | 1/11 [00:00<00:00, 64.49it/s]
Epoch 38:   9%|▉         | 1/11 [00:00<00:00, 63.01it/s]
Epoch 38:  18%|█▊        | 2/11 [00:00<00:00, 64.41it/s]
Epoch 38:  18%|█▊        | 2/11 [00:00<00:00, 63.66it/s]
Epoch 38:  27%|██▋       | 3/11 [00:00<00:00, 64.58it/s]
Epoch 38:  27%|██▋       | 3/11 [00:00<00:00, 64.01it/s]
Epoch 38:  36%|███▋      | 4/11 [00:00<00:00, 64.59it/s]
Epoch 38:  36%|███▋      | 4/11 [00:00<00:00, 64.21it/s]
Epoch 38:  45%|████▌     | 5/11 [00:00<00:00, 63.35it/s]
Epoch 38:  45%|████▌     | 5/11 [00:00<00:00, 63.04it/s]
Epoch 38:  55%|█████▍    | 6/11 [00:00<00:00, 62.77it/s]
Epoch 38:  55%|█████▍    | 6/11 [00:00<00:00, 62.53it/s]
Epoch 38:  64%|██████▎   | 7/11 [00:00<00:00, 62.70it/s]
Epoch 38:  64%|██████▎   | 7/11 [00:00<00:00, 62.49it/s]
Epoch 38:  73%|███████▎  | 8/11 [00:00<00:00, 62.63it/s]
Epoch 38:  73%|███████▎  | 8/11 [00:00<00:00, 62.45it/s]
Epoch 38:  82%|████████▏ | 9/11 [00:00<00:00, 61.82it/s]
Epoch 38:  82%|████████▏ | 9/11 [00:00<00:00, 61.66it/s]
Epoch 38:  91%|█████████ | 10/11 [00:00<00:00, 62.02it/s]
Epoch 38:  91%|█████████ | 10/11 [00:00<00:00, 61.88it/s]
Epoch 38: 100%|██████████| 11/11 [00:00<00:00, 62.11it/s]
Epoch 38: 100%|██████████| 11/11 [00:00<00:00, 61.98it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 347.79it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 333.01it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 333.78it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 336.06it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 345.69it/s]


Epoch 38: 100%|██████████| 11/11 [00:00<00:00, 56.26it/s]
Epoch 38: 100%|██████████| 11/11 [00:00<00:00, 56.13it/s]
Epoch 38:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 39:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 39:   9%|▉         | 1/11 [00:00<00:00, 67.48it/s]
Epoch 39:   9%|▉         | 1/11 [00:00<00:00, 65.82it/s]
Epoch 39:  18%|█▊        | 2/11 [00:00<00:00, 65.70it/s]
Epoch 39:  18%|█▊        | 2/11 [00:00<00:00, 64.91it/s]
Epoch 39:  27%|██▋       | 3/11 [00:00<00:00, 64.44it/s]
Epoch 39:  27%|██▋       | 3/11 [00:00<00:00, 63.94it/s]
Epoch 39:  36%|███▋      | 4/11 [00:00<00:00, 63.63it/s]
Epoch 39:  36%|███▋      | 4/11 [00:00<00:00, 63.25it/s]
Epoch 39:  45%|████▌     | 5/11 [00:00<00:00, 63.39it/s]
Epoch 39:  45%|████▌     | 5/11 [00:00<00:00, 63.10it/s]
Epoch 39:  55%|█████▍    | 6/11 [00:00<00:00, 63.31it/s]
Epoch 39:  55%|█████▍    | 6/11 [00:00<00:00, 63.07it/s]
Epoch 39:  64%|██████▎   | 7/11 [00:00<00:00, 63.39it/s]
Epoch 39:  64%|██████▎   | 7/11 [00:00<00:00, 63.19it/s]
Epoch 39:  73%|███████▎  | 8/11 [00:00<00:00, 63.68it/s]
Epoch 39:  73%|███████▎  | 8/11 [00:00<00:00, 63.50it/s]
Epoch 39:  82%|████████▏ | 9/11 [00:00<00:00, 64.03it/s]
Epoch 39:  82%|████████▏ | 9/11 [00:00<00:00, 63.87it/s]
Epoch 39:  91%|█████████ | 10/11 [00:00<00:00, 64.00it/s]
Epoch 39:  91%|█████████ | 10/11 [00:00<00:00, 63.85it/s]
Epoch 39: 100%|██████████| 11/11 [00:00<00:00, 64.08it/s]
Epoch 39: 100%|██████████| 11/11 [00:00<00:00, 63.95it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 359.90it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 336.01it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 336.11it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 338.81it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 348.69it/s]


Epoch 39: 100%|██████████| 11/11 [00:00<00:00, 57.93it/s]
Epoch 39: 100%|██████████| 11/11 [00:00<00:00, 57.78it/s]
Epoch 39:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 40:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 40:   9%|▉         | 1/11 [00:00<00:00, 69.50it/s]
Epoch 40:   9%|▉         | 1/11 [00:00<00:00, 67.91it/s]
Epoch 40:  18%|█▊        | 2/11 [00:00<00:00, 68.14it/s]
Epoch 40:  18%|█▊        | 2/11 [00:00<00:00, 67.30it/s]
Epoch 40:  27%|██▋       | 3/11 [00:00<00:00, 66.79it/s]
Epoch 40:  27%|██▋       | 3/11 [00:00<00:00, 66.25it/s]
Epoch 40:  36%|███▋      | 4/11 [00:00<00:00, 66.23it/s]
Epoch 40:  36%|███▋      | 4/11 [00:00<00:00, 65.83it/s]
Epoch 40:  45%|████▌     | 5/11 [00:00<00:00, 66.32it/s]
Epoch 40:  45%|████▌     | 5/11 [00:00<00:00, 65.96it/s]
Epoch 40:  55%|█████▍    | 6/11 [00:00<00:00, 66.26it/s]
Epoch 40:  55%|█████▍    | 6/11 [00:00<00:00, 65.99it/s]
Epoch 40:  64%|██████▎   | 7/11 [00:00<00:00, 65.71it/s]
Epoch 40:  64%|██████▎   | 7/11 [00:00<00:00, 65.49it/s]
Epoch 40:  73%|███████▎  | 8/11 [00:00<00:00, 65.68it/s]
Epoch 40:  73%|███████▎  | 8/11 [00:00<00:00, 65.48it/s]
Epoch 40:  82%|████████▏ | 9/11 [00:00<00:00, 65.71it/s]
Epoch 40:  82%|████████▏ | 9/11 [00:00<00:00, 65.55it/s]
Epoch 40:  91%|█████████ | 10/11 [00:00<00:00, 66.04it/s]
Epoch 40:  91%|█████████ | 10/11 [00:00<00:00, 65.87it/s]
Epoch 40: 100%|██████████| 11/11 [00:00<00:00, 65.53it/s]
Epoch 40: 100%|██████████| 11/11 [00:00<00:00, 65.40it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 373.86it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 349.74it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 348.74it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 349.45it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 356.85it/s]


Epoch 40: 100%|██████████| 11/11 [00:00<00:00, 59.28it/s]
Epoch 40: 100%|██████████| 11/11 [00:00<00:00, 59.13it/s]
Epoch 40:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 41:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 41:   9%|▉         | 1/11 [00:00<00:00, 69.41it/s]
Epoch 41:   9%|▉         | 1/11 [00:00<00:00, 67.78it/s]
Epoch 41:  18%|█▊        | 2/11 [00:00<00:00, 66.75it/s]
Epoch 41:  18%|█▊        | 2/11 [00:00<00:00, 65.95it/s]
Epoch 41:  27%|██▋       | 3/11 [00:00<00:00, 65.98it/s]
Epoch 41:  27%|██▋       | 3/11 [00:00<00:00, 65.45it/s]
Epoch 41:  36%|███▋      | 4/11 [00:00<00:00, 65.76it/s]
Epoch 41:  36%|███▋      | 4/11 [00:00<00:00, 65.31it/s]
Epoch 41:  45%|████▌     | 5/11 [00:00<00:00, 65.42it/s]
Epoch 41:  45%|████▌     | 5/11 [00:00<00:00, 65.12it/s]
Epoch 41:  55%|█████▍    | 6/11 [00:00<00:00, 65.40it/s]
Epoch 41:  55%|█████▍    | 6/11 [00:00<00:00, 65.15it/s]
Epoch 41:  64%|██████▎   | 7/11 [00:00<00:00, 65.33it/s]
Epoch 41:  64%|██████▎   | 7/11 [00:00<00:00, 65.10it/s]
Epoch 41:  73%|███████▎  | 8/11 [00:00<00:00, 64.79it/s]
Epoch 41:  73%|███████▎  | 8/11 [00:00<00:00, 64.60it/s]
Epoch 41:  82%|████████▏ | 9/11 [00:00<00:00, 64.47it/s]
Epoch 41:  82%|████████▏ | 9/11 [00:00<00:00, 64.30it/s]
Epoch 41:  91%|█████████ | 10/11 [00:00<00:00, 64.51it/s]
Epoch 41:  91%|█████████ | 10/11 [00:00<00:00, 64.36it/s]
Epoch 41: 100%|██████████| 11/11 [00:00<00:00, 64.59it/s]
Epoch 41: 100%|██████████| 11/11 [00:00<00:00, 64.45it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 360.74it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 342.90it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 330.63it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 333.57it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 343.53it/s]


Epoch 41: 100%|██████████| 11/11 [00:00<00:00, 58.29it/s]
Epoch 41: 100%|██████████| 11/11 [00:00<00:00, 58.13it/s]
Epoch 41:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 42:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 42:   9%|▉         | 1/11 [00:00<00:00, 60.49it/s]
Epoch 42:   9%|▉         | 1/11 [00:00<00:00, 59.17it/s]
Epoch 42:  18%|█▊        | 2/11 [00:00<00:00, 61.20it/s]
Epoch 42:  18%|█▊        | 2/11 [00:00<00:00, 60.53it/s]
Epoch 42:  27%|██▋       | 3/11 [00:00<00:00, 61.81it/s]
Epoch 42:  27%|██▋       | 3/11 [00:00<00:00, 61.36it/s]
Epoch 42:  36%|███▋      | 4/11 [00:00<00:00, 63.08it/s]
Epoch 42:  36%|███▋      | 4/11 [00:00<00:00, 62.72it/s]
Epoch 42:  45%|████▌     | 5/11 [00:00<00:00, 63.62it/s]
Epoch 42:  45%|████▌     | 5/11 [00:00<00:00, 63.33it/s]
Epoch 42:  55%|█████▍    | 6/11 [00:00<00:00, 63.95it/s]
Epoch 42:  55%|█████▍    | 6/11 [00:00<00:00, 63.70it/s]
Epoch 42:  64%|██████▎   | 7/11 [00:00<00:00, 62.94it/s]
Epoch 42:  64%|██████▎   | 7/11 [00:00<00:00, 62.69it/s]
Epoch 42:  73%|███████▎  | 8/11 [00:00<00:00, 63.04it/s]
Epoch 42:  73%|███████▎  | 8/11 [00:00<00:00, 62.85it/s]
Epoch 42:  82%|████████▏ | 9/11 [00:00<00:00, 62.99it/s]
Epoch 42:  82%|████████▏ | 9/11 [00:00<00:00, 62.83it/s]
Epoch 42:  91%|█████████ | 10/11 [00:00<00:00, 63.16it/s]
Epoch 42:  91%|█████████ | 10/11 [00:00<00:00, 63.01it/s]
Epoch 42: 100%|██████████| 11/11 [00:00<00:00, 63.46it/s]
Epoch 42: 100%|██████████| 11/11 [00:00<00:00, 63.33it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 365.90it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 342.53it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 341.26it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.88it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.91it/s]


Epoch 42: 100%|██████████| 11/11 [00:00<00:00, 57.46it/s]
Epoch 42: 100%|██████████| 11/11 [00:00<00:00, 57.32it/s]
Epoch 42:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 43:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 43:   9%|▉         | 1/11 [00:00<00:00, 69.50it/s]
Epoch 43:   9%|▉         | 1/11 [00:00<00:00, 67.90it/s]
Epoch 43:  18%|█▊        | 2/11 [00:00<00:00, 68.28it/s]
Epoch 43:  18%|█▊        | 2/11 [00:00<00:00, 67.47it/s]
Epoch 43:  27%|██▋       | 3/11 [00:00<00:00, 67.46it/s]
Epoch 43:  27%|██▋       | 3/11 [00:00<00:00, 66.92it/s]
Epoch 43:  36%|███▋      | 4/11 [00:00<00:00, 67.12it/s]
Epoch 43:  36%|███▋      | 4/11 [00:00<00:00, 66.71it/s]
Epoch 43:  45%|████▌     | 5/11 [00:00<00:00, 66.93it/s]
Epoch 43:  45%|████▌     | 5/11 [00:00<00:00, 66.57it/s]
Epoch 43:  55%|█████▍    | 6/11 [00:00<00:00, 66.97it/s]
Epoch 43:  55%|█████▍    | 6/11 [00:00<00:00, 66.70it/s]
Epoch 43:  64%|██████▎   | 7/11 [00:00<00:00, 67.05it/s]
Epoch 43:  64%|██████▎   | 7/11 [00:00<00:00, 66.84it/s]
Epoch 43:  73%|███████▎  | 8/11 [00:00<00:00, 67.54it/s]
Epoch 43:  73%|███████▎  | 8/11 [00:00<00:00, 67.36it/s]
Epoch 43:  82%|████████▏ | 9/11 [00:00<00:00, 67.97it/s]
Epoch 43:  82%|████████▏ | 9/11 [00:00<00:00, 67.81it/s]
Epoch 43:  91%|█████████ | 10/11 [00:00<00:00, 68.32it/s]
Epoch 43:  91%|█████████ | 10/11 [00:00<00:00, 68.17it/s]
Epoch 43: 100%|██████████| 11/11 [00:00<00:00, 68.20it/s]
Epoch 43: 100%|██████████| 11/11 [00:00<00:00, 68.06it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 366.70it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 351.34it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 350.12it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 349.72it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 357.60it/s]


Epoch 43: 100%|██████████| 11/11 [00:00<00:00, 61.35it/s]
Epoch 43: 100%|██████████| 11/11 [00:00<00:00, 61.19it/s]
Epoch 43:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 44:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 44:   9%|▉         | 1/11 [00:00<00:00, 64.82it/s]
Epoch 44:   9%|▉         | 1/11 [00:00<00:00, 63.50it/s]
Epoch 44:  18%|█▊        | 2/11 [00:00<00:00, 65.62it/s]
Epoch 44:  18%|█▊        | 2/11 [00:00<00:00, 64.83it/s]
Epoch 44:  27%|██▋       | 3/11 [00:00<00:00, 66.99it/s]
Epoch 44:  27%|██▋       | 3/11 [00:00<00:00, 66.52it/s]
Epoch 44:  36%|███▋      | 4/11 [00:00<00:00, 68.07it/s]
Epoch 44:  36%|███▋      | 4/11 [00:00<00:00, 67.66it/s]
Epoch 44:  45%|████▌     | 5/11 [00:00<00:00, 68.39it/s]
Epoch 44:  45%|████▌     | 5/11 [00:00<00:00, 68.07it/s]
Epoch 44:  55%|█████▍    | 6/11 [00:00<00:00, 67.79it/s]
Epoch 44:  55%|█████▍    | 6/11 [00:00<00:00, 67.53it/s]
Epoch 44:  64%|██████▎   | 7/11 [00:00<00:00, 66.45it/s]
Epoch 44:  64%|██████▎   | 7/11 [00:00<00:00, 66.22it/s]
Epoch 44:  73%|███████▎  | 8/11 [00:00<00:00, 65.59it/s]
Epoch 44:  73%|███████▎  | 8/11 [00:00<00:00, 65.40it/s]
Epoch 44:  82%|████████▏ | 9/11 [00:00<00:00, 65.63it/s]
Epoch 44:  82%|████████▏ | 9/11 [00:00<00:00, 65.45it/s]
Epoch 44:  91%|█████████ | 10/11 [00:00<00:00, 65.66it/s]
Epoch 44:  91%|█████████ | 10/11 [00:00<00:00, 65.51it/s]
Epoch 44: 100%|██████████| 11/11 [00:00<00:00, 65.68it/s]
Epoch 44: 100%|██████████| 11/11 [00:00<00:00, 65.54it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 358.55it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 338.56it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 334.30it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 339.08it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 348.03it/s]


Epoch 44: 100%|██████████| 11/11 [00:00<00:00, 59.26it/s]
Epoch 44: 100%|██████████| 11/11 [00:00<00:00, 59.12it/s]
Epoch 44:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 45:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 45:   9%|▉         | 1/11 [00:00<00:00, 61.62it/s]
Epoch 45:   9%|▉         | 1/11 [00:00<00:00, 60.15it/s]
Epoch 45:  18%|█▊        | 2/11 [00:00<00:00, 62.27it/s]
Epoch 45:  18%|█▊        | 2/11 [00:00<00:00, 61.52it/s]
Epoch 45:  27%|██▋       | 3/11 [00:00<00:00, 63.12it/s]
Epoch 45:  27%|██▋       | 3/11 [00:00<00:00, 62.66it/s]
Epoch 45:  36%|███▋      | 4/11 [00:00<00:00, 63.23it/s]
Epoch 45:  36%|███▋      | 4/11 [00:00<00:00, 62.86it/s]
Epoch 45:  45%|████▌     | 5/11 [00:00<00:00, 63.45it/s]
Epoch 45:  45%|████▌     | 5/11 [00:00<00:00, 63.16it/s]
Epoch 45:  55%|█████▍    | 6/11 [00:00<00:00, 63.89it/s]
Epoch 45:  55%|█████▍    | 6/11 [00:00<00:00, 63.65it/s]
Epoch 45:  64%|██████▎   | 7/11 [00:00<00:00, 64.33it/s]
Epoch 45:  64%|██████▎   | 7/11 [00:00<00:00, 64.13it/s]
Epoch 45:  73%|███████▎  | 8/11 [00:00<00:00, 64.79it/s]
Epoch 45:  73%|███████▎  | 8/11 [00:00<00:00, 64.61it/s]
Epoch 45:  82%|████████▏ | 9/11 [00:00<00:00, 65.28it/s]
Epoch 45:  82%|████████▏ | 9/11 [00:00<00:00, 65.12it/s]
Epoch 45:  91%|█████████ | 10/11 [00:00<00:00, 65.40it/s]
Epoch 45:  91%|█████████ | 10/11 [00:00<00:00, 65.25it/s]
Epoch 45: 100%|██████████| 11/11 [00:00<00:00, 65.55it/s]
Epoch 45: 100%|██████████| 11/11 [00:00<00:00, 65.41it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 356.81it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 340.76it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 343.87it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 346.43it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 354.40it/s]


Epoch 45: 100%|██████████| 11/11 [00:00<00:00, 59.21it/s]
Epoch 45: 100%|██████████| 11/11 [00:00<00:00, 59.05it/s]
Epoch 45:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 46:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 46:   9%|▉         | 1/11 [00:00<00:00, 70.40it/s]
Epoch 46:   9%|▉         | 1/11 [00:00<00:00, 68.84it/s]
Epoch 46:  18%|█▊        | 2/11 [00:00<00:00, 70.22it/s]
Epoch 46:  18%|█▊        | 2/11 [00:00<00:00, 69.43it/s]
Epoch 46:  27%|██▋       | 3/11 [00:00<00:00, 69.14it/s]
Epoch 46:  27%|██▋       | 3/11 [00:00<00:00, 68.60it/s]
Epoch 46:  36%|███▋      | 4/11 [00:00<00:00, 68.31it/s]
Epoch 46:  36%|███▋      | 4/11 [00:00<00:00, 67.91it/s]
Epoch 46:  45%|████▌     | 5/11 [00:00<00:00, 67.75it/s]
Epoch 46:  45%|████▌     | 5/11 [00:00<00:00, 67.44it/s]
Epoch 46:  55%|█████▍    | 6/11 [00:00<00:00, 67.78it/s]
Epoch 46:  55%|█████▍    | 6/11 [00:00<00:00, 67.53it/s]
Epoch 46:  64%|██████▎   | 7/11 [00:00<00:00, 68.07it/s]
Epoch 46:  64%|██████▎   | 7/11 [00:00<00:00, 67.86it/s]
Epoch 46:  73%|███████▎  | 8/11 [00:00<00:00, 68.57it/s]
Epoch 46:  73%|███████▎  | 8/11 [00:00<00:00, 68.38it/s]
Epoch 46:  82%|████████▏ | 9/11 [00:00<00:00, 68.94it/s]
Epoch 46:  82%|████████▏ | 9/11 [00:00<00:00, 68.77it/s]
Epoch 46:  91%|█████████ | 10/11 [00:00<00:00, 69.03it/s]
Epoch 46:  91%|█████████ | 10/11 [00:00<00:00, 68.87it/s]
Epoch 46: 100%|██████████| 11/11 [00:00<00:00, 68.93it/s]
Epoch 46: 100%|██████████| 11/11 [00:00<00:00, 68.78it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 388.79it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 358.76it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 344.96it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 345.08it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 349.96it/s]


Epoch 46: 100%|██████████| 11/11 [00:00<00:00, 61.82it/s]
Epoch 46: 100%|██████████| 11/11 [00:00<00:00, 61.66it/s]
Epoch 46:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 47:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 47:   9%|▉         | 1/11 [00:00<00:00, 68.00it/s]
Epoch 47:   9%|▉         | 1/11 [00:00<00:00, 66.30it/s]
Epoch 47:  18%|█▊        | 2/11 [00:00<00:00, 66.42it/s]
Epoch 47:  18%|█▊        | 2/11 [00:00<00:00, 65.41it/s]
Epoch 47:  27%|██▋       | 3/11 [00:00<00:00, 65.59it/s]
Epoch 47:  27%|██▋       | 3/11 [00:00<00:00, 65.06it/s]
Epoch 47:  36%|███▋      | 4/11 [00:00<00:00, 65.50it/s]
Epoch 47:  36%|███▋      | 4/11 [00:00<00:00, 65.13it/s]
Epoch 47:  45%|████▌     | 5/11 [00:00<00:00, 65.79it/s]
Epoch 47:  45%|████▌     | 5/11 [00:00<00:00, 65.48it/s]
Epoch 47:  55%|█████▍    | 6/11 [00:00<00:00, 65.75it/s]
Epoch 47:  55%|█████▍    | 6/11 [00:00<00:00, 65.49it/s]
Epoch 47:  64%|██████▎   | 7/11 [00:00<00:00, 65.85it/s]
Epoch 47:  64%|██████▎   | 7/11 [00:00<00:00, 65.63it/s]
Epoch 47:  73%|███████▎  | 8/11 [00:00<00:00, 65.80it/s]
Epoch 47:  73%|███████▎  | 8/11 [00:00<00:00, 65.60it/s]
Epoch 47:  82%|████████▏ | 9/11 [00:00<00:00, 65.69it/s]
Epoch 47:  82%|████████▏ | 9/11 [00:00<00:00, 65.52it/s]
Epoch 47:  91%|█████████ | 10/11 [00:00<00:00, 65.75it/s]
Epoch 47:  91%|█████████ | 10/11 [00:00<00:00, 65.60it/s]
Epoch 47: 100%|██████████| 11/11 [00:00<00:00, 65.60it/s]
Epoch 47: 100%|██████████| 11/11 [00:00<00:00, 65.46it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 374.06it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 351.86it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 349.25it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 352.00it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 359.67it/s]


Epoch 47: 100%|██████████| 11/11 [00:00<00:00, 59.24it/s]
Epoch 47: 100%|██████████| 11/11 [00:00<00:00, 59.10it/s]
Epoch 47:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 48:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 48:   9%|▉         | 1/11 [00:00<00:00, 64.94it/s]
Epoch 48:   9%|▉         | 1/11 [00:00<00:00, 63.66it/s]
Epoch 48:  18%|█▊        | 2/11 [00:00<00:00, 65.34it/s]
Epoch 48:  18%|█▊        | 2/11 [00:00<00:00, 64.63it/s]
Epoch 48:  27%|██▋       | 3/11 [00:00<00:00, 66.24it/s]
Epoch 48:  27%|██▋       | 3/11 [00:00<00:00, 65.75it/s]
Epoch 48:  36%|███▋      | 4/11 [00:00<00:00, 66.67it/s]
Epoch 48:  36%|███▋      | 4/11 [00:00<00:00, 66.24it/s]
Epoch 48:  45%|████▌     | 5/11 [00:00<00:00, 67.10it/s]
Epoch 48:  45%|████▌     | 5/11 [00:00<00:00, 66.79it/s]
Epoch 48:  55%|█████▍    | 6/11 [00:00<00:00, 67.18it/s]
Epoch 48:  55%|█████▍    | 6/11 [00:00<00:00, 66.84it/s]
Epoch 48:  64%|██████▎   | 7/11 [00:00<00:00, 66.75it/s]
Epoch 48:  64%|██████▎   | 7/11 [00:00<00:00, 66.53it/s]
Epoch 48:  73%|███████▎  | 8/11 [00:00<00:00, 66.55it/s]
Epoch 48:  73%|███████▎  | 8/11 [00:00<00:00, 66.36it/s]
Epoch 48:  82%|████████▏ | 9/11 [00:00<00:00, 66.52it/s]
Epoch 48:  82%|████████▏ | 9/11 [00:00<00:00, 66.36it/s]
Epoch 48:  91%|█████████ | 10/11 [00:00<00:00, 66.36it/s]
Epoch 48:  91%|█████████ | 10/11 [00:00<00:00, 66.21it/s]
Epoch 48: 100%|██████████| 11/11 [00:00<00:00, 66.61it/s]
Epoch 48: 100%|██████████| 11/11 [00:00<00:00, 66.47it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 377.97it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 355.10it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 353.20it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 355.07it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 362.10it/s]


Epoch 48: 100%|██████████| 11/11 [00:00<00:00, 60.23it/s]
Epoch 48: 100%|██████████| 11/11 [00:00<00:00, 60.08it/s]
Epoch 48:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 49:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 49:   9%|▉         | 1/11 [00:00<00:00, 68.14it/s]
Epoch 49:   9%|▉         | 1/11 [00:00<00:00, 66.62it/s]
Epoch 49:  18%|█▊        | 2/11 [00:00<00:00, 66.11it/s]
Epoch 49:  18%|█▊        | 2/11 [00:00<00:00, 65.31it/s]
Epoch 49:  27%|██▋       | 3/11 [00:00<00:00, 66.15it/s]
Epoch 49:  27%|██▋       | 3/11 [00:00<00:00, 65.65it/s]
Epoch 49:  36%|███▋      | 4/11 [00:00<00:00, 66.44it/s]
Epoch 49:  36%|███▋      | 4/11 [00:00<00:00, 66.06it/s]
Epoch 49:  45%|████▌     | 5/11 [00:00<00:00, 66.54it/s]
Epoch 49:  45%|████▌     | 5/11 [00:00<00:00, 66.24it/s]
Epoch 49:  55%|█████▍    | 6/11 [00:00<00:00, 66.67it/s]
Epoch 49:  55%|█████▍    | 6/11 [00:00<00:00, 66.41it/s]
Epoch 49:  64%|██████▎   | 7/11 [00:00<00:00, 66.78it/s]
Epoch 49:  64%|██████▎   | 7/11 [00:00<00:00, 66.56it/s]
Epoch 49:  73%|███████▎  | 8/11 [00:00<00:00, 66.83it/s]
Epoch 49:  73%|███████▎  | 8/11 [00:00<00:00, 66.64it/s]
Epoch 49:  82%|████████▏ | 9/11 [00:00<00:00, 66.82it/s]
Epoch 49:  82%|████████▏ | 9/11 [00:00<00:00, 66.65it/s]
Epoch 49:  91%|█████████ | 10/11 [00:00<00:00, 66.83it/s]
Epoch 49:  91%|█████████ | 10/11 [00:00<00:00, 66.67it/s]
Epoch 49: 100%|██████████| 11/11 [00:00<00:00, 66.87it/s]
Epoch 49: 100%|██████████| 11/11 [00:00<00:00, 66.71it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 369.31it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 345.24it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 331.12it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 330.27it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 340.82it/s]


Epoch 49: 100%|██████████| 11/11 [00:00<00:00, 60.13it/s]
Epoch 49: 100%|██████████| 11/11 [00:00<00:00, 59.98it/s]
Epoch 49:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 50:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 50:   9%|▉         | 1/11 [00:00<00:00, 67.45it/s]
Epoch 50:   9%|▉         | 1/11 [00:00<00:00, 65.92it/s]
Epoch 50:  18%|█▊        | 2/11 [00:00<00:00, 65.61it/s]
Epoch 50:  18%|█▊        | 2/11 [00:00<00:00, 64.86it/s]
Epoch 50:  27%|██▋       | 3/11 [00:00<00:00, 65.78it/s]
Epoch 50:  27%|██▋       | 3/11 [00:00<00:00, 65.15it/s]
Epoch 50:  36%|███▋      | 4/11 [00:00<00:00, 66.48it/s]
Epoch 50:  36%|███▋      | 4/11 [00:00<00:00, 66.13it/s]
Epoch 50:  45%|████▌     | 5/11 [00:00<00:00, 66.96it/s]
Epoch 50:  45%|████▌     | 5/11 [00:00<00:00, 66.67it/s]
Epoch 50:  55%|█████▍    | 6/11 [00:00<00:00, 67.54it/s]
Epoch 50:  55%|█████▍    | 6/11 [00:00<00:00, 67.28it/s]
Epoch 50:  64%|██████▎   | 7/11 [00:00<00:00, 67.68it/s]
Epoch 50:  64%|██████▎   | 7/11 [00:00<00:00, 67.46it/s]
Epoch 50:  73%|███████▎  | 8/11 [00:00<00:00, 67.71it/s]
Epoch 50:  73%|███████▎  | 8/11 [00:00<00:00, 67.51it/s]
Epoch 50:  82%|████████▏ | 9/11 [00:00<00:00, 67.88it/s]
Epoch 50:  82%|████████▏ | 9/11 [00:00<00:00, 67.69it/s]
Epoch 50:  91%|█████████ | 10/11 [00:00<00:00, 68.02it/s]
Epoch 50:  91%|█████████ | 10/11 [00:00<00:00, 67.87it/s]
Epoch 50: 100%|██████████| 11/11 [00:00<00:00, 68.18it/s]
Epoch 50: 100%|██████████| 11/11 [00:00<00:00, 68.05it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 375.80it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 353.59it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 352.54it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 354.14it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 360.89it/s]


Epoch 50: 100%|██████████| 11/11 [00:00<00:00, 61.52it/s]
Epoch 50: 100%|██████████| 11/11 [00:00<00:00, 61.36it/s]
Epoch 50:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 51:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 51:   9%|▉         | 1/11 [00:00<00:00, 72.53it/s]
Epoch 51:   9%|▉         | 1/11 [00:00<00:00, 71.08it/s]
Epoch 51:  18%|█▊        | 2/11 [00:00<00:00, 72.63it/s]
Epoch 51:  18%|█▊        | 2/11 [00:00<00:00, 71.86it/s]
Epoch 51:  27%|██▋       | 3/11 [00:00<00:00, 72.48it/s]
Epoch 51:  27%|██▋       | 3/11 [00:00<00:00, 71.95it/s]
Epoch 51:  36%|███▋      | 4/11 [00:00<00:00, 72.42it/s]
Epoch 51:  36%|███▋      | 4/11 [00:00<00:00, 72.01it/s]
Epoch 51:  45%|████▌     | 5/11 [00:00<00:00, 71.86it/s]
Epoch 51:  45%|████▌     | 5/11 [00:00<00:00, 71.52it/s]
Epoch 51:  55%|█████▍    | 6/11 [00:00<00:00, 71.58it/s]
Epoch 51:  55%|█████▍    | 6/11 [00:00<00:00, 71.31it/s]
Epoch 51:  64%|██████▎   | 7/11 [00:00<00:00, 71.25it/s]
Epoch 51:  64%|██████▎   | 7/11 [00:00<00:00, 71.02it/s]
Epoch 51:  73%|███████▎  | 8/11 [00:00<00:00, 71.05it/s]
Epoch 51:  73%|███████▎  | 8/11 [00:00<00:00, 70.84it/s]
Epoch 51:  82%|████████▏ | 9/11 [00:00<00:00, 71.00it/s]
Epoch 51:  82%|████████▏ | 9/11 [00:00<00:00, 70.82it/s]
Epoch 51:  91%|█████████ | 10/11 [00:00<00:00, 70.99it/s]
Epoch 51:  91%|█████████ | 10/11 [00:00<00:00, 70.78it/s]
Epoch 51: 100%|██████████| 11/11 [00:00<00:00, 70.70it/s]
Epoch 51: 100%|██████████| 11/11 [00:00<00:00, 70.55it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 372.53it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 354.26it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 351.92it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 352.11it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 360.80it/s]


Epoch 51: 100%|██████████| 11/11 [00:00<00:00, 63.50it/s]
Epoch 51: 100%|██████████| 11/11 [00:00<00:00, 63.34it/s]
Epoch 51:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 52:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 52:   9%|▉         | 1/11 [00:00<00:00, 71.73it/s]
Epoch 52:   9%|▉         | 1/11 [00:00<00:00, 70.10it/s]
Epoch 52:  18%|█▊        | 2/11 [00:00<00:00, 70.48it/s]
Epoch 52:  18%|█▊        | 2/11 [00:00<00:00, 69.70it/s]
Epoch 52:  27%|██▋       | 3/11 [00:00<00:00, 70.18it/s]
Epoch 52:  27%|██▋       | 3/11 [00:00<00:00, 69.56it/s]
Epoch 52:  36%|███▋      | 4/11 [00:00<00:00, 70.14it/s]
Epoch 52:  36%|███▋      | 4/11 [00:00<00:00, 69.73it/s]
Epoch 52:  45%|████▌     | 5/11 [00:00<00:00, 69.73it/s]
Epoch 52:  45%|████▌     | 5/11 [00:00<00:00, 69.38it/s]
Epoch 52:  55%|█████▍    | 6/11 [00:00<00:00, 69.39it/s]
Epoch 52:  55%|█████▍    | 6/11 [00:00<00:00, 69.12it/s]
Epoch 52:  64%|██████▎   | 7/11 [00:00<00:00, 69.49it/s]
Epoch 52:  64%|██████▎   | 7/11 [00:00<00:00, 69.28it/s]
Epoch 52:  73%|███████▎  | 8/11 [00:00<00:00, 69.39it/s]
Epoch 52:  73%|███████▎  | 8/11 [00:00<00:00, 69.20it/s]
Epoch 52:  82%|████████▏ | 9/11 [00:00<00:00, 69.70it/s]
Epoch 52:  82%|████████▏ | 9/11 [00:00<00:00, 69.53it/s]
Epoch 52:  91%|█████████ | 10/11 [00:00<00:00, 69.99it/s]
Epoch 52:  91%|█████████ | 10/11 [00:00<00:00, 69.84it/s]
Epoch 52: 100%|██████████| 11/11 [00:00<00:00, 70.11it/s]
Epoch 52: 100%|██████████| 11/11 [00:00<00:00, 69.97it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 350.87it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 340.92it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 343.95it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 346.98it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 356.76it/s]


Epoch 52: 100%|██████████| 11/11 [00:00<00:00, 62.99it/s]
Epoch 52: 100%|██████████| 11/11 [00:00<00:00, 62.81it/s]
Epoch 52:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 53:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 53:   9%|▉         | 1/11 [00:00<00:00, 71.49it/s]
Epoch 53:   9%|▉         | 1/11 [00:00<00:00, 69.88it/s]
Epoch 53:  18%|█▊        | 2/11 [00:00<00:00, 70.68it/s]
Epoch 53:  18%|█▊        | 2/11 [00:00<00:00, 69.89it/s]
Epoch 53:  27%|██▋       | 3/11 [00:00<00:00, 71.01it/s]
Epoch 53:  27%|██▋       | 3/11 [00:00<00:00, 70.51it/s]
Epoch 53:  36%|███▋      | 4/11 [00:00<00:00, 70.96it/s]
Epoch 53:  36%|███▋      | 4/11 [00:00<00:00, 70.56it/s]
Epoch 53:  45%|████▌     | 5/11 [00:00<00:00, 70.88it/s]
Epoch 53:  45%|████▌     | 5/11 [00:00<00:00, 70.54it/s]
Epoch 53:  55%|█████▍    | 6/11 [00:00<00:00, 70.59it/s]
Epoch 53:  55%|█████▍    | 6/11 [00:00<00:00, 70.32it/s]
Epoch 53:  64%|██████▎   | 7/11 [00:00<00:00, 70.26it/s]
Epoch 53:  64%|██████▎   | 7/11 [00:00<00:00, 69.97it/s]
Epoch 53:  73%|███████▎  | 8/11 [00:00<00:00, 70.03it/s]
Epoch 53:  73%|███████▎  | 8/11 [00:00<00:00, 69.83it/s]
Epoch 53:  82%|████████▏ | 9/11 [00:00<00:00, 69.93it/s]
Epoch 53:  82%|████████▏ | 9/11 [00:00<00:00, 69.75it/s]
Epoch 53:  91%|█████████ | 10/11 [00:00<00:00, 69.89it/s]
Epoch 53:  91%|█████████ | 10/11 [00:00<00:00, 69.70it/s]
Epoch 53: 100%|██████████| 11/11 [00:00<00:00, 69.81it/s]
Epoch 53: 100%|██████████| 11/11 [00:00<00:00, 69.67it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 393.24it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 364.90it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 359.42it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 358.46it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 366.78it/s]


Epoch 53: 100%|██████████| 11/11 [00:00<00:00, 62.91it/s]
Epoch 53: 100%|██████████| 11/11 [00:00<00:00, 62.72it/s]
Epoch 53:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 54:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 54:   9%|▉         | 1/11 [00:00<00:00, 71.59it/s]
Epoch 54:   9%|▉         | 1/11 [00:00<00:00, 70.07it/s]
Epoch 54:  18%|█▊        | 2/11 [00:00<00:00, 71.78it/s]
Epoch 54:  18%|█▊        | 2/11 [00:00<00:00, 71.02it/s]
Epoch 54:  27%|██▋       | 3/11 [00:00<00:00, 72.03it/s]
Epoch 54:  27%|██▋       | 3/11 [00:00<00:00, 71.51it/s]
Epoch 54:  36%|███▋      | 4/11 [00:00<00:00, 72.13it/s]
Epoch 54:  36%|███▋      | 4/11 [00:00<00:00, 71.70it/s]
Epoch 54:  45%|████▌     | 5/11 [00:00<00:00, 71.71it/s]
Epoch 54:  45%|████▌     | 5/11 [00:00<00:00, 71.37it/s]
Epoch 54:  55%|█████▍    | 6/11 [00:00<00:00, 71.30it/s]
Epoch 54:  55%|█████▍    | 6/11 [00:00<00:00, 71.03it/s]
Epoch 54:  64%|██████▎   | 7/11 [00:00<00:00, 71.13it/s]
Epoch 54:  64%|██████▎   | 7/11 [00:00<00:00, 70.88it/s]
Epoch 54:  73%|███████▎  | 8/11 [00:00<00:00, 70.61it/s]
Epoch 54:  73%|███████▎  | 8/11 [00:00<00:00, 70.40it/s]
Epoch 54:  82%|████████▏ | 9/11 [00:00<00:00, 70.46it/s]
Epoch 54:  82%|████████▏ | 9/11 [00:00<00:00, 70.28it/s]
Epoch 54:  91%|█████████ | 10/11 [00:00<00:00, 70.54it/s]
Epoch 54:  91%|█████████ | 10/11 [00:00<00:00, 70.38it/s]
Epoch 54: 100%|██████████| 11/11 [00:00<00:00, 70.56it/s]
Epoch 54: 100%|██████████| 11/11 [00:00<00:00, 70.41it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 381.96it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 353.34it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 354.25it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 355.07it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 361.98it/s]


Epoch 54: 100%|██████████| 11/11 [00:00<00:00, 63.46it/s]
Epoch 54: 100%|██████████| 11/11 [00:00<00:00, 63.30it/s]
Epoch 54:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 55:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 55:   9%|▉         | 1/11 [00:00<00:00, 72.42it/s]
Epoch 55:   9%|▉         | 1/11 [00:00<00:00, 70.86it/s]
Epoch 55:  18%|█▊        | 2/11 [00:00<00:00, 71.50it/s]
Epoch 55:  18%|█▊        | 2/11 [00:00<00:00, 70.69it/s]
Epoch 55:  27%|██▋       | 3/11 [00:00<00:00, 69.57it/s]
Epoch 55:  27%|██▋       | 3/11 [00:00<00:00, 69.02it/s]
Epoch 55:  36%|███▋      | 4/11 [00:00<00:00, 69.26it/s]
Epoch 55:  36%|███▋      | 4/11 [00:00<00:00, 68.87it/s]
Epoch 55:  45%|████▌     | 5/11 [00:00<00:00, 69.23it/s]
Epoch 55:  45%|████▌     | 5/11 [00:00<00:00, 68.91it/s]
Epoch 55:  55%|█████▍    | 6/11 [00:00<00:00, 69.21it/s]
Epoch 55:  55%|█████▍    | 6/11 [00:00<00:00, 68.93it/s]
Epoch 55:  64%|██████▎   | 7/11 [00:00<00:00, 69.18it/s]
Epoch 55:  64%|██████▎   | 7/11 [00:00<00:00, 68.92it/s]
Epoch 55:  73%|███████▎  | 8/11 [00:00<00:00, 69.39it/s]
Epoch 55:  73%|███████▎  | 8/11 [00:00<00:00, 69.20it/s]
Epoch 55:  82%|████████▏ | 9/11 [00:00<00:00, 69.66it/s]
Epoch 55:  82%|████████▏ | 9/11 [00:00<00:00, 69.46it/s]
Epoch 55:  91%|█████████ | 10/11 [00:00<00:00, 68.60it/s]
Epoch 55:  91%|█████████ | 10/11 [00:00<00:00, 68.43it/s]
Epoch 55: 100%|██████████| 11/11 [00:00<00:00, 68.45it/s]
Epoch 55: 100%|██████████| 11/11 [00:00<00:00, 68.29it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 359.53it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 340.35it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 342.08it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.40it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 350.88it/s]


Epoch 55: 100%|██████████| 11/11 [00:00<00:00, 61.53it/s]
Epoch 55: 100%|██████████| 11/11 [00:00<00:00, 61.37it/s]
Epoch 55:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 56:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 56:   9%|▉         | 1/11 [00:00<00:00, 69.34it/s]
Epoch 56:   9%|▉         | 1/11 [00:00<00:00, 67.31it/s]
Epoch 56:  18%|█▊        | 2/11 [00:00<00:00, 65.78it/s]
Epoch 56:  18%|█▊        | 2/11 [00:00<00:00, 65.02it/s]
Epoch 56:  27%|██▋       | 3/11 [00:00<00:00, 63.59it/s]
Epoch 56:  27%|██▋       | 3/11 [00:00<00:00, 63.10it/s]
Epoch 56:  36%|███▋      | 4/11 [00:00<00:00, 64.30it/s]
Epoch 56:  36%|███▋      | 4/11 [00:00<00:00, 63.94it/s]
Epoch 56:  45%|████▌     | 5/11 [00:00<00:00, 64.22it/s]
Epoch 56:  45%|████▌     | 5/11 [00:00<00:00, 63.93it/s]
Epoch 56:  55%|█████▍    | 6/11 [00:00<00:00, 64.25it/s]
Epoch 56:  55%|█████▍    | 6/11 [00:00<00:00, 63.97it/s]
Epoch 56:  64%|██████▎   | 7/11 [00:00<00:00, 64.32it/s]
Epoch 56:  64%|██████▎   | 7/11 [00:00<00:00, 64.10it/s]
Epoch 56:  73%|███████▎  | 8/11 [00:00<00:00, 64.56it/s]
Epoch 56:  73%|███████▎  | 8/11 [00:00<00:00, 64.37it/s]
Epoch 56:  82%|████████▏ | 9/11 [00:00<00:00, 64.83it/s]
Epoch 56:  82%|████████▏ | 9/11 [00:00<00:00, 64.67it/s]
Epoch 56:  91%|█████████ | 10/11 [00:00<00:00, 64.91it/s]
Epoch 56:  91%|█████████ | 10/11 [00:00<00:00, 64.76it/s]
Epoch 56: 100%|██████████| 11/11 [00:00<00:00, 64.73it/s]
Epoch 56: 100%|██████████| 11/11 [00:00<00:00, 64.59it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 360.96it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 339.02it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 339.13it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.46it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 352.40it/s]


Epoch 56: 100%|██████████| 11/11 [00:00<00:00, 58.55it/s]
Epoch 56: 100%|██████████| 11/11 [00:00<00:00, 58.41it/s]
Epoch 56:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 57:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 57:   9%|▉         | 1/11 [00:00<00:00, 62.64it/s]
Epoch 57:   9%|▉         | 1/11 [00:00<00:00, 61.49it/s]
Epoch 57:  18%|█▊        | 2/11 [00:00<00:00, 65.32it/s]
Epoch 57:  18%|█▊        | 2/11 [00:00<00:00, 64.69it/s]
Epoch 57:  27%|██▋       | 3/11 [00:00<00:00, 66.31it/s]
Epoch 57:  27%|██▋       | 3/11 [00:00<00:00, 65.85it/s]
Epoch 57:  36%|███▋      | 4/11 [00:00<00:00, 66.85it/s]
Epoch 57:  36%|███▋      | 4/11 [00:00<00:00, 66.48it/s]
Epoch 57:  45%|████▌     | 5/11 [00:00<00:00, 64.54it/s]
Epoch 57:  45%|████▌     | 5/11 [00:00<00:00, 64.26it/s]
Epoch 57:  55%|█████▍    | 6/11 [00:00<00:00, 64.63it/s]
Epoch 57:  55%|█████▍    | 6/11 [00:00<00:00, 64.38it/s]
Epoch 57:  64%|██████▎   | 7/11 [00:00<00:00, 64.24it/s]
Epoch 57:  64%|██████▎   | 7/11 [00:00<00:00, 64.04it/s]
Epoch 57:  73%|███████▎  | 8/11 [00:00<00:00, 64.48it/s]
Epoch 57:  73%|███████▎  | 8/11 [00:00<00:00, 64.31it/s]
Epoch 57:  82%|████████▏ | 9/11 [00:00<00:00, 63.77it/s]
Epoch 57:  82%|████████▏ | 9/11 [00:00<00:00, 63.62it/s]
Epoch 57:  91%|█████████ | 10/11 [00:00<00:00, 64.32it/s]
Epoch 57:  91%|█████████ | 10/11 [00:00<00:00, 64.18it/s]
Epoch 57: 100%|██████████| 11/11 [00:00<00:00, 63.60it/s]
Epoch 57: 100%|██████████| 11/11 [00:00<00:00, 63.47it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 375.53it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 351.84it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 338.94it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 343.64it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 352.53it/s]


Epoch 57: 100%|██████████| 11/11 [00:00<00:00, 57.56it/s]
Epoch 57: 100%|██████████| 11/11 [00:00<00:00, 57.41it/s]
Epoch 57:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 58:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 58:   9%|▉         | 1/11 [00:00<00:00, 61.38it/s]
Epoch 58:   9%|▉         | 1/11 [00:00<00:00, 59.97it/s]
Epoch 58:  18%|█▊        | 2/11 [00:00<00:00, 60.35it/s]
Epoch 58:  18%|█▊        | 2/11 [00:00<00:00, 59.63it/s]
Epoch 58:  27%|██▋       | 3/11 [00:00<00:00, 61.24it/s]
Epoch 58:  27%|██▋       | 3/11 [00:00<00:00, 60.81it/s]
Epoch 58:  36%|███▋      | 4/11 [00:00<00:00, 60.04it/s]
Epoch 58:  36%|███▋      | 4/11 [00:00<00:00, 59.74it/s]
Epoch 58:  45%|████▌     | 5/11 [00:00<00:00, 61.36it/s]
Epoch 58:  45%|████▌     | 5/11 [00:00<00:00, 61.12it/s]
Epoch 58:  55%|█████▍    | 6/11 [00:00<00:00, 62.63it/s]
Epoch 58:  55%|█████▍    | 6/11 [00:00<00:00, 62.43it/s]
Epoch 58:  64%|██████▎   | 7/11 [00:00<00:00, 63.13it/s]
Epoch 58:  64%|██████▎   | 7/11 [00:00<00:00, 62.95it/s]
Epoch 58:  73%|███████▎  | 8/11 [00:00<00:00, 63.91it/s]
Epoch 58:  73%|███████▎  | 8/11 [00:00<00:00, 63.74it/s]
Epoch 58:  82%|████████▏ | 9/11 [00:00<00:00, 64.15it/s]
Epoch 58:  82%|████████▏ | 9/11 [00:00<00:00, 63.98it/s]
Epoch 58:  91%|█████████ | 10/11 [00:00<00:00, 64.41it/s]
Epoch 58:  91%|█████████ | 10/11 [00:00<00:00, 64.28it/s]
Epoch 58: 100%|██████████| 11/11 [00:00<00:00, 63.70it/s]
Epoch 58: 100%|██████████| 11/11 [00:00<00:00, 63.57it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 360.03it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 345.68it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 348.08it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 350.14it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 358.07it/s]


Epoch 58: 100%|██████████| 11/11 [00:00<00:00, 57.79it/s]
Epoch 58: 100%|██████████| 11/11 [00:00<00:00, 57.65it/s]
Epoch 58:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 59:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 59:   9%|▉         | 1/11 [00:00<00:00, 67.51it/s]
Epoch 59:   9%|▉         | 1/11 [00:00<00:00, 66.05it/s]
Epoch 59:  18%|█▊        | 2/11 [00:00<00:00, 66.87it/s]
Epoch 59:  18%|█▊        | 2/11 [00:00<00:00, 66.13it/s]
Epoch 59:  27%|██▋       | 3/11 [00:00<00:00, 66.26it/s]
Epoch 59:  27%|██▋       | 3/11 [00:00<00:00, 65.68it/s]
Epoch 59:  36%|███▋      | 4/11 [00:00<00:00, 65.19it/s]
Epoch 59:  36%|███▋      | 4/11 [00:00<00:00, 64.79it/s]
Epoch 59:  45%|████▌     | 5/11 [00:00<00:00, 64.81it/s]
Epoch 59:  45%|████▌     | 5/11 [00:00<00:00, 64.47it/s]
Epoch 59:  55%|█████▍    | 6/11 [00:00<00:00, 64.72it/s]
Epoch 59:  55%|█████▍    | 6/11 [00:00<00:00, 64.48it/s]
Epoch 59:  64%|██████▎   | 7/11 [00:00<00:00, 64.14it/s]
Epoch 59:  64%|██████▎   | 7/11 [00:00<00:00, 63.90it/s]
Epoch 59:  73%|███████▎  | 8/11 [00:00<00:00, 63.39it/s]
Epoch 59:  73%|███████▎  | 8/11 [00:00<00:00, 63.21it/s]
Epoch 59:  82%|████████▏ | 9/11 [00:00<00:00, 62.94it/s]
Epoch 59:  82%|████████▏ | 9/11 [00:00<00:00, 62.79it/s]
Epoch 59:  91%|█████████ | 10/11 [00:00<00:00, 62.54it/s]
Epoch 59:  91%|█████████ | 10/11 [00:00<00:00, 62.40it/s]
Epoch 59: 100%|██████████| 11/11 [00:00<00:00, 62.66it/s]
Epoch 59: 100%|██████████| 11/11 [00:00<00:00, 62.55it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 398.96it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 367.81it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 362.77it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 360.64it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 368.50it/s]


Epoch 59: 100%|██████████| 11/11 [00:00<00:00, 56.98it/s]
Epoch 59: 100%|██████████| 11/11 [00:00<00:00, 56.82it/s]
Epoch 59:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 60:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 60:   9%|▉         | 1/11 [00:00<00:00, 59.60it/s]
Epoch 60:   9%|▉         | 1/11 [00:00<00:00, 58.57it/s]
Epoch 60:  18%|█▊        | 2/11 [00:00<00:00, 58.57it/s]
Epoch 60:  18%|█▊        | 2/11 [00:00<00:00, 58.01it/s]
Epoch 60:  27%|██▋       | 3/11 [00:00<00:00, 59.34it/s]
Epoch 60:  27%|██▋       | 3/11 [00:00<00:00, 58.94it/s]
Epoch 60:  36%|███▋      | 4/11 [00:00<00:00, 59.82it/s]
Epoch 60:  36%|███▋      | 4/11 [00:00<00:00, 59.51it/s]
Epoch 60:  45%|████▌     | 5/11 [00:00<00:00, 60.35it/s]
Epoch 60:  45%|████▌     | 5/11 [00:00<00:00, 60.11it/s]
Epoch 60:  55%|█████▍    | 6/11 [00:00<00:00, 60.76it/s]
Epoch 60:  55%|█████▍    | 6/11 [00:00<00:00, 60.57it/s]
Epoch 60:  64%|██████▎   | 7/11 [00:00<00:00, 61.09it/s]
Epoch 60:  64%|██████▎   | 7/11 [00:00<00:00, 60.92it/s]
Epoch 60:  73%|███████▎  | 8/11 [00:00<00:00, 61.19it/s]
Epoch 60:  73%|███████▎  | 8/11 [00:00<00:00, 61.04it/s]
Epoch 60:  82%|████████▏ | 9/11 [00:00<00:00, 61.08it/s]
Epoch 60:  82%|████████▏ | 9/11 [00:00<00:00, 60.94it/s]
Epoch 60:  91%|█████████ | 10/11 [00:00<00:00, 60.96it/s]
Epoch 60:  91%|█████████ | 10/11 [00:00<00:00, 60.84it/s]
Epoch 60: 100%|██████████| 11/11 [00:00<00:00, 60.63it/s]
Epoch 60: 100%|██████████| 11/11 [00:00<00:00, 60.52it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 386.25it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 347.33it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 347.84it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 348.21it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 353.80it/s]


Epoch 60: 100%|██████████| 11/11 [00:00<00:00, 55.16it/s]
Epoch 60: 100%|██████████| 11/11 [00:00<00:00, 55.03it/s]
Epoch 60:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 61:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 61:   9%|▉         | 1/11 [00:00<00:00, 58.80it/s]
Epoch 61:   9%|▉         | 1/11 [00:00<00:00, 57.64it/s]
Epoch 61:  18%|█▊        | 2/11 [00:00<00:00, 57.90it/s]
Epoch 61:  18%|█▊        | 2/11 [00:00<00:00, 57.36it/s]
Epoch 61:  27%|██▋       | 3/11 [00:00<00:00, 58.18it/s]
Epoch 61:  27%|██▋       | 3/11 [00:00<00:00, 57.82it/s]
Epoch 61:  36%|███▋      | 4/11 [00:00<00:00, 58.45it/s]
Epoch 61:  36%|███▋      | 4/11 [00:00<00:00, 58.19it/s]
Epoch 61:  45%|████▌     | 5/11 [00:00<00:00, 58.53it/s]
Epoch 61:  45%|████▌     | 5/11 [00:00<00:00, 58.31it/s]
Epoch 61:  55%|█████▍    | 6/11 [00:00<00:00, 58.35it/s]
Epoch 61:  55%|█████▍    | 6/11 [00:00<00:00, 58.14it/s]
Epoch 61:  64%|██████▎   | 7/11 [00:00<00:00, 58.13it/s]
Epoch 61:  64%|██████▎   | 7/11 [00:00<00:00, 57.97it/s]
Epoch 61:  73%|███████▎  | 8/11 [00:00<00:00, 57.88it/s]
Epoch 61:  73%|███████▎  | 8/11 [00:00<00:00, 57.74it/s]
Epoch 61:  82%|████████▏ | 9/11 [00:00<00:00, 57.46it/s]
Epoch 61:  82%|████████▏ | 9/11 [00:00<00:00, 57.33it/s]
Epoch 61:  91%|█████████ | 10/11 [00:00<00:00, 57.17it/s]
Epoch 61:  91%|█████████ | 10/11 [00:00<00:00, 57.05it/s]
Epoch 61: 100%|██████████| 11/11 [00:00<00:00, 56.94it/s]
Epoch 61: 100%|██████████| 11/11 [00:00<00:00, 56.84it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 400.18it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 367.73it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 361.16it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 360.52it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 368.24it/s]


Epoch 61: 100%|██████████| 11/11 [00:00<00:00, 52.28it/s]
Epoch 61: 100%|██████████| 11/11 [00:00<00:00, 52.17it/s]
Epoch 61:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 62:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 62:   9%|▉         | 1/11 [00:00<00:00, 54.60it/s]
Epoch 62:   9%|▉         | 1/11 [00:00<00:00, 53.54it/s]
Epoch 62:  18%|█▊        | 2/11 [00:00<00:00, 52.84it/s]
Epoch 62:  18%|█▊        | 2/11 [00:00<00:00, 52.34it/s]
Epoch 62:  27%|██▋       | 3/11 [00:00<00:00, 51.78it/s]
Epoch 62:  27%|██▋       | 3/11 [00:00<00:00, 51.45it/s]
Epoch 62:  36%|███▋      | 4/11 [00:00<00:00, 51.38it/s]
Epoch 62:  36%|███▋      | 4/11 [00:00<00:00, 51.14it/s]
Epoch 62:  45%|████▌     | 5/11 [00:00<00:00, 51.12it/s]
Epoch 62:  45%|████▌     | 5/11 [00:00<00:00, 50.92it/s]
Epoch 62:  55%|█████▍    | 6/11 [00:00<00:00, 50.35it/s]
Epoch 62:  55%|█████▍    | 6/11 [00:00<00:00, 50.19it/s]
Epoch 62:  64%|██████▎   | 7/11 [00:00<00:00, 49.92it/s]
Epoch 62:  64%|██████▎   | 7/11 [00:00<00:00, 49.78it/s]
Epoch 62:  73%|███████▎  | 8/11 [00:00<00:00, 49.72it/s]
Epoch 62:  73%|███████▎  | 8/11 [00:00<00:00, 49.60it/s]
Epoch 62:  82%|████████▏ | 9/11 [00:00<00:00, 49.53it/s]
Epoch 62:  82%|████████▏ | 9/11 [00:00<00:00, 49.43it/s]
Epoch 62:  91%|█████████ | 10/11 [00:00<00:00, 49.31it/s]
Epoch 62:  91%|█████████ | 10/11 [00:00<00:00, 49.22it/s]
Epoch 62: 100%|██████████| 11/11 [00:00<00:00, 49.08it/s]
Epoch 62: 100%|██████████| 11/11 [00:00<00:00, 49.00it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 315.76it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 317.98it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 324.59it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 332.39it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 342.99it/s]


Epoch 62: 100%|██████████| 11/11 [00:00<00:00, 45.30it/s]
Epoch 62: 100%|██████████| 11/11 [00:00<00:00, 45.21it/s]
Epoch 62:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 63:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 63:   9%|▉         | 1/11 [00:00<00:00, 44.36it/s]
Epoch 63:   9%|▉         | 1/11 [00:00<00:00, 43.68it/s]
Epoch 63:  18%|█▊        | 2/11 [00:00<00:00, 45.04it/s]
Epoch 63:  18%|█▊        | 2/11 [00:00<00:00, 44.68it/s]
Epoch 63:  27%|██▋       | 3/11 [00:00<00:00, 45.16it/s]
Epoch 63:  27%|██▋       | 3/11 [00:00<00:00, 44.91it/s]
Epoch 63:  36%|███▋      | 4/11 [00:00<00:00, 45.05it/s]
Epoch 63:  36%|███▋      | 4/11 [00:00<00:00, 44.86it/s]
Epoch 63:  45%|████▌     | 5/11 [00:00<00:00, 44.99it/s]
Epoch 63:  45%|████▌     | 5/11 [00:00<00:00, 44.84it/s]
Epoch 63:  55%|█████▍    | 6/11 [00:00<00:00, 44.91it/s]
Epoch 63:  55%|█████▍    | 6/11 [00:00<00:00, 44.79it/s]
Epoch 63:  64%|██████▎   | 7/11 [00:00<00:00, 44.77it/s]
Epoch 63:  64%|██████▎   | 7/11 [00:00<00:00, 44.67it/s]
Epoch 63:  73%|███████▎  | 8/11 [00:00<00:00, 44.65it/s]
Epoch 63:  73%|███████▎  | 8/11 [00:00<00:00, 44.56it/s]
Epoch 63:  82%|████████▏ | 9/11 [00:00<00:00, 44.58it/s]
Epoch 63:  82%|████████▏ | 9/11 [00:00<00:00, 44.50it/s]
Epoch 63:  91%|█████████ | 10/11 [00:00<00:00, 44.51it/s]
Epoch 63:  91%|█████████ | 10/11 [00:00<00:00, 44.44it/s]
Epoch 63: 100%|██████████| 11/11 [00:00<00:00, 44.41it/s]
Epoch 63: 100%|██████████| 11/11 [00:00<00:00, 44.33it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 352.23it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 333.97it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 338.51it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.53it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 350.40it/s]


Epoch 63: 100%|██████████| 11/11 [00:00<00:00, 41.38it/s]
Epoch 63: 100%|██████████| 11/11 [00:00<00:00, 41.30it/s]
Epoch 63:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 64:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 64:   9%|▉         | 1/11 [00:00<00:00, 44.70it/s]
Epoch 64:   9%|▉         | 1/11 [00:00<00:00, 44.02it/s]
Epoch 64:  18%|█▊        | 2/11 [00:00<00:00, 43.91it/s]
Epoch 64:  18%|█▊        | 2/11 [00:00<00:00, 43.58it/s]
Epoch 64:  27%|██▋       | 3/11 [00:00<00:00, 43.24it/s]
Epoch 64:  27%|██▋       | 3/11 [00:00<00:00, 43.00it/s]
Epoch 64:  36%|███▋      | 4/11 [00:00<00:00, 42.80it/s]
Epoch 64:  36%|███▋      | 4/11 [00:00<00:00, 42.63it/s]
Epoch 64:  45%|████▌     | 5/11 [00:00<00:00, 42.49it/s]
Epoch 64:  45%|████▌     | 5/11 [00:00<00:00, 42.35it/s]
Epoch 64:  55%|█████▍    | 6/11 [00:00<00:00, 42.27it/s]
Epoch 64:  55%|█████▍    | 6/11 [00:00<00:00, 42.15it/s]
Epoch 64:  64%|██████▎   | 7/11 [00:00<00:00, 41.91it/s]
Epoch 64:  64%|██████▎   | 7/11 [00:00<00:00, 41.82it/s]
Epoch 64:  73%|███████▎  | 8/11 [00:00<00:00, 41.77it/s]
Epoch 64:  73%|███████▎  | 8/11 [00:00<00:00, 41.68it/s]
Epoch 64:  82%|████████▏ | 9/11 [00:00<00:00, 41.59it/s]
Epoch 64:  82%|████████▏ | 9/11 [00:00<00:00, 41.52it/s]
Epoch 64:  91%|█████████ | 10/11 [00:00<00:00, 41.45it/s]
Epoch 64:  91%|█████████ | 10/11 [00:00<00:00, 41.38it/s]
Epoch 64: 100%|██████████| 11/11 [00:00<00:00, 41.20it/s]
Epoch 64: 100%|██████████| 11/11 [00:00<00:00, 41.14it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 347.47it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 334.14it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 337.31it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 341.13it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 351.66it/s]


Epoch 64: 100%|██████████| 11/11 [00:00<00:00, 38.54it/s]
Epoch 64: 100%|██████████| 11/11 [00:00<00:00, 38.48it/s]
Epoch 64:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 65:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 65:   9%|▉         | 1/11 [00:00<00:00, 39.24it/s]
Epoch 65:   9%|▉         | 1/11 [00:00<00:00, 38.68it/s]
Epoch 65:  18%|█▊        | 2/11 [00:00<00:00, 38.78it/s]
Epoch 65:  18%|█▊        | 2/11 [00:00<00:00, 38.50it/s]
Epoch 65:  27%|██▋       | 3/11 [00:00<00:00, 38.96it/s]
Epoch 65:  27%|██▋       | 3/11 [00:00<00:00, 38.77it/s]
Epoch 65:  36%|███▋      | 4/11 [00:00<00:00, 38.79it/s]
Epoch 65:  36%|███▋      | 4/11 [00:00<00:00, 38.59it/s]
Epoch 65:  45%|████▌     | 5/11 [00:00<00:00, 38.58it/s]
Epoch 65:  45%|████▌     | 5/11 [00:00<00:00, 38.47it/s]
Epoch 65:  55%|█████▍    | 6/11 [00:00<00:00, 38.61it/s]
Epoch 65:  55%|█████▍    | 6/11 [00:00<00:00, 38.52it/s]
Epoch 65:  64%|██████▎   | 7/11 [00:00<00:00, 38.76it/s]
Epoch 65:  64%|██████▎   | 7/11 [00:00<00:00, 38.68it/s]
Epoch 65:  73%|███████▎  | 8/11 [00:00<00:00, 38.88it/s]
Epoch 65:  73%|███████▎  | 8/11 [00:00<00:00, 38.82it/s]
Epoch 65:  82%|████████▏ | 9/11 [00:00<00:00, 38.90it/s]
Epoch 65:  82%|████████▏ | 9/11 [00:00<00:00, 38.84it/s]
Epoch 65:  91%|█████████ | 10/11 [00:00<00:00, 38.89it/s]
Epoch 65:  91%|█████████ | 10/11 [00:00<00:00, 38.84it/s]
Epoch 65: 100%|██████████| 11/11 [00:00<00:00, 38.88it/s]
Epoch 65: 100%|██████████| 11/11 [00:00<00:00, 38.84it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 376.75it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 338.91it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 340.36it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 342.80it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 345.19it/s]


Epoch 65: 100%|██████████| 11/11 [00:00<00:00, 36.54it/s]
Epoch 65: 100%|██████████| 11/11 [00:00<00:00, 36.48it/s]
Epoch 65:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 66:   0%|          | 0/11 [00:00<?, ?it/s]
Epoch 66:   9%|▉         | 1/11 [00:00<00:00, 38.49it/s]
Epoch 66:   9%|▉         | 1/11 [00:00<00:00, 37.94it/s]
Epoch 66:  18%|█▊        | 2/11 [00:00<00:00, 37.80it/s]
Epoch 66:  18%|█▊        | 2/11 [00:00<00:00, 37.53it/s]
Epoch 66:  27%|██▋       | 3/11 [00:00<00:00, 37.65it/s]
Epoch 66:  27%|██▋       | 3/11 [00:00<00:00, 37.47it/s]
Epoch 66:  36%|███▋      | 4/11 [00:00<00:00, 37.39it/s]
Epoch 66:  36%|███▋      | 4/11 [00:00<00:00, 37.25it/s]
Epoch 66:  45%|████▌     | 5/11 [00:00<00:00, 37.31it/s]
Epoch 66:  45%|████▌     | 5/11 [00:00<00:00, 37.21it/s]
Epoch 66:  55%|█████▍    | 6/11 [00:00<00:00, 37.32it/s]
Epoch 66:  55%|█████▍    | 6/11 [00:00<00:00, 37.23it/s]
Epoch 66:  64%|██████▎   | 7/11 [00:00<00:00, 37.19it/s]
Epoch 66:  64%|██████▎   | 7/11 [00:00<00:00, 37.10it/s]
Epoch 66:  73%|███████▎  | 8/11 [00:00<00:00, 36.99it/s]
Epoch 66:  73%|███████▎  | 8/11 [00:00<00:00, 36.93it/s]
Epoch 66:  82%|████████▏ | 9/11 [00:00<00:00, 36.98it/s]
Epoch 66:  82%|████████▏ | 9/11 [00:00<00:00, 36.92it/s]
Epoch 66:  91%|█████████ | 10/11 [00:00<00:00, 37.02it/s]
Epoch 66:  91%|█████████ | 10/11 [00:00<00:00, 36.97it/s]
Epoch 66: 100%|██████████| 11/11 [00:00<00:00, 37.02it/s]
Epoch 66: 100%|██████████| 11/11 [00:00<00:00, 36.98it/s]

Validation: |          | 0/? [00:00<?, ?it/s]

Validation:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]

Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 368.86it/s]

Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 344.64it/s]

Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 346.87it/s]

Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 348.67it/s]

Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 355.82it/s]


Epoch 66: 100%|██████████| 11/11 [00:00<00:00, 34.94it/s]
Epoch 66: 100%|██████████| 11/11 [00:00<00:00, 34.89it/s]
Epoch 66: 100%|██████████| 11/11 [00:00<00:00, 34.83it/s]

Validation: |          | 0/? [00:00<?, ?it/s]
Validation:   0%|          | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0:   0%|          | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0:  20%|██        | 1/5 [00:00<00:00, 331.07it/s]
Validation DataLoader 0:  40%|████      | 2/5 [00:00<00:00, 346.21it/s]
Validation DataLoader 0:  60%|██████    | 3/5 [00:00<00:00, 350.14it/s]
Validation DataLoader 0:  80%|████████  | 4/5 [00:00<00:00, 354.01it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 359.99it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 344.68it/s]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     Validate metric           DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
        val_loss             2.769702672958374
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Changed graph_conv_layers in AttentionWeightedGNN
Changed dropout_prob in AttentionWeightedGNN
Reset fused layers in AttentionWeightedGNN
/home/docs/checkouts/readthedocs.org/user_builds/fusilli/envs/latest/lib/python3.9/site-packages/lightning/pytorch/loops/fit_loop.py:298: The number of training batches (1) is smaller than the logging interval Trainer(log_every_n_steps=2). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.

Training: |          | 0/? [00:00<?, ?it/s]
Training:   0%|          | 0/1 [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/1 [00:00<?, ?it/s]
Epoch 0: 100%|██████████| 1/1 [00:00<00:00,  2.60it/s]
Epoch 0: 100%|██████████| 1/1 [00:00<00:00,  2.59it/s, v_num=dGNN]
Epoch 0: 100%|██████████| 1/1 [00:00<00:00,  1.47it/s, v_num=dGNN, val_loss=2.49e+4]
Epoch 0: 100%|██████████| 1/1 [00:00<00:00,  1.47it/s, v_num=dGNN, val_loss=2.49e+4, train_loss=1.48e+5]
Epoch 0:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=2.49e+4, train_loss=1.48e+5]
Epoch 1:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=2.49e+4, train_loss=1.48e+5]
Epoch 1: 100%|██████████| 1/1 [00:00<00:00,  3.08it/s, v_num=dGNN, val_loss=2.49e+4, train_loss=1.48e+5]
Epoch 1: 100%|██████████| 1/1 [00:00<00:00,  3.08it/s, v_num=dGNN, val_loss=2.49e+4, train_loss=1.48e+5]
Epoch 1: 100%|██████████| 1/1 [00:00<00:00,  1.70it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=1.48e+5]
Epoch 1: 100%|██████████| 1/1 [00:00<00:00,  1.70it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=8.7e+4]
Epoch 1:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=8.7e+4]
Epoch 2:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=8.7e+4]
Epoch 2: 100%|██████████| 1/1 [00:00<00:00,  3.34it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=8.7e+4]
Epoch 2: 100%|██████████| 1/1 [00:00<00:00,  3.33it/s, v_num=dGNN, val_loss=2.28e+3, train_loss=8.7e+4]
Epoch 2: 100%|██████████| 1/1 [00:00<00:00,  1.76it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=8.7e+4]
Epoch 2: 100%|██████████| 1/1 [00:00<00:00,  1.76it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=4.76e+4]
Epoch 2:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=4.76e+4]
Epoch 3:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=4.76e+4]
Epoch 3: 100%|██████████| 1/1 [00:00<00:00,  2.69it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=4.76e+4]
Epoch 3: 100%|██████████| 1/1 [00:00<00:00,  2.69it/s, v_num=dGNN, val_loss=1.32e+4, train_loss=4.76e+4]
Epoch 3: 100%|██████████| 1/1 [00:00<00:00,  1.48it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=4.76e+4]
Epoch 3: 100%|██████████| 1/1 [00:00<00:00,  1.48it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=3.23e+4]
Epoch 3:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=3.23e+4]
Epoch 4:   0%|          | 0/1 [00:00<?, ?it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=3.23e+4]
Epoch 4: 100%|██████████| 1/1 [00:00<00:00,  2.67it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=3.23e+4]
Epoch 4: 100%|██████████| 1/1 [00:00<00:00,  2.67it/s, v_num=dGNN, val_loss=1.15e+4, train_loss=3.23e+4]
Epoch 4: 100%|██████████| 1/1 [00:00<00:00,  1.48it/s, v_num=dGNN, val_loss=4.3e+3, train_loss=3.23e+4]
Epoch 4: 100%|██████████| 1/1 [00:00<00:00,  1.48it/s, v_num=dGNN, val_loss=4.3e+3, train_loss=2.16e+4]
Epoch 4: 100%|██████████| 1/1 [00:00<00:00,  1.44it/s, v_num=dGNN, val_loss=4.3e+3, train_loss=2.16e+4]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     Validate metric           DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
         MAE_val            59.173885345458984
         R2_val             -515.9340209960938
        val_loss              4301.3505859375
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

It worked! Let’s have a look at the model structure to see what changes have been made.

print("Attention Weighted MLP:\n", datamodule.graph_maker_instance.AttentionWeightingMLPInstance)
print("Fusion model:\n", trained_model_list[0].model)

Attention Weighted MLP:
 AttentionWeightMLP(
  (weighting_layers): ModuleDict(
    (Layer 1): Sequential(
      (0): Linear(in_features=784, out_features=500, bias=True)
      (1): ReLU()
    )
    (Layer 2): Sequential(
      (0): Linear(in_features=500, out_features=128, bias=True)
      (1): ReLU()
    )
    (Layer 3): Sequential(
      (0): Linear(in_features=128, out_features=128, bias=True)
      (1): ReLU()
    )
    (Layer 4): Sequential(
      (0): Linear(in_features=128, out_features=500, bias=True)
      (1): ReLU()
    )
    (Layer 5): Sequential(
      (0): Linear(in_features=500, out_features=784, bias=True)
      (1): ReLU()
    )
  )
  (fused_layers): Sequential(
    (0): Linear(in_features=784, out_features=256, bias=True)
    (1): ReLU()
    (2): Dropout(p=0.15, inplace=False)
    (3): Linear(in_features=256, out_features=64, bias=True)
    (4): ReLU()
    (5): Dropout(p=0.15, inplace=False)
  )
  (final_prediction): Sequential(
    (0): Linear(in_features=64, out_features=1, bias=True)
  )
)
Fusion model:
 AttentionWeightedGNN(
  (graph_conv_layers): Sequential(
    (0): ChebConv(392, 50, K=3, normalization=sym)
    (1): ChebConv(50, 100, K=3, normalization=sym)
    (2): ChebConv(100, 130, K=3, normalization=sym)
  )
  (final_prediction): Sequential(
    (0): Linear(in_features=130, out_features=1, bias=True)
  )
)

You can see that the input features to the final_prediction layer changed to fit with our modification to the graph_conv_layers output features!

What happens when the modifications are incorrect?

Let’s see what happens when we try to modify an attribute that doesn’t exist.

layer_mods = {
    "AttentionWeightedGraphMaker": {
        "AttentionWeightingMLPInstance.fake_attribute": nn.Sequential(
            nn.Linear(30, 100),
            nn.ReLU(),
        ),
    }
}

try:
    datamodule = prepare_fusion_data(prediction_task=prediction_task,
                                     fusion_model=AttentionWeightedGNN,
                                     data_paths=data_paths,
                                     output_paths=output_paths,
                                     layer_mods=layer_mods,
                                     max_epochs=5)
except Exception as error:
    print(error)

Layer group AttentionWeightingMLPInstance.fake_attribute not found in AttentionWeightedGraphMaker

What about modifying an attribute with the wrong data type?

dropout_prob should be an float and between 0 and 1.
graph_conv_layers should be an nn.Sequential of graph convolutional layers.
edge_probability_threshold should be a float between 0 and 100.
AttentionWeightingMLPInstance.weighting_layers should be an nn.ModuleDict

layer_mods = {
    "AttentionWeightedGraphMaker": {
        "AttentionWeightingMLPInstance.weighting_layers": nn.Sequential(
            nn.Linear(392, 75),
            nn.ReLU(),
            nn.Linear(75, 75),
            nn.ReLU(),
            nn.Linear(75, 30),
            nn.ReLU()
        ),
    }
}

try:
    prepare_fusion_data(prediction_task=prediction_task,
                        fusion_model=AttentionWeightedGNN,
                        data_paths=data_paths,
                        output_paths=output_paths,
                        layer_mods=layer_mods,
                        max_epochs=5)
except Exception as error:
    print(error)

Changed weighting_layers in AttentionWeightedGraphMaker
('Incorrect data type for the modifications: Attribute weighting_layers must be of type ModuleDict, not dtype Sequential.',)

layer_mods = {
    "AttentionWeightedGraphMaker": {
        "edge_probability_threshold": "two",
    }
}

try:
    prepare_fusion_data(prediction_task=prediction_task,
                        fusion_model=AttentionWeightedGNN,
                        data_paths=data_paths,
                        output_paths=output_paths,
                        layer_mods=layer_mods,
                        max_epochs=5)
except Exception as error:
    print(error)

Changed edge_probability_threshold in AttentionWeightedGraphMaker
('Incorrect data type for the modifications: Attribute edge_probability_threshold must be of type int, not dtype str.',)

What about modifying multiple attributes with the conflicting modifications?

For this, let’s switch to looking at the ConcatTabularFeatureMaps model. This model concatenates the feature maps of the two modalities and then passes them through a prediction layer.

We can modify the layers that each tabular modality goes through before being concatenated, as well as the layers that come after the concatenation.

The output features of our modified mod1_layers and mod2_layers are 100 and 128, so the input features of the fused_layers should be 228. However, we’ve set the input features of the fused_layers to be 25.

Let’s see what happens when we try to modify the model in this way. It should throw an error when the data is passed through the model.

layer_mods = {
    "ConcatTabularFeatureMaps": {
        "mod1_layers": nn.ModuleDict(
            {
                "layer 1": nn.Sequential(
                    nn.Linear(392, 300),
                    nn.ReLU(),
                ),
                "layer 2": nn.Sequential(
                    nn.Linear(300, 128),
                    nn.ReLU(),
                ),
                "layer 3": nn.Sequential(
                    nn.Linear(128, 128),
                    nn.ReLU(),
                ),
            }
        ),
        "mod2_layers": nn.ModuleDict(
            {
                "layer 1": nn.Sequential(
                    nn.Linear(392, 300),
                    nn.ReLU(),
                ),
                "layer 2": nn.Sequential(
                    nn.Linear(300, 128),
                    nn.ReLU(),
                ),
                "layer 3": nn.Sequential(
                    nn.Linear(128, 100),
                    nn.ReLU(),
                ),
            }
        ),
        "fused_layers": nn.Sequential(
            nn.Linear(25, 150),
            nn.ReLU(),
            nn.Linear(150, 75),
            nn.ReLU(),
            nn.Linear(75, 50),
            nn.ReLU(),
        ),
    },
}

# get the data and train the model

from fusilli.fusionmodels.tabularfusion.concat_feature_maps import ConcatTabularFeatureMaps

datamodule = prepare_fusion_data(prediction_task=prediction_task,
                                 fusion_model=ConcatTabularFeatureMaps,
                                 data_paths=data_paths,
                                 output_paths=output_paths,
                                 layer_mods=layer_mods,
                                 max_epochs=5)
trained_model_list = train_and_save_models(
    data_module=datamodule,
    fusion_model=ConcatTabularFeatureMaps,
    layer_mods=layer_mods,
    max_epochs=5,
)

Changed mod1_layers in ConcatTabularFeatureMaps
Changed mod2_layers in ConcatTabularFeatureMaps
Changed fused_layers in ConcatTabularFeatureMaps
Reset fused layers in ConcatTabularFeatureMaps

Training: |          | 0/? [00:00<?, ?it/s]
Training:   0%|          | 0/50 [00:00<?, ?it/s]
Epoch 0:   0%|          | 0/50 [00:00<?, ?it/s]
Epoch 0:   2%|▏         | 1/50 [00:00<00:00, 72.96it/s]
Epoch 0:   2%|▏         | 1/50 [00:00<00:00, 71.27it/s, v_num=Maps]
Epoch 0:   4%|▍         | 2/50 [00:00<00:00, 81.93it/s, v_num=Maps]
Epoch 0:   4%|▍         | 2/50 [00:00<00:00, 80.88it/s, v_num=Maps]
Epoch 0:   6%|▌         | 3/50 [00:00<00:00, 84.03it/s, v_num=Maps]
Epoch 0:   6%|▌         | 3/50 [00:00<00:00, 83.29it/s, v_num=Maps]
Epoch 0:   8%|▊         | 4/50 [00:00<00:00, 86.70it/s, v_num=Maps]
Epoch 0:   8%|▊         | 4/50 [00:00<00:00, 86.09it/s, v_num=Maps]
Epoch 0:  10%|█         | 5/50 [00:00<00:00, 88.70it/s, v_num=Maps]
Epoch 0:  10%|█         | 5/50 [00:00<00:00, 88.20it/s, v_num=Maps]
Epoch 0:  12%|█▏        | 6/50 [00:00<00:00, 90.55it/s, v_num=Maps]
Epoch 0:  12%|█▏        | 6/50 [00:00<00:00, 90.13it/s, v_num=Maps]
Epoch 0:  14%|█▍        | 7/50 [00:00<00:00, 91.85it/s, v_num=Maps]
Epoch 0:  14%|█▍        | 7/50 [00:00<00:00, 91.49it/s, v_num=Maps]
Epoch 0:  16%|█▌        | 8/50 [00:00<00:00, 92.99it/s, v_num=Maps]
Epoch 0:  16%|█▌        | 8/50 [00:00<00:00, 92.64it/s, v_num=Maps]
Epoch 0:  18%|█▊        | 9/50 [00:00<00:00, 93.79it/s, v_num=Maps]
Epoch 0:  18%|█▊        | 9/50 [00:00<00:00, 93.50it/s, v_num=Maps]
Epoch 0:  20%|██        | 10/50 [00:00<00:00, 94.02it/s, v_num=Maps]
Epoch 0:  20%|██        | 10/50 [00:00<00:00, 93.75it/s, v_num=Maps]
Epoch 0:  22%|██▏       | 11/50 [00:00<00:00, 94.61it/s, v_num=Maps]
Epoch 0:  22%|██▏       | 11/50 [00:00<00:00, 94.36it/s, v_num=Maps]
Epoch 0:  24%|██▍       | 12/50 [00:00<00:00, 95.31it/s, v_num=Maps]
Epoch 0:  24%|██▍       | 12/50 [00:00<00:00, 95.09it/s, v_num=Maps]
Epoch 0:  26%|██▌       | 13/50 [00:00<00:00, 95.80it/s, v_num=Maps]
Epoch 0:  26%|██▌       | 13/50 [00:00<00:00, 95.59it/s, v_num=Maps]
Epoch 0:  28%|██▊       | 14/50 [00:00<00:00, 96.22it/s, v_num=Maps]
Epoch 0:  28%|██▊       | 14/50 [00:00<00:00, 96.01it/s, v_num=Maps]
Epoch 0:  30%|███       | 15/50 [00:00<00:00, 96.56it/s, v_num=Maps]
Epoch 0:  30%|███       | 15/50 [00:00<00:00, 96.37it/s, v_num=Maps]
Epoch 0:  32%|███▏      | 16/50 [00:00<00:00, 96.84it/s, v_num=Maps]
Epoch 0:  32%|███▏      | 16/50 [00:00<00:00, 96.66it/s, v_num=Maps]
Epoch 0:  34%|███▍      | 17/50 [00:00<00:00, 96.73it/s, v_num=Maps]
Epoch 0:  34%|███▍      | 17/50 [00:00<00:00, 96.56it/s, v_num=Maps]
Epoch 0:  36%|███▌      | 18/50 [00:00<00:00, 97.01it/s, v_num=Maps]
Epoch 0:  36%|███▌      | 18/50 [00:00<00:00, 96.85it/s, v_num=Maps]
Epoch 0:  38%|███▊      | 19/50 [00:00<00:00, 97.10it/s, v_num=Maps]
Epoch 0:  38%|███▊      | 19/50 [00:00<00:00, 96.94it/s, v_num=Maps]
Epoch 0:  40%|████      | 20/50 [00:00<00:00, 97.25it/s, v_num=Maps]
Epoch 0:  40%|████      | 20/50 [00:00<00:00, 97.10it/s, v_num=Maps]
Epoch 0:  42%|████▏     | 21/50 [00:00<00:00, 97.37it/s, v_num=Maps]
Epoch 0:  42%|████▏     | 21/50 [00:00<00:00, 97.22it/s, v_num=Maps]
Epoch 0:  44%|████▍     | 22/50 [00:00<00:00, 97.48it/s, v_num=Maps]
Epoch 0:  44%|████▍     | 22/50 [00:00<00:00, 97.35it/s, v_num=Maps]
Epoch 0:  46%|████▌     | 23/50 [00:00<00:00, 97.53it/s, v_num=Maps]
Epoch 0:  46%|████▌     | 23/50 [00:00<00:00, 97.39it/s, v_num=Maps]
Epoch 0:  48%|████▊     | 24/50 [00:00<00:00, 97.32it/s, v_num=Maps]
Epoch 0:  48%|████▊     | 24/50 [00:00<00:00, 97.15it/s, v_num=Maps]
Epoch 0:  50%|█████     | 25/50 [00:00<00:00, 97.34it/s, v_num=Maps]
Epoch 0:  50%|█████     | 25/50 [00:00<00:00, 97.22it/s, v_num=Maps]
Epoch 0:  52%|█████▏    | 26/50 [00:00<00:00, 97.33it/s, v_num=Maps]
Epoch 0:  52%|█████▏    | 26/50 [00:00<00:00, 97.21it/s, v_num=Maps]
Epoch 0:  54%|█████▍    | 27/50 [00:00<00:00, 97.34it/s, v_num=Maps]
Epoch 0:  54%|█████▍    | 27/50 [00:00<00:00, 97.23it/s, v_num=Maps]
Epoch 0:  56%|█████▌    | 28/50 [00:00<00:00, 97.47it/s, v_num=Maps]
Epoch 0:  56%|█████▌    | 28/50 [00:00<00:00, 97.36it/s, v_num=Maps]
Epoch 0:  58%|█████▊    | 29/50 [00:00<00:00, 97.58it/s, v_num=Maps]
Epoch 0:  58%|█████▊    | 29/50 [00:00<00:00, 97.47it/s, v_num=Maps]
Epoch 0:  60%|██████    | 30/50 [00:00<00:00, 97.63it/s, v_num=Maps]
Epoch 0:  60%|██████    | 30/50 [00:00<00:00, 97.53it/s, v_num=Maps]
Epoch 0:  62%|██████▏   | 31/50 [00:00<00:00, 97.51it/s, v_num=Maps]
Epoch 0:  62%|██████▏   | 31/50 [00:00<00:00, 97.41it/s, v_num=Maps]
Epoch 0:  64%|██████▍   | 32/50 [00:00<00:00, 97.48it/s, v_num=Maps]
Epoch 0:  64%|██████▍   | 32/50 [00:00<00:00, 97.39it/s, v_num=Maps]
Epoch 0:  66%|██████▌   | 33/50 [00:00<00:00, 97.55it/s, v_num=Maps]
Epoch 0:  66%|██████▌   | 33/50 [00:00<00:00, 97.45it/s, v_num=Maps]
Epoch 0:  68%|██████▊   | 34/50 [00:00<00:00, 97.55it/s, v_num=Maps]
Epoch 0:  68%|██████▊   | 34/50 [00:00<00:00, 97.46it/s, v_num=Maps]
Epoch 0:  70%|███████   | 35/50 [00:00<00:00, 97.63it/s, v_num=Maps]
Epoch 0:  70%|███████   | 35/50 [00:00<00:00, 97.53it/s, v_num=Maps]
Epoch 0:  72%|███████▏  | 36/50 [00:00<00:00, 97.74it/s, v_num=Maps]
Epoch 0:  72%|███████▏  | 36/50 [00:00<00:00, 97.66it/s, v_num=Maps]
Epoch 0:  74%|███████▍  | 37/50 [00:00<00:00, 97.90it/s, v_num=Maps]
Epoch 0:  74%|███████▍  | 37/50 [00:00<00:00, 97.82it/s, v_num=Maps]
Epoch 0:  76%|███████▌  | 38/50 [00:00<00:00, 97.85it/s, v_num=Maps]
Epoch 0:  76%|███████▌  | 38/50 [00:00<00:00, 97.78it/s, v_num=Maps]
Epoch 0:  78%|███████▊  | 39/50 [00:00<00:00, 98.02it/s, v_num=Maps]
Epoch 0:  78%|███████▊  | 39/50 [00:00<00:00, 97.95it/s, v_num=Maps]
Epoch 0:  80%|████████  | 40/50 [00:00<00:00, 98.21it/s, v_num=Maps]
Epoch 0:  80%|████████  | 40/50 [00:00<00:00, 98.15it/s, v_num=Maps]
Epoch 0:  82%|████████▏ | 41/50 [00:00<00:00, 98.44it/s, v_num=Maps]
Epoch 0:  82%|████████▏ | 41/50 [00:00<00:00, 98.38it/s, v_num=Maps]
Epoch 0:  84%|████████▍ | 42/50 [00:00<00:00, 98.62it/s, v_num=Maps]
Epoch 0:  84%|████████▍ | 42/50 [00:00<00:00, 98.56it/s, v_num=Maps]
Epoch 0:  86%|████████▌ | 43/50 [00:00<00:00, 98.77it/s, v_num=Maps]
Epoch 0:  86%|████████▌ | 43/50 [00:00<00:00, 98.70it/s, v_num=Maps]
Epoch 0:  88%|████████▊ | 44/50 [00:00<00:00, 98.91it/s, v_num=Maps]
Epoch 0:  88%|████████▊ | 44/50 [00:00<00:00, 98.84it/s, v_num=Maps]
Epoch 0:  90%|█████████ | 45/50 [00:00<00:00, 98.89it/s, v_num=Maps]
Epoch 0:  90%|█████████ | 45/50 [00:00<00:00, 98.83it/s, v_num=Maps]
Epoch 0:  92%|█████████▏| 46/50 [00:00<00:00, 99.01it/s, v_num=Maps]
Epoch 0:  92%|█████████▏| 46/50 [00:00<00:00, 98.95it/s, v_num=Maps]
Epoch 0:  94%|█████████▍| 47/50 [00:00<00:00, 99.12it/s, v_num=Maps]
Epoch 0:  94%|█████████▍| 47/50 [00:00<00:00, 99.06it/s, v_num=Maps]
Epoch 0:  96%|█████████▌| 48/50 [00:00<00:00, 99.23it/s, v_num=Maps]
Epoch 0:  96%|█████████▌| 48/50 [00:00<00:00, 99.17it/s, v_num=Maps]
Epoch 0:  98%|█████████▊| 49/50 [00:00<00:00, 99.16it/s, v_num=Maps]
Epoch 0:  98%|█████████▊| 49/50 [00:00<00:00, 99.10it/s, v_num=Maps]
Epoch 0: 100%|██████████| 50/50 [00:00<00:00, 99.03it/s, v_num=Maps]
Epoch 0: 100%|██████████| 50/50 [00:00<00:00, 98.96it/s, v_num=Maps]
Epoch 0: 100%|██████████| 50/50 [00:00<00:00, 93.01it/s, v_num=Maps, val_loss=4.920]
Epoch 0: 100%|██████████| 50/50 [00:00<00:00, 92.86it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 0:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   2%|▏         | 1/50 [00:00<00:00, 93.84it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   2%|▏         | 1/50 [00:00<00:00, 91.06it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   4%|▍         | 2/50 [00:00<00:00, 98.10it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   4%|▍         | 2/50 [00:00<00:00, 96.61it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   6%|▌         | 3/50 [00:00<00:00, 99.70it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   6%|▌         | 3/50 [00:00<00:00, 98.61it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   8%|▊         | 4/50 [00:00<00:00, 99.59it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:   8%|▊         | 4/50 [00:00<00:00, 98.71it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  10%|█         | 5/50 [00:00<00:00, 98.09it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  10%|█         | 5/50 [00:00<00:00, 97.43it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  12%|█▏        | 6/50 [00:00<00:00, 98.62it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  12%|█▏        | 6/50 [00:00<00:00, 98.08it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  14%|█▍        | 7/50 [00:00<00:00, 98.81it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  14%|█▍        | 7/50 [00:00<00:00, 98.35it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  16%|█▌        | 8/50 [00:00<00:00, 98.70it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  16%|█▌        | 8/50 [00:00<00:00, 98.30it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  18%|█▊        | 9/50 [00:00<00:00, 98.76it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  18%|█▊        | 9/50 [00:00<00:00, 98.41it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  20%|██        | 10/50 [00:00<00:00, 98.99it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  20%|██        | 10/50 [00:00<00:00, 98.67it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  22%|██▏       | 11/50 [00:00<00:00, 99.11it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  22%|██▏       | 11/50 [00:00<00:00, 98.84it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  24%|██▍       | 12/50 [00:00<00:00, 99.10it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  24%|██▍       | 12/50 [00:00<00:00, 98.85it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  26%|██▌       | 13/50 [00:00<00:00, 99.63it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  26%|██▌       | 13/50 [00:00<00:00, 99.40it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  28%|██▊       | 14/50 [00:00<00:00, 99.96it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  28%|██▊       | 14/50 [00:00<00:00, 99.73it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  30%|███       | 15/50 [00:00<00:00, 100.25it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  30%|███       | 15/50 [00:00<00:00, 100.01it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  32%|███▏      | 16/50 [00:00<00:00, 100.49it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  32%|███▏      | 16/50 [00:00<00:00, 100.29it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  34%|███▍      | 17/50 [00:00<00:00, 100.68it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  34%|███▍      | 17/50 [00:00<00:00, 100.50it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  36%|███▌      | 18/50 [00:00<00:00, 100.84it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  36%|███▌      | 18/50 [00:00<00:00, 100.66it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  38%|███▊      | 19/50 [00:00<00:00, 100.67it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  38%|███▊      | 19/50 [00:00<00:00, 100.49it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  40%|████      | 20/50 [00:00<00:00, 100.77it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  40%|████      | 20/50 [00:00<00:00, 100.61it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  42%|████▏     | 21/50 [00:00<00:00, 100.99it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  42%|████▏     | 21/50 [00:00<00:00, 100.84it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  44%|████▍     | 22/50 [00:00<00:00, 101.06it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  44%|████▍     | 22/50 [00:00<00:00, 100.89it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  46%|████▌     | 23/50 [00:00<00:00, 101.19it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  46%|████▌     | 23/50 [00:00<00:00, 101.05it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  48%|████▊     | 24/50 [00:00<00:00, 101.28it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  48%|████▊     | 24/50 [00:00<00:00, 101.14it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  50%|█████     | 25/50 [00:00<00:00, 101.34it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  50%|█████     | 25/50 [00:00<00:00, 101.20it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  52%|█████▏    | 26/50 [00:00<00:00, 101.19it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  52%|█████▏    | 26/50 [00:00<00:00, 101.06it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  54%|█████▍    | 27/50 [00:00<00:00, 101.04it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  54%|█████▍    | 27/50 [00:00<00:00, 100.92it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  56%|█████▌    | 28/50 [00:00<00:00, 100.89it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  56%|█████▌    | 28/50 [00:00<00:00, 100.73it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  58%|█████▊    | 29/50 [00:00<00:00, 100.75it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  58%|█████▊    | 29/50 [00:00<00:00, 100.64it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  60%|██████    | 30/50 [00:00<00:00, 100.74it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  60%|██████    | 30/50 [00:00<00:00, 100.61it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  62%|██████▏   | 31/50 [00:00<00:00, 100.77it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  62%|██████▏   | 31/50 [00:00<00:00, 100.66it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  64%|██████▍   | 32/50 [00:00<00:00, 100.83it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  64%|██████▍   | 32/50 [00:00<00:00, 100.65it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  66%|██████▌   | 33/50 [00:00<00:00, 100.48it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  66%|██████▌   | 33/50 [00:00<00:00, 100.39it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  68%|██████▊   | 34/50 [00:00<00:00, 100.51it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  68%|██████▊   | 34/50 [00:00<00:00, 100.41it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  70%|███████   | 35/50 [00:00<00:00, 100.46it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  70%|███████   | 35/50 [00:00<00:00, 100.36it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  72%|███████▏  | 36/50 [00:00<00:00, 100.48it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  72%|███████▏  | 36/50 [00:00<00:00, 100.39it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  74%|███████▍  | 37/50 [00:00<00:00, 100.55it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  74%|███████▍  | 37/50 [00:00<00:00, 100.46it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  76%|███████▌  | 38/50 [00:00<00:00, 100.63it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  76%|███████▌  | 38/50 [00:00<00:00, 100.53it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  78%|███████▊  | 39/50 [00:00<00:00, 100.60it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  78%|███████▊  | 39/50 [00:00<00:00, 100.52it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  80%|████████  | 40/50 [00:00<00:00, 100.37it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  80%|████████  | 40/50 [00:00<00:00, 100.29it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  82%|████████▏ | 41/50 [00:00<00:00, 100.34it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  82%|████████▏ | 41/50 [00:00<00:00, 100.25it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  84%|████████▍ | 42/50 [00:00<00:00, 100.20it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  84%|████████▍ | 42/50 [00:00<00:00, 100.08it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  86%|████████▌ | 43/50 [00:00<00:00, 100.01it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  86%|████████▌ | 43/50 [00:00<00:00, 99.92it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  88%|████████▊ | 44/50 [00:00<00:00, 99.82it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  88%|████████▊ | 44/50 [00:00<00:00, 99.74it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  90%|█████████ | 45/50 [00:00<00:00, 99.77it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  90%|█████████ | 45/50 [00:00<00:00, 99.70it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  92%|█████████▏| 46/50 [00:00<00:00, 99.68it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  92%|█████████▏| 46/50 [00:00<00:00, 99.61it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  94%|█████████▍| 47/50 [00:00<00:00, 99.55it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  94%|█████████▍| 47/50 [00:00<00:00, 99.48it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  96%|█████████▌| 48/50 [00:00<00:00, 99.57it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  96%|█████████▌| 48/50 [00:00<00:00, 99.50it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  98%|█████████▊| 49/50 [00:00<00:00, 99.57it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1:  98%|█████████▊| 49/50 [00:00<00:00, 99.50it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1: 100%|██████████| 50/50 [00:00<00:00, 99.51it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1: 100%|██████████| 50/50 [00:00<00:00, 99.44it/s, v_num=Maps, val_loss=4.920, train_loss=9.640]
Epoch 1: 100%|██████████| 50/50 [00:00<00:00, 94.30it/s, v_num=Maps, val_loss=5.130, train_loss=9.640]
Epoch 1: 100%|██████████| 50/50 [00:00<00:00, 94.15it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 1:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   2%|▏         | 1/50 [00:00<00:00, 90.65it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   2%|▏         | 1/50 [00:00<00:00, 87.85it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   4%|▍         | 2/50 [00:00<00:00, 94.57it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   4%|▍         | 2/50 [00:00<00:00, 93.06it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   6%|▌         | 3/50 [00:00<00:00, 96.30it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   6%|▌         | 3/50 [00:00<00:00, 95.28it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   8%|▊         | 4/50 [00:00<00:00, 97.72it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:   8%|▊         | 4/50 [00:00<00:00, 96.96it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  10%|█         | 5/50 [00:00<00:00, 99.23it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  10%|█         | 5/50 [00:00<00:00, 98.60it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  12%|█▏        | 6/50 [00:00<00:00, 99.50it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  12%|█▏        | 6/50 [00:00<00:00, 99.00it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  14%|█▍        | 7/50 [00:00<00:00, 100.83it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  14%|█▍        | 7/50 [00:00<00:00, 100.42it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  16%|█▌        | 8/50 [00:00<00:00, 101.99it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  16%|█▌        | 8/50 [00:00<00:00, 101.63it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  18%|█▊        | 9/50 [00:00<00:00, 102.46it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  18%|█▊        | 9/50 [00:00<00:00, 102.13it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  20%|██        | 10/50 [00:00<00:00, 103.14it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  20%|██        | 10/50 [00:00<00:00, 102.84it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  22%|██▏       | 11/50 [00:00<00:00, 103.73it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  22%|██▏       | 11/50 [00:00<00:00, 103.46it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  24%|██▍       | 12/50 [00:00<00:00, 104.21it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  24%|██▍       | 12/50 [00:00<00:00, 103.95it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  26%|██▌       | 13/50 [00:00<00:00, 104.13it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  26%|██▌       | 13/50 [00:00<00:00, 103.87it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  28%|██▊       | 14/50 [00:00<00:00, 103.95it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  28%|██▊       | 14/50 [00:00<00:00, 103.67it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  30%|███       | 15/50 [00:00<00:00, 103.67it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  30%|███       | 15/50 [00:00<00:00, 103.44it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  32%|███▏      | 16/50 [00:00<00:00, 103.42it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  32%|███▏      | 16/50 [00:00<00:00, 103.16it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  34%|███▍      | 17/50 [00:00<00:00, 102.67it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  34%|███▍      | 17/50 [00:00<00:00, 102.45it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  36%|███▌      | 18/50 [00:00<00:00, 102.40it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  36%|███▌      | 18/50 [00:00<00:00, 102.21it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  38%|███▊      | 19/50 [00:00<00:00, 102.22it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  38%|███▊      | 19/50 [00:00<00:00, 102.03it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  40%|████      | 20/50 [00:00<00:00, 101.81it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  40%|████      | 20/50 [00:00<00:00, 101.63it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  42%|████▏     | 21/50 [00:00<00:00, 101.90it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  42%|████▏     | 21/50 [00:00<00:00, 101.74it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  44%|████▍     | 22/50 [00:00<00:00, 101.95it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  44%|████▍     | 22/50 [00:00<00:00, 101.80it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  46%|████▌     | 23/50 [00:00<00:00, 101.97it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  46%|████▌     | 23/50 [00:00<00:00, 101.82it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  48%|████▊     | 24/50 [00:00<00:00, 101.93it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  48%|████▊     | 24/50 [00:00<00:00, 101.79it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  50%|█████     | 25/50 [00:00<00:00, 102.07it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  50%|█████     | 25/50 [00:00<00:00, 101.94it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  52%|█████▏    | 26/50 [00:00<00:00, 101.63it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  52%|█████▏    | 26/50 [00:00<00:00, 101.50it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  54%|█████▍    | 27/50 [00:00<00:00, 101.32it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  54%|█████▍    | 27/50 [00:00<00:00, 101.19it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  56%|█████▌    | 28/50 [00:00<00:00, 101.11it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  56%|█████▌    | 28/50 [00:00<00:00, 100.99it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  58%|█████▊    | 29/50 [00:00<00:00, 100.82it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  58%|█████▊    | 29/50 [00:00<00:00, 100.69it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  60%|██████    | 30/50 [00:00<00:00, 100.54it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  60%|██████    | 30/50 [00:00<00:00, 100.42it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  62%|██████▏   | 31/50 [00:00<00:00, 100.47it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  62%|██████▏   | 31/50 [00:00<00:00, 100.36it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  64%|██████▍   | 32/50 [00:00<00:00, 100.61it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  64%|██████▍   | 32/50 [00:00<00:00, 100.51it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  66%|██████▌   | 33/50 [00:00<00:00, 100.75it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  66%|██████▌   | 33/50 [00:00<00:00, 100.65it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  68%|██████▊   | 34/50 [00:00<00:00, 100.64it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  68%|██████▊   | 34/50 [00:00<00:00, 100.54it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  70%|███████   | 35/50 [00:00<00:00, 100.66it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  70%|███████   | 35/50 [00:00<00:00, 100.56it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  72%|███████▏  | 36/50 [00:00<00:00, 100.73it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  72%|███████▏  | 36/50 [00:00<00:00, 100.64it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  74%|███████▍  | 37/50 [00:00<00:00, 100.77it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  74%|███████▍  | 37/50 [00:00<00:00, 100.68it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  76%|███████▌  | 38/50 [00:00<00:00, 100.80it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  76%|███████▌  | 38/50 [00:00<00:00, 100.72it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  78%|███████▊  | 39/50 [00:00<00:00, 100.88it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  78%|███████▊  | 39/50 [00:00<00:00, 100.78it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  80%|████████  | 40/50 [00:00<00:00, 100.92it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  80%|████████  | 40/50 [00:00<00:00, 100.83it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  82%|████████▏ | 41/50 [00:00<00:00, 100.64it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  82%|████████▏ | 41/50 [00:00<00:00, 100.55it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  84%|████████▍ | 42/50 [00:00<00:00, 100.56it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  84%|████████▍ | 42/50 [00:00<00:00, 100.47it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  86%|████████▌ | 43/50 [00:00<00:00, 100.46it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  86%|████████▌ | 43/50 [00:00<00:00, 100.38it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  88%|████████▊ | 44/50 [00:00<00:00, 100.38it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  88%|████████▊ | 44/50 [00:00<00:00, 100.30it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  90%|█████████ | 45/50 [00:00<00:00, 100.24it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  90%|█████████ | 45/50 [00:00<00:00, 100.16it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  92%|█████████▏| 46/50 [00:00<00:00, 100.19it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  92%|█████████▏| 46/50 [00:00<00:00, 100.12it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  94%|█████████▍| 47/50 [00:00<00:00, 100.14it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  94%|█████████▍| 47/50 [00:00<00:00, 100.06it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  96%|█████████▌| 48/50 [00:00<00:00, 99.98it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  96%|█████████▌| 48/50 [00:00<00:00, 99.92it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  98%|█████████▊| 49/50 [00:00<00:00, 100.05it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2:  98%|█████████▊| 49/50 [00:00<00:00, 99.98it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2: 100%|██████████| 50/50 [00:00<00:00, 100.11it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2: 100%|██████████| 50/50 [00:00<00:00, 100.05it/s, v_num=Maps, val_loss=5.130, train_loss=3.760]
Epoch 2: 100%|██████████| 50/50 [00:00<00:00, 95.19it/s, v_num=Maps, val_loss=3.510, train_loss=3.760]
Epoch 2: 100%|██████████| 50/50 [00:00<00:00, 95.04it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 2:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   2%|▏         | 1/50 [00:00<00:00, 93.35it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   2%|▏         | 1/50 [00:00<00:00, 90.46it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   4%|▍         | 2/50 [00:00<00:00, 97.31it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   4%|▍         | 2/50 [00:00<00:00, 95.80it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   6%|▌         | 3/50 [00:00<00:00, 99.11it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   6%|▌         | 3/50 [00:00<00:00, 97.86it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   8%|▊         | 4/50 [00:00<00:00, 99.32it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:   8%|▊         | 4/50 [00:00<00:00, 98.52it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  10%|█         | 5/50 [00:00<00:00, 98.64it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  10%|█         | 5/50 [00:00<00:00, 98.01it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  12%|█▏        | 6/50 [00:00<00:00, 99.50it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  12%|█▏        | 6/50 [00:00<00:00, 98.94it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  14%|█▍        | 7/50 [00:00<00:00, 100.00it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  14%|█▍        | 7/50 [00:00<00:00, 99.52it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  16%|█▌        | 8/50 [00:00<00:00, 100.05it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  16%|█▌        | 8/50 [00:00<00:00, 99.62it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  18%|█▊        | 9/50 [00:00<00:00, 100.17it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  18%|█▊        | 9/50 [00:00<00:00, 99.79it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  20%|██        | 10/50 [00:00<00:00, 100.31it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  20%|██        | 10/50 [00:00<00:00, 99.96it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  22%|██▏       | 11/50 [00:00<00:00, 100.36it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  22%|██▏       | 11/50 [00:00<00:00, 100.05it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  24%|██▍       | 12/50 [00:00<00:00, 99.43it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  24%|██▍       | 12/50 [00:00<00:00, 99.14it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  26%|██▌       | 13/50 [00:00<00:00, 99.38it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  26%|██▌       | 13/50 [00:00<00:00, 99.12it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  28%|██▊       | 14/50 [00:00<00:00, 99.63it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  28%|██▊       | 14/50 [00:00<00:00, 99.39it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  30%|███       | 15/50 [00:00<00:00, 99.80it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  30%|███       | 15/50 [00:00<00:00, 99.56it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  32%|███▏      | 16/50 [00:00<00:00, 99.75it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  32%|███▏      | 16/50 [00:00<00:00, 99.55it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  34%|███▍      | 17/50 [00:00<00:00, 99.97it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  34%|███▍      | 17/50 [00:00<00:00, 99.76it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  36%|███▌      | 18/50 [00:00<00:00, 100.06it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  36%|███▌      | 18/50 [00:00<00:00, 99.88it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  38%|███▊      | 19/50 [00:00<00:00, 99.84it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  38%|███▊      | 19/50 [00:00<00:00, 99.63it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  40%|████      | 20/50 [00:00<00:00, 99.77it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  40%|████      | 20/50 [00:00<00:00, 99.59it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  42%|████▏     | 21/50 [00:00<00:00, 99.74it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  42%|████▏     | 21/50 [00:00<00:00, 99.58it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  44%|████▍     | 22/50 [00:00<00:00, 99.87it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  44%|████▍     | 22/50 [00:00<00:00, 99.73it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  46%|████▌     | 23/50 [00:00<00:00, 100.03it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  46%|████▌     | 23/50 [00:00<00:00, 99.89it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  48%|████▊     | 24/50 [00:00<00:00, 100.27it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  48%|████▊     | 24/50 [00:00<00:00, 100.14it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  50%|█████     | 25/50 [00:00<00:00, 100.47it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  50%|█████     | 25/50 [00:00<00:00, 100.34it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  52%|█████▏    | 26/50 [00:00<00:00, 100.33it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  52%|█████▏    | 26/50 [00:00<00:00, 100.20it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  54%|█████▍    | 27/50 [00:00<00:00, 100.32it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  54%|█████▍    | 27/50 [00:00<00:00, 100.18it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  56%|█████▌    | 28/50 [00:00<00:00, 100.15it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  56%|█████▌    | 28/50 [00:00<00:00, 100.02it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  58%|█████▊    | 29/50 [00:00<00:00, 100.14it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  58%|█████▊    | 29/50 [00:00<00:00, 100.02it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  60%|██████    | 30/50 [00:00<00:00, 100.24it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  60%|██████    | 30/50 [00:00<00:00, 100.13it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  62%|██████▏   | 31/50 [00:00<00:00, 100.41it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  62%|██████▏   | 31/50 [00:00<00:00, 100.31it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  64%|██████▍   | 32/50 [00:00<00:00, 100.58it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  64%|██████▍   | 32/50 [00:00<00:00, 100.47it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  66%|██████▌   | 33/50 [00:00<00:00, 100.44it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  66%|██████▌   | 33/50 [00:00<00:00, 100.35it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  68%|██████▊   | 34/50 [00:00<00:00, 100.52it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  68%|██████▊   | 34/50 [00:00<00:00, 100.41it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  70%|███████   | 35/50 [00:00<00:00, 100.68it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  70%|███████   | 35/50 [00:00<00:00, 100.59it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  72%|███████▏  | 36/50 [00:00<00:00, 100.86it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  72%|███████▏  | 36/50 [00:00<00:00, 100.77it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  74%|███████▍  | 37/50 [00:00<00:00, 101.04it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  74%|███████▍  | 37/50 [00:00<00:00, 100.95it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  76%|███████▌  | 38/50 [00:00<00:00, 101.20it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  76%|███████▌  | 38/50 [00:00<00:00, 101.12it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  78%|███████▊  | 39/50 [00:00<00:00, 101.28it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  78%|███████▊  | 39/50 [00:00<00:00, 101.20it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  80%|████████  | 40/50 [00:00<00:00, 101.26it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  80%|████████  | 40/50 [00:00<00:00, 101.18it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  82%|████████▏ | 41/50 [00:00<00:00, 101.36it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  82%|████████▏ | 41/50 [00:00<00:00, 101.29it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  84%|████████▍ | 42/50 [00:00<00:00, 101.54it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  84%|████████▍ | 42/50 [00:00<00:00, 101.46it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  86%|████████▌ | 43/50 [00:00<00:00, 101.66it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  86%|████████▌ | 43/50 [00:00<00:00, 101.59it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  88%|████████▊ | 44/50 [00:00<00:00, 101.73it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  88%|████████▊ | 44/50 [00:00<00:00, 101.65it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  90%|█████████ | 45/50 [00:00<00:00, 101.82it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  90%|█████████ | 45/50 [00:00<00:00, 101.75it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  92%|█████████▏| 46/50 [00:00<00:00, 101.90it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  92%|█████████▏| 46/50 [00:00<00:00, 101.83it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  94%|█████████▍| 47/50 [00:00<00:00, 101.85it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  94%|█████████▍| 47/50 [00:00<00:00, 101.78it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  96%|█████████▌| 48/50 [00:00<00:00, 101.95it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  96%|█████████▌| 48/50 [00:00<00:00, 101.88it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  98%|█████████▊| 49/50 [00:00<00:00, 102.04it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3:  98%|█████████▊| 49/50 [00:00<00:00, 101.97it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3: 100%|██████████| 50/50 [00:00<00:00, 102.11it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3: 100%|██████████| 50/50 [00:00<00:00, 102.04it/s, v_num=Maps, val_loss=3.510, train_loss=2.840]
Epoch 3: 100%|██████████| 50/50 [00:00<00:00, 96.85it/s, v_num=Maps, val_loss=3.350, train_loss=2.840]
Epoch 3: 100%|██████████| 50/50 [00:00<00:00, 96.70it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 3:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   0%|          | 0/50 [00:00<?, ?it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   2%|▏         | 1/50 [00:00<00:00, 98.46it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   2%|▏         | 1/50 [00:00<00:00, 95.21it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   4%|▍         | 2/50 [00:00<00:00, 101.86it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   4%|▍         | 2/50 [00:00<00:00, 100.35it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   6%|▌         | 3/50 [00:00<00:00, 104.08it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   6%|▌         | 3/50 [00:00<00:00, 103.05it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   8%|▊         | 4/50 [00:00<00:00, 105.07it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:   8%|▊         | 4/50 [00:00<00:00, 104.28it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  10%|█         | 5/50 [00:00<00:00, 105.79it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  10%|█         | 5/50 [00:00<00:00, 105.15it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  12%|█▏        | 6/50 [00:00<00:00, 105.54it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  12%|█▏        | 6/50 [00:00<00:00, 105.02it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  14%|█▍        | 7/50 [00:00<00:00, 106.18it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  14%|█▍        | 7/50 [00:00<00:00, 105.73it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  16%|█▌        | 8/50 [00:00<00:00, 106.73it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  16%|█▌        | 8/50 [00:00<00:00, 106.32it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  18%|█▊        | 9/50 [00:00<00:00, 106.90it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  18%|█▊        | 9/50 [00:00<00:00, 106.53it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  20%|██        | 10/50 [00:00<00:00, 107.07it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  20%|██        | 10/50 [00:00<00:00, 106.74it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  22%|██▏       | 11/50 [00:00<00:00, 107.33it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  22%|██▏       | 11/50 [00:00<00:00, 107.02it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  24%|██▍       | 12/50 [00:00<00:00, 107.46it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  24%|██▍       | 12/50 [00:00<00:00, 107.16it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  26%|██▌       | 13/50 [00:00<00:00, 106.62it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  26%|██▌       | 13/50 [00:00<00:00, 106.35it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  28%|██▊       | 14/50 [00:00<00:00, 106.66it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  28%|██▊       | 14/50 [00:00<00:00, 106.42it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  30%|███       | 15/50 [00:00<00:00, 106.78it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  30%|███       | 15/50 [00:00<00:00, 106.56it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  32%|███▏      | 16/50 [00:00<00:00, 107.05it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  32%|███▏      | 16/50 [00:00<00:00, 106.85it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  34%|███▍      | 17/50 [00:00<00:00, 107.20it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  34%|███▍      | 17/50 [00:00<00:00, 107.01it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  36%|███▌      | 18/50 [00:00<00:00, 107.41it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  36%|███▌      | 18/50 [00:00<00:00, 107.24it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  38%|███▊      | 19/50 [00:00<00:00, 107.57it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  38%|███▊      | 19/50 [00:00<00:00, 107.39it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  40%|████      | 20/50 [00:00<00:00, 107.16it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  40%|████      | 20/50 [00:00<00:00, 106.97it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  42%|████▏     | 21/50 [00:00<00:00, 106.92it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  42%|████▏     | 21/50 [00:00<00:00, 106.74it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  44%|████▍     | 22/50 [00:00<00:00, 106.73it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  44%|████▍     | 22/50 [00:00<00:00, 106.57it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  46%|████▌     | 23/50 [00:00<00:00, 106.66it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  46%|████▌     | 23/50 [00:00<00:00, 106.50it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  48%|████▊     | 24/50 [00:00<00:00, 106.65it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  48%|████▊     | 24/50 [00:00<00:00, 106.51it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  50%|█████     | 25/50 [00:00<00:00, 106.65it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  50%|█████     | 25/50 [00:00<00:00, 106.50it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  52%|█████▏    | 26/50 [00:00<00:00, 106.71it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  52%|█████▏    | 26/50 [00:00<00:00, 106.59it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  54%|█████▍    | 27/50 [00:00<00:00, 106.64it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  54%|█████▍    | 27/50 [00:00<00:00, 106.52it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  56%|█████▌    | 28/50 [00:00<00:00, 106.81it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  56%|█████▌    | 28/50 [00:00<00:00, 106.69it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  58%|█████▊    | 29/50 [00:00<00:00, 106.94it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  58%|█████▊    | 29/50 [00:00<00:00, 106.83it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  60%|██████    | 30/50 [00:00<00:00, 107.07it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  60%|██████    | 30/50 [00:00<00:00, 106.96it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  62%|██████▏   | 31/50 [00:00<00:00, 107.16it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  62%|██████▏   | 31/50 [00:00<00:00, 107.06it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  64%|██████▍   | 32/50 [00:00<00:00, 107.18it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  64%|██████▍   | 32/50 [00:00<00:00, 107.08it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  66%|██████▌   | 33/50 [00:00<00:00, 107.26it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  66%|██████▌   | 33/50 [00:00<00:00, 107.16it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  68%|██████▊   | 34/50 [00:00<00:00, 107.17it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  68%|██████▊   | 34/50 [00:00<00:00, 107.07it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  70%|███████   | 35/50 [00:00<00:00, 107.24it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  70%|███████   | 35/50 [00:00<00:00, 107.13it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  72%|███████▏  | 36/50 [00:00<00:00, 107.32it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  72%|███████▏  | 36/50 [00:00<00:00, 107.23it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  74%|███████▍  | 37/50 [00:00<00:00, 107.38it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  74%|███████▍  | 37/50 [00:00<00:00, 107.29it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  76%|███████▌  | 38/50 [00:00<00:00, 107.39it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  76%|███████▌  | 38/50 [00:00<00:00, 107.31it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  78%|███████▊  | 39/50 [00:00<00:00, 107.40it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  78%|███████▊  | 39/50 [00:00<00:00, 107.31it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  80%|████████  | 40/50 [00:00<00:00, 107.45it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  80%|████████  | 40/50 [00:00<00:00, 107.36it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  82%|████████▏ | 41/50 [00:00<00:00, 107.12it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  82%|████████▏ | 41/50 [00:00<00:00, 107.04it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  84%|████████▍ | 42/50 [00:00<00:00, 107.17it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  84%|████████▍ | 42/50 [00:00<00:00, 107.10it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  86%|████████▌ | 43/50 [00:00<00:00, 107.23it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  86%|████████▌ | 43/50 [00:00<00:00, 107.15it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  88%|████████▊ | 44/50 [00:00<00:00, 107.30it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  88%|████████▊ | 44/50 [00:00<00:00, 107.23it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  90%|█████████ | 45/50 [00:00<00:00, 107.38it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  90%|█████████ | 45/50 [00:00<00:00, 107.31it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  92%|█████████▏| 46/50 [00:00<00:00, 107.46it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  92%|█████████▏| 46/50 [00:00<00:00, 107.39it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  94%|█████████▍| 47/50 [00:00<00:00, 107.55it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  94%|█████████▍| 47/50 [00:00<00:00, 107.48it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  96%|█████████▌| 48/50 [00:00<00:00, 107.50it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  96%|█████████▌| 48/50 [00:00<00:00, 107.43it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  98%|█████████▊| 49/50 [00:00<00:00, 107.56it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4:  98%|█████████▊| 49/50 [00:00<00:00, 107.49it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4: 100%|██████████| 50/50 [00:00<00:00, 107.50it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4: 100%|██████████| 50/50 [00:00<00:00, 107.43it/s, v_num=Maps, val_loss=3.350, train_loss=1.840]
Epoch 4: 100%|██████████| 50/50 [00:00<00:00, 101.72it/s, v_num=Maps, val_loss=3.150, train_loss=1.840]
Epoch 4: 100%|██████████| 50/50 [00:00<00:00, 101.56it/s, v_num=Maps, val_loss=3.150, train_loss=1.530]
Epoch 4: 100%|██████████| 50/50 [00:00<00:00, 98.67it/s, v_num=Maps, val_loss=3.150, train_loss=1.530]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     Validate metric           DataLoader 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
         MAE_val            1.2130459547042847
         R2_val             0.5892401337623596
        val_loss             3.149669885635376
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Wow it still works! Let’s have a look at what the model structure looks like to see what changes have been made to keep the model valid.

print(trained_model_list[0].model)

ConcatTabularFeatureMaps(
  (mod1_layers): ModuleDict(
    (layer 1): Sequential(
      (0): Linear(in_features=392, out_features=300, bias=True)
      (1): ReLU()
    )
    (layer 2): Sequential(
      (0): Linear(in_features=300, out_features=128, bias=True)
      (1): ReLU()
    )
    (layer 3): Sequential(
      (0): Linear(in_features=128, out_features=128, bias=True)
      (1): ReLU()
    )
  )
  (mod2_layers): ModuleDict(
    (layer 1): Sequential(
      (0): Linear(in_features=392, out_features=300, bias=True)
      (1): ReLU()
    )
    (layer 2): Sequential(
      (0): Linear(in_features=300, out_features=128, bias=True)
      (1): ReLU()
    )
    (layer 3): Sequential(
      (0): Linear(in_features=128, out_features=100, bias=True)
      (1): ReLU()
    )
  )
  (fused_layers): Sequential(
    (0): Linear(in_features=228, out_features=150, bias=True)
    (1): ReLU()
    (2): Linear(in_features=150, out_features=75, bias=True)
    (3): ReLU()
    (4): Linear(in_features=75, out_features=50, bias=True)
    (5): ReLU()
  )
  (final_prediction): Sequential(
    (0): Linear(in_features=50, out_features=1, bias=True)
  )
)

As you can see, a few corrections have been made to the modifications:

The fused_layers has been modified to have the correct number of nodes in the first layer to match the concatenated feature maps from the two modalities.

In general, there are checks in the fusion models to make sure that the modifications are valid. If the input number of nodes to a modification is not correct, then the model will automatically calculate the correct number of nodes and correct the modification.

This is the case for quite a few modifications, but potentially not all of them so please be careful! Make sure to print out the model structure to check that the modifications have been made correctly and see what changes have been made to keep the model valid.

# removing checkpoints

for file in os.listdir(output_paths["checkpoints"]):
    # remove file
    os.remove(os.path.join(output_paths["checkpoints"], file))

Total running time of the script: (0 minutes 21.393 seconds)

Gallery generated by Sphinx-Gallery