What's the purpose of eval_model function?

[SailSabnis]

Under 03. PyTorch Computer Vision > 4. Make predictions and get Model 0 results¶

The eval_model function is exactly like a testing loop. The only difference is in test loop we mention test_dataloader directly, while here we have functionalised it. But if these two codes are exactly the same, why the need to create this function?
If we just want to print it differently, can't we just add the following print line (from the eval_model fiunction) to the test_loop - print(f"model_name: {model.class.name}\n--model_loss:{test_loss.item()}\n--model_accuracy: {test_acc.item()}")

trying to understand the purpose of eval_model function

compare the two below -

Test Loop

### Testing
    # Setup variables for accumulatively adding up loss and accuracy 
    test_loss, test_acc = 0, 0 
    model_0.eval()
    with torch.inference_mode():
        for X, y in test_dataloader:
            # 1. Forward pass
            test_pred = model_0(X)
           
            # 2. Calculate loss (accumatively)
            test_loss += loss_fn(test_pred, y) # accumulatively add up the loss per epoch

            # 3. Calculate accuracy (preds need to be same as y_true)
            test_acc += accuracy_fn(y_true=y, y_pred=test_pred.argmax(dim=1))
        
        # Calculations on test metrics need to happen inside torch.inference_mode()
        # Divide total test loss by length of test dataloader (per batch)
        test_loss /= len(test_dataloader)

        # Divide total accuracy by length of test dataloader (per batch)
        test_acc /= len(test_dataloader)

    ## Print out what's happening
    print(f"\nTrain loss: {train_loss:.5f} | Test loss: {test_loss:.5f}, Test acc: {test_acc:.2f}%\n")

eval_model function

def eval_model(model: torch.nn.Module, 
               data_loader: torch.utils.data.DataLoader, 
               loss_fn: torch.nn.Module, 
               accuracy_fn):
    """Returns a dictionary containing the results of model predicting on data_loader.

    Args:
        model (torch.nn.Module): A PyTorch model capable of making predictions on data_loader.
        data_loader (torch.utils.data.DataLoader): The target dataset to predict on.
        loss_fn (torch.nn.Module): The loss function of model.
        accuracy_fn: An accuracy function to compare the models predictions to the truth labels.

    Returns:
        (dict): Results of model making predictions on data_loader.
    """
    loss, acc = 0, 0
    model.eval()
    with torch.inference_mode():
        for X, y in data_loader:
            # Make predictions with the model
            y_pred = model(X)
            
            # Accumulate the loss and accuracy values per batch
            loss += loss_fn(y_pred, y)
            acc += accuracy_fn(y_true=y, 
                                y_pred=y_pred.argmax(dim=1)) # For accuracy, need the prediction labels (logits -> pred_prob -> pred_labels)
        
        # Scale loss and acc to find the average loss/acc per batch
        loss /= len(data_loader)
        acc /= len(data_loader)
        
    return {"model_name": model.__class__.__name__, # only works when model was created with a class
            "model_loss": loss.item(),
            "model_acc": acc}

# Calculate model 0 results on test dataset
model_0_results = eval_model(model=model_0, data_loader=test_dataloader,
    loss_fn=loss_fn, accuracy_fn=accuracy_fn
)
model_0_results

[LuluW8071]

In ur model architecture, u might use batch normalization, layer normalization, and dropout layers. Setting the model to evaluation mode turns off these layers and creates a separate environment for testing or prediction.

Similarly, during training, u use model.train() to activate all defined layers. If u look at ur training and testing loop code, model.train() and model.eval() will be constantly switching between these modes

[SailSabnis]

Hi, thank you for responding. I am aware of model.train() and model.eval() and the switching between these. My question is around the custom eval_model() function Daniel created right after creating the train and test loop. My purpose does it solve except for printing the model name, loss and accu? These things could be printed right in the test loop as well. What am I missing?

[LuluW8071]

The function does similar job as test loop, just returns the model_name, acc, loss. It was done to store the metrics of a model in a variable by calling the eval function so later on it would be easy to compare the 3 models evaluation metrics(u will find bar plot of 3 model comparison later on the video).

[SailSabnis]

Many thanks! Clear.