individual predictions are different from batch predictions

[wzhang682]

for the computer vision exercise 3

num_to_plot = 5 
for i in range(num_to_plot):

  img = test_data[i][0]
  label = test_data[i][1]

  model_pred_logits = model_gpu(img.unsqueeze(dim=0).to(device))
  model_pred_probs = torch.softmax(model_pred_logits, dim=1)
  model_pred_label = torch.argmax(model_pred_probs, dim=1)

  plt.figure()
  plt.imshow(img.squeeze(), cmap="gray")
  plt.title(f"Truth: {label} | Pred: {model_pred_label.cpu().item()}") 
  plt.axis(False);   

The results of label are 7，2，1，0，4

from tqdm.auto import tqdm
model_gpu.eval()
y_preds = []
with torch.inference_mode():
  for batch, (X, y) in tqdm(enumerate(test_dataloader)):
    # Make sure data on right device
    X, y = X.to(device), y.to(device)
    # Forward pass
    y_pred_logits = model_gpu(X)
    # Logits -> Pred probs -> Pred label
    y_pred_labels = torch.argmax(torch.softmax(y_pred_logits, dim=1), dim=1)
    # Append the labels to the preds list
    y_preds.append(y_pred_labels)
  y_preds=torch.cat(y_preds).cpu()
len(y_preds) y_preds[:10]

But here use the same model the first five result changes to tensor([0, 4, 6, 5, 7, 5, 9, 4, 9, 8])) why this happen

[LuluW8071]

Two things could happen here:

• Either the test data loader images are randomized due to shuffle
• Or batch inconsistency(either pass images one at a time for inference or pass same batch_size as u trained before)

[mrdbourke]

Hi @wzhang682 ,

I agree with @LuluW8071 , are you sure the order of the images are the same in each case?

Have you tried to compare the predictions image single vs image batch?

The test_dataloader may have shuffled the data, hence the different outcomes.