I am writing a CNN for binary classification MedMNIST data: https://medmnist.com/, specifically the Lung Nodule 3D dataset (N=1633 and 7:1:2). Currently, my model is not training at all; it is either fixed at constant accuracy or randomly fluctuating from around 0.2 to 0.8. What I have attempted so far:

  1. Tweaking the learning rate and other hyperparameters.
  2. Changing the activation function between layers
  3. Changing the number of filters
  4. Changing the optimiser and loss function (currently I am using Adam and CrossEntropyLoss respectively)
  5. Introducing weight into the loss function, since the data is imbalanced by around 3:1.
  6. Various other small tweaks to the CNN architecture, like padding.

I have stuck with the general architecture shown below:

class Net(nn.Module):
    def __init__(self):
        self.conv1 = nn.Conv3d(num_channels, 28, kernel_size=(3,3,3), padding=1)
        self.conv2 = nn.Conv3d(28, 56, kernel_size=(3,3,3), padding=1)
        self.dropout1 = nn.Dropout(0.5)
        self.fc1 = nn.Linear(153664, 64)
        self.fc2 = nn.Linear(64, num_classes)
    def forward(self, x):
      # Pass data through conv1
        x = self.conv1(x)
      # Use the rectified-linear activation function over x
        x = F.relu(x)

        x = self.conv2(x)
        x = F.relu(x)

        x = F.max_pool3d(x, (2,2,2))
        x = self.dropout1(x)
        x = torch.flatten(x, 1)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)

      # Apply softmax to x
        output = F.log_softmax(x, dim=1)
        return output

Any suggestions on how I could adapt this would be greatly appreciated. I'm at a loss because I'd kind of expect the model to at least learn something, with the aforementioned tweaks being used to squeeze out a final bit of optimisation.

  • $\begingroup$ Not sure if it's the key issue, but you shouldn't be applying a log softmax in the model if you're using CrossEntropyLoss, as CrossEntropyLoss includes the softmax operation. Your current setup is applying the softmax twice $\endgroup$
    – Karl
    Mar 14 at 2:12


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.