0
$\begingroup$

I am using Ensemble PyTorch to train a voting classifier. My dataset includes around 60k records. I trained a Neural Network with Cross-entropy loss. Below is my model architecture

class Classifier(nn.Module):

    def __init__(self):
        super(Classifier, self).__init__()
        self.linear1 = nn.Linear(10, 128)
        self.linear2 = nn.Linear(128, 128)
        self.linear3 = nn.Linear(128, 60)
    
    def forward(self, data):
        data = data.view(data.size(0), -1)
        output = F.relu(self.linear1(data))
        output = F.relu(self.linear2(output))
        output = self.linear3(output)
        return output

I am using Adam optimizer, CosineAnnealingLR scheduler, number of estimators for ensemble is 10, batch size is 100 and I am running it for 150 epochs

criterion = nn.CrossEntropyLoss()
model.set_criterion(criterion)
model.set_optimizer('Adam', lr=1e-3, weight_decay=5e-4)
model.set_scheduler(
    "CosineAnnealingLR",
    T_max=epochs,
)

I am getting below learning curves for training/validation loss and accuracy. The validation loss is very high but it is decreasing very slowly and it is much much higher than training loss. I suspect that it is overfitting and that is why the validation loss is too high. Is it indeed the case of overfitting or underfitting(I am confused here as the validation accuracy is quite good and it is not deviating too much from the training accuracy). Please guide me on how can I reduce the validation loss and if you can please provide any pointers on improving the model.

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ this is a well-written question but sharing more information about the problem you are trying to solve can be very useful for others to be able to help you: - can you show a sample of the data you are working with? - what is the target variable? consider renaming the columns for privacy if needed and showing distributions when possible $\endgroup$
    – Just trying
    Mar 6 at 15:10
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Mar 6 at 15:12

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.