0
$\begingroup$

I'm trying to create a pytorch neural network capable of recognizing peaks in 2D graphs. Previously, I was able to get a result close to what I wanted, but it was not ideal and did not give a satisfactory result on some data.

Old results:

Medium result

Bad result

So I thought why not add more data? And i added about 300 more graphs to the 30 on which the training was initially carried out. Since then I haven't been able to get anything even close to the results I got earlier.

New results: New results

Criterion, optimizer and number of epochs i'm using, notice that the number is epochs is being multiplied by the number of graphs so 330 * n_epochs is the real epochs number.

criterion = nn.BCELoss()  # Binary Cross Entropy Loss for binary classification
optimizer = optim.Adam(model.parameters(), lr=0.001)
epochs = 1500
BATCH_SIZE = 64

NN itself:

class Classifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer1 = nn.Linear(BATCH_SIZE, 64)
        self.act1 = nn.ReLU()
        self.layer2 = nn.Linear(64, 64)
        self.act2 = nn.ReLU()
        # self.layer3 = nn.Linear(128, 64)
        # self.act3 = nn.ReLU()
        self.output = nn.Linear(64, 1)
        self.sigmoid = nn.Sigmoid()
 
    def forward(self, x):
        x = self.act1(self.layer1(x))
        x = self.act2(self.layer2(x))
        # x = self.act3(self.layer3(x))
        x = self.sigmoid(self.output(x))
        return x
 
model = Classifier()
model = model.to(torch.float64)
print(model)
model.to(device) # the device variable was set above to be "cuda" if available or "cpu" if not
model = nn.DataParallel(model)
next(model.parameters()).device

Example of training data: How training data looks like

Result array for each batch:

[1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 1. 0. 0. 0.]

Perhaps I am using the wrong approach; for each graph I separately train the neural network for n number of epochs. However, I think that this is not the point because the graphs are divided into batches in any case. In addition, I tried to train the neural network on 300 graphs combined into one graph and this also did not bring results.

Also, the train loss shows that the neural network is very well trained, but this is clearly not the case.

Epoch: 1400 | Train loss: 3.838743537139037e-245

I started changing other parameters of the neural network, changing the number of neurons and layers of the activation function, epochs, and batch size. But nothing led to the result that I received before. What did I do wrong and what should I do to get something close to the desired result again?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.