-1
$\begingroup$

I have scoured the internet and documentation, but have not yet been able to find a simple explanation of how to load data and/or simply type data into pytorch.

I am just learning this library, and I believe being able to put in my own custom data would make learning much more accessible.

For example, instead of the example I find everywhere, in which dataloader loads an MNIST dataset, how can I get data working with the rest of my pytorch file in a much simpler way?

     INPUTS   OUTPUTS 

data = [[[1,1,0], [1,1]], [[1,1,1], [1,0]], [[1,0,0], [0,0]], ... [[0,0,0], [0,1]]]

  • Where there are 3 inputs in the input layer, and 2 outputs at the output layer. This example shows 4 distinct examples of training data.

I'd even be happy for a straightforward way to load in a CSV.

All i really need, at the end of the day, is a little help understanding what a dataset in pytorch looks like, and how I can create and edit them to aid in my attempts at learning more about this library.

Thank you very much for any help you might offer.

$\endgroup$
0
$\begingroup$

You can input data from any form (e.g. csv, json files, txt files) into simple python list structures and then convert to pytorch tensors...

It is useful to store your input and output data as separate pytorch tensors:

    X = [[1,1,0], [1,1,1]]
    y = [[1,1], [1,0]]

    X = torch.FloatTensor(X)
    y = torch.FloatTensor(y)

    Xy_data = TensorDataset(X, y)

The TensorDataset is useful for storing the corresponding inputs and outputs together (use DataLoader to help with automatic batching)

A typical implementation has the following structure:

    loaded_data = DataLoader(Xy_data, batch_size = bs, shuffle = True)
    model = your_model()
    for epoch in range(epochs):
        model.train()
        for xb, yb in loaded_data:
            y_predicted = model(xb)
            loss = some_loss_function(yb, y_predicted)

            # Update gradients and do back propagation...
$\endgroup$
  • $\begingroup$ Thank you very much for your help. This is exactly what I was looking for. I’m sorry if the question was a bit trivial, but I honestly could not find a simple answer. $\endgroup$ – Colin Gladue Aug 16 '19 at 2:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.