ANN and prediction

Question

I have a list of objects. Each objects contains longitude, latitude and a list of words.

What I want to do is to predict the location based on the text contained in the object (similar texts should have similar location). Right now I’m using cosine similarity to calculate the similarity between the objects text but I’m stuck how I can use that information to train my neural network. I have a matrix containing each object and how many time each word appeared in that object. F.x. if I had these two objects

Obj C:   54.123, 10.123,   [This is a text for object C]
Obj B:   57.321, 11.113,   [This is a another text for object B]

Then I have something like the following matrix

       This is a text for object C another B
ObjC:   1   1  1   1   1    1    1   0     0
ObjB:   1   1  1   1   1    1    0   1     1

I would also have something like, for the distance between the two objects (note, that the numbers are not real)

        ObjC    ObjB
ObjC    1       0.25
ObjB    0.25    1

I have looked at how I use neural network to either classify things into groups (like A,B,C) or predict something like a housing price, but nothing that I find helpful for my problem.

I would consider the prediction right if it is within some distance X, since I’m dealing with location. This might be a stupid question, but someone point me to the right direction.

Answer 1

Looks like you are trying to predict two different things (lat and longitude) with same input. Therefore, this is a problem of predicting multiple output. However, it is easy to understand the basic with only one output. I will describe it that way.

Let's say, your target is just one variable Y, which is your location. After transformation (possibly TF-IDF), you have a feature matrix X where one row represents one sample and one column represents one feature. What you need to do now is train an ANN with the input X and target Y. This is the way an ANN will train:

1. It will randomly initiate weight matrices for input and outside side of your network. If you have just one hidden layer, the number of weight matrix is 2.
2. ANN will pass your input through some kind of activation function, get some output from them and then forward those outputs to output layer for further processing. Look up some tutorial for the actual maths. But this is the intuition.
3. Finally, based on the weights and further processing, ANN will generate some output. They are not within acceptable margin in the first iteration in most cases. Therefore, ANN needs to refine those randomly selected weights. This is where the backpropagation algorithm kicks in. It allows ANN to learn the most suitable weights for the best prediction.
4. This backpropagation algorithm requires a cost function which it tries to minimize. This cost function is usually RMS error function. But this cost function can be anything according to your choice. Cost functions take (mostly) two inputs. One of them is your true location (from training/validation data) and the other is predicted location that ANN got after the processing. As ANN tries to minimize the difference between predicted output and true output, it finds suitable weights for the network.
5. When the error from your cost function reduces to a desired value (or a predefined number of iteration has been passed, or a tolerance level has been reached), ANN stops training and returns you those learned weight matrices.
6. Once you have these final weight matrices, you can apply them on any new data to get the prediction. These weight matrices are your ANN.

As you have a multiple output problem, you need an implementation of ANN which support minimizing two errors at the same time. This is similar to multi-objective optimization. Python can help you here. See this sknn.mlp documentation for details on use. It will take multiple output, each on its own column in the target matrix. Therefore, your Y will be of shape n * 2, where n is the number of samples in your dataset. Just call fit method with your existing X and Y of the above format. Once the training is done, you get your minimum viable ANN. How to optimize the performance of ANN is a different story though :)