5
$\begingroup$

I want my neural network to learn to predict the square $n+1$ number having $n$ number. I am considering a regression problem. That's what I'm doing:

from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense, Embedding, Dropout
import numpy as np

x = np.array([[int(i)] for i in range(1001)])
y = np.array([x*x for x in range(1001)])

model = Sequential()
model.add(Dense(100, activation = 'relu', input_dim = 1))
model.add(Dense(50, activation = 'relu'))
model.add(Dense(10, activation = 'relu'))
model.add(Dense(1))

model.compile(loss='mse',optimizer='adam', metrics=['mae'])
model.fit(x,y,epochs= 2500)
pred = model.predict([1001])
print(pred)

However, as a result, I get [[ 1000166.8125]] instead 1002001.

Update:

x = np.array([[int(i)] for i in range(80001)])
y = np.array([x*x for x in range(80001)])
print(x)
print(y)
model = Sequential()
model.add(Dense(20, activation = 'relu', input_dim = 1))
model.add(Dense(20, activation = 'relu'))
model.add(Dense(1))

adam = optimizers.Adam(lr=0.0002,beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False,)
model.compile(loss='mse',optimizer=adam, metrics=['mae'])
model.fit(x,y,epochs= 3000)
pred = model.predict([80001])
print(pred)
model.save_weights("test.h5")
model_json = model.to_json()
json_file = open("test.json", "w")
json_file.write(model_json)
json_file.close()

result: [[ 4.81360333e+09]]

$\endgroup$
  • 3
    $\begingroup$ Vaalizaadeh already provided a good answer to your question. If you want to get a feeling for the most common hyperparameters of DNN's, I'd suggest, you also check out the Tensorflow Playground. Also, check out this answer. $\endgroup$ – georg-un May 6 '19 at 7:30
  • 1
    $\begingroup$ I want to push the second link in @georg_un's answer; it's pretty much exactly this question. In particular, a neural net seems unlikely to do a good job when the test data is outside the range of the training data (your 10001). The NN is trying very hard to fit the parabolic curve inside the training window, but will ignore everything outside that window: it has no incentive to do that "right". ( (It will probably ultimately produce a linear function, extending what it does at the ends of your training window.) $\endgroup$ – Ben Reiniger May 7 '19 at 1:34
6
$\begingroup$

Decrease the number of hidden layers; you can omit the dense layer with $50$ neurons. Furthermore, train your network more. You should also provide more data. It is not much at the moment.

Your current architecture is very deep for such a relatively easy task. Consequently, it needs more train time. You can just decrease the size of the current model by diminishing the number of hidden layers and neurons. For instance, use the following setting to see how you can train very fast and have a good accuracy.

model = Sequential()
model.add(Dense(20, activation = 'relu', input_dim = 1))
model.add(Dense(20, activation = 'relu'))
model.add(Dense(1))
| improve this answer | |
$\endgroup$
  • $\begingroup$ did everything as you said. A data set of 10.000 squares and your architecture. I get: 10001^2 = [[ 99884920.]] instead 100020001.How big should the data set be? Is it possible to solve this problem using machine learning methods? $\endgroup$ – SoH May 6 '19 at 7:29
  • 1
    $\begingroup$ Yes, you can. It is a very easy task. Try to train your network more and decrease the learning rate. The default value is relatively high. $\endgroup$ – Media May 6 '19 at 7:34
  • $\begingroup$ updated question. I could not get a good result. $\endgroup$ – SoH May 6 '19 at 11:13
  • $\begingroup$ @SoH It took so much time to find out what is really occurring. please first take a look at here. I will update the answer showing you what really is happening. The provided link can help you. $\endgroup$ – Media May 6 '19 at 13:26
  • $\begingroup$ thanks! Apparently, to achieve a normal result is impossible. $\endgroup$ – SoH May 6 '19 at 16:17
3
$\begingroup$

Because neural networks with a sufficiently large hidden layer can approximate arbitrary functions only on compact sets (this is one of the first things you can learn when you try to read some literature about neural networks). Train your neural network on a range from 0 to 100 and then ask the square of 78.

The much more interesting problem of learning the algorithm of multiplication is a part of an active research area called algorithm learning, for example check this paper.

| improve this answer | |
$\endgroup$
2
$\begingroup$

By its very nature, Neural Networks are used to only approximate one given function. So it will work quite well for tasks where you do not need very precise information to perform, for example to recognise a picture you do not need to actually know every details of the picture. However, in your case, you really want to know the exact value of a specific function F (here n-> n^2). Thus in general you can not do that, except if your NN already contains the function F in itself (or with parameters). For example, usually NN uses very transcendental functions as tanh, for which I do not see how can produce a polynomial for you.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.