Keep in mind that the Accuracy measure is measuring whether the values are Exactly The Same. I.e. it is a classification measure, whereas approximating a sin curve is much better suited for measurement as a regression problem.
That said:
In evaluating the network's performance, what is the network actually doing? Let's take the network and perform a little visual analysis on its performance:
import matplotlib.pyplot as plt
preds = model.predict(x)
plt.plot(x, y, 'b', x, preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()
Hmm. The model seems to be minimizing error by simply getting closer and closer to guessing 0 for every value, rather than approximating the function. There are several hypotheses here to explain this. One is that the network is not complex enough to model the function. In order to test this, let's simplify the function- that is, let's bring the range down to one sine cycle:
x = np.arange(0, math.pi*2, 0.1)
y = np.sin(x)
And try to train the network again:
Not wonderful, but a better fit, certainly.
How about with 100 epochs instead of 10?
How about with 1000 epochs?
This is, of course, very interesting. After 1000 epochs, our networks is able to roughly approximate the downward curve from 1:0 ($\pi/2$: $\pi$) of the sine response, but not the initial upward curve 0:1 (0:$\pi/2$) or the region in which the function is negative ($\pi$:$2\pi$).
This result begs the question- what will it look like after 10000 epochs?
Not significantly better. It looks like we'll have to change the architecture of the network (more layers, more neurons, and/or different activation functions) to improve beyond this point.
To inform this architecture change, let's take a look at the sigmoid activation function:
Uh-oh. The value of this sigmoid function can only ever be in the range 0:1, and the range of the sin function is -1:1.
To correct this, let's just normalize the sin response between 0 and 1:
y = (np.sin(x)+1)/2
Now, the network performs much better than before after 1000 epochs:
And 10000:
After 100000 epochs, it's roughly perfect:
Even still, this advancement doesn't help much on the larger sin range (after 1000 epochs):
x = np.arange(0, 100, 1)
y = (np.sin(x)+1)/2
If we, however, take the model trained on a single sin curve and further train it on the larger range, we begin to see progress after 1000 epochs:
x = np.arange(0, 100, .1)
y = (np.sin(x)+1)/2
model_copy = model
model_copy.fit(x, y, epochs=1000, batch_size=8, verbose=0)
model_copy_preds = model_copy.predict(x)
And more so after 10000 epochs:
And more so, well into the third repetition after 100000 epochs:
So, with careful training, our network with a single layer of sigmoid activations appears to be learning to generalize the sin curve. Further investigation could find a limit to that generalization, certainly.
For reproduction:
import numpy as np
from keras.layers import Dense
from keras.models import Sequential
import matplotlib.pyplot as plt
import math
x = np.arange(0, math.pi*2, .1)
y = (np.sin(x)+1)/2
model = Sequential([
Dense(10, input_shape=(1,)),
Activation('sigmoid'),
Dense(1)
])
model.compile(loss='mean_squared_error', optimizer='SGD', metrics=['mean_squared_error'])
model.fit(x, y, epochs=100000, batch_size=8, verbose=0)
preds = model.predict(x)
plt.plot(x, y, 'b', x, preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()
x = np.arange(0, 100, .1)
y = (np.sin(x)+1)/2
model_copy = model
model_copy.fit(x, y, epochs=10000, batch_size=8, verbose=0)
model_copy_preds = model_copy.predict(x)
plt.plot(x, y, 'b', x, model_copy_preds, 'r--')
plt.ylabel('Y / Predicted Value')
plt.xlabel('X Value')
plt.show()