# Learning sum of a function using Keras

I am fitting a function $$y:R^3\to R$$ using the following model in Keras.

from keras.models import Sequential
from keras.layers import Dense

model = Sequential()
model.add(Dense(50, input_dim=3, activation='tanh'))
model.add(Dense(1, activation='linear', kernel_initializer='normal'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=10, batch_size=8)
predictions = model.predict(x_test)

To train, I used a data set consisting of $$\{\mathbf{x}_i\}_{i=1}^N$$ and $$\{y_i\}_{i=1}^N$$. This worked as expected.

Now imagine a slightly different situation where for each sample $$i$$ I have a pair of $$\mathbf{x}_{i,1}$$ and $$\mathbf{x}_{i,2}$$ ($$\mathbf{x}_{i,\alpha} \in R^3$$) and my target data point corresponding to $$i$$ is modelled as $$y_i=z(\mathbf{x}_{i,1})+z(\mathbf{x}_{i,2})$$, with $$z:R^3 \to R$$. Is it possible to define such a model in Keras, and how to do it? If not Keras, any other framework?

## 2 Answers

If you had two different functions $$z_1$$ and $$z_2$$, I would agree with Piotr: you could have two parallel networks, one for each part of the input, that join together at the end. This would minimize (though not eliminate) information from $$x_{i,1}$$ bleeding into the fitting of $$z_2$$.

But with $$z_1=z_2=z$$, we'd like to enforce that those two parallel networks are the same. The idea of using the same weights for different parts of the input remind of convolutions, and I think that will work here: let $$x_{i,1},x_{i,2}$$ be the columns of a $$3\times 2$$ input, apply 1-dimensional width-1 convolutional layers instead of dense ones, and add a "Sum Pooling" layer at the end. (Without writing something custom, you could manage that in Keras with average pooling and manually deal with the factor of 2.)

• Thanks for this. I sort of understand and see in principle how it should work. In practice I will need to play around with this a bit... – albapa Dec 3 '19 at 20:44

Sounds like you need functional model instead of sequential. Read more here.