2
$\begingroup$

Disclaimer: I am almost a complete novice when it comes to tensorflow, keras, coding in general, and neural networks/data science.

While reading papers on novel architectures for neural nets, I see diagrams and such describing their ideas, and then they present their results, with no code shown. While learning to apply neural nets, I just load in the data, build a model by stacking layers like LSTM, Dense, etc, and training it. In other words I can't see how to do anything that isn't straight out of the box.

What tools, libraries, etc, do researchers use to implement these architectures when the layers aren't as simple as model.add(Dense(1))

For example, how might we implement the SeqMO algorithm described in this paper? https://arxiv.org/pdf/1806.05357.pdf

$\endgroup$

1 Answer 1

3
$\begingroup$

In this particular case, I don't know how are they implementing these complex layers, but in Keras/TensorFlow you can define your own layers by inheriting from tf.keras.layers.Layer. For example, you could define a custom dense layer as (example from the documentation)

class MyDenseLayer(tf.keras.layers.Layer):
  def __init__(self, num_outputs):
    super(MyDenseLayer, self).__init__()
    self.num_outputs = num_outputs
  def build(self, input_shape):
    self.kernel = self.add_weight("kernel",
                                  shape=[int(input_shape[-1]),
                                         self.num_outputs])
  def call(self, inputs):
    return tf.matmul(inputs, self.kernel)

If you're interested in this topic I recommend you paperswithcode.com, where you can find a lot of code implementations of research papers. Hope it helps :)


EDIT:

Apparently, for your particular example the code is available here, but they're using PyTorch, not Keras.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.