2
$\begingroup$

how can I make this training faster ? when I call the fit method on a 100 x 100 matrix goes very slow

my model it's a sequential

h = self.model.fit(
        inputs,
        targets,
        epochs=epochs,
        batch_size=16,
        verbose=1,
)

this is my matrix

def build(n):
    mat=np.ones(N*N)
    return mat.reshape((N,N))

this is the of my Qtraining

qt = Qtraining(model,
               env,
               n_epoch=200,
               max_memory=500,
               data_size=100,
               name='model100')

This is the experience method

def get_data(self, data_size=10):
        env_size = self.memory[0][0].shape[1]  # env_state 1d size (1st element of episode)
        mem_size = len(self.memory)
        data_size = min(mem_size, data_size)
        inputs = np.zeros((data_size, env_size)) # metti Nsize righe di 0 , e envSize elementi 0
        targets = np.zeros((data_size, self.num_actions))#
        for i, j in enumerate(np.random.choice(range(mem_size), data_size, replace=False)):
            env_state, action, reward, next_env_state, game_over = self.memory[j]
            inputs[i] = env_state
            # There should be no target values for actions not taken.
            # Thou shalt not correct actions not taken #deep (quote by Eder Santana)
            targets[i] = self.predict(env_state)
            # Q_sa = derived policy = max quality env/action = max_a' Q(s', a')
            Q_sa = np.max(self.predict(next_env_state))
            if game_over:
                targets[i, action] = reward
            else:
                # reward + gamma * max_a' Q(s', a')
                targets[i, action] = reward + self.discount * Q_sa
       
        return inputs, targets

using a 50 x 50 matrix I get 2500 cells, and in the construction of the neural network I have a 2500x2500 parameters + 2500 for a total of 6252500. I think that slows down operations. and there are only 3 Danse layers, the last one is size 4 because of the possible actions that are 4. Is it possible to reduce the time of operation by adding more Danse layers?

this is my model

def build_model(env, **opt):
loss = opt.get('loss', 'mse')
a = opt.get('alpha', 0.24)
model = Sequential()
esize = env.maze.size
model.add(Dense(esize, input_shape=(esize,)))
model.add(LeakyReLU(alpha=a))
model.add(Dense(esize))
model.add(LeakyReLU(alpha=a))
model.add(Dense(num_actions))
model.compile(optimizer='adam', loss='mse')
return model
$\endgroup$
3
  • $\begingroup$ Hello, your build function seems to have an error. You should use n instead of N inside the function. Besides, I think you can directly return np.ones(N) to have a correct shape. $\endgroup$
    – TQA
    May 18, 2021 at 17:38
  • $\begingroup$ thank you very much for your answer, but I think I have identified the problem although I do not know how to improve. $\endgroup$ May 18, 2021 at 20:18
  • $\begingroup$ I write a "problem" as an answer to my question $\endgroup$ May 18, 2021 at 20:19

2 Answers 2

1
$\begingroup$

There are a couple ways to speed up your code:

  • Use an existing framework. It appears you are doing reinforcement learning (RL). keras-rl is an existing framework for RL which may be more efficient than your code.
  • Having everything in computer memory all the time. It will speed up your code to define a smaller gameplay memory replay buffer and only use a subset of the environment that is currently relevant.
$\endgroup$
1
$\begingroup$

You can use the TensorFlow GPU module. Keras is built on top of TensorFlow and it helps a lot. It is quit difficult to install, but I can help you with that. In total I got a 40x speed-up: https://www.tensorflow.org/install/gpu

VID: https://www.youtube.com/watch?v=IubEtS2JAiY or https://www.youtube.com/watch?v=hHWkvEcDBO0

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.