2
$\begingroup$

I'm trying to make a bot to the famous "Icy Tower" game. I rebuilt the game using pygame and I'm trying to build the bot using Python-NEAT.

Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.

example

The problem I'm facing is that the bot isn't learning good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).

when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.

I tried several input configurations to make the bot perform better. but nothing really helped.

these are the inputs I tried to mess around with:

  • pos.x, pos.y
  • velocity.x, velocity.y
  • isOnPlatform (bool)
  • [plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations)
  • [prev.x, prev.y] (2-6 previous character positions)

I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.

Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.

[NEAT]
fitness_criterion     = max
fitness_threshold     = 10000
pop_size              = 70
reset_on_extinction   = False

[DefaultGenome]
# node activation options
activation_default      = tanh
activation_mutate_rate  = 0.0
activation_options      = tanh

# node aggregation options
aggregation_default     = sum
aggregation_mutate_rate = 0.0
aggregation_options     = sum

# node bias options
bias_init_mean          = 0.0
bias_init_stdev         = 1.0
bias_max_value          = 30.0
bias_min_value          = -30.0
bias_mutate_power       = 0.5
bias_mutate_rate        = 0.7
bias_replace_rate       = 0.1

# genome compatibility options
compatibility_disjoint_coefficient = 1.0
compatibility_weight_coefficient   = 0.5

# connection add/remove rates
conn_add_prob           = 0.5
conn_delete_prob        = 0.5

# connection enable options
enabled_default         = True
enabled_mutate_rate     = 0.01

feed_forward            = True
initial_connection      = full

# node add/remove rates
node_add_prob           = 0.2
node_delete_prob        = 0.2

# network parameters
num_hidden              = 6
num_inputs              = 11
num_outputs             = 3

# node response options
response_init_mean      = 1.0
response_init_stdev     = 0.0
response_max_value      = 30.0
response_min_value      = -30.0
response_mutate_power   = 0.0
response_mutate_rate    = 0.0
response_replace_rate   = 0.0

# connection weight options
weight_init_mean        = 0.0
weight_init_stdev       = 1.0
weight_max_value        = 30
weight_min_value        = -30
weight_mutate_power     = 0.5
weight_mutate_rate      = 0.8
weight_replace_rate     = 0.1

[DefaultSpeciesSet]
compatibility_threshold = 3.0

[DefaultStagnation]
species_fitness_func = max
max_stagnation       = 3
species_elitism      = 2

[DefaultReproduction]
elitism            = 3
survival_threshold = 0.2
  • Note 1: the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...

  • ..Note 2: the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combos.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.