Every generation a population of 70 characters tries to jump to the next platform and increase their fitness. right now the fitness is the number of platforms they jumped on, each platform gives +10.
The problem I'm facing is that the bot isn't learning good enough after 1000 generations the best score was around 200 (it can get to 200 even at the first few generations by mistake. 200 means 20 platforms which is not a lot).
when I look at the characters jumping it looks like they just always jump and go left or right and not deliberately aiming to the next platform.
I tried several input configurations to make the bot perform better. but nothing really helped.
these are the inputs I tried to mess around with:
- pos.x, pos.y
- velocity.x, velocity.y
- isOnPlatform (bool)
- [plat.x, plat.y, plat.width] (list of the 3-7 next platforms locations)
- [prev.x, prev.y] (2-6 previous character positions)
I'm not so proficient with neuroevolution and I'm probably doing something wrong. glad if you could explain what's causing the bot to be so bad or what's not helping him to learn properly.
Although I think that the fitness function and the inputs should be the only problem I'm attaching the python-NEAT config file.
[NEAT] fitness_criterion = max fitness_threshold = 10000 pop_size = 70 reset_on_extinction = False [DefaultGenome] # node activation options activation_default = tanh activation_mutate_rate = 0.0 activation_options = tanh # node aggregation options aggregation_default = sum aggregation_mutate_rate = 0.0 aggregation_options = sum # node bias options bias_init_mean = 0.0 bias_init_stdev = 1.0 bias_max_value = 30.0 bias_min_value = -30.0 bias_mutate_power = 0.5 bias_mutate_rate = 0.7 bias_replace_rate = 0.1 # genome compatibility options compatibility_disjoint_coefficient = 1.0 compatibility_weight_coefficient = 0.5 # connection add/remove rates conn_add_prob = 0.5 conn_delete_prob = 0.5 # connection enable options enabled_default = True enabled_mutate_rate = 0.01 feed_forward = True initial_connection = full # node add/remove rates node_add_prob = 0.2 node_delete_prob = 0.2 # network parameters num_hidden = 6 num_inputs = 11 num_outputs = 3 # node response options response_init_mean = 1.0 response_init_stdev = 0.0 response_max_value = 30.0 response_min_value = -30.0 response_mutate_power = 0.0 response_mutate_rate = 0.0 response_replace_rate = 0.0 # connection weight options weight_init_mean = 0.0 weight_init_stdev = 1.0 weight_max_value = 30 weight_min_value = -30 weight_mutate_power = 0.5 weight_mutate_rate = 0.8 weight_replace_rate = 0.1 [DefaultSpeciesSet] compatibility_threshold = 3.0 [DefaultStagnation] species_fitness_func = max max_stagnation = 3 species_elitism = 2 [DefaultReproduction] elitism = 3 survival_threshold = 0.2
Note 1: the previous character positions are the position in the previous frame, and if the game runs at 60 fps the previous position is not that different from the current one...
..Note 2: the game score is a bit more complex than just jumping on platforms, the bot should also be rewarded for combos that can make him jump higher. the combo system is already implemented but I first want to see the bot aiming to the next platform before he learns to jump combos.