It looks like there is special structure in your images that might allow you to do better than using generic image inpainting methods.
It's hard to know from just one example image, but it is possible that you might be able to build a satisfactory method using only very simple features of the input image. In particular, you could train a classifier that, given a black pixel, tries to predict the correct color for that pixel.
In your specific example, some features that look useful are: "the most common color in the same column", "the most common color in the same row", "the color of the closest non-black pixel above your pixel (in the same column)", "the color of the closest non-black pixel below your pixel (in the same column)", "the color of the closest non-black pixel to the right of your pixel (in the same row)", "the color of the closest non-black pixel to the left of your pixel (in the same row)". I suggest trying to train a simple classifer on a few simple features like this, and see how well it works.
If you want to get fancy, you could try modelling this as a Markov random field over a grid graph. In other words, each pixel is a vertex in the graph, and two pixels are connected by an edge if they are adjacent. There are then standard methods to learn the underlying density functions given some training examples, and once you know the density functions, you could then use maximum likelihood methods to infer the values of the black pixels.