How to replace NaN values for image data?

Question

My data set has a total of 200 columns, where each column corresponds to the same pixel in all of my images. In total, I have 48,500 rows. The labels for the data range from 0-9.

The data looks something like this:

raw_0   raw_1   raw_2   raw_3   raw_4
0   120.0   133.0   96.0    155.0   66.0
1   159.0   167.0   163.0   185.0   160.0
2   45.0    239.0   66.0    252.0   NaN
3   126.0   239.0   137.0   NaN 120.0
4   226.0   222.0   153.0   235.0   171.0
5   169.0   81.0    100.0   44.0    104.0
6   154.0   145.0   76.0    134.0   175.0
7   77.0    35.0    105.0   108.0   112.0
8   104.0   55.0    113.0   90.0    107.0
9   97.0    253.0   255.0   251.0   141.0
10  224.0   227.0   84.0    214.0   57.0
11  NaN 13.0    51.0    50.0    NaN
12  82.0    213.0   61.0    98.0    59.0
13  NaN 40.0    84.0    7.0 39.0
14  129.0   103.0   65.0    159.0   NaN
15  123.0   128.0   116.0   198.0   111.0

Each column has around 5% missing values and I want to fill in these NaN values with something meaningful. However, I'm not sure how to go about this. Any suggestions would be welcome.

Thank you!

Answer 1

Given you have images stretched out as columns in a table with ~48,500 rows, I am assuming you have the raw images that are 220x220 in dimension.

You can use a function available via OpenCV called inpaint, which will restore missing pixel values (for example black pixels of degraded photos).

Here is an image example. Top-left shows the image with missing values (in black). Top-right shows just the missing values (the mask). Bottom-left and bottom-right are the final output, comparing two different algorithms for filling the images.

I would suggest trying both methods on your images to see what looks best.

Have a look at the Documentation for more details on the algorithms themselves. Here is the documentation of the actual function.

As for code, it will look something like this:

import opencv as cv    # you will need to install OpenCV

dst = cv.inpaint(img, mask, 3, cv.INPAINT_TELEA)

• the first argument is your image with missing values
• the second is the mask, with locations of where missing pixels are, i.e. which pixels should be filled/interpolated.
• third is the radius around missing pixels to fill
• fourth is the flag for the algorithm to use (see link above for two alternatives)

For each image, you can generate the mask with something like this:

mask = image[np.isnan(image)]

Note: '==' doesn't work with np.nan

Answer 2

There are multiple ways to go after this. You can do mean imputation, median imputation, mode imputation or most common value imputation. Calculate one of the above value for either rows or columns depending on how your data is structured. One of the simplest ways to fill Nan's are df.fillna in pandas

Answer 3

for any (x,y) if NAN you can impute to average of surrounding pixels as:

if((x==0  & y==0):
 return (x+1)+(y+1))/2 

else if(x==x_max & y==y_max):
 return (x-1)+(y-1))/2

else if(x==0 & y==y_max):
 return (x+1)+(y-1))/2

else if(x==x_max & y==0):
 return (x-1)+(y+1))/2

else if(x==0):
 return ((x+1)+(y-1)+(y+1))/3

else if(x==x_max):
 return ((x-1)+(y-1)+(y+1))/3

else if(y==0):
 return ((x+1)+(x-1)+(y+1))/3

else if(y==y_max):
 return ((x-1)+(x+1)+(y-1))/3

else :
  return  ((x-1)+(x+1)+(y-1)+(y+1))/4 

Answer 4

If adjacent rows are adjacent pixels that I'd just use the average value of the adjacent pixels. That seems like it would make sense for an image, and would certainly be hard for the human eye to see.