I'm trying to create something to help solve this problem:
My goal is that, given two images where one is an edit of the other, produce a system that outputs which one is most likely to be the original.
I have attempted to solve the problem in the following way.
Collect a data set of random images that one may find in their social media blog. (10,000 images)
Apply random lossy transformations to the image: resizing (with various scaling algorithms), cropping, lossy compression (JPEG)
Preprocess input into training data:
Collect file size, resolution, file format, lossy compression quality
Add jitter (to prevent overfitting by recognizing specific images), jitter varies per 8x8 sample
Output is a fixed number of floats per datum
Break the image into 8x8 chunks (the size is chosen to coincide with JPEG MCU size)
Select a fixed number of chunks with the most entropy (changes in pixels)
Select a color channel
Divide by 255.0
Output is 64 floats (one per pixel)
Using the pixel data from above, perform a DCT transform
Take the absolute value of the DCT value
Output is 64 floats
Feed the above data into a neural network.
I don't know if it makes sense or not, but I started with a straight-forward model and did a simple hill-climbing hypersearch which added/removed/edited layers, and it produced the following model:
Each sample (and associated metadata) is fed as a single input.
Fit the model to the labels 0.0 or 1.0, with 0.0 representing original (less noise) and 1.0 representing edited (more noise).
Even though these labels don't describe any quality of the image itself, the goal is to use them to indicate some kind of ordering. The idea here is to "pull" the weights towards 0 for images known to have less noise and towards 1 for those with more noise, which I hope will cause the model to output higher values for images which have been edited.
Currently the model's accuracy is 88% (i.e. it correctly produces higher values for our edited versions for 88% of input image pairs).
My question is if there is a better way to represent ordering in this case other than labeling pairs 0 and 1. One potential issue with this is that the model is penalized if it outputs a score >1 or <0 (though currently avoided using sigmoid activation if that makes sense). Also, if there are other obvious improvements in the whole process. Thank you!