How to train a neural network where computing the loss requires multiple object values?

Question

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image.

My understanding is (please forgive me I'm a novice here) a neural network would suit this purpose.

I want to train a function:

def myHyperParameterFunction(a,b,c,...):
  return (one,two,three,...)

(This being a neural network starting with keras.layers.InputLayer(input_shape=(n,)) and ending with keras.layers.Dense(m))

Which produces hyper-parameters for another algorithm:

def myAlgorithm(image, (one,two,three,...)):
  return newImage

By using myAlgorithm a cost function can be formed:

def lossFunction(true_image, image, (one,two,three,...)):
  return true_image-myAlgorithm(image, (one,two,three,...)) 

Looking at the tensorflow documentation it seems I can only defined a loss function in terms of 2 variables y_true and y_pred, this seems to make this impossible. Possibly I could get around this by y_true being an index to arrays which contain images and true_images, but this feels a rather awkward solution (notably the object type of y_true cannot be numpy.ndarray so I cannot package both images in).

In summary, it seems the problem is that to compute the loss requires an additional external value (that being image, where y_pred is (one,two,three,...) and y_true is true_image). I am not bound to tensorflow, any tool or framework which helps with this I would be interested in.

How could I solve this?