Can we use prior information to improve the results of a neural network? If so, how do we incorporate it into the learning process?

Let $$f_\alpha(x)$$ be the function $$f_\alpha(x) = x^2 + \alpha\sin(x),$$ on the interval $$[-5,5]$$. Suppose $$\alpha = 2$$, and our goal is to learn the function $$f_2(x)$$ using some form of neural network. $$f_2$$ looks like this:

We are given a set of noisy measurements of $$f_2$$ from which we would like to learn $$f_2$$. Specifically, we are given a set of random samples $$\{y_i,x_i\}_{i=1}^n$$ with $$y_i = f_2(x_i) + N(0,\sigma),$$ where $$N(0,\sigma)$$ is normally distributed noise with standard deviation $$\sigma$$:

Now we can use the samples $$\{y_i,x_i\}_{i=1}^n$$ to estimate the unknown function $$f_2$$. However, before doing that, note that the general behavior of $$f_2$$ is captured well by the function $$f_0(x) = x^2$$:

Now suppose that instead of just being given the set of noisy samples $$\{y_i,x_i\}_{i=1}^n$$ of $$f_2$$ we are given both the noisy samples and the function $$f_0$$ which we are told is a decent approximation of $$f_2$$. In other words, we have some prior knowledge on what the function $$f_2$$ generally looks like.

Can we somehow incorporate this prior knowledge of $$f_2$$ into our neural network learning process so that we can get a better estimate of $$f_2$$ than estimating it based on just the noisy samples on their own?

If so, what are our options for incorporating this knowledge into a neural network? Does the type of neural network (CNN, RNN, etc...) affect the way we incorporate the prior information?

P.S. I am coming from statistics/mathematics and while I understand the general principles of neural networks I have only just started using them.

P.P.S. Here is the Matlab code for the images

rng(123);

N = 100;
x = linspace(-5,5,N);
alpha = 2;
sigma = 4;

f_0 = x.^2;
f_alpha = x.^2 + alpha*sin(5*x);
f_sigma = f_alpha + sigma*randn(N,1).';

figure
hold on, grid on
plot(x,f_0,'k--')
plot(x,f_alpha,'b')
plot(x,f_sigma,'r.')