I think you are trying to do cross validation with hyperparameter tuning, so here is how it's done with k-fold CV (from this answer):

You can split your data into 2 datasets: training and test. k-folds cross validation takes a model (and specified hyperparameters) and partitions the training dataset into k equally sized subsets. Then, it does the following k times:

• Trains the model on k-1 of the subsets
• Evaluates the model accuracy on the subset that wasn’t trained on.

It then reports the average error. To do hyperparameter tuning, do the steps above using every time a different hyperparameter combinations. Then, choose the set of parameters for which k-folds reports the lowest error. However, be careful to not excessively minimize the k-folds error, since it will often lead to overfitting.

Ultimately, we want a measure of how well our final model will generalize. This is why we created the test set at the beginning—evaluating the model’s accuracy on this set is a useful estimation of its success.

So, k-fold doesn't mean k different models, but k folds of the dataset!

I think you are trying to do cross validation with hyperparameter tuning, so here is how it's done with k-fold CV (from this answer):

You can split your data into 2 datasets: training and test. k-folds cross validation takes a model (and specified hyperparameters) and partitions the training dataset into k equally sized subsets. Then, it does the following k times:

• Trains the model on k-1 of the subsets
• Evaluates the model accuracy on the subset that wasn’t trained on.

It then reports the average error. To do hyperparameter tuning, do the steps above using every time a different hyperparameter combinations. Then, choose the set of parameters for which k-folds reports the lowest error. However, be careful to not excessively minimize the k-folds error, since it will often lead to overfitting.

Ultimately, we want a measure of how well our final model will generalize. This is why we created the test set at the beginning—evaluating the model’s accuracy on this set is a useful estimation of its success.

So, k-fold doesn't mean k different models, but k folds of the dataset!

To reply to your questions:

Will the training of these individual model be as usual(backward propagation)?

Yes each training is as usual, just changes the training set and the hyperparameters.

What do I initialise each model with? (Random Weights?)

Yes, (still) as usual in neural networks. You are not re-using old weights.

After completing the cross-validation, I have the best model(say B) with me now, what does it mean to train this model on the entire dataset? (Does it mean, I initialise the weights of the new model being trained on the whole dataset, with those of B).

Well, you are mistakenly exchanging again weights and hyperparameters, but, if you have a very big dataset and cross validating on the entire dataset takes too long, you can:

• take a portion of your dataset (maybe 10%), let's call it A
• Use A to find the best hyperparameters using k-fold CV as I described before
• Now you can use the entire dataset for training (except a test set) the model using those best hyperparameters. With the hope that's the real best model.