If I have a set of N data, each individual data has 4 features. I do not know a priori the relations that could exist (or don't exist) between the features.
Is it possible to generate new data from my initial set that will respect the implicit relations that could exist between the features without having to find explicitly these relations ?
What you want is a generative model.
A simple generative model from the deep learning family are autoencoders: neural networks that receive as input your data and are trained to output that very same data.
There are different types of autoencoders. One of the most simple are contractive autoencoders, that have a bottle neck layer, that is, a layer with very few units. For your case, with only 4 input features, you may have a 2 unit (or even 1 unit, you may try to tune this hyperparameter) hidden layer as bottleneck.
Once fully trained, you only take the part of the autoencoder from the bottleneck to the output, and feed it with random numbers as input, and expect to get as output data that follows the same distribution as the original inputs.
The idea is that the training has allowed the net to learn representations of the input data distributions in the form of latent variables.
Depending on the distribution of your input data, a simple contractive autoencoder may not be able to properly learn good representations. More advanced variants include denoising autoencoders, sparse autoencoders and variational autoencoders.
Other generative models that are currently very trendy are Generative Adversarial Networks.
