Somebody trained a machine learning model successfully with some data A.
Is it possible to reverse engineer that machine learning model in such a way, that I get insights about that data A or at least parts of it?
With insights I mean: to get an idea about a single row in my data A like: what values or value ranges did certain attributes in that row have.
Extra question: may it perhaps depend on the kind of machine learning model (whether it is a neural networks model or another one)?
Yes, this is possible to various extents.
The most striking example is in predictive text models, as in the xkcd comic:
See these two Berkeley AI Research blog posts on the topic, and the associated papers. As mentioned there, one important idea in this area is differential privacy. This is a very strong condition; as described in section 2.1 of this paper:
the attacker can know any amount of information about an individual, and even know every single other data point in the dataset, and still not be able to detect the presence of the targeted individual
This level of privacy is failed by even simple linear models; a quick search finds multiple papers about differentially-private versions of logistic regression, and the above link surveys such questions for decision trees.
To your less stringent sense of privacy, you would need to have some way to identify individual rows, or else extracting insights about any given row would be impossible. Knowing some subset of the rows' values may be enough to uniquely identify it, and then some knowledge about the model can give information about the other values in the row as above. Having access to the actual model information can help a lot, from k-nearest-neighbors which has to store the actual training data, to linear models where some information can be gleaned from the coefficients; in most research areas though, we're limited to treating the model as black-box, with at best the ability to query the model freely.
One thing came to mind while preparing this answer, but I don't know if there's anything formalized about it: training data is generally "nicer" than random numbers; perhaps you know that in all the training data, a feature is rounded to the hundredths place. Then even with a simple polynomial regression you may get some interesting hacks. Say the model has a single feature $x$ with a target $y$, and both are rounded to the hundredths place in the training set, and suppose the model interpolates the training data (generally a bad idea, but sometimes fine with say neural networks). An attacker can query $\hat{f}(x)$ for every hundredth-rounded $x$ (in some realistic range), and keep a list of those whose outputs happen to also have no nonzero decimals after the hundredths place; those are the only possible training points because the model interpolated the training set.
