4
$\begingroup$

I'm training a Neural Network for pattern recognition. I have a matrix of examples of size ($N$x$4$) with $N$ examples and $4$ variables.

When I train the Network the Number of examples used for training vs. Cross-entropy, the performance of the Network it's not improving when adding new examples.

I suspect that the examples are highly correlated between them and adding new examples is not giving new information to the network, but I don't figure how to measure that correlation.

Each row of the matrix can be labeled $\mathbf{X}_{i}$. Suppose zero mean, the covariance matrix between examples would be $\mathbf{R}_{ij} = E[\mathbf{X}_{i} \mathbf{X}_{j}^T]$. I can also calculate the covariance matrix of $\mathbf{X}$, whose elements are $\mathbf{R}_{ij}$ but in that case I would have a matrix of $\sim 300000$x$300000$.

Is there a way of calculate some average correlation or an other measure of correlation between examples?.

Thanks.

EDIT: I don't want to know the correlation matrix of variables, I want to know the correlation between examples.

EDIT2: This is the learning curve:

enter image description here

$\endgroup$
  • $\begingroup$ What is the topology of your network? You have very few features and many rows, maybe increasing the complexity will help, although the (big) difference in CV and train scores is weird $\endgroup$ – Jan van der Vegt Nov 22 '16 at 8:26
1
$\begingroup$

When I train the Network the Number of examples used for training vs. Cross-entropy, the performance of the Network it's not improving when adding new examples.

You are assuming improving new samples will always improve the performance. This is not true in many cases. Did you plot a learning curve to see how the performance is improving with respect to increasing number of samples? You probably should try to do this before delving deep into sample correlation.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes, I edited the question and I attached the learning curve. Thanks. $\endgroup$ – juan9793 Nov 20 '16 at 21:48
  • $\begingroup$ It looks to me that your training cross-entropy is already fairly low beginning from the first sample size. It may just indicate that NN has learned everything it could learn from the first batch of sample. You probably need to do some feature engineering, transformation and perhaps, try another algorithm/technique. $\endgroup$ – Sal Nov 20 '16 at 22:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.