2
$\begingroup$

I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error. The machine has 16G RAM, so I don't think this should have happened. I tried with some logging where the error came and found that the data got loaded completely into numpy arrays. And it was the algorithm itself that caused the error. Can someone help me with this or sugggest more memory efficient algorithm for feature selection, or is subsampling the data my only option. Are there some deterministic good subsampling techniques.

EDIT: Relevant code piece:

n=8;
y=mydata[:,0];
X=mydata[:,[1,2,3,4,5,6,7,8]];
#print y;
#print X;
print "here";
omp = OrthogonalMatchingPursuit(n_nonzero_coefs=5,copy_X = False, normalize=True);
omp.fit(X,y);
coef = omp.coef_;
print omp.coef_;
idx_r, = coef.nonzero();
for id in idx_r:
        print coef[id], vars[id],"\n";

The error I get:

File "/usr/local/lib/python2.7/dist-packages/sklearn/base.py", line 324, in score
return r2_score(y, self.predict(X), sample_weight=sample_weight)
File "/usr/local/lib/python2.7/dist-packages/sklearn/metrics/metrics.py", line 2332, in r2_score
numerator = (weight * (y_true - y_pred) ** 2).sum(dtype=np.float64)
MemoryError
Improve this question
$\endgroup$
3
  • $\begingroup$ There's not much information here. You should provide more detailed diagnostic information, including how you are invoking OMP and with what settings. $\endgroup$
    – Sean Owen
    Nov 3, 2014 at 8:27
  • $\begingroup$ Edited with relevant piece of code and the error as well $\endgroup$
    – sshanks
    Nov 4, 2014 at 9:52
  • $\begingroup$ It's not hard to imagine that 16G of data would be too much for 16G of memory. Do you read the data straight into numpy arrays? If not, you're doubling the memory footprint. Can you use sparse matrices? $\endgroup$
    – jamesmf
    Nov 13, 2015 at 13:09

1 Answer 1

Reset to default
0
$\begingroup$

One option is to set precompute to True, which will precompute the Gram and Xy matrix. It would be something like:

from sklearn.linear_model import OrthogonalMatchingPursuit

omp = OrthogonalMatchingPursuit(precompute=True, n_nonzero_coefs=5,copy_X = False, normalize=True)

Also upgrading to Python 3 might help with memory issues.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.