Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a large dataset with characters and 90000 intances and I have the error ValueError: array is too big when I have the following code before the plot_kmeans_digits.py code:
data2=list(csv.DictReader(open('C:\diabeticdata.csv', 'rU'))) vec = DictVectorizer() data = vec.fit_transform(data2).toarray()
Do you know how I can solve this error?
Thanks in advance.
The problem is that you're converting a large sparse dataset into a dense array via toarray(). Don't do that :) Why do you need it in dense array format? Try posting more of your code to see if we can figure out better ways to solve the problem. With any large dataset, you want to stay sparse as long as possible and only work with smaller sets of dense data (if dense is needed at all).
