I have a sparse matrix, $X$, created by TfidfVectorizer and its size is $(500000, 200000)$. I want to convert $X$ to a data frame but I'm always getting a memory error.
pd.read_csv(X.toarray().astype("float32"), columns=tokens, chunksize=...).
And it seems that when I convert $X$ to a numpy array using
X.toarray(), I get an error.
Can someone tell me what is an easy solution for this? Is there anyway I can create a sparse dataframe from $X$ without memory error?
I have been running my codes on Google Colab Pro and I think it provides me less than 100 GB Ram.