# Low memory error while performing degree 2 polynomial regression on (3000*1835) sized array

I am working on a problem to predict the revenue, a film will generate. Some of the features available in the data set are json collection for the crew, cast which worked in the film. I applied onehotencoding to these columns.
As a result, I have a (3000*1835) sized array. This too I got after extracting only director's data from 'Crew' columns and applying PCA with 60% variance retention.
But, when I apply polynomial regression, I get the below mentioned error:

\$\lib\site-packages\sklearn\model_selection_validation.py:532: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: MemoryError: Unable to allocate 30.2 GiB for an array with shape (2400, 1686366) and data type float64

I am using the code as shown below for polynomial regression:

polyFeature = PolynomialFeatures(degree=2)
linearRegression = LinearRegression()
pipeline = Pipeline([('polyFeature',polyFeature),('linearRegression',linearRegression)])
score = cross_val_score(pipeline,XTrain,YTrain,n_jobs=4,cv=5)


I am using a system with 6 cores, 32 GB RAM.

• 1. No sense in using polynomial features to binary features (one hot encoded) 2. If you use polynomial features I think it is because you have proven that the relationship between your features and your target is not linear, so why not to use a non linear model like a tree base model instead? (In such case no need for polynomial features) – Julio Jesus Mar 24 at 21:11