3
It looks like that your prediction is clamping at 750.
Be mindful of the fact that Tree can't predict a Regression value that is outside the range it has been trained on.
So, first of all, please assure that your data doesn't have a trend.
2
Of course, making them smaller will cause loss of information. Since before you had N x M data points for each image to describe the content of the image, but after resizing you will have n x m (Where n and N denote the number of rows in an image, m and M denotes the number of columns in an image. Also, n <= N and/or m <= M). That is, a small number of ...
1
It sounds like you are looking for an unsupervised learning approach (meaning you don't need to manually label your data).
Something like k-means clustering could work well. This would allow you to group you comments into k distinct clusters. You could then view counts of comments in those clusters and explore the clusters to determine their meaning.
In ...
1
A slightly hacky way to get there maybe but you can do this to get what you want from the second table;
df2['count'] = 1
pivot = df.pivot_table(df, index='userid', columns='productid', values = 'count').reset_index()
pivot = pivot.fillna(0)
You would then want to merge this to the first dataset like this;
finaldf = pd.merge(df1, pivot, left_on='userid', ...
1
If my understanding is right, you have a regression problem, with categorical features with high cardinality and "outliers" (or just big numbers).
How have you encoded categories? Target Encoding?
There is another option that is not encoding with the mean but with the median that on some cases can perform better.
On this notebook , you can see an ...
1
In a first iteration, use a sentence encoder.You can find pre-trained model on tensorflowhub (https://www.tensorflow.org/hub/tutorials/semantic_similarity_with_tf_hub_universal_encoder), spacy (https://spacy.io/universe/project/spacy-universal-sentence-encoder) or huggingface (https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens) to name a ...
1
Thank you for the above answer!
That definitely works. However, I found a more efficient way in terms of computation using np.rolling
df['D'] = df['A'].rolling(min_periods=1, window=3).mean()
df['B'] = np.where(df['B'].isnull,df['D'],df['B'])
np.rolling helps to compute the cumulative sum of previous n values.
np.where helps to apply some output based on a ...
1
Strictly theoretically it makes no difference on DNN, I answered it today here and I said:
Here is why: We already know mathematically that NN can approximate any function. So lets say that we have Input X. X is highly correlated, than we can apply a decorrelation technique out there. Main Thing is, you get X` that has different numerical representation. ...
Only top voted, non community-wiki answers of a minimum length are eligible
