2
Both pandas and readr allow flexibility on the marker used to indicate missing values, allowing to specify values that should be treated as missing when reading csvs:
readr https://readr.tidyverse.org/reference/read_delim.html
pandas https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html
given that, maybe you can handle missing ...
2
In your confusion matrix kernel , you passed two arguments. Y_pred and y_true. They are supposed to have same dimensionality. But they are different. That's why the error occured.
2
Figured out an easy way to do this.
First we will just select PID from the real data.
Then we will just sample 0.75 % of these PID and save these point as training PID and the rest as testing PID.
We will thne find the intersection between this list and the real data using PID.
1
So, your question is to instantiate a new data frame df2 from another data frame df1, by simply selecting rows.
You can do this by indexing. What is great by pandas DataFrames is that you can index a DataFrame using a list of indices.
df2 = df1.iloc[[list of indices],:]
1
Hi Soumyadeep and welcome to Data Science/Stack Exchange
What you are describing is called regression imputation, and it is a valid method to use on missing data. However, if the data is sparse (lots of missing values), this issue will be more difficult to handle.
In general, missing data can be handled in several ways (row deletion, imputation, substitution,...
1
To Tokenise, clean up symbols (i.e. Normalise), etc. just use one of the widely used NLP libraries, they should be able to do most of the work for you.
Examples include:
NTLK
Spacy
SparkNLP
.. and many more. Perhaps look up some articles comparing their strengths and weaknesses on Google to decide what's best with your project.
As for the detecting English ...
1
If you decided to remove outliers. Please remove them before the split(even not only before a split, it's better to do the entire analysis(stat-testing, visualization) again after removing them, you may find interesting things by doing this).
If you remove outliers in only any one of train/test set it will create more problems.
(EX: An outlier in train set ...
1
One option is to reframe it as a word embedding problem. Emojis can be embedded in a vector space along with comments and hashtags. Then distance measures and clustering can be used to find the emojis that are associated with different sentiments.
Only top voted, non community-wiki answers of a minimum length are eligible
