2
You can try with this:
import pandas as pd
import nltk
df = pd.DataFrame({'frases': ['Do not let the day end without having grown a little,', 'without having been happy, without having increased your dreams', 'Do not let yourself be overcomed by discouragement.','We are passion-full beings.']})
df['tokenized'] = df.apply(lambda row: nltk.word_tokenize(row[...
2
There is not a reshape problem.
You need to transform your text in a set of features, say, vectorize it in the same way you created your dataset, in this case using TF-IDF.
Just prepare a query vector applying the same TF-IDF and will work.
2
You cannot do that directly because your training data is not a text but set of features extracted from the text.You need to convert the text to list of features and then try to predict it
2
A quick search of the source code, e.g. RandomForestClassifier's, doesn't find anywhere that sample_weight gets saved as a class attribute. I suspect that was a conscious decision: the sample weights are directly tied to the dataset, which also doesn't get saved for later use; that's why sample_weight appears in the fit method rather than the class ...
1
Reason for the discrepancy
Two aspects have to be considered regarding the split:
Is the split done in a stratified manner? (it should)
Is the data shuffled? (it should)
The line
X_train, X_test, y_train, y_test, i_train, i_test = train_test_split(feature_matrix, y, indices, test_size=0.33, random_state=random_state)
splits the data in a stratified ...
1
Accuracy is a statistic that you can compute on a dataset if you know the true labels. For a single image, the accuracy is either 0% or 100% based on if you get it right or wrong.
In the newer versions of Keras, the predict method returns the probabilities of the classes, what you want to print is (if I guess correctly), the probability score for the best ...
1
You could always call on the pandas DataFrame's columns and work with that.
values = df['price'] * df['quantity']
sum(values)
if you want more information, I recommend https://stackoverflow.com/questions/14059094/i-want-to-multiply-two-columns-in-a-pandas-dataframe-and-add-the-result-into-a-n
1
If I understand the problem correctly, you want to fill all missing values in the Fare column by the median value of the Fare column where Pclass=3. This can be achieved by putting the extra row filter test['Pclass']=3 on median of the fare column, see below.
test['Fare'] = test['Fare'].fillna(test.loc[test['Pclass']=3,'Fare'].median())
1
Experimentally: using cross-validation on a subset of your training data, compute the performance of every option that you want to consider. Then select the best option and train the final model using this option.
// different settings for hyper-parameters,
// for instance different pruning criteria:
hpSet = { hp1, hp2, ...}
trainSet, testSet = split(...
1
If you have a one dim data, why do you need to use K-means?
In such a case, to detect the outlier I would recommend creating a simple histogram and then based on its shape you can visually find the outliers. To get a proper outlier threshold you can use np.quantile() function.
1
I think you have misunderstood the koalas library. You can say its Pandas on Distributed System. You can use Koalas similar to pandas. There are few drawbacks with respect to APIs which is documented in their docs and few articles already written on medium.
You can do your EDA and straight away use them in all the libraries you have mentioned.
Recent ...
1
You are using different random starting points each time you run k-means (random_state=None) This means you may get different clusterings, different clustering metrics each time. That's expected. What you may wish to do is average the results over several different runs to get a more reliable estimate of the loss at each k.
Only top voted, non community-wiki answers of a minimum length are eligible
