Shaido
  • Member for 4 years, 8 months
  • Last seen more than a month ago
Set value for column based on two other columns in pandas dataframe
Accepted answer
2 votes

You can use groupby together with shift and cumsum as follows: df['header_contract'] = df['contract'] + '_' + df.sort_values(['contract', 'date']).\ groupby('contract')["date"].\ apply(...

View answer
How do I print full date in the x axis of the line plot here?
Accepted answer
1 votes

For better control over the x-axis formatting, you can use the matplotlib.dates methods. In your case, MonthLocator and DateFormatter could be of interest. These can be used to adjust the x-axis as ...

View answer
pairwise_distances with Cosine and weighting
1 votes

Instead of using pairwise_distances you can use the pdist method to compute the distances. This will use the distance.cosine which supports weights for the values. import numpy as np from scipy....

View answer
Find matches between columns and manipulate data based on match
Accepted answer
0 votes

One possible way to solve this would be to create a dictionary of which components should be changed and then use replace. So in the example above, we want to create a dictionary {'B': 'A'} since all ...

View answer
Pandas datetime error when reading from excel file
Accepted answer
4 votes

Using pandas, first make sure you have a datetime column: df['DT'] = pd.to_datetime(df['DT']) To remove the milliseconds, a possible solution is to use round to obtain a specified frequency (in this ...

View answer
Dataset from sequence of messages
Accepted answer
4 votes

First, find out when the user switch and give a separate id to each message group: df['group_id'] = ((df['user'] != df['user'].shift()).cumsum()) user message group_id A ...

View answer
Numpy arithmetic operation between two columns
Accepted answer
1 votes

These types of operations can easily be done using both numpy or pandas. However, in this case I would recommend pandas since it is more intuitive. Using the example array we can create a pandas ...

View answer
Why is the result of CountVectorizer * TfidfVectorizer.idf_ different from TfidfVectorizer.fit_transform()?
1 votes

TfidfVectorizer will by default normalize each row. From the documentation we can see that: norm : ‘l1’, ‘l2’ or None, optional (default=’l2’) Each output row will have unit norm, either: * ‘l2’: ...

View answer
Pyspark Matrix Transformation
Accepted answer
1 votes

The way to do this in PySpark is to use groupBy and pivot. Since you don't want to do any actual aggregation, just the pivot, you can use first here. from pyspark.sql.functions import first (df....

View answer
How to shuffle only a fraction of a column in a Pandas dataframe?
Accepted answer
0 votes

I don't think there is any idiomatic way of doing this since it's quite unusual operation, normally the whole row or column should be shuffled. What you are doing looks like a good approach. The ...

View answer
Conditional Statement to update columns based on range
1 votes

One elegant way to solve this is by using numpy.select. This function takes a list of conditions and a list of choices and then pick the choice where the first condition is true. An advantage is that ...

View answer
Difference between sklearn make_pipeline and imblearn make_pipeline
Accepted answer
1 votes

The imblearn package contains a lot of different samplers for easy over- or under-sampling of data. These samplers can not be placed in a standard sklearn pipeline. To allow for using a pipeline ...

View answer
How to calculate Cumulative Sum with Groupby in Python?
Accepted answer
10 votes

There are multiple entries for each group so you need to aggregate the data twice, in other words, use groupby twice. Once to get the sum for each group and once to calculate the cumulative sum of ...

View answer
ESC-50 Audio data for binary classifier
1 votes

On the github page of the ESC-50 dataset, there is a list of tried classifiers as well as links to the relevant papers. The best ones are all using some kind of deep learning, mostly CNNs, and ...

View answer
Confusion Matrix - Get Items FP/FN/TP/TN - Python
4 votes

Create a method that does the printing for you: def print_confusion_matrix(y_true, y_pred): cm = confusion_matrix(y_true, y_pred) print('True positive = ', cm[0][0]) print('False positive ...

View answer
Custom conditional loss function in Keras
1 votes

You should be able to solve this with currying. Make a function that takes the label as input and returns a function which takes y_true and y_pred as input. Note that the label needs to be a constant ...

View answer