New answers tagged dataframe
1
You can use the .shift method for this:
df["B"] = df["B"].shift(2)
The value used for the first two rows (NA or something else) can be controlled using the fill_value argument.
0
In your code, you passed smallDF.index directly. The interpreter considers this as a list of column names. So your code will be interpreted like below. But 4,8,105,107 are not columns. They are indexes.
bigDF[[4,8,105,107] = smallDF
To locate a row based on the index, use the loc function. Try using below.
bigDF.loc[smallDF.index]=smallDF
1
The binary features obtained from one-hot encoding a categorical feature must be obtained from the training set only. This implies that any new value in the test set cannot be used.
I recommend the following method: before encoding the variable in the training set, discard all the rare values (for examples the ones which have a frequency lower than 3) and ...
0
I suggest reading only the index from the CSV file and do your modification and copy it back instead of reading the entire CSV file.
You can do that with:
df = pd.read_csv("sample.csv", names=column_names)
0
Instead of df.index.map(), use panda's .str accessor so that the slicing is vectorized. That will speed up processing on each chunk.
First of all, reading and then writing seems not to be very efficient since every line will be fully overwritten, which is not necessary, right?
Unless the entries in your index are of uniform length (so the final character ...
0
You can read a file line by line, process each line and write to a new file line by line, this is probably not the most efficient way, but will certainly solve the RAM issue.
For example:
with open("my_file.csv") as f_in, open("new_file.csv", "w") as f_out:
for line in f_in:
new_line = line # do you processing here
...
0
I prefer to read the maximum value index of each group and change the value of the candidate:
df.loc[df.groupby('Constiteuncy').idxmax().values.ravel(), 'candidate'] = 'won'
But if you prefer to use apply and lambda as you mentioned, you could try this:
index_max = df.groupby(['Constiteuncy'])['Vote'].apply(lambda x: x.idxmax())
df.loc[index_max , '...
Top 50 recent answers are included
