Which is preferable pandas or numpy?
It's totally up to you, if you really need to increase speed, go for numpy arrays, although at the same time the code will tend to get unwieldy and prone to errors, because you won't be able to keep track of feature columns easily. On the contrary, pandas will make your life easy when coding, so this choice is up to you.
What is the best way of creating folds for cross-validation?
Again there is no best algorithm or approach in machine learning that can be used in every case, it all depends on your preferences and needs. So I'll give you several options. I assume you are dealing with classification problem so I'll advise only on that,
- You can use sklearn.model_selection.StratifiedKFold function, as in this example,
kf = StratifiedKFold(n_splits=5, shuffle=True, random_state=43)
for train_idx, test_idx in kf.split(X, y):
X_train, X_valid = X.iloc[train_idx], X.iloc[test_idx]
y_train, y_valid = y.iloc[train_idx], y.iloc[test_idx]
this option gives you more control over the code while debugging, and what's more important it stratifies the result according to y labels. Stratification here makes sure that distribution of values in y_train and y_valid will repeat the distribution in y proportionally, or simply put if one half of values in y were 1s and another half 0s, y_train and y_valid will have the same distribution of half being 1s and another half being 0s.
With this function, you can use whichever scoring function you want.
- Predefine cross-validation folds beforehand,
kf = StratifiedKFold(n_splits=5, shuffle=True, random_state=43)
for fold, (t_, v_) in enumerate(kf.split(X, y)):
# where X = df.drop('target'), y = df.target
df.loc[v_, 'fold'] = fold
these folds can be used as following:
folds = set(df['fold'].astype(int).unique())
for fold in folds:
df_train, df_valid = df[~(df.fold==fold)], df[(df.fold==fold)]
X_train, y_train = df_train.drop(['target', 'fold'], axis=1), df_train.target
X_valid, y_valid = df_valid.drop(['target', 'fold'], axis=1), df_valid.target
this approach apparently may increase the speed of computations, as there is no need to compute folds at every step using complex algorithm, instead all we do is select rows with the current fold. Additionally, this approach makes reproducibility of the result possible, especially if you save the dataset with predefined folds to csv, and so that other people will get the same cross-validation folds.
Another important thing, this approach makes it possible to compute different folds in parallel. This may come in handy, with neural nets used in deep learning.
Here, you can use whichever scoring function you want as well.
- Use sklearn.model_selection.cross_val_score function as it was mentioned in the previous answer.
This function is not exactly what you want, because it evaluates a score by cross-validation, while you were asking about ways of cross validation itself. Although I could hardly imagine what you can use cross-validation for apart from evaluating a score. Anyway, let's address pros and cons of this function.
Pros: simple to use, stratification is possible.
Cons: apparently, this is a black-box function, you can't debug it, unless on local machine, even there it is hard to do that. It is slower than second approach for mentioned reasons. And it is restricted to scoring methods listed here, in case you want use different scoring.
UPDATE:
In your newly added code, you're slicing the datasets wrong,
train_setosa1 = iris_setosa.iloc[40, :]
test_setosa1 = iris_setosa.iloc[-10, :]
you are selecting only one row for each dataset, I suspect you were trying to slice it so that first 40 rows were included in train_setosa1 and the rest in test_setosa1, here's how you do that,
train_setosa1 = iris_setosa.iloc[:40]
test_setosa1 = iris_setosa.iloc[40:]
But even in this case it is not proper cross-validation, because the whole point of that is using randomness of selection, and usually machines are better at this than people are.
Here's equivalent implementation of your code,
iris_species = ["Iris-setosa", "Iris-virginica", "Iris-versicolor"]
kf = StratifiedKFold(n_splits=5, shuffle=True, random_state=43)
for species in iris_species:
species_df = data[data["Species"] == species]
X = species_df.drop('Species', axis=1)
y = species_df.Species
for train_idx, valid_idx in kf.split(X, y):
X_train, X_valid = X.iloc[train_idx], X.iloc[valid_idx]
y_train, y_valid = y.iloc[train_idx], y.iloc[valid_idx]
# do whatever it is you want with training and validation data here
pass
Cheers,
