How to fetch Kaggle data from python code?
Install kaggle package
C:\Users\TalgatHafiz> pip install kaggle
login to your Kaggle account
click on the icon in the upper right corner -> My Account
Scroll down to API section
Click "Create New API Token"
"kaggle.json" file is created and saved locally
Create ".kaggle" dir
OpenML has a gallery of different use case examples, including browsing and downloading datasets through python, and running benchmarks:
When you want to benchmark new algorithms, this is the gist:
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import ...
Here is some script for "openml" collection of datasets.
Hopefully one can provide something similar for other databases.
#see docs: https://docs.openml.org/Python-guide/
!pip install openml
import numpy as np
import pandas as pd
# Get information on all collection of openml datasets:
datalist = openml.datasets....
In your case you have 3 options for training your model:
Make a class for each card number, so 13 classes.
Make a class for 1, 2, 3, king, queen, jack and one bucket class for "other", so 7 classes.
Make a class for 1, 2, 3, king, queen, and jack, so 6 classes. And whenever the model is not confident of any, assume it is "other".
Normalizing so that "all the observations have the same importance" is kinda ambiguous and ill-defined. In any case, it would be strongly advised to avoid re-inventing the wheel, and use one of the several scalers available out there (e.g. in the sklearn.preprocessing module).
Here is an example using MinMaxScaler, which will re-scale your data in [...
Apparently the paper introducing the dataset mentions a list of "online appendices" (table 1 p 363) which seems to contain details about the categories. However I wasn't able to find these appendices in the additional material.
 Since it appears that there is no existing source for the original list of topics, I think your best bet is to ...
You probably should
conduct a missing values analysis to see what is the percentage of missing per column (figure below, from dataprep package)
Decide a threshold according to which you may want to completely drop a column or not (depending on how your analysis or model treats nans as well)
For the columns that are not dropped, you should impute the missing ...