Questions tagged [kaggle]
Kaggle is an online community for data scientists and machine learning practitioners owned by Google.
116
questions
0
votes
0
answers
11
views
can someone explain how to create new features using feature interactions?
There is this notebook solving housing prices.
https://www.kaggle.com/code/jesucristo/1-house-prices-solution-top-1/notebook?scriptVersionId=12846740
and it had this bit of code, can anyone explain ...
- 1
0
votes
1
answer
24
views
Where can I practice multivariate outlier detection?
Can anyone provide me with a dataset, hopefully on Kaggle, where I can practice my skills in outlier analysis?
I have been studying this topic for quite a while, but I can't find a case study to apply ...
- 11
0
votes
1
answer
29
views
Need help understanding how this Neural Network is working
This is a model I came across, and I need some help understanding how it works
It uses South German Credit Prediction data set from Kaggle
...
1
vote
1
answer
15
views
Changing the predicted variable from price to price/km due to better visual correlation
I'm working on a dataset of Uber Rides from Kaggle. Of the important variables there are pickup and drop-off coordinates, passenger count, datetime of pickup, distance and the final price. I'm ...
1
vote
2
answers
93
views
Massive difference in accuracy of KNN depending on random_state
pardon the noob question but I am baffled by the following behavior. My model has MASSIVELY different results based on the random seed. I want to train a KNN classifier on the famous Kaggle Titanic ...
- 113
4
votes
2
answers
219
views
Why SMOTE is not used in prize-winning Kaggle solutions?
Synthetic Minority Over-sampling Technique SMOTE, is a well known method to tackle imbalanced datasets. There are many papers with a lot of citations out-there claiming that it is used to boost ...
- 5,376
0
votes
0
answers
16
views
Why feature engineering and filling NaN's reduce score?
I used CatBoost for InClass Kaggle competition.
I have tried various strategies to filling NaN values. Convert float binary variables to categorical.
Add new categorical features (from age, for ...
- 115
1
vote
1
answer
38
views
Difference between model score on test part and Kaggle public score
I tested my CatBoostModel model on part of data and get 0.92 score, but Kaggle public score was 0.9. I found new hyperparameters via randomsearch, new model score was 0.925, but on Kaggle score fell ...
- 115
0
votes
1
answer
38
views
Why performance varies among validate set, public testset and private testset?
When practicing with classical kaggle competitions, such and Titanic, House pricing, and so on, I followed the traditional process that I learned from textbook:
split training data into trainig set ...
- 3
0
votes
1
answer
144
views
Keras ImageDataGenerator unable to find images
I'm trying to add image data to a Kaggle notebook so I can run a convolutional neural network but I'm having trouble doing this via ImageDataGenerator. This is the ...
- 25
0
votes
0
answers
28
views
Not able to download image dataset from Kaggle api
I want to download only train dataset
Dataset link here
Kaggle API version - 1.5.12
...
- 43
0
votes
0
answers
56
views
GloVe Embedding Matrix "could not broadcast input array from shape (0) into shape (300)"
I'm working on Quora Question Pairs data set.
I'm trying to get embedding matrix for GloVe with the following code:
...
- 438
2
votes
1
answer
112
views
Loading medical imaging data from multiple folders
I have a fairly basic mathematical and implementational understanding of ML algorithms and CNNs, and I am trying to think of an approach for this task: https://www.kaggle.com/c/rsna-miccai-brain-tumor-...
- 99
1
vote
1
answer
32
views
Creating and training a Multilayer perceptron model with few data
Are there any ways to create a deep multilayer perceptron model that is capable of making accurate regression predictions based on the training done using around 1000 unique data?
I'm currently ...
- 13
0
votes
0
answers
23
views
Can kaggle notebook run without websocket? It is stuck at session starting on laptop
I am running a laptop in which internet works only by connecting a VPN. However, seems like that VPN does not like below websocket which is required to run Kaggle Notebooks. I dont have other option ...
- 784
3
votes
2
answers
276
views
How to pose a non-trivial ML task for students?
As I am working in teaching, I would like to provide my students a (basic) machine learning classification challenge in the next semester. I really like the idea of giving them a challenge on an ...
- 31
2
votes
0
answers
31
views
When to stop the final model training?
Let's say I'm participating in a Kaggle image recognition competition.
Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
- 21
3
votes
1
answer
75
views
Can data leakage be sometimes acceptable?
I have recently started using kaggle and I have stumbled on a few examples of practices I would consider do be data leakage. Many of them were done by people well established on the platform and I ...
- 115
0
votes
0
answers
154
views
Why am I getting good validation scores, but poor test scores in Kaggle competition
I am participating in a Kaggle multiclass classification competition. The submissions will be scored based on the 'logloss' score. I am using Keras and Scikit libraries and a deep learning network ...
- 101
0
votes
1
answer
314
views
Good model but bad confusion matrix?
I am trying to understand the code here.
The output [12] shows that the model accuracy is above 90% even for the validation set, but the confusion matrix in [16] ca not even achieve 50% accuracy, and ...
- 31
1
vote
0
answers
26
views
About the Licence of Shared Codes on the Kaggle Site [closed]
I have one job offer and there are many codes on the Kaggle site based of that job offer which could be used for the company usage like the result of this google search, and Salomon Tetelepta codes:
...
1
vote
0
answers
35
views
Recommendation System with ALS Implicit
I created a model for Recommending top 10 items to users similar to the approach used here https://github.com/benfred/implicit/blob/master/examples/lastfm.py
I wanted to evaluate the model using NDCG ...
- 21
1
vote
0
answers
26
views
Predicted probabilities of Multi Label Classification
I'm currently working on a Kaggle Competition wich objective is to predict probabilities of an ID belonging to each class. There are 4 posible classes.
The data is tabular and because it's a Kaggle '...
- 11
-1
votes
1
answer
100
views
Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?
I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
2
votes
1
answer
23
views
Image classification vs medical grading problems
For image classification problems like cat vs dogs, the output layer is 2.
Image classification problems like diabetic retinopathy seem to be more of a grading classifier. Although the targets range ...
- 23
0
votes
1
answer
23
views
Parse documents to obtain subjective sentiment
I'm working on a project which deals with MRC (Machine Reading Comprehension).
I would like a machine to read an article and give me the sentiments based on a provided token.
For Instance given the ...
- 101
0
votes
1
answer
25
views
Trying to run a kaggle notebook
I'm found an interesting problem on kaggle and more or less solved it with my limited knowledge of machine learning. I was curious how other people solved and checked the solution with the highest ...
0
votes
0
answers
14
views
Size drastically increased when images converted to hdf5 format
I converted Kaggle Dog vs Cat dataset from image to hdf5 format, in order to fasten the learning time. To my surprise, 540MB training data converted to more than 30GB of data in hdf5 file format. Why ...
- 659
0
votes
1
answer
18
views
Searchable list of Kaggle challenges
Is there a way to search Kaggle for a list of current and prior challenges, or an external site that does that?
When I search on Kaggle it will only bring up solution notebooks and datasets, it doesn'...
- 165
1
vote
1
answer
25
views
How to find appropliate algorithm to bulid a model for natural language based two data [closed]
What I would like to do
I would like to create a model to infer nationality from name and created the below data frame combining two dataset from Kaggle.
Titanic: Machine Learning from Disaster （input/...
- 129
1
vote
1
answer
69
views
Dropping missing rows in two dataframes
I have two files :
Test_data - contains the features of a dataset to find predictions for
Submission_data - contains two columns : The index column for test data and another column for its ...
- 75
0
votes
0
answers
22
views
Is there a certain threshold over which to accept or reject predictors based on correlation values with the target variable?
I have been working on the Titanic dataset. After some feature manipulation, I printed out the correlation values between my target variable Survived and all the ...
- 1
2
votes
1
answer
343
views
How do I read the cord_19_embeddings_2020-07-16.csv from the COVID-19 Open Research Dataset Challenge (CORD-19) on Kaggle?
What I'm trying to do
I wanted to use the CORD19 word embeddings csv to map them to certain findings from the rest of the dataset, but as we can see there are no stings in the first column.
The way I ...
- 151
6
votes
3
answers
4k
views
Kaggle notebook Vs Google Colab
What are the major differences between Kaggle notebook and Google Colab notebook?
To work on a dataset my first step is to start a Kaggle notebook but then I cant help thinking what could be the ...
- 61
1
vote
2
answers
208
views
How to find correlation between categorical data and continuous data
I'm working on imputing null values in the Titanic dataset.
The 'Embarked' column has some. I do NOT want to just set them all to the most common value, ...
- 13
0
votes
1
answer
40
views
Dropping attributes leads to better classifier accuracy? (Titanic Set)
I am currently tackling the Titanic Dataset on Kaggle. The goal is to find a classifier that can predict whether a passenger will survive or die.
The dataset has features that I believe are strongly ...
- 1
2
votes
2
answers
193
views
How to divide a dataset for training and testing when the features and targets are in two different files?
I am trying to divide a dataset into training dataset and testing dataset for multi-label classification. The datset I am working on is this one. It is divided into a file which contains the features ...
0
votes
0
answers
88
views
Using word embeddings for kaggle?
Not sure, if this is the right forum so redirect me if it wrong.
I have started on an NLP problem in kaggle. There i have word embeddings from google news, wiki, glove
in a zipped folder. I want to ...
- 1
1
vote
2
answers
37
views
Is this over-fitting or something else?
I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost.
I split Kaggle's 'training' data into 'training' ...
- 289
1
vote
0
answers
64
views
How to train a simple Machine learning model in batches?
I have a dataset for multiclass classification of text data. The number of samples in training data are 1,20,000. If I extract features using TF-IDF vectoriser of the sklearn library it gives about 80,...
- 135
1
vote
0
answers
27
views
What to do when feature engineering and parameter tuning don't add to the base model performance
I've been working on using LogisticRegression from scikit to try the Titanic Kaggle comp.
I've found something interesting, and that is that no amount of feature engineering and paramater tuning is ...
- 289
1
vote
0
answers
63
views
Multiple choice gap-fill question (with distractors) dataset for evaluating NLP algorithms
I am looking for a standard gap-filling multiple-choice exercise (with distractors) dataset that can be used to evaluate the NLP gap-filling ML algorithms. I expect the dataset to contain questions ...
- 43
2
votes
1
answer
178
views
Sklearn Pipeline for mixed features: numerical and (skewed) categorical
I am working on a dataset from Kaggle (housing price prediction). I have done some pre-processing on the data (missing values, category aggregation, selecting ordinal vs one-hot). I am trying to ...
- 65
2
votes
1
answer
615
views
Extract features from Decision tree leaf nodes
Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree.
[Video Link][1]
I ...
- 175
4
votes
1
answer
351
views
AUC ROC metric on a Kaggle competition
I am trying to learn data modeling by working on a dataset from Kaggle competition. As the competition was closed 2 years back, I am asking my question here. The competition uses AUC-ROC as the ...
- 65
0
votes
1
answer
54
views
Python and Titanic competition how to get the median of specific range of values where class is 3
I am trying to solve Kaggle's titanic competition. In the test set, there is only 1 row having Fare value as null. It's so easy to replace it with median or average of all Fare values.
But I am ...
- 163
-1
votes
1
answer
126
views
Meaning of 'hue" in seaborn [closed]
I know what "hue" does a little
I'm studying kaggle and the image is about bike sharing demand analysis
What I want to know is
shouldn't the second images's sum of each point's y axis be the first ...
- 3
2
votes
1
answer
111
views
Importing .ipnyb file from Kaggle into local Jupyter
Total beginner question here, please let me know if it would be more appropriate somewhere else.
I just created my first iPython notebook in Kaggle and I downloaded the ipnyb file. Now I have ...
- 123
3
votes
1
answer
223
views
Kaggle Titanic submission score is higher than local accuracy score
This is the starter challenge, Titanic. The original question I posted on Kaggle is here. However, nobody really gives any insightful advice so I am turning to the powerful Stackoverflow community.
...
- 31
0
votes
1
answer
20
views
How to define the adequate cash prize sizing for hosting a Kaggle or similar compeition?
If you take Kaggle as a well known example of data science competition size, how do you know what is an adequate budget for the cash prize size? At least to determine an order of magnitude, given I am ...
- 381