Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [kaggle]

Kaggle is an online community for data scientists and machine learning practitioners owned by Google.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

can someone explain how to create new features using feature interactions?

There is this notebook solving housing prices. https://www.kaggle.com/code/jesucristo/1-house-prices-solution-top-1/notebook?scriptVersionId=12846740 and it had this bit of code, can anyone explain ...
user avatar
  • 1
0 votes
1 answer
24 views

Where can I practice multivariate outlier detection?

Can anyone provide me with a dataset, hopefully on Kaggle, where I can practice my skills in outlier analysis? I have been studying this topic for quite a while, but I can't find a case study to apply ...
user avatar
0 votes
1 answer
29 views

Need help understanding how this Neural Network is working

This is a model I came across, and I need some help understanding how it works It uses South German Credit Prediction data set from Kaggle ...
user avatar
1 vote
1 answer
15 views

Changing the predicted variable from price to price/km due to better visual correlation

I'm working on a dataset of Uber Rides from Kaggle. Of the important variables there are pickup and drop-off coordinates, passenger count, datetime of pickup, distance and the final price. I'm ...
user avatar
1 vote
2 answers
93 views

Massive difference in accuracy of KNN depending on random_state

pardon the noob question but I am baffled by the following behavior. My model has MASSIVELY different results based on the random seed. I want to train a KNN classifier on the famous Kaggle Titanic ...
user avatar
  • 113
4 votes
2 answers
219 views

Why SMOTE is not used in prize-winning Kaggle solutions?

Synthetic Minority Over-sampling Technique SMOTE, is a well known method to tackle imbalanced datasets. There are many papers with a lot of citations out-there claiming that it is used to boost ...
user avatar
0 votes
0 answers
16 views

Why feature engineering and filling NaN's reduce score?

I used CatBoost for InClass Kaggle competition. I have tried various strategies to filling NaN values. Convert float binary variables to categorical. Add new categorical features (from age, for ...
user avatar
1 vote
1 answer
38 views

Difference between model score on test part and Kaggle public score

I tested my CatBoostModel model on part of data and get 0.92 score, but Kaggle public score was 0.9. I found new hyperparameters via randomsearch, new model score was 0.925, but on Kaggle score fell ...
user avatar
0 votes
1 answer
38 views

Why performance varies among validate set, public testset and private testset?

When practicing with classical kaggle competitions, such and Titanic, House pricing, and so on, I followed the traditional process that I learned from textbook: split training data into trainig set ...
user avatar
0 votes
1 answer
144 views

Keras ImageDataGenerator unable to find images

I'm trying to add image data to a Kaggle notebook so I can run a convolutional neural network but I'm having trouble doing this via ImageDataGenerator. This is the ...
user avatar
0 votes
0 answers
28 views

Not able to download image dataset from Kaggle api

I want to download only train dataset Dataset link here Kaggle API version - 1.5.12 ...
user avatar
0 votes
0 answers
56 views

GloVe Embedding Matrix "could not broadcast input array from shape (0) into shape (300)"

I'm working on Quora Question Pairs data set. I'm trying to get embedding matrix for GloVe with the following code: ...
user avatar
  • 438
2 votes
1 answer
112 views

Loading medical imaging data from multiple folders

I have a fairly basic mathematical and implementational understanding of ML algorithms and CNNs, and I am trying to think of an approach for this task: https://www.kaggle.com/c/rsna-miccai-brain-tumor-...
user avatar
1 vote
1 answer
32 views

Creating and training a Multilayer perceptron model with few data

Are there any ways to create a deep multilayer perceptron model that is capable of making accurate regression predictions based on the training done using around 1000 unique data? I'm currently ...
user avatar
0 votes
0 answers
23 views

Can kaggle notebook run without websocket? It is stuck at session starting on laptop

I am running a laptop in which internet works only by connecting a VPN. However, seems like that VPN does not like below websocket which is required to run Kaggle Notebooks. I dont have other option ...
user avatar
3 votes
2 answers
276 views

How to pose a non-trivial ML task for students?

As I am working in teaching, I would like to provide my students a (basic) machine learning classification challenge in the next semester. I really like the idea of giving them a challenge on an ...
user avatar
2 votes
0 answers
31 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
user avatar
3 votes
1 answer
75 views

Can data leakage be sometimes acceptable?

I have recently started using kaggle and I have stumbled on a few examples of practices I would consider do be data leakage. Many of them were done by people well established on the platform and I ...
user avatar
  • 115
0 votes
0 answers
154 views

Why am I getting good validation scores, but poor test scores in Kaggle competition

I am participating in a Kaggle multiclass classification competition. The submissions will be scored based on the 'logloss' score. I am using Keras and Scikit libraries and a deep learning network ...
user avatar
  • 101
0 votes
1 answer
314 views

Good model but bad confusion matrix?

I am trying to understand the code here. The output [12] shows that the model accuracy is above 90% even for the validation set, but the confusion matrix in [16] ca not even achieve 50% accuracy, and ...
user avatar
  • 31
1 vote
0 answers
26 views

About the Licence of Shared Codes on the Kaggle Site [closed]

I have one job offer and there are many codes on the Kaggle site based of that job offer which could be used for the company usage like the result of this google search, and Salomon Tetelepta codes: ...
user avatar
1 vote
0 answers
35 views

Recommendation System with ALS Implicit

I created a model for Recommending top 10 items to users similar to the approach used here https://github.com/benfred/implicit/blob/master/examples/lastfm.py I wanted to evaluate the model using NDCG ...
user avatar
  • 21
1 vote
0 answers
26 views

Predicted probabilities of Multi Label Classification

I'm currently working on a Kaggle Competition wich objective is to predict probabilities of an ID belonging to each class. There are 4 posible classes. The data is tabular and because it's a Kaggle '...
user avatar
  • 11
-1 votes
1 answer
100 views

Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?

I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
user avatar
2 votes
1 answer
23 views

Image classification vs medical grading problems

For image classification problems like cat vs dogs, the output layer is 2. Image classification problems like diabetic retinopathy seem to be more of a grading classifier. Although the targets range ...
user avatar
0 votes
1 answer
23 views

Parse documents to obtain subjective sentiment

I'm working on a project which deals with MRC (Machine Reading Comprehension). I would like a machine to read an article and give me the sentiments based on a provided token. For Instance given the ...
user avatar
  • 101
0 votes
1 answer
25 views

Trying to run a kaggle notebook

I'm found an interesting problem on kaggle and more or less solved it with my limited knowledge of machine learning. I was curious how other people solved and checked the solution with the highest ...
user avatar
0 votes
0 answers
14 views

Size drastically increased when images converted to hdf5 format

I converted Kaggle Dog vs Cat dataset from image to hdf5 format, in order to fasten the learning time. To my surprise, 540MB training data converted to more than 30GB of data in hdf5 file format. Why ...
user avatar
  • 659
0 votes
1 answer
18 views

Searchable list of Kaggle challenges

Is there a way to search Kaggle for a list of current and prior challenges, or an external site that does that? When I search on Kaggle it will only bring up solution notebooks and datasets, it doesn'...
user avatar
1 vote
1 answer
25 views

How to find appropliate algorithm to bulid a model for natural language based two data [closed]

What I would like to do I would like to create a model to infer nationality from name and created the below data frame combining two dataset from Kaggle. Titanic: Machine Learning from Disaster (input/...
user avatar
  • 129
1 vote
1 answer
69 views

Dropping missing rows in two dataframes

I have two files : Test_data - contains the features of a dataset to find predictions for Submission_data - contains two columns : The index column for test data and another column for its ...
user avatar
0 votes
0 answers
22 views

Is there a certain threshold over which to accept or reject predictors based on correlation values with the target variable?

I have been working on the Titanic dataset. After some feature manipulation, I printed out the correlation values between my target variable Survived and all the ...
user avatar
2 votes
1 answer
343 views

How do I read the cord_19_embeddings_2020-07-16.csv from the COVID-19 Open Research Dataset Challenge (CORD-19) on Kaggle?

What I'm trying to do I wanted to use the CORD19 word embeddings csv to map them to certain findings from the rest of the dataset, but as we can see there are no stings in the first column. The way I ...
user avatar
6 votes
3 answers
4k views

Kaggle notebook Vs Google Colab

What are the major differences between Kaggle notebook and Google Colab notebook? To work on a dataset my first step is to start a Kaggle notebook but then I cant help thinking what could be the ...
user avatar
  • 61
1 vote
2 answers
208 views

How to find correlation between categorical data and continuous data

I'm working on imputing null values in the Titanic dataset. The 'Embarked' column has some. I do NOT want to just set them all to the most common value, ...
user avatar
0 votes
1 answer
40 views

Dropping attributes leads to better classifier accuracy? (Titanic Set)

I am currently tackling the Titanic Dataset on Kaggle. The goal is to find a classifier that can predict whether a passenger will survive or die. The dataset has features that I believe are strongly ...
user avatar
  • 1
2 votes
2 answers
193 views

How to divide a dataset for training and testing when the features and targets are in two different files?

I am trying to divide a dataset into training dataset and testing dataset for multi-label classification. The datset I am working on is this one. It is divided into a file which contains the features ...
user avatar
0 votes
0 answers
88 views

Using word embeddings for kaggle?

Not sure, if this is the right forum so redirect me if it wrong. I have started on an NLP problem in kaggle. There i have word embeddings from google news, wiki, glove in a zipped folder. I want to ...
user avatar
  • 1
1 vote
2 answers
37 views

Is this over-fitting or something else?

I recently put together an entry for the House Prices Kaggle competition for beginners. I decided to try my hand at understanding and using XGBoost. I split Kaggle's 'training' data into 'training' ...
user avatar
1 vote
0 answers
64 views

How to train a simple Machine learning model in batches?

I have a dataset for multiclass classification of text data. The number of samples in training data are 1,20,000. If I extract features using TF-IDF vectoriser of the sklearn library it gives about 80,...
user avatar
  • 135
1 vote
0 answers
27 views

What to do when feature engineering and parameter tuning don't add to the base model performance

I've been working on using LogisticRegression from scikit to try the Titanic Kaggle comp. I've found something interesting, and that is that no amount of feature engineering and paramater tuning is ...
user avatar
1 vote
0 answers
63 views

Multiple choice gap-fill question (with distractors) dataset for evaluating NLP algorithms

I am looking for a standard gap-filling multiple-choice exercise (with distractors) dataset that can be used to evaluate the NLP gap-filling ML algorithms. I expect the dataset to contain questions ...
user avatar
2 votes
1 answer
178 views

Sklearn Pipeline for mixed features: numerical and (skewed) categorical

I am working on a dataset from Kaggle (housing price prediction). I have done some pre-processing on the data (missing values, category aggregation, selecting ordinal vs one-hot). I am trying to ...
user avatar
2 votes
1 answer
615 views

Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
user avatar
4 votes
1 answer
351 views

AUC ROC metric on a Kaggle competition

I am trying to learn data modeling by working on a dataset from Kaggle competition. As the competition was closed 2 years back, I am asking my question here. The competition uses AUC-ROC as the ...
user avatar
0 votes
1 answer
54 views

Python and Titanic competition how to get the median of specific range of values where class is 3

I am trying to solve Kaggle's titanic competition. In the test set, there is only 1 row having Fare value as null. It's so easy to replace it with median or average of all Fare values. But I am ...
user avatar
  • 163
-1 votes
1 answer
126 views

Meaning of 'hue" in seaborn [closed]

I know what "hue" does a little I'm studying kaggle and the image is about bike sharing demand analysis What I want to know is shouldn't the second images's sum of each point's y axis be the first ...
user avatar
2 votes
1 answer
111 views

Importing .ipnyb file from Kaggle into local Jupyter

Total beginner question here, please let me know if it would be more appropriate somewhere else. I just created my first iPython notebook in Kaggle and I downloaded the ipnyb file. Now I have ...
user avatar
  • 123
3 votes
1 answer
223 views

Kaggle Titanic submission score is higher than local accuracy score

This is the starter challenge, Titanic. The original question I posted on Kaggle is here. However, nobody really gives any insightful advice so I am turning to the powerful Stackoverflow community. ...
user avatar
  • 31
0 votes
1 answer
20 views

How to define the adequate cash prize sizing for hosting a Kaggle or similar compeition?

If you take Kaggle as a well known example of data science competition size, how do you know what is an adequate budget for the cash prize size? At least to determine an order of magnitude, given I am ...
user avatar