Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Time Series Analysis / Modeling with auto_arima

I recently dived into Time Series and was attempting at doing some data analysis and modeling. I'm using the following dataframe ...
0
votes
1answer
18 views

Canny edge detection not working on Gaussian blurred images

I am trying to detect edges on this lane image. First blurred the image using Gaussian filter and applied Canny edge detection but it gives only blank image without detecting edges. I have done like ...
0
votes
1answer
15 views

Question about reshaping array size for KNN Classifiers

I keep trying to run a new set of data through my KNN Classifier but would recieve the message: ValueError: query data dimension must match training data dimension ...
1
vote
0answers
22 views

Why is Tensorflow LSTM training slower on a machine with far better components?

Training an LSTM using the exact code and dataset on two different machines with different components yields different results in terms of training time. However, for my case, the results were the ...
1
vote
1answer
23 views

XGBoost - feature importance just depends on the location of the feature in the data

I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
0
votes
0answers
15 views

Transfer learning with Keras for medical image classification

Good afternoon; I'm trying to do Transfer Learning from pre-trained model on imagenet to solve a classification task with Lung CT slices. These slices are stored in ...
0
votes
3answers
39 views

How do i get dummies for this dataset

I am using an udemy course for MachineLearning and I am trying to form a dummy for my variable the column is Country I want to change to France Germany Spain France ...
0
votes
1answer
10 views

How do feature selection on a sparse matrix?

Say I want to do features selection on a sparse matrix, i.e., 10,000 rows x 1500 features, but the matrix is mostly sparse. Let's say the features are all numeric and the target is binary and discrete....
0
votes
0answers
10 views

How could I implement POS Tagger using HMM with Viterbi Algorithm in Python(IPython)?

I was trying to implement a POS tagger for resource-scarce language. For the implementation, I've my own manually tagged corpus in a quite small amount. Having this corpus, I've been dealing with the ...
1
vote
0answers
25 views

XGBoost model performance barely budging, and max_depth is basically irrelevant

I have a dataset that includes 62 features and around 1 million observations. The 62 features are mostly socioeconomic status indicators for students as they start a school year. The labels are the ...
0
votes
1answer
22 views

plot gridsearch csv results how?

how can i plot my results from gridsearch csv? ...
1
vote
1answer
17 views

Image classification using cnn [closed]

I did image classification using CNN and it successfully classified the images but How to save predicted images to separate folder for example i have two classes cat and dog after prediction how to ...
3
votes
1answer
49 views

Why would GradientBoostClassifier do better than XGBoostClassifier? [duplicate]

I am working on the Kaggle home loan model and interestingly enough, the GradientBoostClassifier has a considerably better score than XGBClassifier. At the same time it seems to not overfit as much. (...
0
votes
1answer
16 views

Predicting evolution of an unknown mathematical function with machine learning?

Is it possible to predict how a mathematical function evolves using ML? I am studying radioactive decay of polonium 210 and I have gathered data for its decay throughout a couple of weeks. I was ...
0
votes
2answers
32 views

Validation Curve Interpretation

I have been reading about the validation_curve function from scikit learn. When I run this it takes too long. Therefore, I am plotting the results from grid search ...
1
vote
0answers
19 views

How to extract embeddings of categorical variables

I am little bit confused about encoding categorical variables. There are other posts/blogposts on this issue but none is talking about the problem I am facing. I have a dataset with mixed variables (i....
1
vote
0answers
17 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found here: https://medium.com/@ageitgey/natural-...
1
vote
1answer
41 views

How Calculate Effect (percentage) label of the input variables on the output variable by BernoulliNB

a description problem below. I have 10 words like X1 , X2 , X3 , ... , X10 and three Label like short , long , hold. My problem is that how calculate Effect (percentage) label of the input variables ...
0
votes
0answers
12 views

Digitize graphs. Pull out data from points in a scatter plot

I have a project where I would like to extract data from a series of scatter plots that are image files (Jpeg or png). The plots are similar but the axes scales are not always exactly the same. I have ...
0
votes
1answer
11 views

How to enable GPU on GradientBoostingClassifier?

Is there a way to enable GPU on GradientBoostingClassifier?
2
votes
1answer
23 views

Detect passive voice in headlines

To detect passive voice in sentences, we can use the spacy module to tag each token in the sentence, then build a classifier to classify it as passive or active based on conventional grammatical rules,...
0
votes
1answer
81 views
1
vote
2answers
42 views

Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group

In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables. I have tried an XGB binary ...
0
votes
0answers
20 views

How to remove outliers from each group within a boxplot?

I am wanting to remove all the outliers for each individual ID in a boxplot. I have tried the following code: ''' ...
0
votes
0answers
14 views

Prioritized Experience Replay - which version is correct?

After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER). Example code taken from here ...
0
votes
1answer
13 views

How to get back stationary data from non stationary data after performing diff operations in var (time series)?

I have applied var model on multiple features ,but var model accepts the stationary data so i have converted non-stationary data to stationary data by doing diff() ,given the data to var model. what ...
1
vote
1answer
17 views

How Calculate Effect (percentage) label of the input variables on the output variable by DecisionTreeClassifier

a description problem below. I have 10 words like X1 , X2 , X3 , ... , X10 and three Label like short , long , hold. My problem is that how calculate Effect (percentage) label of the input variables ...
0
votes
0answers
10 views

XGBoost parameters that will overweight errors for a specific decile

My goal is to make predictions about the bottom decile of my dataset. If I can predict the bottom decile values well (ie. minimize the errors there) then I am not worried about making accurate ...
4
votes
2answers
88 views

ExtraTreesRegressor criterion

As I understand, ExtraTreesRegressor from sklearn works by doing random splits instead of minimizing a metric like gini for ...
0
votes
1answer
24 views

What is the distribution of the goal feature in this dataset

https://www.kaggle.com/kemical/kickstarter-projects I was checking the distribution of the "goal" feature in this 2016 KickStarter data set. I figure goal would be more normally distributed ...
0
votes
0answers
20 views

Debugging Reinforcement Learning Model

I'm new to RL and I'm attempting to train an RL agent to play MsPacman in PyTorch. I've adapted the code from this tutorial on the PyTorch page for my problem. The DQN has the following architecture: <...
0
votes
0answers
7 views

Siamese network do not compare pictures correctly

I have a siamese network trained for recognizing products. Some of these products are recognized properly but some of theme not. Images in each class are similar (For testing this architecture I had ...
0
votes
1answer
23 views

How to calculate change in Loss Function w.r.t Weights for a fully connected NN

I read this article: Optimal Brain Damage by Yann LeCun, John S. Denker and Sara A. Solla. where the authors discuss about estimating the saliency of each weight of a neural network, which they define ...
0
votes
0answers
13 views

What's the best approach to classify labeled stockchart images , based on their curve

I'm working on an exercise where I have selected a lot of different stock chart images, showing the week closing of around 1200 companies at two different periode in time. I have stock charts from ...
-1
votes
1answer
21 views

How to explain the results from this kmeans?

I got the following results by using k-means algorithm. There are $10$ elements in Cluster $0$ and $3$ elements in Cluster $1$. Do you think it makes sense and it might be an acceptable result? How ...
0
votes
1answer
16 views

Clustering with k-means for text classification based on similarity

I have a column that contains all texts that I would like to cluster in order to find some patterns/similarity among each other. ...
1
vote
0answers
15 views

Stasmodels unexpected behaviour

I am using statsmodels in python to perform quantile regression. I am running quantile 99 and quantile 1 to see the extremes quantile of my distribution. ...
0
votes
1answer
9 views

How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only

I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
0
votes
1answer
7 views

Can we calculate AUC for deep learning based regression task

There is a paper [ref.attaced], where they used a deep learning based regression and evaluated using mse and AUC. The targets are continuous values such as 1,2,3 up to 16 and has been normalized ...
0
votes
0answers
11 views

Why siamese network using ResNet50 architecture has worse results than network trained from beginning?

I am trying to build product recognition tool based on ResNet50 architecture as below ...
0
votes
1answer
15 views

Scrapping data from a CSV file [closed]

So, I'm a rookie at web scrapping and Python in general, and I'm having quite a hard time learning it. I'm trying to get some data from a csv file, but so far I could only split lines and columns. I ...
0
votes
0answers
10 views

Getting different graph after converting tflearn model to keras

I have the following code written using tflearn: ...
1
vote
1answer
20 views

Keeping DataFrame in Memory while working - R vs Python in VS Code

I am considering migrating from R-Studio to Python. So I am just starting Python with VS Code as my editor. My purpose is mainly to analyze data, build predictive models and make such models available ...
2
votes
1answer
28 views

Find non overlapping area between two kde plots in python

I was attempting to determine whether a feature is important or not base on its kde distribution for target variable. I am aware how to plot the kde plot and guess after looking at the plots, but is ...
1
vote
2answers
49 views

Facial recognition on offline images

I want to do facial recognition on wide varieties of images captured at various ages of my family members. Below are some of the questions I have. If a person uses glasses of different types, do I ...
0
votes
0answers
18 views

get the size of dataset after applying a filter from tf.data.Dataset

How can I get the size or the len of the dataset after applying a filter. Using tf.data.experimental.cardinality give -2, and this is not what I am looking for!! I want to know how many filtered ...
0
votes
1answer
25 views

Multivariate, Multi-step LSTM time series forecast

I'm trying to predict the Pollution using a Multivariate and Multi-step LSTM code, I've been following this tutorial. I've been following the code until the end, but couldn't understand where the ...
2
votes
1answer
160 views

How to setup and run Conda on Google Colab

I am interested in using Google Colab for data modeling. How do I install conda, create an environment and run python in a notebook? I did some searching and found some helpful hints, but had several ...
0
votes
0answers
9 views

Getting error ValueError: Found input variables with inconsistent numbers of samples: [6, 21]

I'm trying to evaluate the perfomance of my algorith by getting the Mean absolute error, mean squared error and root mean squared error. But my output is value error ValueError: Found input variables ...
1
vote
0answers
16 views

pandas memory issues

What are some good "best practices" for dealing with large datasets in pandas and numpy? Typically i read data from csv files and use numpy and pandas to prepare them for machine learning and AI ...

1 2
3
4 5
93