Questions tagged [python]
Use for data science questions related to the programming language Python. Not intended for general coding questions (which should be asked on Stack Overflow).
6,584
questions
0
votes
1
answer
6k
views
GridSearchCV() to fine tune outputs ValueError and FitFailedWarning
I would like to fine tune some parameters for my linear SVM.
This is the code:
...
CommunityBot
- 1
1
vote
1
answer
59
views
Why is this binary state format so much more efficient than a non-binary one?
I am teaching myself Deep Q Learning (and machine learning in general) using python and tensorflow to play Snake. My initial learning came mostly from this article. I have rewritten it in my own ...
CommunityBot
- 1
0
votes
1
answer
2k
views
Jupyter Notebook not importing pandas module
I am running Jupyter on a server on a virtual environment. I then tunnel my connection so I can access Jupyter on my browser.
When I SSH into the server, I can use the Panda module in both Ipython ...
CommunityBot
- 1
0
votes
1
answer
110
views
Text Classification misclassifying?
I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0).
My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
CommunityBot
- 1
0
votes
1
answer
19
views
Python Tensorflow - Predict human vs horses images always same value
I'm trying to follow a tutorial about Tensorflow in Python and computer vision.
In this exercise I'm using a pre-trained model (InceptionV3) and some image augmentation about a datasets of humans vs ...
0
votes
2
answers
131
views
Test data missing values
I've been given a test and train data set, and I have null values in my test data set. What do I do with it?
CommunityBot
- 1
0
votes
4
answers
95
views
How to order a python dataframe by adding the row values?
I have the following dataframe:
M1 M2 M4 M5
N1 45 46 54 57
N2 32 36 29 56
N3 56 44 40 55
N4 57 43 42 54
How is it possible to order ...
- 101
0
votes
2
answers
414
views
Ways to increase recall in SVM
I am training an SVM on UCI's Bank Marketing Data Set, the bank additional-full.csv. As the data is skewed I am also interested in recall. I am getting accuracy of about 87.95% but my recall is around ...
CommunityBot
- 1
0
votes
0
answers
21
views
Common cross-validation code: why does it work?
The following Python code is common practice when creating a folds column for multi-label stratified k-fold cross-validation:
...
- 101
1
vote
1
answer
847
views
Clustering on binary data
I am working on clustering on binary data which has 25 features,
sample
Feature 1
Feature 2
Feature 3
......
Feature 25
1
1
0
0
011101
1
2
0
1
0
010011
0
3
1
0
1
101001
1
and I have used the ...
CommunityBot
- 1
0
votes
1
answer
1k
views
How to automatically save images from colab to gdrive in seperate folders whose name are as same as the labels of the respective images?
I have a total of 30k images. They are distributed among 32 classes. I want to save these images from google colab to my drive with folder names as their label name. Can it be possible by python code? ...
1
vote
1
answer
72
views
Recommendations based on other products seen
I am trying to develop a basic book recommender system to get in touch with the field and start learning methods and how to prepare the data.
The Dataframe I am using is pretty plain, it has the ...
CommunityBot
- 1
1
vote
2
answers
75
views
How to get different results running sklearn's MeanShift in a single program? (Python3)
I ran into a quirk with sklearn's MeanShift that I don't know how to get around.
MeanShift doesn't predictably give the same results on every run, so I wanted to run it multiple times within one ...
CommunityBot
- 1
0
votes
1
answer
343
views
How to add Error Bar to Matplotlib line plot?
I have the following dataset which I use to plot a line plot. The plot is obtained as the mean of values obtained from the data. I want to add error bars to this ...
CommunityBot
- 1
1
vote
1
answer
20
views
Difficulty in Coding research papers
I've done a couple of beginner-level competitions on Kaggle, and I thought I should implement research papers in computer vision (YOLOv1) but I'm finding it difficult to code it. How should I proceed?
...
- 11
0
votes
0
answers
16
views
What is the steps to generate data from a plot and use them to have a predication using python?
I have a plot that represent a BH curve for magnatic material. The material have a behavior for each H value for two different temperature 25 C and 100 C. Figure 1. I need to extract the data for each ...
- 1
0
votes
2
answers
134
views
Can I do the below problem with ML?
My dataset has 3 fields ColumnX, ColumnA, Column B as below.
Column X, Column_A , Column_B
AA, 1 , KEYWORD1 KEYWORD2 KEYWORD3
AA, 2 , KEYWORD4 KEYWORD5 KEYWORD6
XX, 3 , ...
CommunityBot
- 1
0
votes
0
answers
19
views
Regression with time series data
I want to predict temperature when time (datetime type, hourly data for five months) and humidity is given. Before starting in python, I created a regression model in excel. But instead of predicting ...
- 1
1
vote
1
answer
456
views
Generating synthetic data based off existing real data (in Python)
I am looking for an approach to generate synthetic data for anomaly detection. We have real data, but want to inject anomalies to battle-test the model (the real data is too limited for likely future ...
CommunityBot
- 1
1
vote
1
answer
102
views
Predicting the next occurrence based on binary
I have no experience in statistics or machine learning. I have a True/False binary array describing occupation of open public spaces
...
CommunityBot
- 1
0
votes
0
answers
30
views
How can I improve the inefficiency so the function actually completes?
How can I improve the efficiency of the following line:
X_df=pd.concat([X_df,pd.concat([pd.concat([dyn_df,stat],axis=1),demo],axis=1)],axis=0)
Which can be found ...
- 101
0
votes
1
answer
217
views
Tensorflow 2.11 error: AttributeError: module 'tensorflow._api.v2.compat.v2.__internal__' has no attribute 'register_load_context_function'
I had to update Tensorflow to the currently latest version 2.11. when importing i get "AttributeError: module 'tensorflow._api.v2.compat.v2.internal' has no attribute '...
CommunityBot
- 1
0
votes
0
answers
8
views
Facing Problems in Dilation and Erosion of Segmented Images of Knee X-Rays
I am working on a project to grade the severity of Knee Osteoarthritis using X-Ray Images. Before feeding the images into the machine learning model and enhancing the features of Knee osteoarthritis i....
0
votes
1
answer
373
views
How do I get my Neural network to ignore certain values?
I was wondering if there was a way that I can get my CNN encoder-decoder neural network to completely ignore certain values in my data (2d images).
There are some pixel values of 0 that never change ...
CommunityBot
- 1
3
votes
1
answer
184
views
Pandas: Calculating the best seller to incorporate number of houses sold
I have been analysing seller data and trying to get insights. I have written a groupby statement to get the average price of selling for every seller
...
CommunityBot
- 1
1
vote
2
answers
79
views
Probabilities in KERAS
I'm struggle with a problem OF class probabilities (binary, 0 and 1). Dont know why but after 100 epoches the probabilities became 0 or 1 (like the class to predict).
Maybe something is not working ...
CommunityBot
- 1
0
votes
0
answers
9
views
GridSearchCV using best paramters
i have got a question, i want to use GridSearch on a MLP and my question is after finding out best parameters do i have to train my model with the data again with the best parameters ?
0
votes
1
answer
50
views
Can I cluster an aggregated data-set (grouped by) and apply dimensionality reduction?
I have data of sales, however it is in the millions, about 500M rows. I aggregate this data by factors such as location, shoptype, country_of_shop, cardtype, and then the aggregated statistic is:
...
CommunityBot
- 1
1
vote
1
answer
239
views
Predicting churn - deal with missing dates in time series and improve modelling result
This is the follow up question for General approach on time series for customer retention/churn in retail.
I have a time series of data in the following form:
...
CommunityBot
- 1
0
votes
0
answers
11
views
How to split a LGBM.DataSet into trainning/validating/testing sets?
It looks like that LightGBM.train() can accept two arguments i.e. train_set and valid_sets, then it can do validating operations ...
- 101
3
votes
3
answers
2k
views
How to prepare data for LSTM time series prediction
I have a binary classification task for time series data.
Every 14 rows in my CSV is relevant to one time slot. How should I prepare this data to be used in LSTM? In other word how to feed the model ...
CommunityBot
- 1
1
vote
2
answers
58
views
How to vectorize and speed-up double for-loop for pandas dataframe when doing text similarity scoring
I have the following dataframe:
...
CommunityBot
- 1
0
votes
2
answers
160
views
How to use function from other Colab file?
I’m coming from a math background and learning some data science. I am very new to some details of this stuff.
Working with colab, I’m trying to organize my sample simulating functions in such a way ...
CommunityBot
- 1
1
vote
1
answer
83
views
How to Proceed with Tokenized text content which is given in number?
I have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed?
As I am encountering such data set the first time, ...
CommunityBot
- 1
0
votes
0
answers
15
views
Curve Fitting for multiple scatter plot from different csv dataset
I have successfully plotted multiple scatter plot from different dataset into a plot with the figure below.
The next agenda is curve fitting for this scatter plots and then proceeding to analyze using ...
- 1
0
votes
1
answer
241
views
Sparse matrix after vectorization giving size = 1
I am working on a NLP problem https://www.kaggle.com/c/nlp-getting-started. I want to perform vectorization after train_test_split but when I do that, the resulting ...
1
vote
2
answers
178
views
Problem importing CNTK in Azure jupyter notebook
Vineeth Sai indicated in this that with the following code:
pip install cntk
the problem is solved. However, I am getting the error shown in attached image:
CommunityBot
- 1
0
votes
1
answer
70
views
How to Manipulate data for multiple visits per person?
I have a query to solve. I have data regarding customers and number of visits done to them. These are in two tables. So I want to join two table and create different features so that I can find better/...
CommunityBot
- 1
0
votes
1
answer
93
views
Using text classification for system calls
I'm working on a project in which I should classify System calls sequences, my dataset is represented as sequences of integers (from 1 to 340). To do the classification I have inspired from Text ...
CommunityBot
- 1
0
votes
1
answer
641
views
How do feature selection on a sparse matrix?
Say I want to do features selection on a sparse matrix, i.e., 10,000 rows x 1500 features, but the matrix is mostly sparse. Let's say the features are all numeric and the target is binary and discrete....
CommunityBot
- 1
0
votes
1
answer
55
views
How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only
I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
CommunityBot
- 1
0
votes
2
answers
80
views
Predicting a relative distribution between the available instances
I am working on a model that is supposed to predict how a given order volume distributes over the available articles in a retail scenario. For simplicity's sake, let's say I'm a retailer that buys ...
CommunityBot
- 1
0
votes
0
answers
4
views
Is there a way for CTC to output different types of blanks?
I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
1
vote
1
answer
192
views
Python equivalent of Wolfram Language Query for JSON?
The Wolfram Language has a Query function that can traverse data structures and apply functions at different levels of the structure. I am working with multi-level ...
CommunityBot
- 1
2
votes
1
answer
39
views
Reference implementation of q-learning in Python
I'm a machine learning newbie, trying to learn Q-learning. I read a few texts and I get the general gist, but what I'd really love to see is a simple example of a Q-learning algorithm in Python that I ...
- 1
0
votes
3
answers
234
views
Perform fuzzy lookup to partially match the address column
I have two separate datasets (eg. dataset1.xlsx and dataset2.xlsx). Dataset1 has 2 columns, serial number and service address. Similarly dataset2 has 2 columns service address and customer number. The ...
CommunityBot
- 1
3
votes
1
answer
1k
views
Keras bug NasNetlarge no top
I am trying to use NasNetlarge in Keras without the top but I cant get rid of the top:
...
CommunityBot
- 1
6
votes
1
answer
8k
views
Keras - Implementation of custom loss function with multiple outputs
I am trying to replicate (a way smaller version) the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following:
$$l = (z -...
CommunityBot
- 1
2
votes
1
answer
387
views
Custom connections between layers Keras
I would like to manually define connections in neural network between layers using TensorFlow and keras with Python. By default connections are beween all pairs of neurons. I need to make connections ...
CommunityBot
- 1
0
votes
1
answer
537
views
Set column names to column value in pandas forloop, add prefix to column names
I have a dataframe with repeat column names, and I am trying to add a prefix label to column names with an appropriate identifier that comes from a column within the dataframe. I have this data frame:
...
CommunityBot
- 1