Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1
vote
1answer
30 views

Comparing excel data sets in Pandas

Pretty new to Python, but as an SEO I'm looking at the benefits of using notebooks in my workflow. I've got two excel files which I've cleaned and imported into a new notebook using pandas. I'm ...
1
vote
0answers
8 views

Scipy.optimize getting back x0 for optimization of input to recurrent neural network

I need help with this optimization problem which is either not getting solved at all or is taking a copious amount of time. I am trying to find optimized input to an RNN (GRU) model of a process ...
0
votes
1answer
24 views

How to load pre-processed .txt data and use it for training convolutional networks?

I have a lot of '.txt' (pre-processed) files related to C3D and I need to use this to train and test while using CNN model. How does one go about this? Most of data in each .txt files are formatted ...
2
votes
1answer
61 views

Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score: $F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$ The problem of the F1-score is that it is not ...
1
vote
1answer
55 views

NLP Feature creation from phrase matching

I'm building a model to classify email content, to decide whether the email should lead to a JIRA ticket being "Raised" or "Not Raised". The problem I am having is the data is highly imbalanced with ...
0
votes
1answer
80 views

CVNN and Tensorflow

How the algorithm use Complex Value Neural Network ( all complex value, like: input, weight,bias and output) in python
0
votes
0answers
14 views

How to create POS Embeddings for input?

I am trying to append POS embeddings to an input word embedding (which is from Glove). In order to do that, I need represent POS tags as embeddings. I saw the link in this channel already; however I ...
2
votes
0answers
11 views

Using an unmerged branch from StatsModels for Anaconda

I am interested in using a statistical test from StatsModels, one that is not available in the default Anaconda StatsModels folder on my drive. See this blog, which as some point mentions, almost ...
2
votes
1answer
48 views

How to validate regex based Resume parser efficiently

I am using rule based logic to extract features from resume. Basically I am trying to find if the candidate switched the company in less than 1 year. So I have the code in place to find it using ...
1
vote
0answers
22 views

Filter and plot values using matplotlib

In order to understand data a bit more, I want to plot one column from my csv file using matplotlib as a dotted graph This is how my my csv file looks like. I want to filter the csv with ...
0
votes
1answer
39 views

Getting Rid of duplicate data

I extract data from a website, on which I have no control. The data is multiplied. The original report should look like this. ...
0
votes
1answer
467 views

Group small values in a pie chart

When plotting a Pie chart from a dictionary using Python's matplotlib, I get some classes that are too mashed together because of their small size. Is there a way to group the smallest values together ...
1
vote
3answers
616 views

convert dataframe without header to dictionary with a row of number

I am trying to put a dataframe into dictionary ,with the first column as the key ,the numbers in a row would be the value . how should I do it ? I tried to_dict but did not work , my df has no header ...
4
votes
1answer
399 views

How to implement patternet in python as it is in matlab?

NOTE: This question was first posted in cross-validated website but I was instructed to move it off that website as it was not a good fit. I am new in implementation of machine learning, neural ...
0
votes
0answers
14 views
0
votes
2answers
23 views

Binary classification for variable outputs

I am creating a binary classifier in Keras and here's the code ...
-1
votes
0answers
16 views

Keras - Preparing the Embedding layer with Word2Vec [closed]

I have a word2vec model. But I can't implement word2vec with keras embedding layer. In this tutorial, Preparing the Embedding layer section, they used GloVe. I want to use word2vec instead. How can I ...
1
vote
1answer
32 views

very important features, but rarely input

The features are of capital gain and capital loss, but very small amounts of people have one or the other. As far as I can see it may not signify any great difference, though I'm not sure how I would ...
0
votes
2answers
58 views

Dummy encoding the categorical variables using the changed version of OneHotEncoder

This is my code, I was trying to dummy encode the first column of X using OneHotEncoder but it was showing error and the documentation page of OneHotEncoder says that it has been changed and I wasn't ...
0
votes
0answers
25 views

How to fill missing value based on other column in Pandas dataframe? [closed]

For the below sample dataframe, I would like to map the respective state with zone and fill na's with the respective zone names. How can we code for the below instead manually filling missing values? ...
-1
votes
0answers
20 views

How to update package in my Anaconda Environment? [closed]

I am trying to update numpy from 1.15 to 1.18. Using the command conda install -c anaconda numpy did not help as the only thing that was updated are the 'ca-...
2
votes
1answer
24 views

Unable to learn weights of a Word2Vec model

I was going to implement a word embedding model - namely Word2Vec - by following this TensorFlow tutorial and adapting the code a little bit. Unfortunately, though, my model won't learn anything. I've ...
-2
votes
1answer
32 views

Which is the correct method for outlier analysis on a dataset for modelling?

I'm trying to build a regression model but my data-set have many outliers points which I need to analyze and then remove them. There are two ways to do it, 1) First do all the analysis on every ...
1
vote
1answer
42 views

Representing geospatial information

I am trying to train a model to predict the location of a storm at a given time. The dataset includes the longitude and latitude of the storm at the given "timestamps" but I am not sure if that is the ...
0
votes
3answers
118 views

pandas apply function with multiple condition?

if i want to apply lambda with multiple condition how should I do it? df.train.age.apply(lambda x:0 (if x>=0 and x<500)) or is there much better methods?
2
votes
1answer
250 views

Randomstate and kmeans issues

I try to cluster a dataframe of 227 rows in 5 clusters using kmeans algorithm. Each time I run my code I got different labels and different clusters which make my analysis afterwards a bit tricky. ...
2
votes
1answer
24 views

Trying to return more than just the top result from sklearn NearestNeighbors

I'm trying to compare a list of names (duplicated into a clean file and a messy file). I then compare the files against each other. My problem is that it returns only the top 1 result for each, ...
1
vote
1answer
20 views

Forecasting - 3 independent variables

I have 3 independent input variables: time, temperature and pressure, and one output - usefulness. Example: Number of the sample; Time; Temperature; Pressure; Usefulness 1;1;30;50;9% 2;3;50;50;80% ...
1
vote
1answer
244 views

String handling by OneHotEncoder

I am reading everywhere on new questions and blogs that since version 0.20, OneHotEncoder is able to handle string features. Moreover, the documentation is what looks more ambiguous. Here are the ...
1
vote
1answer
375 views

K-means clustering for one class classification

I want to know if I can use the k-means clustering algorithm for a one class classification (as in the case of one class SVM), which means I have data for 2 classes, and I labelled only the one class ...
5
votes
2answers
953 views

Question about sklearn's StratifiedShuffleSplit

I'm reading through the book Hands-On Machine Learning with Scikit-Learn and Tensorflow by Aurélien Géron. In a regression project on California Housing Prices, he goes over the concept of stratified ...
2
votes
1answer
41 views

Why do probabilities sum to one and how can I set optimal threshold level?

I am working on a text classification use case. The training data has two classes, so the XBBoostClassifier and onevsrest model is classifying the test data into either of the two classes. But my ...
1
vote
0answers
35 views

Can anyone direct me on how to got to [closed]

I am a newbie to python. I have uploaded about 2000 images into my google drive, assigned a variable to all content in the folder and tried to split the folder's content into training and testing, ...
-1
votes
3answers
82 views

Can tanh be used as an output for a binary classifier?

I am creating a binary classifier in Keras and here;s the code ...
1
vote
0answers
26 views

Need help regarding my thesis project related to Data Analysis [closed]

I am working on a project with a company that manufactures Injection molding machines. I am to perform data analysis on some selected parameters as a part of my thesis to tell the company if from ...
0
votes
4answers
4k views

Unable to generate error bars with seaborn

Can you please help me add error bars to my graph? This is the csv: ...
0
votes
1answer
573 views

Remove columns with a certain number of consecutive zeros

I'm using pandas and I am dealing with time series of sales. What I would like to do is to remove the columns where a certain number of consecutive zeros appear, since forecasting for sparse series ...
1
vote
1answer
149 views

Isolation forest - grouped by

I'm trying to use isolation forest algorithm for outliers detection. Data has 2 columns: id and REV. Below code gives me ...
2
votes
1answer
28 views

Why does MAE differ after prediction (Neural Network)?

I'm having trouble understanding what's happening in the following code. I already have defined x_train, y_train, x_val, y_val and x_test which define my training, validation and test sets. I'm using ...
0
votes
1answer
137 views

error while running lasso.py

The following is the error code generated while running lasso.py. Can anybody help in fixing the same. Here is the code: ...
1
vote
1answer
18 views

multiclass classification

I want to build an ml model, which can when given a text input, can predict predefined tags or labels for the text. I already built one such model, but the problem with that is that it only predicts ...
1
vote
0answers
20 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
1
vote
1answer
28 views

How to get the conflicting user from the user list

I have a data file like this: User Item Ram Dog sam Dog Ram wolf max lion zed fox sam cat zed cat max dog max cat I want to check for conflict:...
0
votes
1answer
99 views

Kmeans cluster validation when I have labeled test data

I'm trying to implement the unsupervised k-means algorithm for sentiment analysis of imdb movie dataset created by stanford. The steps that I followed is : 1) Load the comments 2) Apply tokenization ...
0
votes
1answer
99 views

Pytorch CrossEntropyLoss expected long but got float

have you done some research before asking the question? Yes. I have done a lot of online searching, and others had similar problems. There solution was to use .float() when entering into the loss ...
2
votes
1answer
399 views
0
votes
1answer
226 views

Using categorial_crossentropy to train a model in keras

I'm a novice in machine learning. I was following this Keras blog to train image classifier using Keras. Though this blog only demonstrates how to train only two classes using binary_crossentropy, I ...
0
votes
1answer
192 views

how to implement Excel vlookup on Python

I have a dataset A and dataset B, both data has a column in common which is the ID, i want to check if dataset A ID values can be found in dataset B ID values. Also get the records of dataset B that ...
1
vote
1answer
18 views

Passing the output of a model as input to an other model , computer vision?

i want to try my hands at a project where i have to detect the brands + the models of sportswears worn by persons participating to a marathon , the project is quite complex , so my first approach ...
3
votes
1answer
41 views

Data wrangling for a big set of docx files advice!

I'm looking for some advice on a data wrangling problem I'm trying to solve. I've spent a week solid taking different approaches and nothing seems to be quite perfect. Just FYI, this is my first big (...