Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1
vote
1answer
18 views

multiclass classification

I want to build an ml model, which can when given a text input, can predict predefined tags or labels for the text. I already built one such model, but the problem with that is that it only predicts ...
1
vote
0answers
20 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
1
vote
1answer
28 views

How to get the conflicting user from the user list

I have a data file like this: User Item Ram Dog sam Dog Ram wolf max lion zed fox sam cat zed cat max dog max cat I want to check for conflict:...
0
votes
1answer
100 views

Kmeans cluster validation when I have labeled test data

I'm trying to implement the unsupervised k-means algorithm for sentiment analysis of imdb movie dataset created by stanford. The steps that I followed is : 1) Load the comments 2) Apply tokenization ...
0
votes
1answer
100 views

Pytorch CrossEntropyLoss expected long but got float

have you done some research before asking the question? Yes. I have done a lot of online searching, and others had similar problems. There solution was to use .float() when entering into the loss ...
2
votes
1answer
400 views
0
votes
1answer
226 views

Using categorial_crossentropy to train a model in keras

I'm a novice in machine learning. I was following this Keras blog to train image classifier using Keras. Though this blog only demonstrates how to train only two classes using binary_crossentropy, I ...
0
votes
1answer
196 views

how to implement Excel vlookup on Python

I have a dataset A and dataset B, both data has a column in common which is the ID, i want to check if dataset A ID values can be found in dataset B ID values. Also get the records of dataset B that ...
1
vote
1answer
18 views

Passing the output of a model as input to an other model , computer vision?

i want to try my hands at a project where i have to detect the brands + the models of sportswears worn by persons participating to a marathon , the project is quite complex , so my first approach ...
0
votes
1answer
1k views

How do I change the marker edge color of a seaborn relplot?

I am trying to output a complex facet grid plot in the format of the following image: But the problem is that I don't want the edge color of the markers to be white, I want it to be the face color. ...
3
votes
1answer
145 views

Fast Python implementation of the gradient descent

I'm looking for fast Python implémentations of gradient descent optimization algorithm. I have a convex problem , with no constraint, so for now I'm using the BFGS algorithm implemented in scikit-...
3
votes
3answers
1k views

Input contains NaN, infinity or a value too large for dtype('float32'). Without ruining Dataset

I have a datasheet that is all in Binary but sometimes there are missing cases. In one of the inputs, there is a blank, I don't want to completely remove the whole column because some of the rows have ...
3
votes
1answer
31 views

What is a good method for detecting local minims and maxims?

I'm using kernel density estimation in order to compute probability density function for item (triangles in the figure) occurrence. Using this output, i want to find all the local minims and maxims. ...
3
votes
1answer
95 views

Pandas DataFrame Rollup Error

I am trying to compute Percentage Contributions on a pandas dataframe. I have a data frame which has three columns (Buyer, ...
2
votes
1answer
42 views

Machine learning model with simultaneous function optimization

Consider the following scenario. I am a sculpturer and customers ask me for what price I am willing to provide them with some statues. Their request for sculptures can vary in difficulty, quantity, ...
1
vote
0answers
19 views

Organize the Dataset

I am working on a neural network(Variational Autoencoder) to separate music instruments I don't know how to organize my data I have a folder full of spectrograms of the songs as a mix and the ...
0
votes
2answers
22 views

Support Vector Machine analysis - Python, how do I determing whether to use linear, square, or other types of SVM models?

how do I determine whether to use linear, square, or other types of SVM models? under which criteria should an SVM model be applied in the first place?
1
vote
0answers
15 views

How to tune the hyperparameters of XGBoost and RF? [closed]

How to tune the hyperparameters of XGBoost and RF in python?
0
votes
1answer
61 views

GD for logistic regression isn't stable. Why?

Here's my (incomplete) implementation for linear regression using GD: ...
1
vote
1answer
753 views

Getting negative r2_score with new set of dimensions

I am trying to predict flight take off delay using my current dataset. At this point of time, I only have four dimensions. ...
2
votes
0answers
24 views

Parameter optimization and selection in dynamic neural networks

I have used a Bayesian optimization to tune machine learning parameters. The optimized parameters are "Hidden layer size" and "...
1
vote
2answers
38 views

Predict the average temperature for next 30 years

Objective: I want to predict the average temperature for next 30 years. Q1: What type of dataset is suitable for this (what columns should it contain) Q2: What are the independent variables for ...
4
votes
1answer
3k views

Memory error on using data generator in keras

I am using the following augmentations on dataset of size 9 GB: ...
1
vote
1answer
29 views

Implementing “full convolution” to find gradient w.r.t the convolution layer inputs

I've been trying to implement "full convolution" w.r.t to convolution layer inputs. According to this article, it looks like this: So, I wrote this function: ...
1
vote
1answer
19 views

(pre-trained) python package for semantic word similarity

I am searching for a python package that calculates the semantic similarity between words. I do not want to train a model (what most packages seem to offer) - the package should have been pre-trained ...
3
votes
0answers
30 views

Sources to learn Anomaly Detection

Project Goal- Create an Anomaly Detection tool. As the first step I worked on some python code that would parse my XML files into a Dataframe in csv file. The next step is to go into machine ...
-1
votes
1answer
39 views

NumPy — ImportError: PyInit__multiarray_umath [closed]

I'm facing the below error while importing NumPy library. I have tried all of the suggestions advised by Python in its error log, but no luck. Strangely, when I use Python's OS library and print the "...
1
vote
1answer
33 views

Detecting unique icon on video in real time

For a project I am working on I'd like to be able to detect unique icons/barcodes in video footage. Suppose you have 10 people in the frame where each person is wearing a t-shirt with a similar but ...
0
votes
0answers
26 views

What is the difference between “fit_transform” and “transform” methods when using “SimpleImputer”? [duplicate]

I have following code, I am not able to understand the difference between use of fit_transform() and transform() method in this ...
1
vote
1answer
29 views

How to use weights from our data set properly? Should we use them at all?

The census data set I'm using: https://archive.ics.uci.edu/ml/datasets/Adult So, I'm currently using this census data to make observations and predict whether someone is married or not. However, when ...
0
votes
0answers
19 views

detect table of content in PDFs

I have PDF text data that I want to use for NLP tasks. Now during preprocessing, I came to the conclusion that it is not worth analyzing the table of content. It merely generates a bunch of ...
1
vote
1answer
29 views

PCA Regression Problem

I have a regression problem whereby my data has 21 features and I wish to apply dimensionality reduction using PCA. As far as I know, all the tutorials I have seen so far use PCA for classification ...
0
votes
1answer
127 views

How to run a model in an application using gpu (without CUDA)

I am looking for a way to utilize a computer’s gpu without using cuda (or any installable software). The reason for this is I have an application where the neural network runs on a user’s computer and ...
0
votes
1answer
34 views

How can I imporve accuracy for text classification and mapping using SVM?

I am working on a problem where I need to predict the text corresponding to another text in my training data file. For example: if I have value like the software in one of my columns and another ...
2
votes
1answer
25 views

Ridge regression model creation using grid-search and cross validation

I created python code for ridge regression.For that I used cross validation and grid-search technique in together. i got output result. I want check whether my regression model building steps correct ...
0
votes
1answer
41 views

LSTM with input of actual time step

I'm working on an implementation of LSTM neural network to forecast energy consumption. I have a dataset with load, series of weather parameters and indicator of it's bank holiday or not. I first ...
3
votes
0answers
20 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
0
votes
2answers
419 views

Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?

I'm building a CNN to make a binary classification (1 or zero). For this, I'm using the cost function sigmoid_cross_entropy_with_logits. But for some reason, the ...
3
votes
1answer
60 views

Best anomaly detection algorithm based on two conditions

Choosing the right anomaly detection algorithm seems quite hard at the moment. It might be because I am bombarded with so many alternatives likes clustering, K-Means DBSCAN and so many others. On my ...
2
votes
1answer
30 views

Multiclassification Error: NotFittedError: This MultiLabelBinarizer instance is not fitted yet

After picking the model, when I try to use it, I am getting error - "NotFittedError: This MultiLabelBinarizer instance is not fitted yet. Call 'fit' with appropriate arguments before using this ...
3
votes
1answer
30 views

How to properly store a list of pandas dataframes, so that I can import them in a new python script?

I'm trying to make a GUI in PyQt5 to present the results of my research. The results are around 800 pandas dataframes, stored in lists (so that I can call them individually according to the list index)...
2
votes
2answers
28 views

Method to assess text credibility

I am searching for an automated method (ideally a python package) that produces a score to assess the credibility of a given text (e.g. from a webpage). I am not searching for: text complexity ...
0
votes
1answer
45 views

CNN 1D for time series. Value Error. I did read all the possible answers in StackOverflow regardiing this problem

I been fighting with this problem for 2 weeks now. And I extensively research for solutions here and in other sites. I have a dataset of 4 dimensions device_id, time and longitude and latitude. After ...
3
votes
4answers
77 views

Machine Learning methods for finding outliers

I have a csv files of thousands of lines, the data is put down into a Dataframe of columns. Some of the column have text information while others might have numbers. I want to detect anomalies or ...
1
vote
0answers
27 views

What is the formula to calculate the precision, recall, f-measure with macro, micro, none for multi-label classification in sklearn metrics?

I am working in the problem of multi-label classification tasks. But I would not able to understand the formula for calculating the precision, recall, and f-measure with macro, micro, and none. ...
0
votes
1answer
22 views

How to get different results running sklearn's MeanShift in a single program? (Python3)

I ran into a quirk with sklearn's MeanShift that I don't know how to get around. MeanShift doesn't predictably give the same results on every run, so I wanted to run it multiple times within one ...
0
votes
1answer
4k views

Cluster tendency using Hopkins statistic implementation in Python

The Hopkins statistic, is a statistic which gives a value which indicates the cluster tendency, in other words: how well the data can be clustered. If the value is between {0.01, ...,0.3}, the data ...
0
votes
1answer
11 views

the library 'transformers' works also with older version of Tensorflow?

i am working with Tensorflow version 1.14 and i would like to use the bert embedding. In order to do so, i was thinking to use the transformers library( https://pypi.org/project/transformers/) but i ...
10
votes
2answers
34k views

How can I fit categorical data types for random forest classification?

I need to find the accuracy of a training dataset by applying Random Forest Algorithm. But my the type of my data set are both categorical and numeric. When I tried to fit those data, I get an error. ...
0
votes
2answers
63 views

Unsupervised Clustering of 6-d vectors

I have 17k vectors, each having 6 points, I want to cluster the vectors based on nature of points, eg. linearly increasing into one cluster, convex in another cluster, concave in another, decreasing ...