Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
30
votes
1answer
55k views

What is the best Keras model for multi-class classification?

I am working on research, where need to classify one of three event WINNER=(win, draw, lose) ...
1
vote
1answer
54 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
2
votes
0answers
37 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
0
votes
1answer
1k views

Python sample function TypeError: '<' not supported between instances of 'str' and 'int'

i GOT this ERROR when doing sampling stratify of a sample dataframe code is like this : ...
1
vote
1answer
184 views

How can I adjust the legend when visualizing clusters in two dimensions?

How can I change the legend as we can see now the legend has some cluster numbers missing. How can I adjust the legend so that it can show all the cluster numbers (such as Cluster 1, Cluster 2 etc, no ...
2
votes
2answers
92 views

Prediction with tensorflow keras

I'm trying to predict age from a given picture. I built the model below but the problem is that I'm getting very large loss value with low ...
0
votes
1answer
1k views

Accumulators in Spark (PySpark) without global variables?

BACKGROUND Consider the following textbook example which uses accumulators to add vectors. ...
0
votes
1answer
23 views

Algorithm for user profiling without distinct profiles

I am trying to design an algorithm that takes in a new user with the variables department, location, job_role etc. and I want a machine-learning algorithm to decide ...
0
votes
1answer
34 views

confusing regarding to kmeans clulstering for data correlation

I am trying to think through my process before doing any real coding. However, got really confused easily. Say I have 100 instruments and I know their price movements every day for a year. So I can ...
0
votes
3answers
646 views

Keras y-labels range between 0 and 1 instead of binary?

I have X values and corresponding y-labels, until now I used to round my labels <0.5 to 0 and >0.5 to 1. Is it possible to use values between 0 and 1 for "y train"? Using Keras and Tensorflow. ...
0
votes
2answers
37 views

Weight pruning of CNN

I was confused when i was reading about weight pruning on CNN. Is it applied for all the layers including convolutional layers or only it is done for dense layers?
6
votes
3answers
420 views

What is difference between Multi-class One vs All and Multilabel Classification?

Although Multi class is different from Multi label classification, whats difference does adding One vs All make in Multi-class. Edit 1: http://scikit-learn.org/stable/modules/multiclass.html#...
0
votes
1answer
77 views

Explaination of the anomalies detected [closed]

I have a dataset with a mixture of categorical and numeric variables. I have converted categorical variables into one hot encoding. I have used Isolation forest algorithm to extract 5% of the ...
1
vote
2answers
286 views

Pre-process data images before training OneClassSVM and decrease number of features

I want to train a OneClassSVM() using sklearn, and I have a set of around 800 images in my training set. I am using opencv to read the images and resize them to constant dimensions (960x540) and then ...
1
vote
1answer
23 views

Automatic legal document reviewing advice

I am lawyer from SK. I also like programming and Maths. Which technology should i use for this task? Automatic legal document reviewing. I have a big data set of specific contracts in slovak. Now I ...
0
votes
1answer
349 views

Can I use Tensorboard to manage-fjobs and visualize learning on non-Tensorflow algoithrms? (e.g. Scikit?)

I am still searching for a great tool that manages jobs and visualizes learning from my models. Tensorboard is obviously one option given it's massive support. But is it possible to organize jobs ...
3
votes
3answers
481 views

Time series analysis on small dataset

I am new to time series and therefore trying to feel my way around. I am required to do some analysis and possibly come up with a model. However, I have at most 2 years worth of quarterly data (at ...
0
votes
1answer
31 views

Algorithmically extract seasonality in time series data

Suppose you are trying to measure if the seasonality of a particular event stream is consistent i.e. the events in a time series happen more or less in pattern like fashion. How can you ...
6
votes
5answers
628 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
2
votes
2answers
2k views

Spliting keras model into multiple GPU's

Dear fellow Data Scientists. I'm having a problem with splitting model into multiple GPU's. I have read something about "towering" in native tensorflow but my whole architecture is already written in ...
1
vote
1answer
61 views

Orange Web Services - Problems installing imblearn module for Python Script

I am trying to implement SMOTE in Orange Web Services, but having troubles with the imblearn module. Following screenshot is from Orange Python Script widget: ...
1
vote
2answers
86 views
0
votes
1answer
102 views

Keras LSTM model not performant

I have trained a LSTM model to detect fake domain names. My dataset is like this: domain,fake google, 0 bezqcuoqzcjloc,1 ... with 50% normal and 50% fake ...
1
vote
0answers
20 views

Lemmatization of pandas column using Wordnet after POS

I have a pandas column df_travail[line_text] with text. I want to lemmatize each word of this column. First I Lowercase the text : ...
2
votes
0answers
11 views

Keras model predicts differently when predicting based on video frame vs. individual snapshot

I'm training a binary image classification model in Keras, with the end goal of parsing through entire videos. I have a script that reads in a few snapshots from training videos and lets the model ...
3
votes
2answers
1k views

Analyze performance Poisson regression model on a time series(count forecasting)

I have tried to build a model to forecast the count of a particular variable.The model that was used for the purpose was poisson .Unfortunately ,i don't have enough stat knowledge to analyze the model ...
1
vote
3answers
3k views

What should I use if I have millions of possible values for a feature in a sklearn predictive model?

I am trying to create a large model. One of the features is categorical, and it has almost 100 million entries. I have looked at sklearn LabelEncoder, but I am concerned that it will still create an ...
4
votes
1answer
266 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
2
votes
1answer
62 views

How to perform (modified) t-test for multiple variables and multiple models on Python (Machine Learning)

I have created and analyzed around 16 machine learning models using WEKA. Right now, I have a CSV file which shows the models' metrics (such as percent_correct, F-measure, recall, precision, etc.). I ...
2
votes
1answer
26 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
2
votes
0answers
17 views

Correct interpretation of summary_plot shap graph

While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples - Shap value 0 for a feature ...
4
votes
2answers
159 views

Text topic classification in tensorflow

I want to create a CNN in tensorflow that does the following: Classify a recipe headline and find out the topic. For instance <...
0
votes
1answer
380 views

Keras multi-label time-series classification considering time-series as an input image vector

I am trying to build a multi-class classifier using Keras. I am not quite sure I have implemented it correctly. Data is like this label time-series variables [0:25728} ...
2
votes
1answer
30 views

Skin Detection From a Face Image

I have photos of people's faces, and I want to detect each person's skin color for later use. I don't want to use predetermined color ranges, instead I want to determine each person's color and then ...
2
votes
2answers
67 views

KMeans clustering for Image Data

I am trying to cluster the sample of Imagenet Dataset using K-Means clustering. In this approach, I have used the below 2 approaches to get the optimal number of clusters. Elbow method From the ...
0
votes
1answer
212 views

how to find holiday effect on revenue?

I have 2 datasets from 2013-2017 for each day. a) Revenue generated by Locations and date. b) Holiday name and date I would like to know how each holiday is impacting the revenue by location. I am ...
2
votes
1answer
261 views

Siamese network return same loss value while training

I have ~9k training samples, the same pair vectors are labeled as 0 and not same pair samples are labeled as 1. I trained it for 100 epochs for the first 3 epochs the loss value is fluctuating, then ...
0
votes
2answers
126 views

Unable to resolve Type error using Tokenizer.tokenize from NLTK

I want to tokenize text data and am unable to proceed due to a type error, am unable to know how to proceed to rectify the error, To give some context - all the columns - Resolution code','Resolution ...
1
vote
0answers
14 views

regression model assumption [migrated]

I try to check whether my regression model is follow regression assumption or not? for that I did below python code but response is error. can someone explain how it wrong ...
2
votes
0answers
514 views

Multidimensional scaling producing different results for different seeds

I took the data from here and wanted to play around with multidimensional scaling with this data. The data looks like this: In particular, I want to plot the cities in a 2D space, and see how much it ...
2
votes
1answer
36 views

Differences between normalization and standarization in multiple regression

Consider the following question regarding multiple regression 1) Can someone explain why we have to transform dependent variable using log-transformation (Normalization) when appear positive skewed y ...
0
votes
1answer
28 views

Add ID information from one dataframe to every row in another dataframe without a common key

I am trying to combine two dataframes into the second dataframe, but duplicating the first dataframe into every row of dataframe two. The first dataframe (df1) contains the identifing information, ...
0
votes
1answer
43 views

How do I find all the drastic changes in the stock and fetch news for those days of the stock?

I am analyzing MindTree company's stocks as an internship project using python pandas and have been given the task to find all the drastic changes by looking at the produced plot for the stock and ...
2
votes
1answer
782 views

R NbClust for Python?

Is somebody know a Python equivalent to R NbClust? I'm searching for a way to determine the optimal clusters number for many 'similar datasets' and NbClust sounds good but it's for R and i'm a Python ...
1
vote
0answers
93 views

I used Stargazer in Python. How to convert output to LaTex?

Hello I typically use R but is there a way how to use stargazer in python For example when doing an arima model ...
1
vote
0answers
55 views

Finding p values and estimates of external variables in facebook Prophet for forecasting in Python

I am using facebook Prophet for multivariate forecasting which has an objecting of forecasting prices. My target variable is affected by n number of external variables too. I am using add_regressor ...
0
votes
1answer
128 views

Why would Pytorch be non-deterministic for batch only?

I am training a LSTM network on CPU and can achieve deterministic results when not using a dataloader. But when I used Pytorchs dataloader I achieve non-deterministic training error results, despite ...
0
votes
1answer
49 views

bad input shape (5634, 2)

I tried everything and I am not sure how to resolve the following error: ValueError: bad input shape (5634, 2) This is my first machine learning example so please bear with me. This is the python ...