Questions tagged [python]

Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).

Filter by
Sorted by
Tagged with
1
vote
2answers
42 views

Is One Hot Encoding Vectorized?

Sorry for such a weird question - I do not even know if this makes sense, however I thought of this during my intro to Python course at uni and have wondered about it since. So I have some ...
2
votes
2answers
372 views

How to extract and classify data from a column in excel?

I have a column in an Excel sheet that contains a lot of data separated by || delimiters. The data can be classified to some classes like Entity, IFSC codes, ...
1
vote
1answer
758 views

keep_dims is deprecated, use keepdims instead

I downloaded: !git clone https://www.github.com/matterport/Mask_RCNN.git os.chdir('Mask_RCNN') And I've got an error: which version I should have of Keras? <...
2
votes
0answers
23 views

Omnibus and R square improvements for OLS model

Checking on this community if any one can help on below problem posted by me on stats.stackexchange. https://stats.stackexchange.com/q/441653/266047 Detailed question is as below: ...
3
votes
2answers
32 views

TensorFlow Sigmoid activation function as output layer - value interpretation

My TensorFlow model has the following structure. It aims to solve a binary classification problem where the labels are either 0 or 1. The output layer uses a sigmoid activation function with 1 output. ...
0
votes
1answer
68 views

Mixed effect random forest model for Python Windows

Does anybody know if there is a Mixed effect random forest model for Python Windows? The merf package https://anaconda.org/search?q=merf+ seems to only be available on a linux environment? thanks!
2
votes
1answer
34 views

Machine learning dataframe dimension concept vs NumPy dimension

From Machine Learning for Absolute Beginners: A Plain English Introduction: Contained in each column is a feature. A feature is also known as variable, a dimension or an attribute - but they all ...
2
votes
1answer
23 views

Maximize one data point

I am completely new to data science and looking to narrow down the search and reduce the learning curve required to solve problems like the one given below I have a data set with 7 columns , Column ...
2
votes
2answers
23 views

Classification when variables are in ranges

I want to classify my data and some of my variables are ranges. I classify location so for example, school, the hours that people are at school are from 7:00 to 14:00, some of my variables are ...
0
votes
1answer
194 views

train_test_split ValueError: Input contains NaN

I tried to do a stratified sampling by way of train_test_split in order to save myself some trouble later. So I wrote the following lines: ...
0
votes
0answers
27 views

Different extraction pipeline for train and test

I'm trying to create a production-ready ML model. The problem is as follows: Training data: Database A + Plus python aggregations. Testing data: Database B + Plus python aggregations. Both ...
2
votes
1answer
22 views

reduction of model accuracy while using PCA for a regression problem

I am trying to build a prection problem to predict the fare of flights. My data set has several catergorical variables like class,hour,day of week, day of month, month of year etc. I am using multiple ...
2
votes
1answer
33 views

How can I perform categorical encoding when the dataset is too large for memory?

I generally do preprocessing before fitting estimators using Scikit-Learn. My latest project is using significantly more data than I have used in the past, and whilst I know I can use online learning ...
0
votes
1answer
27 views

How neural style transfer work in pytorch?

I am using this pytorch script to learn and understand neural style transfer. I understood most part of the code but having some hard time understanding some parts of the code. In ...
3
votes
1answer
2k views

Export Python Code from Orange

Since Orange is just running python code under the hood... is it possible to set up and do my analysis in Orange but then export or output the underlying code for documentation or replication?
1
vote
1answer
70 views

ValueError from statsmodels ExponentialSmoothing

I've been having a frustrating issue with the ExponentialSmoothing module from statsmodels. My data is a pandas series with 74 weekly data points that looks like this: ...
30
votes
1answer
55k views

What is the best Keras model for multi-class classification?

I am working on research, where need to classify one of three event WINNER=(win, draw, lose) ...
1
vote
1answer
53 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
2
votes
0answers
37 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
0
votes
1answer
1k views

Python sample function TypeError: '<' not supported between instances of 'str' and 'int'

i GOT this ERROR when doing sampling stratify of a sample dataframe code is like this : ...
1
vote
1answer
175 views

How can I adjust the legend when visualizing clusters in two dimensions?

How can I change the legend as we can see now the legend has some cluster numbers missing. How can I adjust the legend so that it can show all the cluster numbers (such as Cluster 1, Cluster 2 etc, no ...
2
votes
2answers
85 views

Prediction with tensorflow keras

I'm trying to predict age from a given picture. I built the model below but the problem is that I'm getting very large loss value with low ...
0
votes
1answer
1k views

Accumulators in Spark (PySpark) without global variables?

BACKGROUND Consider the following textbook example which uses accumulators to add vectors. ...
0
votes
1answer
23 views

Algorithm for user profiling without distinct profiles

I am trying to design an algorithm that takes in a new user with the variables department, location, job_role etc. and I want a machine-learning algorithm to decide ...
0
votes
1answer
34 views

confusing regarding to kmeans clulstering for data correlation

I am trying to think through my process before doing any real coding. However, got really confused easily. Say I have 100 instruments and I know their price movements every day for a year. So I can ...
0
votes
3answers
636 views

Keras y-labels range between 0 and 1 instead of binary?

I have X values and corresponding y-labels, until now I used to round my labels <0.5 to 0 and >0.5 to 1. Is it possible to use values between 0 and 1 for "y train"? Using Keras and Tensorflow. ...
0
votes
2answers
35 views

Weight pruning of CNN

I was confused when i was reading about weight pruning on CNN. Is it applied for all the layers including convolutional layers or only it is done for dense layers?
6
votes
3answers
407 views

What is difference between Multi-class One vs All and Multilabel Classification?

Although Multi class is different from Multi label classification, whats difference does adding One vs All make in Multi-class. Edit 1: http://scikit-learn.org/stable/modules/multiclass.html#...
0
votes
1answer
76 views

Explaination of the anomalies detected

I have a dataset with a mixture of categorical and numeric variables. I have converted categorical variables into one hot encoding. I have used Isolation forest algorithm to extract 5% of the ...
1
vote
2answers
286 views

Pre-process data images before training OneClassSVM and decrease number of features

I want to train a OneClassSVM() using sklearn, and I have a set of around 800 images in my training set. I am using opencv to read the images and resize them to constant dimensions (960x540) and then ...
1
vote
1answer
23 views

Automatic legal document reviewing advice

I am lawyer from SK. I also like programming and Maths. Which technology should i use for this task? Automatic legal document reviewing. I have a big data set of specific contracts in slovak. Now I ...
0
votes
1answer
344 views

Can I use Tensorboard to manage-fjobs and visualize learning on non-Tensorflow algoithrms? (e.g. Scikit?)

I am still searching for a great tool that manages jobs and visualizes learning from my models. Tensorboard is obviously one option given it's massive support. But is it possible to organize jobs ...
3
votes
3answers
463 views

Time series analysis on small dataset

I am new to time series and therefore trying to feel my way around. I am required to do some analysis and possibly come up with a model. However, I have at most 2 years worth of quarterly data (at ...
0
votes
1answer
31 views

Algorithmically extract seasonality in time series data

Suppose you are trying to measure if the seasonality of a particular event stream is consistent i.e. the events in a time series happen more or less in pattern like fashion. How can you ...
6
votes
5answers
625 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
2
votes
2answers
2k views

Spliting keras model into multiple GPU's

Dear fellow Data Scientists. I'm having a problem with splitting model into multiple GPU's. I have read something about "towering" in native tensorflow but my whole architecture is already written in ...
1
vote
1answer
61 views

Orange Web Services - Problems installing imblearn module for Python Script

I am trying to implement SMOTE in Orange Web Services, but having troubles with the imblearn module. Following screenshot is from Orange Python Script widget: ...
1
vote
2answers
86 views
0
votes
1answer
101 views

Keras LSTM model not performant

I have trained a LSTM model to detect fake domain names. My dataset is like this: domain,fake google, 0 bezqcuoqzcjloc,1 ... with 50% normal and 50% fake ...
1
vote
0answers
19 views

Lemmatization of pandas column using Wordnet after POS

I have a pandas column df_travail[line_text] with text. I want to lemmatize each word of this column. First I Lowercase the text : ...
2
votes
0answers
10 views

Keras model predicts differently when predicting based on video frame vs. individual snapshot

I'm training a binary image classification model in Keras, with the end goal of parsing through entire videos. I have a script that reads in a few snapshots from training videos and lets the model ...
3
votes
2answers
1k views

Analyze performance Poisson regression model on a time series(count forecasting)

I have tried to build a model to forecast the count of a particular variable.The model that was used for the purpose was poisson .Unfortunately ,i don't have enough stat knowledge to analyze the model ...
1
vote
3answers
3k views

What should I use if I have millions of possible values for a feature in a sklearn predictive model?

I am trying to create a large model. One of the features is categorical, and it has almost 100 million entries. I have looked at sklearn LabelEncoder, but I am concerned that it will still create an ...
4
votes
1answer
263 views

Scikit-learn average_precision_score() vs. auc score of precision_recall_curve()

I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the ...
2
votes
1answer
61 views

How to perform (modified) t-test for multiple variables and multiple models on Python (Machine Learning)

I have created and analyzed around 16 machine learning models using WEKA. Right now, I have a CSV file which shows the models' metrics (such as percent_correct, F-measure, recall, precision, etc.). I ...
2
votes
1answer
25 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
2
votes
0answers
15 views

Correct interpretation of summary_plot shap graph

While through the various resources online to understand the shap plots, I ended up slightly confused. Find below my interpretation of the overall plot given in examples - Shap value 0 for a feature ...
4
votes
2answers
156 views

Text topic classification in tensorflow

I want to create a CNN in tensorflow that does the following: Classify a recipe headline and find out the topic. For instance <...