Questions tagged [python]
Use for data science questions related to the programming language Python. Not intended for general coding questions (-> stackoverflow).
3,764
questions
3
votes
0answers
2k views
+50
Doc2vec to calculate cosine similarity - absolutely inaccurate
I'm trying to modify the Doc2vec tutorial to calculate cosine similarity and take Pandas dataframes instead of .txt documents. I ...
0
votes
0answers
7 views
How to fit integer output in neural network
I have a data set that contains 104 input features and 96 output values to be predicted, the input features are floating values normalized to [-1,1], and each out put value would be one integer ...
-1
votes
0answers
6 views
Multiple integer output for neural network [closed]
I have a data set that contains 104 input features and 96 output values to be predicted, the input features are floating values normalized to [-1,1], and each out put value would be one integer ...
2
votes
3answers
542 views
Detect the time at which deviation occurs in time series data
I working on multivariate time series data. I have sensor data generated by a machine every time it is operated. Data set consists of machine_ID(machines of same model), hours_ operated, measurements ...
0
votes
1answer
145 views
Integration of Sentiment analysis in CRM
What is the process for integrating sentiment analysis in a CRM? What I am searching for is a system which analyzes the customer comments or reviews using the CRM and finds out the customer sentiment ...
0
votes
1answer
184 views
Getting parameters of the best model with crossvalidation in with SparkMLLib
I am having trouble accessing the parameters of estimators of model in SparkMLlib. More precisely my problem is:
I have a logistic regression model for which I want to find the best regularization ...
1
vote
1answer
51 views
ValueError from statsmodels ExponentialSmoothing
I've been having a frustrating issue with the ExponentialSmoothing module from statsmodels.
My data is a pandas series with 74 weekly data points that looks like this:
...
0
votes
0answers
10 views
Searching for ressource recommendations (books,papers) to validate the results of a Topic Model (LDA)?
Hi I am building a Topic Model Process with Python.
To do this I am using the LDA.
However I am having trouble to determine the ideal amount of topics.
Currently I am using the CoherenceModel ...
3
votes
1answer
529 views
How do I create a Keras custom loss function for a one-hot-encoded binary classifier?
I have a CNN binary classifier with one-hot-encoded labels that I've written using Keras and it's just not training to the metric I want to encourage. My data is very imbalanced (91% class 0, 9% class ...
1
vote
1answer
15 views
python tsne.transform does not exist?
I am trying to transform two datasets: x_train and x_test using tsne. I assume the way to do this is to fit tsne to x_train, and then transform x_test and x_train. But, I am not able to transform any ...
0
votes
0answers
11 views
Python Algorithm for adding and expanding new rows in excel file [closed]
Is there some kind of Python Algorithm for adding and expanding new rows in excel file, where we can +1 new city for this example and it will create multiple row with this city name AND include the ...
2
votes
1answer
43 views
How can I know the name of the features selected by a Deep Belief Network?
I want to use DBN to reduce the 41 features of nslkdd dataset after transforming nominal data to numeric the number of features increases from 41 to 121 . I used 3 RBMs (121-50-10) now I want to know ...
1
vote
1answer
37 views
Anomaly detection - relation between thresholds and anomalies
I'm developing an anomaly detection program in Python.
Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day.
Then, using thresholds, find ...
2
votes
2answers
5k views
Creating Balanced Dataset Using Scikits
I have a classic User-Item dataset where each row (i.e., (user, item)) indicates the action of a user clicking/selecting an item....
5
votes
1answer
6k views
Which convolution should I use? Conv2d or Conv1d
I've dataset which contains dlib landmark points of the faces. I'm using keras to train a model.
The dataset shape is (length_of_dataset,68,2). I know that I've two options.
The first is using ...
1
vote
1answer
42 views
Why is the difference of datetime = zero in the following dataframe?
This issue that I am facing is very simple yet weird and has troubled me to no end.
I have a dataframe as follows :
...
2
votes
1answer
18 views
What is Continuous Ranked Probability Score (CRPS)?
I came across some evolution metric at Kaggle:
Continuous Ranked Probability Score (CRPS):
Mathematically,
$C = \frac{1}{199N} \sum_{m=1}^{N} \sum_{n=-99}^{99} (P(y \le n) -H(n - Y_m))^2,$
where P ...
0
votes
1answer
33 views
3
votes
1answer
100 views
What is cohen kappa metric, implementation in Python?
Can somebody explain indetail explanation on Quadratic Kappa Metric/cohen kappa metric with implementation in Python
0
votes
2answers
25 views
customer segmentation with unbalanced data
I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
1
vote
0answers
12 views
How to create list of multi value in a list format of each field in a single column?
I am very new to programming and python; coming from functional background. I am currently using a tutorial: https://www.analyticsvidhya.com/blog/2019/04/predicting-movie-genres-nlp-multi-label-...
1
vote
0answers
16 views
1
vote
2answers
446 views
error when input function called with shape (2,2)
I am new to Tensorflow and machine learning.
I am trying to use high level API from Tensorflow.
Please tell me what i am doing wrong.
...
0
votes
3answers
462 views
Python Code to find the number of hapax legomena in a Text or Words_List
In corpus linguistics, a hapax legomenon is a word that occurs only once within a context, either in the written record of an entire language, in the works of an author, or in a single text. The term ...
3
votes
6answers
63k views
I got the following error : 'DataFrame' object has no attribute 'data'
I am trying to get the 'data' and the 'target' of the iris setosa database, but I can't. For example, when I load the iris setosa directly from sklearn datasets I get a good result:
Program:
...
0
votes
1answer
22 views
Algorithmically extract seasonality in time series data
Suppose you are trying to measure if the seasonality of a particular event stream is consistent i.e. the events in a time series happen more or less in pattern like fashion. How can you ...
0
votes
4answers
166 views
Damerau-Levenshtein Edit Distance in Python
I found some python codes on Damerau Levensthein edit distance through google, but when i look at their comments, many said that the algorithms were incorrect. I'm confused.
Can someone share a ...
2
votes
1answer
18 views
Generating random numbers from best probability distributions?
I have done some statistical distributions on real-world data. The distribution fitting gives me the below results (lognormal fits the best on data based on chi-squared test):
...
0
votes
0answers
11 views
Pruning of SVM classifier [closed]
I am trying to perform the pruning technique for SVM classifier.
I couldn't find any python code examples for SVM pruning to get inspired.
15
votes
3answers
3k views
Difference between OrdinalEncoder and LabelEncoder
I was going through the official documentation of scikit-learn learn after going through a book on ML and came across the following thing:
In the Documentation it is given about ...
14
votes
3answers
12k views
Looking for good package for anomaly detection in time series
Is there a comprehensive open source package (preferably in python or R) that can be used for anomaly detection in time series?
There is a one class SVM package in scikit-learn but it is not for time ...
0
votes
0answers
8 views
Implement neural network to train my data and get best 6 output from the testing dataset [closed]
I have a trainer dataset which has the average run, strike rate , runs scored and other characteristics of some player. I want to train my data in such a way so that from my tester data I get 6 best ...
0
votes
1answer
22 views
How would I load a database with multiple inconsistent headers?
I have a database with multiple headers like so:
...
0
votes
0answers
9 views
Error :UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 124: invalid start byte [closed]
Error:
^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 124: invalid start byte
I am getting this above error, any help is highly ...
35
votes
3answers
35k views
Multi GPU in keras
How we can program in the keras library (or tensorflow) to partition training on multiple GPUs? Let's say that you are in an Amazon ec2 instance that has 8 GPU's and you would like to use all of them ...
1
vote
1answer
25 views
When to use deep learning for java as opposed to python
I have been asked to explore options to build deep learning based applications using java, so i happend to browse a website called dl4j (https://deeplearning4j.org) which has got implemantations of ...
4
votes
1answer
458 views
HMMLearn Predict Next Observed Event
From my understanding you can use the transition matrix to predict the probability of going from the last predicted hidden state(state t), to the t+1 hidden state. My confusion is how in code format ...
1
vote
3answers
56 views
Detect if word is «common English» word or slang word
I have a huge list of short phrases, for example:
...
-1
votes
0answers
30 views
NameError: name 'randn' is not defined [closed]
I tried to run this code and it gave me an error saying name 'randn' is not defined
Kindly have a look on the code :
...
0
votes
1answer
18 views
Cannot fig out error in my gradient function implementation in python
Im trying to implement following gradient descent function in Python for logistic regression:
$∇θ(−logL)=−X^T (y−e^{Xθ})$
This is my python implementation:
...
1
vote
1answer
33 views
Which classification model to use on large, high-dimensional dataset?
I face a classification task: with several features a target features is to be predicted. I'm working with python.
My dataset includes 60 features from which I picked 16 which I think could be ...
1
vote
4answers
43k views
Using pandas, check a column for matching text and update new column if TRUE
My objective: Using pandas, check a column for matching text [not exact] and update new column if TRUE.
From a csv file, a data frame was created and values of a particular column - COLUMN_to_Check, ...
1
vote
0answers
19 views
LSTM for multiple time series regression with extremely large ranges
I have the following question for those which encountered the same dilemma as me:
My target is to develop a LSTM RNN for multi-step prediction for multiple time series representing daily sales of ...
0
votes
1answer
148 views
Random seed in Machine learning model comparison
I wanna ask very important question about the random population generation gin splitting the dataset in machine learning classification models;
For example to more explain, i used seed =1 a,d i got ...
2
votes
0answers
18 views
Implementing a Kernel Adaptive Filtering model explained in a paper
In this paper, Stock price prediction using kernel adaptive filtering
within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the ...
1
vote
1answer
1k views
IndexError: index 804 is out of bounds for axis 0 with size 800
i installed a self driving car project from superdatascience site , when i open the map using terminal after a while the map window close up or it closes directly after i maximize the map window and ...
1
vote
1answer
319 views
tensorflow categorical data with vocabulary list - Expected binary or Unicode string, got [0,1,2,…]
I'm brand new to machine learning (having just completed the google machine learning crash course) and thought it would be good to try my hand at a Kaggle competition as a good starter to some real ...
0
votes
1answer
23 views
How can I override the values of 1 column in a big dataframe using a small second dataframe? [closed]
I have a Pandas dataframe (donations_df) that contains thousands of donations. Each donation row has many columns, but the two key ones for my question are:
A <...
0
votes
3answers
3k views
Unable to generate error bars with seaborn
Can you please help me add error bars to my graph?
This is the csv:
...
1
vote
1answer
19 views
Functions in scikit that detect outliers automatically?
I know a way to visualize outliers is to make a box plot, but wanted to know if scikit had any quick ways to detect outliers for each variable?
