# All Questions

13,799 questions
5 views

### Converting string_id to number_id

I have column with movie ids like this: tt0984332 tt0984332 tt0847742 ttnanana1 I need to convert in to numbers that can be inserted into neural network as features, like this: 0 0 1 2 How can I do ...
3 views

### Forecasting a time horizon with machine learning in a recursive manner

I have a time series with hourly values that I predict for one day. My built up machine learning model has a single value as output. So that I have to run the machine learning model 24 times for a ...
7 views

### Using a binary classification, what is the best way to find the accuracy cutoff-threshold to get the final prediction output

Using a binary classification, what is the best way to find the accuracy cutoff-threshold to get the final prediction output?
11 views

### Numpy: how to add zeros while reshaping the array

I have a scenario where I have three numpy arrays - indices = np.array([1,2,3], [4,2,3]) distances = np.array([0.2, 0.3, 0.4], [0.3, 0.4, 0.5]) weights = np.array([...
11 views

I have two dates . 2012-11-20 and today's date .. i want to calculate number of business days between these 2 dates yearwise. 2012:29 till the 31st December 2012 2013:252 2014:252 2015:252 2016:252 ...
9 views

### extracting fields from json using pandas

I found this useful code for extraction of particular field from json but I want to access "weatherDesc" column and values inside it so how to change logic for that and it is also taking only 1 ...
5 views

I have been trying to read this data into python. I tried various techniques but I am not able to organise it in a format that can be used for training a model. Has anybody got any idea how to read ...
8 views

### Historical weather data with machine learning?

My company gave me a task to build some weather forecasting. I have now historical weather data for 10 years (temperature, precipitation in mm, humidity and etc. more than 30 features total). We need ...
6 views

### Number of parameters keras dense layer with a 2D input

I am using 2D data in a classification problem using keras. So I am defining a keras model as following: ...
16 views

### Best Data Science & Machine Learning Conferences 2019?

What are the best data science conferences you have attended? I am looking for a conference that will help me further develop my Data Science and Machine Learning competencies. Topics that are ...
3 views

### Finding Impact of IV on NPS of responses in A survey

I have a dataset like this. Where the first column is the age of usage of product of a user (in days) and second column is the response of the user to the typical NPS survey question: Using these ...
4 views

### Poor performance for unbalanced dataset

Consider a dataset A which has examples for training in a binary classification problem. I have used SVM and applied the weighted method (in MATLAB) since the ...
11 views

### Handling NA Values in the Chicago Crime Rate data set

I am doing a little project on the Chicago Crime Rate data set and I noticed that there are over 600,000 NA values, primarily in the location fields. I feel that ...
6 views

### Class imbalance in one hot encoding for CNN

Hey I am a newbie in Machine Learning. I was building a 2D Convolutional Neural Network for MFCC features for audio classification. The issue i am facing is that there are 2 classes and huge ...
11 views

### What affects the magnitude of lasso penalty of a feature?

Is there a way to intuitively tell if the lasso penalty for a particular feature will be small or large? Consider the following scenario: Imagine we use Lasso regression on a dataset of 100 features ...
6 views

### Is it possible to use word2Vec to derive hyperonymy (hyponymy or ISA relation)?

It's easy to have hyperonymy in WordNet, e.g. to know that "tea" is a case of "beverage". Is it possible to use word2Vec in this way?
6 views

### Are the Doc2vec paper and framework explanation the same?

I am looking into doc2vec since it shows promising results in several articles. When I read the paper by Le & Mikolov, I was under the assumption that the paragraph vector was also used to predict ...
4 views

### Classification accuracy versus mean average precision

In computer vision, I know that the purpose of classification accuracy (CA, calculated in the sense of categorizing images/objects to different classes) is ...
6 views

### YOLO algorithm - understanding training data

I am taking "Convolutional Neural Networks" on Coursera and it is taught by Andrew Ng. I am in week 3 and confused about YOLO algorithm. I checked the course forums on coursera but I am still not ...
9 views

### Machine Learning VGG CNN M 1024 Network

can Someone explain me what is difference between VGG-16 and VGG-CNN-M-1024. I cant find any info. I am going to implement this model in pytorch (It's strange that they dont have any pretrained model ...
37 views

### Tell me some things a data scientist/statistician needs to do that can't be done in R/Stata/SAS/SPSS

Why do I need to learn to program in python or Java and learn data structures? I can't think of a single thing statisticians/data scientists need to do that isn't built in as a package in R/stata.
9 views

### R Keras classification same output

I have an issue with my Keras model which gives me everytimes the same output for prediction classes. I tried to change optimizer, number of epoch, learning rate, but no change. My model accruacy has ...
8 views

### Timedistributed Autoencoder gives strange results in Keras

When I train my autoencoder using 1 sample at a time my model trains as expected. However when I use a timedistributed layer to wrap the autoencoder to do for example 5 time steps it starts off at a ...
9 views

### Does this encoder-decoder LSTM make sense for time series sequence to sequence?

TASK given $\vec x = [x_{t=-3}, x_{t=-2}, x_{t=-1}, x_{t=0}]$ predict $\vec y = [x_{t=1}, x_{t=2}]$ Whith an LSTM encoder-decoder (seq2seq) MODEL NOTE: the ? symbol in the shape of the tensors ...
11 views

### Best framework to detect Emotion

I am slightly greater than a beginner to Data Science. Currently I am trying to build an emotion detection API which will detect emotional state from a short video. I have planned to do it with fast....
13 views

### How to make the model use the data structure as a valuable information? How to help the ML model know that some variables should be used togheter?

I trained some simple sklearn machine learning models that use X (100 values) to predict Y (1 value). (detail: It works fine, but I think we can improve using more dimensions.) Here is a sample: <...
38 views

### Can a recommendation system be built without any user ratings?

I was planning to make an artwork recommendation system as a project by using the WikiArt open source dataset available on kaggle, I'm still looking for datasets which might already have user ratings ...
11 views

### How important is it for each row of data to have the same number of features?

I'm using decision tree learning to try and classify a device based its components. Different devices have a different number of components and the location of these components within the device is ...
9 views

### Finding the best CNN model

I trained two CNN models using different datasets and did 5-fold cross-validation for each using Alexnet, now I want to test the two models together to find the accuracy of the data without training ...
11 views

### How to compare three 3d models for inclusion

I am quite new in data science. Is there any paper or algorithm that solve the following problem: Let assume: two 3D objects o1 and ...
18 views

### density of a synset

I am reading the paper Text Classification Using WordNet Hypernyms http://aclweb.org/anthology/W98-0706 . In it the author gives the definition of synset density as the number of occurrences of a ...
4 views

### Deriving Dirichlet samples from gamma distribution

From wikipedia, it is written that you can generate dirichlet samples from the gamma distribution simply by normalizing the game samples. So in MATLAB, to get dirichlet samples, i would code it in ...
17 views

### Why is there a difference of “ML” vs “MLLIB” in Apache Spark's documentation?

I am trying to figure out which pyspark library to use with Word2Vec and I'm presented with two options according to the pyspark documentation. https://spark.apache.org/docs/2.2.0/mllib-feature-...
11 views

### Cross-validation in r

I have a categorical predictor with 4 categories, 1st and last category have enough frequency but 2nd and 3rd category, each have only 2 observation, also i have binary response variable. How to split ...
13 views

### What is the difference between symmetric bipartite graphs and a complete bipartite graph?

I am studying Restricted Boltzmann Machines (RBMs), and it is described as a symmetrical bipartite graph. Link How is this different from a Complete bipartite ...
5 views

### With two deep learning models, how do I perform Bayesian Model Averaging for better prediction on a test set?

Given two deep learning models that can predict on a test set, what I want to do is use BMA (Bayesian Model Averaging) to average the models to better predict? What exactly is the procedure for this?...
32 views

### How do you pronounce ROC?

In regards to ROC curves, how do you pronounce ROC? I have always spelled out the letters like R-O-C but I sat through a sales demo today where they guy pronounced it as a word like "rock" as in "the ...
9 views

### Clustering multiple timeseries dataframes(Python)

I have about 200 timeseries DataFrames who look about like this: Plotting the data with the expected weights(The red plot is the expected weight) looks about like this: My mission is to cluster ...
14 views

### Huge difference in accuracy when using two datasets (instead of one) for text classification in Python

Recently I started reading more about NLP in order to learn more about the subject. The problem that I've encountered, now that I'm trying to make my own classification algorithm (the text sends a ...
14 views

### LSTM predictions only above or below mean value

I have an interesting problem with the predictions of my keras LSTM model. The goal of my model is to predict the changes in high and low prices for a selected FX rate pair based on the following ...
19 views

### Tensorflow Custom Accuracy

I am writing a neural network that is trainable on a one hot encoded target vector but during test time it is possible to not have one hot (i.e. [1,1,0,1,0]) so as long as my prediction overlaps with ...
15 views

### Signature verification - machine learning

I have to do a project on signature verification. My goal is having a program with two images as inupt (one is a genuine signature of a selected subject and the other one is the one that I want to ...
9 views

### Applying Label From Supervised Learning to Unlabeled Data- Text Classification

I am wondering if anyone has code to following: 1) Apply labels from a previous text classification dataset like this type of data (https://colab.research.google.com/drive/...
23 views

### Do I need to convert booleans to ints to enter them in a machine learning algorithm?

My dataset contains a lot of columns with booleans do I really need to change them so I can insert them into the algorithm? I'm gonna use KNN right now but will test other algorithms later so I'm ...
11 views

### How to select Cut off value for binary classification?

I am a newbie in machine learning, and this question have been asked in an interview . How will you select cut off value for binary classification using KS Score, ROC Curve, Precision recall curve. ...
4 views

### Unsupervised learning use cases for relational data

Most of the unsupervised learning use cases I've previously worked on dealt exclusively with unstructured (text) data - topic modeling and general clustering approaches basically for easier document ...
15 views

### Python: Plotting trajectory on top of SOM u-matrix

Given a data set of many (state describing) vectors and for each of those the timestamp is known. From my application I need to build a SOM of those vectors. Following that, I'd like to plot the ...
5 views

### Whats the Difference between probabilistic programming such as pyro and Belief networks?

I heard about ubers pyro and stumbled upon this Wikipedia Artikel As I understand a bayesian network is the same as a belief netLink to datascience StackExchange post Does someone know how these are ...
Let's take two constants, $\alpha$ and $\beta$, both are given by two functions $f_1(\vec{\theta})$ and $f_2(\vec\theta$) (the model). These functions are known: we have an analytical closed ...