All Questions
8,462
questions with no answers
3
votes
0
answers
223
views
Training deep CNN with noisy dataset
I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering.
The train set captures ...
- 161
3
votes
0
answers
729
views
Xgboost rank:ndcg learning per group or for all dataset
I'm trying to implement xgboost with an objective of rank:ndcg
I want the target to be between 0-3.
In my data for most of the groups, there is only 1 event per ...
- 161
3
votes
0
answers
26
views
looking for databases of audio with labelled 'true' and 'deceptive' sections
For lie-detection research I'm looking for databases of speech audio with labelled 'true' and 'deceptive' sections - does anyone have any leads?
- 225
3
votes
0
answers
57
views
How to use Zero-Shot Learning on text?
Zero-shot learning used to predict the unseen classes using the attributes for each image.
For example, we feed the model pictures of horses and pandas. The model will learn how horses look like and ...
- 131
3
votes
0
answers
223
views
static graphs v.s. dynamic graphs
In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
- 131
3
votes
0
answers
731
views
How to train tensorflow's transformer model on my own data?
https://github.com/tensorflow/models/blob/master/official/transformer has an implementation of transformer model.
I want to train the model on my own data(consisting of two files, src.txt, and tgt.txt)...
3
votes
0
answers
355
views
Multivariate time Series classification - One class
I need your help with time series classification. I have measurements of different medical parameters for patients captured at every one hour. The output label is whether the patient has Acute Kidney ...
- 2,507
3
votes
0
answers
61
views
How to help neuronal network with an other model
I am working on an image classification problem, the input data normally is images to classify, but I thought latitude and longitude would play something on these satellite images. I sorted by ...
- 453
3
votes
0
answers
749
views
Target transformation for tree models
Can anybody explain why/if target variable transformations could help when dealing with tree based models?
I've seen this excellent reply which explains quite well why it shouldn't affect if ...
- 241
3
votes
0
answers
346
views
Layman's explanation of when to use which smoother algorithm/technique: FFT, loess, Savitzky-Golay, etc
As an analytics practitioner, I frequently come across noisy data (e.g. IoT data). When building a model or machine learning algorithm, it can be advantageous to smooth this data. Over the years, I ...
- 130
3
votes
0
answers
187
views
Best linear algebra library for C++?
I have been trying to find the substitute of numpy and perform some linear algebra using C++. Here's a list of the libraries I have encountered:
Eigen
Armadillo
Dlib
GNU Scientific library
Please ...
- 2,355
3
votes
0
answers
2k
views
Using Keras masking layer with 2D convolutions (Conv2D)
I'm trying to design a neural network including time dependent input with different lengths and I'm currently using a Masking layer. This network worked well with TensorFlow version 1.9.0 but after ...
- 31
3
votes
0
answers
52
views
Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?
Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
- 131
3
votes
0
answers
385
views
PCA and FastICA in scikit-learn giving near identical results
So after importing my data, transforming it, and splitting into training and test sets I tried running this script for PCA:
...
- 31
3
votes
0
answers
135
views
What is an intuitive explanation for the Importance Weighted Autoencoder?
I have been reading a paper by Burda et al. on Importance Weighted Autoencoders(IWAE) but I can't quite grasp what they mean by sampling the terms h1...hk. Do they mean you have separate models from ...
- 31
3
votes
0
answers
349
views
How many pairs of image needed for training Siamese Network? And how to augment them?
Hi Deep Learning researchers and engineers,
Does anyone have experience in Siamese Network regarding the training data size? How many pairs of image do I need to train a Siamese Network?
And what ...
- 31
3
votes
0
answers
143
views
Feature Importance Python
My dataset has around 1000 features and 30k rows.
All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
- 243
3
votes
0
answers
418
views
Genarate one hour time interval array using pandas in python (import from csv) to predict next value
I am trying to generate one hour one hour time interval to predict next value according to my data set imported from csv file. Here according to the time it will give outputs include in x column. This ...
- 97
3
votes
0
answers
2k
views
Is regularization only for regression?
I am making a classification model. I understand that the regularization minimizes loss functions. Can i use regularization techniques to minimize loss function in classification if doing so is ...
3
votes
0
answers
295
views
Do I need to fit on train data for truncated SVD and then transform the test data on fitted train data?
Regarding truncated SVD(single value decomposition) do I need to fit on train data and transform the test data on fitted train data? or can I fit on test and transform on test fit?
- 345
3
votes
0
answers
521
views
Convolutional neural network with cross validation in Keras
I want to use K-fold cross-validation on my dataset of images. I am reading the data (images) from a directory.
How do I use cross validation with convolutional neural network in Keras?
- 758
3
votes
0
answers
114
views
Encoder-Decoder Sequence-to-Sequence Model for Translations in Both Directions
Is it possible to use a pre-trained sequence to sequence encoder-decoder model which translates an input text in source language to an output in target language to do an inverse translation? That is, ...
- 193
3
votes
0
answers
120
views
multiple intents for modifying an intent of a sentence?
Say I have a sentence like 'I refuse to fly' or 'I'd like to fly'. I also have a sentence like 'I don't want to sit'. When training custom intents in one of the available NLU engines (rasa/wit/luis), ...
3
votes
0
answers
400
views
For stateful LSTM, does sequence length matter?
With stateful LSTM the entire state is retained between both the sequences in the batch that is submitted, and even between separate batches until ...
- 740
3
votes
0
answers
468
views
Neural Network Prediction regression task, output is a multiple factor of input with same peaks
When I missed some details please point this out. I made a simple sequential LSTM model for regression. The model loss is 3.2145e-06. The data is scaled between 0 and 1. I tried different variations ...
- 51
3
votes
0
answers
415
views
Spatial Transformer Networks and Data Augmentation
We are all familiar with the famous Deep Mind paper STN.
Upon implementation, such as here, did anyone still use input data augementation such as affine transformations?
There are used to make CNN ...
- 263
3
votes
0
answers
516
views
Validation score (f1) remains the same when swapping labels
I have an imbalanced dataset (True labels are ~10x than False labels) and thus use the f_beta score as a metric for model performance, as such:
...
- 133
3
votes
0
answers
986
views
Defining State Representation in Deep Q-Learning
So I am having difficulty difficulty figuring out exactly how I want to represent my environment state in my Deep Q-learning problem.
Premise: There is a 2D grid space of which an agent needs to ...
3
votes
0
answers
205
views
Identifying computer scanned digits
I have digit images as below which I would like to identify:
Some are of slightly worse quality :
The images are not of a fixed resolution but are mostly in the range (80*20 to 130 *40).
Due to ...
- 66
3
votes
0
answers
727
views
Keras single sample prediction returns different values
I'm using a pre-trained ResNet50 model in keras and am trying to see predictions for single samples
The issue is that model.predict() is returning different values,...
- 31
3
votes
0
answers
110
views
Stacking when the the target variable is categorical?
I'm trying to use stacking when predicting for the infamous Iris dataset. Also, I'd like to build to stacked classifier by myself which means I don't want to use mlxtend because it's too "easy" and ...
- 265
3
votes
0
answers
544
views
What are towers in inception architecture and tensorflow?
My understanding of towers in inception architecture and in tensorflow terminology is that they are part of a neural network model for which separate computation can happen on forward phase and ...
3
votes
0
answers
310
views
Importing TensorFlow trained model in OpenCV
I want to import a TensorFlow pretrained model in OpenCV dnn C++ module.
How can this be done? Are there any examples available?
- 31
3
votes
0
answers
39
views
Ethics of using Existing Algorithms in Competitions
Lets say a paper is published which describes a data science algorithm, and the paper is made available on arxiv (no patent or anything else mentioned in paper). The paper is by a university ...
- 571
3
votes
0
answers
1k
views
TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)
So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below
...
- 149
3
votes
0
answers
48
views
Formula to calculate size of Capsule output similar to the formula for CNN?
Is there any formula to find the output dimensions of a capsule network similar to that of a Convolutional Neural Network?
For Example: In CNN, we know that
...
- 151
3
votes
0
answers
93
views
Deriving diagonal approximation of Hessian in a neural network
Consider the equations relating to the diagonal approximation for the hessian matrix for a neural network in "Pattern Recognition and Machine Learning - Christopher Bishop" (on pg. 250 eq. 5.80)
$\...
- 31
3
votes
0
answers
177
views
Next events prediction based on previous events
I have data set of sequences of user executed commands sorted in the order of its occurrence. The data looks like this.
...
- 31
3
votes
0
answers
164
views
Create top 10 index fund based on >100 stocks
Data
I have a zip with ca M=160 csv files, with one csv file per stock. The ticker symbol is in the file name. Each csv file contain the same columns: date, txVolume(USD), txCount, marketcap(USD), ...
- 287
3
votes
0
answers
373
views
Anomaly detection in cooling process data without exact labels
I have a data set where I look at the cooling of a process. The starting temperature may vary between 580 and 180 degrees. I know that at some point the cooling system failed (see examples in the plot)...
- 201
3
votes
0
answers
74
views
Can you have too uniform test data in a feedforward neural network?
I have been playing around trying to implement my own feedforward neural network. To try it out I decided on an easy example. 3 inputs, 3 output. When you send in ...
3
votes
0
answers
499
views
Deep CNN with variable number of classes and "vanishing" data
I am using a deep CNN to predict the class an image belongs to (N classes). However, the number of classes is not stationary. I.e. over the time the network will be used, some new classes may emerge ...
- 131
3
votes
0
answers
110
views
What are best practices for collaborative feature engineering?
I work in a large company on several data science projects. For each of the projects me and my colleagues construct features that have some predictive value for the specific target in that project.
...
- 961
3
votes
0
answers
496
views
Keras - Masking CNNs
I have a 3D tensor on which I apply 2D convolutions. Sometimes, this 3D is padded both in width and height to have a fixed size. How could I apply masking (like with RNNs) so that the gradients ...
- 73
3
votes
0
answers
74
views
Echo-Effect-Metric Network
I have been working on data science project where I am trying to build a metric for how inbred a source is in a network. We hope to apply this to intelligence reporting, in which documents tend to ...
- 81
3
votes
0
answers
5k
views
Custom layer in keras with multiple input and multiple output
I have implemented a custom layer in keras which takes in multiple input and also results to multiple output shape. My code goes as below:
...
- 41
3
votes
0
answers
147
views
Graphical results of Q-Learning: is improvement possible by parameter tweaking?
From left to right:
Maximum Q value for action selection (averaged)
Train error (averaged)
Reward from environment (averaged)
I run double Q-learning.
A behavioral policy is ε-greedy, ε constant ...
- 233
3
votes
0
answers
179
views
How does Api.ai Google dialogueflow classifies "intent" and extracts data from slots
I am trying to build a very naive version of Api.ai, now Google DailogueFlow.
I wanted to know two things.
How DF classifies sentences with entities in it that can be user created and/or things like ...
- 31
3
votes
0
answers
765
views
Keras custom loss - operation on additional data
I am trying to create a custom loss function for a Keras regression task.
I am predicting the points scored per minute in a game, and training on "matches" of variable lengths, in minutes. In an ...
3
votes
0
answers
640
views
Autoencoder behavior with All White/Black MNIST
I am using a stock auto-encoder anomaly detector from Deeplearning4j.
I was getting unexpected results from my own variant of the auto-encoder, which looks for anomalies in my own (non-image) data, ...
- 43