All Questions
36,608
questions
2
votes
2
answers
4k
views
How to properly save and load an intermediate model in Keras?
I'm working with a model that involves 3 stages of 'nesting' of models in Keras.
Conceptually the first is a transfer learning CNN model, for example MobileNetV2. (Model 1) This is then wrapped by a ...
CommunityBot
- 1
0
votes
1
answer
253
views
Varying Image sizes in Tensorflow Malaria dataset | Dealing with unclean tensorflow data
I am trying to build a CNN based image recognition system for the Tensorflow malaria dataset. I loaded the dataset (~27k RGB images) using conventional tensorflow_datasets syntax.
After some data ...
CommunityBot
- 1
0
votes
0
answers
2
views
Logistic Regression Import error
Hi Im using SKlearn for a school project. I cannot importy Logistics regression without getting a numpy float error.
Have not been able to find any solutions online.
1
vote
2
answers
500
views
Hellinger Distance in Gensim
I have set of documents as follows where each document has set of words that represents the content of it.
...
CommunityBot
- 1
0
votes
0
answers
4
views
ML approach for quantifying building quality perception
I'm working on a project to model public perceptions of buildings in a tourism context, focusing on attributes like beauty and mystery. The data I have is a labeled dataset of building photos, each ...
- 101
2
votes
2
answers
303
views
GAN: Discriminator converges, generator learns almost nothing
In my GAN, the discriminator loss goes down steadily, while the generator loss oscillates / does not converge.
I suspect this is due to the vanishing gradient problem. Theory: as the discriminator ...
CommunityBot
- 1
0
votes
2
answers
21
views
Why getting error for this very simple class? [closed]
I am learning classes in python. I do not know why I get error for this very simple class. Where I am doing wrong?
...
- 1
0
votes
1
answer
64
views
Multiple targets in a classification problem
I have a vector of length $n \gt 4$ which has exactly 4 targets, so for example [0, 0, 0, 1, 0, 1, 0, 1, 1]. I would like to know how I can modify the softmax ...
CommunityBot
- 1
2
votes
2
answers
108
views
Idenitity between TD(0) algorithm and Policy Evaluation in Dynamic Programming when alpha is equal to 1
TD(0) algorithm is defined as the iterative update of the following:
$$ V(s) \leftarrow V(s) + \alpha({r + \gamma V(s')} - V(s) ) $$
Now, if we assume alpha to be equal to 1, we get the traditional ...
CommunityBot
- 1
0
votes
0
answers
5
views
Lack of Variability in Predictions from Multivariate LSTM Model
I've been working on a multivariate LSTM model for time series forecasting, but I'm encountering an issue where the predicted output doesn't exhibit enough variability. The predictions tend to be too ...
- 101
1
vote
1
answer
163
views
Semi-supervised anomaly detection
I am currently exploring anomaly detection methods for my work and, basically I have gone through Local Oulier Factor and Isolation Forests, both unsupervised methods.
Now, the thing is, there might ...
CommunityBot
- 1
1
vote
1
answer
142
views
How to solve a non-linear system with the GAUSS-NEWTON algorithm in Python? (Jacobian matrix J, etc.)
I would like to solve a non-linear system (which contains the goals of a football team in previous matches) using the Gauss-Netwon algorithm, in order to find the parameter (of frequency) to use as ...
- 111
3
votes
1
answer
216
views
Pandas: Calculating the best seller to incorporate number of houses sold
I have been analysing seller data and trying to get insights. I have written a groupby statement to get the average price of selling for every seller
...
CommunityBot
- 1
0
votes
3
answers
52
views
Why Relu is correct for CNN?
Relu only passes positive values, so when we calculate the gradients for this layer, we will only get positive gradients. The gradient for the filter weights of this layer is the convolution of the ...
- 1
3
votes
3
answers
2k
views
How to prepare data for LSTM time series prediction
I have a binary classification task for time series data.
Every 14 rows in my CSV is relevant to one time slot. How should I prepare this data to be used in LSTM? In other word how to feed the model ...
CommunityBot
- 1
0
votes
1
answer
257
views
I need sources of interrogative, exclamatory, and imperative sentences
I am working on accumulating a large database of labeled sentences for several projects/experiments. At present I am only using Wikipedia and Project Gutenberg as sources of data. Between these two ...
CommunityBot
- 1
0
votes
1
answer
82
views
How RNN or LSTM delays the input
RNN or LSTM are known to hold the previous timestamp data as "memory" so that short or long range dependencies can be remembered.
But in the following simple keras model, where is that delay ...
CommunityBot
- 1
0
votes
1
answer
504
views
What are the allowed ops for Tensorflow Lite for Microcontrollers?
Currently, I am studying Tensorflow Lite For Microcontrollers (TLFM). I have gone over all the tutorials. Now I will write my own code where I will try to detect some anomalies based on the ...
CommunityBot
- 1
0
votes
1
answer
276
views
How to pass a sequence of 4 images into LSTM and CNN-LSTM
I got an assignment and stuck with it while going down the rabbit hole of learning PyTorch, LSTM, and CNN.
Provided the well-known MNIST library I take combinations of 4 numbers and per combination, ...
CommunityBot
- 1
1
vote
1
answer
844
views
Loss drops to NaN after a short time for a time series classification
here is my model code for a binary classification of a time series:
...
CommunityBot
- 1
0
votes
4
answers
94
views
Bias and variance in the model o in the predictions?
This topic confuses me. In the literature or articles, when talking about bias and variance in automatic learning, specifically in cross-validation, do they refer to the high bias (underfitting) and ...
CommunityBot
- 1
0
votes
1
answer
123
views
How can I take the cross-correlation between two discrete signal with slightly different discrete time points?
I have two discrete signals x1 and x2 with corresponding time points t1 and ...
CommunityBot
- 1
1
vote
1
answer
2k
views
Bayesian optimization with Keras tuner for time series
Goal: trying to use walk-forward validation strategy with keras tuner for time series when training a neural network (mainly LSTM and/or CNN).
Did anyone find a direct way of doing this?
One ...
CommunityBot
- 1
0
votes
1
answer
151
views
Text Classification misclassifying?
I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0).
My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
CommunityBot
- 1
0
votes
1
answer
122
views
What does the below phrase in the lstm blog mean? - Data Science
I am a newbie to data science. I was reading this blog
When I was half way through, I came into this sentence
Further, each series of data has been partitioned into overlapping
windows of 2.56 ...
CommunityBot
- 1
1
vote
1
answer
126
views
Using glue to include information regarding selected observation
I would like my ggplot to display the state I had selected for better clarity but it seems like glue is only seeking for the first observation rather than my desired output.
...
CommunityBot
- 1
1
vote
2
answers
770
views
What is preferred upsampling or zero padding?
When training a CNN one option is either to zero pad an image to make it bigger or upsample it. When should I choose each one?
What criteria is leveraged for choosing a method?
CommunityBot
- 1
0
votes
1
answer
849
views
Merging two datasets with different features for machine learning prediction
I'm trying to create a model which predicts Real estate prices with xgboost in machine learning, my question is : Can i combine two datasets to do it ?
First dataset : 13 features
Second dataset : ...
CommunityBot
- 1
0
votes
0
answers
7
views
Calculating Loss at Different Aggregation Level Than Prediction
I need to evaluate a model based on a loss metric which is at a higher aggregation level than what the model predicts.
There are individual reservations (ID) which are valid over a time period (HOUR). ...
0
votes
0
answers
14
views
Setting derivative to 0 when minimizing expected loss over possible data samples
Suppose, given $x\in \mathbb{R}^d$, you want $\theta$, which is the solution of
$$
\text{argmin}_{\theta} \mathbb{E}_a[L(\theta;x,a)]
$$
where $a \sim \mathcal{N}(x,\Sigma)$ and $L$ measures a (convex)...
1
vote
1
answer
594
views
How can we use the cosine similarity formula on document feature vector without a direction?
In mathematics, a vector has both magnitude and direction.
In data science, for identifying document similarity we convert the document into a feature vector. Then apply cosine angle formula between ...
CommunityBot
- 1
1
vote
1
answer
202
views
Python equivalent of Wolfram Language Query for JSON?
The Wolfram Language has a Query function that can traverse data structures and apply functions at different levels of the structure. I am working with multi-level ...
CommunityBot
- 1
0
votes
0
answers
9
views
Can I modify my training datasets (X_train and Y_train) while fitting the model?
I am new to ML and I am trying to train a forecasting model. The target variable (Y_train) has multiple columns, all of Boolean type. The features table (X_train), according to my approach, in the ...
1
vote
1
answer
76
views
Probability for Nth Place in Race from Bradley-Terry Model Inputs and Outputs
I have created a motorcycle race prediction model that is given pairs of racers and outputs the probability of each rider beating the other in each pairwise comparison. That info is then processed ...
CommunityBot
- 1
2
votes
1
answer
306
views
Building Timeseries models for stock trading having multiple stocks
I have gone through some of the tutorials on the timeseries and all of them have taken one stock for the timeseries and tried to forecast it. My dataset contains many stocks for the time period(each ...
CommunityBot
- 1
0
votes
3
answers
93
views
Is the decision tree the right choice to classify for this dataset?
I have a bunch of tuples like this; [SourceIP, DestinationIP, Port, TimeStamp]
If a destination IP recorded 21, 22, 23 and 80 port (set of 5 tuples) then I will decide something, if it has set of 4 ...
CommunityBot
- 1
2
votes
0
answers
32
views
Why is cross-entropy increasing with accuracy?
I'm making an implementation of the softmax regression and I'm struggling to understand the nature behind the problem of increasing value of Cross-Entropy: $H(y_i, p_i)=-\sum_{i=1}^C y_i log(p_i)$, ...
- 121
1
vote
1
answer
425
views
Need help with prolog program
I am new to prolog and need help on how to write a prolog program. Here is what i am trying to do. I have downloaded the dataset from this link - https://archive.ics.uci.edu/ml/datasets/Blood+...
CommunityBot
- 1
0
votes
1
answer
67
views
How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only
I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
CommunityBot
- 1
0
votes
0
answers
6
views
How to deal with categorical disalignment in test and train in binary classification problems
I have a train and test datasets (600k observations) that have different categories for the same categorical variable.
For example train has the categorical variable Letters having unique categories ...
- 1
0
votes
1
answer
614
views
Visualizing the equation for separating hyperplane
I was wondering if I can visualize with the example the fact that for all points $x$ on the separating hyperplane, the following equation holds true:
$$w^T.x+w_0=0\quad\quad\quad \text{... equation (1)...
CommunityBot
- 1
2
votes
1
answer
164
views
Making an ensemble model for high F1 score
I presently have 2 algorithms that have a numerical output. Using a threshold of 0.9, I get the classification output. Let's say they are:
P (high precision, low recall)
R (high recall, low precision)...
CommunityBot
- 1
2
votes
1
answer
125
views
How to analyze neural network quality in case of overfitting?
I have a Keras neural network that has images both as input and reference data.
My network demonstrates overfitting (for example, train accuracy is about 80% but test accuracy is only up to 70%) due ...
CommunityBot
- 1
0
votes
1
answer
200
views
Gradient descent vs stochastic gradient descent vs mini-batch gradient descent with respect to working step/example
I am trying to understand the working of gradient descent, stochastic gradient descent and mini-batch gradient descent.
In case of gradient descent, gradient is computed on the entire dataset at each ...
CommunityBot
- 1
1
vote
3
answers
134
views
Logistics Demand Forecasting with 20k Different Time Series
I'm trying to tackle a very challenging problem and I would appreciate your help.
My organization has a lot of different items which can be demanded by our clients. Those items can also be returned ...
CommunityBot
- 1
0
votes
2
answers
201
views
Open retail/ecommerce dataset with customers, transactions, products, loyalty, discounts, promotion information?
Does anyone know of an open retail/e-commerce dataset with information about customers, transactions, products, loyalty, discounts, and promotion information?
I am planning to create 360-degree ...
1
vote
1
answer
127
views
Data quality improvement as a part of preprocessing: Imputation
I have a python pandas dataframe representing a superset. The data contains a lot of nulls which I want to overwrite with real values.
the superset has:
both numerical and categorical data
some ...
CommunityBot
- 1
0
votes
0
answers
21
views
How can I read hdf5 files stored as 1-D array. and view them as images?
I have a large image classification dataset stored in the format .hdf5. The dataset has the labels and the images stored in the ...
- 111
1
vote
4
answers
924
views
Why does changing the cluster number change the plot in Kmeans?
This might be a dumb questions but I can't find the answer to it. I don't have the perfect mathematical understanding of kmeans, so apologies if it is.
I'm just wondering why I see a different plot ...
CommunityBot
- 1
0
votes
1
answer
226
views
Issue with using Sparse Data Frame in Mlxtend Apriori function
I am running python 2.7 in anaconda and have installed mlxtend. Based on the latest version of mlxtend, the aprioir class supports sparse dataframe as its input. I have over 500k products that I want ...
CommunityBot
- 1