# All Questions

19,003 questions
Filter by
Sorted by
Tagged with
3 views

### Using a trained Model from Pickle

I trained and saved a model that should predict a sons hight based on his fathers height. I then saved the model to Pickle. I can now load the model and want to use it but unfortunately a second ...
8 views

### How does Neural Network denoise an image?

I understand the mathematical formalism behind how neural networks work as a classifier or perform regression analysis. But I face difficulty to realize how they are such a great denoising instrument. ...
6 views

### What features to use for ML word matching algorithm

I am trying to match descriptions of text. I have made a shonky algorithm that works and produces a 'score' based on how well they matched based on full word matches and words immediately preceding/...
6 views

### Precision score (classification_report) 1/0 not predicting for 0 in logistic modeling using Sklearn

I have a binary outcome for an imported CSV file I am working with. I used the sklearn library to use the logistic model to gather predictions for my target value and also generate probabilities. ...
10 views

### (Feature selection) In which cases it is legitimate to remove features manually?

I am dealing with the feature in which only one category takes up about 90%, the instances of more than 30 other categories are sparse. Is it reasonable to remove this feature before building an ...
10 views

### Supervised Learning: time estimation of bike repair

I would like to train a model that estimate the time a given shop would take to repair for a bike using the data below: shops.csv ...
11 views

### Text similarity with sentence embeddings

I'm trying to calculate similarity between texts with various length. My current approach is following: Using Universal Sentence Encoder I convert text to a set of vectors. I average these vectors to ...
21 views

### How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my ...
7 views

I am not a math expert but have a basic understanding of linear algebra,calculus and probability and understands the math behind backprop. Currently I am trying to learn about policy gradient ...
8 views

### Suggestions on Generative Adversarial Network model to use for simulated Synthetic Aparture Radar images?

I have studied Generative Adversarial Network for some time and currently im looking into Synthetic Aparture Radar Images and simulated SAR images. The problem with simulated SAR images is that they ...
21 views

### Checking if ML model is possible

How can I check if a machine learning model is feasible on a given dataset? What techniques like EDA, correlation etc. can be used to judge if a model is possible i.e. data and predictor variables ...
7 views

### Automatically clear Rstudio temp files when manually stopping background “job” [on hold]

When I manually stop a local background job (that is running a loop over many rstan analyses) on Rstudio, Rstudio/R treat the job as if it's failed (rather than ...
8 views

### How to build a bloom filter based on p-stable locality sensitive hashing functions?

p-stable locality sensitive hashing function:$$h_{a,b}(v)=\lfloor{\frac{a\cdot v+b}{w}\rfloor}$$ a is a vector random choosed from normal distribution, a is uniformly choosed from $[0,w]$, v is a ...
14 views

### How to understand logistic trend parameters proposed by Prophet (Facebook)?

I read the paper and I saw that the logistic trend is defined as below : $$g(t) = \frac{C(t)}{1 + \exp{ -k(t) (t - m(t)) }}$$ Where $k$ and $m$ are respectively a growth rate and an offset parameter....
23 views

### (Feature Selection) different results from L2-based and Tree-based

I am doing feature selection using Sklearn: Tree-based feature selection : RandomForestClassifier.feature_importances_ L2-based feature selection: LogisticRegression.coef_ Target variable is binary ...
15 views

### Why does downsampling leads classification to only predict one class?

I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution : And here are the classification results of my ...
11 views

### Why SQL Server Full memory usage.? [on hold]

Why SQL Server Full memory usage.? If I was Apply size limits then also cross size linit.
20 views

### why does my model take 30 mins per epoch when on my GPU? [on hold]

I am training a model with around 200 images and it usually takes around 12 hours or more to complete. My colleagues' work only takes about a hour and a half to train. I am using windows 10 and ...
13 views

### Difference between binary cross_entropy and mse

model = Sequential() model.add(Dense(16, activation="selu")) model.add(Dense(1, activation="sigmoid")) I am using Keras to create a binary classifier. My data ...
11 views

### Averaging over Irregular Intervals

Given a sample of data where the samples are taken over irregular intervals, what is the most sensible way of going about calculating the mean and Standard deviation? Specifically, suppose we are ...
11 views

### How to get same accuracy with identical models in Keras and Tensorflow?

As we all know Keras backend uses Tensorflow and so it should give out same kind of results when we provide same parameters, hyper-parameters, weights and biases initialisation at each layer, but ...
9 views

### Use sequential pattern mining rules to predict next window of a dataset

Suppose that I am performing Sequential Pattern Mining (maxgap = 1, i.e. rules for consecutive windows) and I ran the following code from arulesSequences in R Studio to determine significant rules ...
14 views

### Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
9 views

### Transform test data when using a persistent model

I'm quite new to data science and only slowly following the necessary steps to get valid results using scikit-learn. As far as I understand you fit and transform the training data and only transform ...
7 views

### Problem with convolution neural network gradient checking

I have implemented a deep learning network : Conv -> Relu -> Maxpool -> flattens -> dense -> softmax. the network has 6178 parameters. I am trying to do gradient checking on my deep learning network. ...
18 views

### TypeError: unhashable type: 'numpy.ndarray''

When I tried to execute the below code for epoch in range(training_epochs): sess.run(training_step, feed_dict={x:train_x,y:train_y}) the following error is ...
31 views

### Dealing with NaN for predictive models

I have data set that has data for patients: Arrival_Date : is when the patient has arrived Seen_By_Nurse : is number of minutes patient take to be seen by nurse since arrival when value is NaN it ...
11 views

### Linear Regression Error in feature matrix step

I'm trying to code the design function used in linear regression using numpy and I get this error: Traceback (most recent call last): File "C:\Users\visha\AppData\Local\Continuum\anaconda3\lib\...
8 views

### Beating Roulette with Neural Networks, YoloV3, and PyTorch

Background: I am in my last semester of electrical engineering, and I am working on my senior design project. The senior design project is a two-semester design project in which students outline, or ...
13 views

### difference between normal skewed distribution and skewed distribution

From what I've read normal skewed is a distribution that has all the properties of normal distribution and is skewed: according to this resource:https://www.statisticshowto.datasciencecentral.com/...
11 views

### Prioritized Experience Replay - for whole episodes

I want to use Prioritized Experience Replay for whole episodes, instead for single transitions. What's the best way to define the priorities as episodes can be of different lengths? Personally I can ...
22 views

### Validation Curve Interpretations for Decision Tree

I'm working on a machine learning class, and we're using supervised learning right now, starting with decision trees. I'm using the UCI Credit Card dataset (whether or not certain people will default ...
7 views

### How can I do the correlation between two estimators?

I'm working with several estimators of all kind. Then, I want to stack these estimators, and the best is if they have low correlation between them. I suppose that the correlation method depends on ...
22 views

### What do you call a feature that always has the same value?

Is there a standard term for a feature that always has the same value, i.e. that can be discarded without loss of information? For example I am trying to classify cats vs dogs, and every example in ...
49 views

### (Python Basic) more elegant way of creating a dictionary

Is there a more elegant way to write a code like this? ...
16 views

### (Scikit-learn) differences between LinearSVC, 'linear' kernel SVC and poly kernel SVC with degree 1

I would like to know the differences between: linearSVC() SVC(kernel='lineaer) ...
9 views

### How to get subsample indices from sklearn.BaggingRegressor

I'm trying to define the number of the repeated samples in sklearn random forest subsamples: Here is my code: ...
7 views

### High accuracy in one v. all, lower accuracy in all vs. all

I am training a classifier (similar to logistic regression) on MNIST. I have 10 one -vs.-all classifiers for each number, each of which independently achieves >90% test set accuracy. However, when I ...
8 views

### Where can I get an untokenized version of GLUE's SST-2 dataset?

On the GLUE faq, they say: Similarly, for SST, the data provided is already tokenized. We're working on obtaining a version that is not tokenized. Feel free to train on other distributions of ...
9 views

### How to generate several elements on image with input parameters

For example, i need to generate circles. I have dataset of images with non-intersecting circles and can generate random circles with DCGAN, but each circle has a different diameter. So I need to ...
12 views

### sklearn: sklearn.linear_model.HuberRegressor vs sklearn.linear_model.ElasticNet

I am experimenting different loss functions for my regression model. I noticed that in the sklearn, there are: sklearn.linear_model.HuberRegressor and sklearn.linear_model.ElasticNet To me, both use ...
7 views

...
23 views

### Data visualisation for big data sets

Working on some voluminous data sets, I have been interested in methods for dimension reduction and plotting. I have stumbled upon this novel technique : UMAP (https://arxiv.org/pdf/1802.03426.pdf), ...
9 views

### How to design n-dimensional feature descriptor similar as the input image?

I am re-writing the H-Net code in Keras for cross-domain image similarity. The network architecture is described in the attached paper. I wrote the encoder and decoder parts but unable to get similar ...
12 views

### How to get Keras accuracy for each step in an epoch like in Tensorflow?

Like in tensorflow I get accuracy for each step - ...
11 views

### Understanding the softmax output in Youtube's recommender

This question has been asked before, but never (that I can see) satisfactorily answered. I'm reading Youtube's paper on their recommender system. The system has two elements, the first of which is a ...
33 views

### Feature selection is not that useful?

I've been doing a few DataScience competitions now, and i'm noticing something quite odd and frustrating to me. Why is it frustrating? Because , in theory, when you read about datascience it's all ...
21 views

### Applying the fourier transform

I have been trying to apply Fourier transform for two days, but haven't been able to because all the examples I've seen so far are not on actual dataset and use standard values that I don't understand ...