All Questions

8,078 questions with no answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
49 views

Gaussian process regressor returns almost identical std for all datapoints

I am using a Gaussian process regressor as the regressor for active learning and I use its standard deviation to choose the next training inctance (the one with the highest std is chosen). However, ...
Ash's user avatar
  • 51
3 votes
0 answers
106 views

SuperLearner Cross validation with iid time series

I created a number of ML models in R and I aim at combining them to form an ensemble. I learned about SuperLearner library which cross validates many models and returns the weight to each model in ...
Vitomir's user avatar
  • 163
3 votes
0 answers
53 views

Information Extraction from image / text - approach?

I need assistance with a ML project I am currently trying to create. I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements ...
oliverbj's user avatar
  • 185
3 votes
0 answers
2k views

How to apply a groupby rolling function to create multiple columns in the dataframe

I am setting up a volume profile series over a stock data. I have implemented the market profile code from this github repo and the link to the data is here and the example here. Some Sample of data ...
Scrappy Coco's user avatar
3 votes
0 answers
418 views

How to encode multiple inputs and multiple outputs

I have a piece of math more complicated than I can understand at the moment, a dynamic graph visualization. It uses a physics metaphor of springs and magnets, where the vertices act as magnets ...
Joshua M. Moore's user avatar
3 votes
0 answers
275 views

Help understanding the Tensorboard histogram names and meaning in an LSTM Model

Can someone please help me understand what the names and shapes of the following tensorboard histogram outputs mean about an LSTM model I coded? Thank you! I understand the terms in the names like ...
Dipankar J Dutta's user avatar
3 votes
0 answers
73 views

Terrible Tensorflow performance on powerful production server

Background I am building a simple face detection API with Python, Flask and the MTCNN face detector. My problem is that the model and API are running really quickly (batch of 100 images takes 0.5-0....
Szőke Péter's user avatar
3 votes
0 answers
537 views

Keras custom metrics - MAP and MRR

I am trying to build a LSTM model in keras where I have one question with 10 answers but only ONE among them is correct. So basically im tring to build a 10 class classification problem. As most of ...
Rohith's user avatar
  • 31
3 votes
0 answers
239 views

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network? Im have a cedar signature data set with 55 peoples signatures(classes) with ...
star's user avatar
  • 1,481
3 votes
0 answers
52 views

Improving a simple trig model

I have some data which I know is well approximated as a trig function, and I can fit it with scipy.optimize.curve_fit as follows: ...
user1887919's user avatar
3 votes
0 answers
260 views

Ising Spin Glass - Optimization

I'm a newbie researcher working on model-based genetic algorithms, mainly linkage learning in both discrete and continuous spaces, using data modeling. I would like to ask you about Ising Spin Glass (...
Piotr Rarus's user avatar
3 votes
0 answers
263 views

Fine tune gpt2 via huggingface API for domain specific LM

i am using the script in the examples folder to fine-tune the LM for a bot meant to deal with insurance related queries. So if someone were to type "i am looking to modify my ..." , the autocomplete ...
Vikram Murthy's user avatar
3 votes
0 answers
26 views

Why RANDOM noise images always predicted as BIRD?

Say I have fine-tuned a 10-classification ResNet18 network on CIFAR-10 and the accuracy on validation set is about 93%. However when feeding into 5000 random noise images (Gaussian noise with the ...
dmrak's user avatar
  • 31
3 votes
0 answers
2k views

Explanation of inductive bias of Candidate Elimination Algorithm

The definition of inductive bias says that The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs ...
JustABeginner's user avatar
3 votes
0 answers
36 views

High dimensional data stream summarization and processing

Can anyone recommend a method for summarizing and processing high dimensional data streams efficiently and effectively for anomaly detection? In fact, I investigated the different methods for data ...
I Sui's user avatar
  • 57
3 votes
0 answers
58 views

Change the way spacy works - Custom properties for training and prediction

Spacy detects the entities using its predefined algorithm. It parses tokens in text considering position of tokens with respect to tokens surrounding it. It also takes into consideration the POS ...
Sandeep Bhutani's user avatar
3 votes
0 answers
412 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
sepehr78's user avatar
3 votes
0 answers
176 views

weighted quantile sketch in xgboost

I am unable to understand what is weighted quantile sketch in xgboost. Can anyone help me give an intuitive understanding of this?
Hardik Gupta's user avatar
3 votes
0 answers
208 views

Using a KMeans to classify URLs: validate the number of cluster and visualise

I'm relatively new to the NLP and DataScience, so apologies for omission or things like this. I've been trying to use the KMeans to classify a list of 1000 of unique URLs containing several keywords ...
Andrea Moro's user avatar
3 votes
0 answers
195 views

How to remove noise using morphological filtering

I have two groups of dots that both contain noise between them: The line that separates the two groups in the picture is diagonal in shape. I tried to use morphological filtering on this image to ...
tamarlev's user avatar
3 votes
0 answers
115 views

Features selection with a lot of dummy variables in R

I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
ccc's user avatar
  • 131
3 votes
0 answers
109 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
user3075342's user avatar
3 votes
0 answers
458 views

Plotting facetgrid in subplot for IRIS dataset

I'm trying to plot 6 selected pair subplots with the combination of facetgrid of seaborn and scatter plot from matplotlib. I'm getting plot, but subplots remains empty whereas facetgrid gets plotted ...
Shadab Hussain's user avatar
3 votes
0 answers
84 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...
Chinmaya B's user avatar
3 votes
0 answers
415 views

Hive query to get all rows where a particular column value lies in a particular precentile

I am trying to filter my rows in hive table named id_counts based on percentile values. Lets considers the following table. ...
Heisenbug's user avatar
  • 411
3 votes
0 answers
92 views

summarizing time series dataset: extract time window sliding, change points, pattern seasonality in time series

I need to detect list of change points in time series dataset (temperature), and I need to split dataset into set of classes (patterns) and detect seasonality of each class (pattern). for example ...
Mina Younan's user avatar
3 votes
0 answers
777 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
Blenz's user avatar
  • 2,084
3 votes
0 answers
159 views

Learn large, variable-size action space for Diplomacy game

I am making an environment using OpenAI gym for Diplomacy, and making an AI for it. In Diplomacy, a player has many units, and each unit has a number of moves available to it. Therefore, the player'...
Daniel Paczuski Bak's user avatar
3 votes
0 answers
1k views

Can you do automated feature engineering in R?

Since now Python has its own deep feature synthesis library, is there anything similar available in R? I know of bounceR but I'm not sure if it does the job as DFS does. Is anyone aware of anything ...
Shiv_90's user avatar
  • 265
3 votes
0 answers
60 views

Structuring a LSTM Layer

I'm trying to improve an NER Bert sequence tagger using LSTM layers in TensorFlow. I'm a bit unclear on the interface and how a LSTM layer should be set up. Currently, I'm taking in 3-5 sentences and ...
bbbbbb's user avatar
  • 41
3 votes
0 answers
304 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
sovon's user avatar
  • 521
3 votes
0 answers
118 views

Computing derivatives for backpropagation across a convolution step

This will be a long post, but I hope it'll be instructive to anyone else in my position. I'm trying to find how the derivatives of the loss function are calculated with respect to the kernels and ...
Shirish's user avatar
  • 299
3 votes
0 answers
222 views

How to measure the stability of hyperparameter selection in a model-building procedure?

For my project I run several model-building-procedures. I use the mean and standard deviation of the test scores in the outer folds as an estimator for the generalizability of the model-building ...
Johannes Wiesner's user avatar
3 votes
0 answers
198 views

How to use zero-inflated negative binomial regression for binary classification task?

I am working on a binary classification problem and I am currently employing XGBoost. The dataset consists of several variables which are count variables. The problem is, these features are highly ...
Rohit Gavval's user avatar
3 votes
0 answers
2k views

Improving recall in XGBoost algorithm

I have highly imbalanced dataset. I am using XGBoost and I got the following results without balancing the dataset out: Precision: 0.87 Recall: 0.79 F1: 0.83 My ...
eemamedo's user avatar
  • 161
3 votes
0 answers
44 views

Deep learning and label noise. Best practices for the real world

Unlike MNIST or other benchmark datasets collected data often come with subpar, inaccurate labels. What are the best practices to help the neural networks to don't overfit the noise? Things that comes ...
ajeje's user avatar
  • 191
3 votes
0 answers
96 views

scikitlearn MLPRegression model conversion to TensorFlow Dnnregressor conversion

I implement custom Neural Network (prediction part only) based on ScikitLearn MLPRegressor model weights with sklearn.preprocessing.StandardScaler normalization of input features Can I convert it ...
Sasha Rabinovich's user avatar
3 votes
0 answers
59 views

Timeseries prediction error measurement. How to deal with diffrent time scales?

I have some time series and a prediction model. Now I would like to measure how good/bad the prediction is for different products. The problem is that for each product the time points (frequency of ...
MichaelRazum's user avatar
3 votes
0 answers
48 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
chupa_kabra's user avatar
3 votes
0 answers
173 views

Dynamic pricing models in freight transportation (logistics) business

I'm not sure this could be an appropriate question for here. I'm a newbie in the field of data science. I'm looking for keywords which can guide me to search the results to implement what I want to ...
Hosang Jeon's user avatar
3 votes
0 answers
313 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
timleathart's user avatar
  • 3,940
3 votes
0 answers
42 views

Back-Translation model for German and English

Do you know of any pre-trained models for back translation between German and English? I am aware that there are ways to include a monolingual corpus into the training of a machine translation model (...
treebased's user avatar
3 votes
0 answers
273 views

Neural networks (keras): predicting a periodic output array

I have a non-linear multiple regression problem where my target arrays have a length of 256 (for a single sample). These arrays have a periodic structure, since it's actually composed of 16 semi-...
Peterukk's user avatar
3 votes
0 answers
60 views

Keras 'cross section' model with time trend

Problem: I have a problem in which cross-sectional features (X) explain a continuous outcome (y). In addition, there is a linear time trend (t) in the data. Using OLS, such a model would write: $y = ...
Peter's user avatar
  • 7,526
3 votes
0 answers
223 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...
Sandeep Bhutani's user avatar
3 votes
0 answers
322 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
DankMasterDan's user avatar
3 votes
0 answers
250 views

Time series pattern recognition

I have measured stress at 1 million points inside a material at 100 time steps. I have made a probability distribution plot for three time steps and I see that the evolution of stress looks like this: ...
user134439's user avatar
3 votes
0 answers
477 views

How to explain a Calibration Plot for many models?

I have a heavy imbalanced dataset with a classification problem. I try to plot the Calibration Curve from the sklearn.calibration package. In specific, I try the ...
Tasos's user avatar
  • 3,930
3 votes
0 answers
202 views

Why does my model only converge when I add a MaxPooling with stride of 1 layer at the beginning?

I have a model that takes an input of inertial sensor data collected at 60 hertz and outputs one of two classes. I have broken the data into 1 min snapshots which seems to be appropriate. I have split ...
Victor 'Chris' Cabral's user avatar
3 votes
0 answers
430 views

Adjust class weights due to class imbalance and class importance Multi class classification XGBoost

With respect to this question and the answer given by @Esmailian, Would anyone be able to let me know if Class B has a higher importance or the positive class ( i.e. it needs to have a higher ...
Michael Schroter's user avatar

15 30 50 per page