Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

Filter by
Sorted by
Tagged with
0
votes
0answers
2 views

Question about Beware of the inherent skew in ranking problems in Google Rule of ML

I try to understand the Rule 35 from Rule of ML from Google. When you switch your ranking algorithm radically enough that different results show up, you have effectively changed the data that your ...
0
votes
0answers
5 views

Does severe multicollinearity affect solving linear regression by gradient descent?

Since OLS may fail when there is severe/near perfect multicollinearity, how would gradient descent perform in such a scenario? Does it converge at the minima? (My guess is, Cost function of linear ...
0
votes
0answers
5 views

Visualization of how K means clustering is used to select K anchor boxes?

I was having trouble understanding how K means clustering is used to select K anchor boxes. We have ground-truth boxes and we run K means clustering on them using IOU as a metric. There is no good ...
0
votes
0answers
10 views

Operation of the k-fold cross-validation method when using caret package

I have a question regarding the operation of the k-fold cross-validation method when using the caret package. As I understand it,...
0
votes
0answers
7 views

Removing the effect of Time series X on time series Y, when their relation is unknown

I am working on a dataset of 6 years measurements of a water quality parameter called 'chla' ( parameter 'X') measured by a sensor for each year from May to October. The parameter has its own trend ...
-1
votes
1answer
7 views

Why does backtranslation work for Neural Machine Translation whereas it decreases for Statistical Machine Translation? [closed]

I've gone through some of the works and it decreases the scores on the evaluation. Any fixes/reasons ?
0
votes
1answer
12 views

how to get the classe names in an image classifer after predection?

I made an image classifier of 80 classes of handwritten numbers then I tested my model and it worked pretty fine, the only problem that I have now is the display of the correct names of these classes. ...
1
vote
1answer
12 views

Can you use two different datasets as train and test sets with countVectorizer and test_train_split?

So I managed to run my code on a combination of train data and validation data, but now I need to create a text file that contains the predictions for the test data and I just don't understand how. Is ...
1
vote
0answers
8 views

Time Series prediction using R

I have a dataset which contains data related to the exchange rate in a certain time period(2013-2015). The dataset has a column date with YYYY/MM/DD format and USD/EUR which contains the exchange rate....
2
votes
1answer
30 views

Is my dataset unbalanced?

So I'm trying to implement Nvidia's end to end driving paper to simply have an agent in Carla follow the lanes. I'm trying to predict the steering angle of the car based on the RGB images from the ...
0
votes
0answers
8 views

Getting a bounding box mask given coordinates of an object in the image

In the paper See the Glass Half Full: Reasoning about Liquid Containers, their Volume and Content, one of the inputs to the model is "a bounding box mask smoothed by a Gaussian kernel". I'm not sure I ...
-1
votes
1answer
17 views

How to select the best features for Support Vector Classification

I have a feature set that contains approximately 2 dozen features of technical analysis indicators. My own domain knowledge tells me that some of these features are better than others for predicitive ...
0
votes
0answers
9 views

Variable with extra small Pearson coefficient has bigger positive impact on ML model performance than variable with bigger Pearson

I made some machine learning models using Python sci-kit learn library and I found some strange situation for me regarding the real importance of some variables (features) to the ML model. I found ...
0
votes
0answers
10 views

Factorization Machines with some some pairwaise effects and some linear effects

I have some covariates $x_1$, $x_2$, $x_3$, .., $x_{10}$. I want a linear term for all these covariates and a pairwise effect for $x_1$ and $x_2$, and I do not want any other pairwise effect. Is there ...
0
votes
1answer
15 views

A Loss of 55.2164 with a sparse_categorical_crossentropy in a sequential neural network?

I'm following Aurélion Géron's book on Machine Learning. The following code tries to evaluate a neural network with a sparse categorical cross entropy loss function, on the Fashion Mnist data set. ...
-1
votes
2answers
21 views

ValueError: Error when checking input: expected dense_9_input to have 2 dimensions, but got array with shape (60000, 28, 28)

I'm doing a regular detection of numbers from photos with MNIST, but when i try to fit my model, it doesn't work, and it dispayed this message... ...
1
vote
1answer
31 views

Recursive Feature Elimination (RFE) with Logistic Regression and little correlation between the features and the target (SKLearn)

I have little experience in the field of Machine Learning, so I apologise in advance for my ignorance and lack of intuitions. I'm trying to build a classifying algorithm to identify if a subject is ...
1
vote
0answers
8 views

Compare the variances of two categorical distributions in a repeated measure design

I ran two model-building procedures with different parameters on the same sample and obtained the selection of my optimized hyperparameter for each outer fold (each of the analyses had 100 outer folds ...
1
vote
0answers
11 views

Recursive Transfer Learning

Is there any methodology called Recursive Transfer Learning? For example, let's consider a situation that we have a lack of data while training a convolution neural network (CNN) for object detection ...
2
votes
1answer
27 views

LSTM: many to one and many to many in time-series prediction

I am trying to predict the trajectory of an object over time using LSTM. I have three different configurations of training and predicting values in my mind and I would like to know what the best ...
1
vote
1answer
10 views

How to use External Data Sets in test set

I have a doubt regarding usage of external datasets like gdp rate, unemployment rate... etc., in test set for time series prediction. These datasets are historical and can be used along with train set,...
0
votes
2answers
37 views

How to make a classification problem into a regression problem?

I have data describing genes which each get 1 of 4 labels, I use this to train models to predict/label other unlabelled genes. I have a huge class imbalance with 10k genes in 1 label and 50-100 genes ...
0
votes
0answers
6 views

Identifying data distribution for my data and model building strategy

i am a beginner for panel data econometrics. Need help of experts of the field deciding if it is panel or pooled data or should i use any other methodology. I have data from wind mill and we use wind ...
0
votes
0answers
11 views

factors in the success of startup fundraising

I have been following this paper published in 2019. There are few things that are ambiguous and not at all clearly explained. On page 7, table 2, Max Portfolio, ...
0
votes
1answer
12 views

How do you choose the min and max for the min-max normalization on a histogram classifier?

Please let me know what to do when there is a value in the testing set is bigger than the max value used to min-max normalize the training set building a histogram classifier. Do I go back and change ...
0
votes
0answers
12 views

How to deal with discrete variables in Multivariable Time Series forecasting?

I am tackling this time series forecasting problem to basically predict number of sales in the future training dataset looks like this: ...
0
votes
0answers
11 views

Time series problem with LSTM is not predicting correctly

. My Initial dataset looks like this: Text sample of Dataset: ...
-2
votes
0answers
6 views

Log normal Distribution

https://www.kaggle.com/nowke9/ipldata After downloading this dataset, I wrote the following commands to load and display the dataset. import math import numpy as np import pandas as pd from scipy ...
0
votes
0answers
10 views

Comparison of Time series models

How to compare two different forecasting models lets say one is the classical statistics-based model and the other is a machine learning-based. Also, let's say if I have to forecast the unit volume ...
0
votes
0answers
6 views

Is data leakage in time series due to both I's of the IID principe or only one?

I am sure that the Independent part of the IID principle gives you data leakage because of the correlation. But the identical part I am not so sure. Identical in time series means that your data is ...
0
votes
0answers
20 views

Estimate optimal parameters [migrated]

We are given a set of $K_1$ vectors $a_{i} \in \mathbb{R}^{N}$, $1 \leq i \leq K_1$; $K_2$ vectors $b_{j} \in \mathbb{R}^{N}$, $1 \leq j \leq K_2$; and a set of $K_1$ matrices $A_{i} \in \mathbb{R}^{N ...
0
votes
1answer
20 views

Aggregating multiple encoded categorical values

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables. I am currently using a dataset with a feature CATEGORY which has a cardinality of ~20,...
0
votes
0answers
6 views

Taking huge time to execute piepeline text classification model using sklearn?

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k ...
2
votes
3answers
29 views

How to normalize data without knowing the min and max values?

I have a Lending club dataset from Kaggle; it contains many different columns: there are for example dummy variables, years, amount of the loan...ect I want to normalize the data in the training and ...
0
votes
0answers
14 views

Further Classification by function [closed]

I am a bit confuse and stuck at a problem, may be someone can guide me in right direction I am doing an analysis and have 3 datasets, supporting, against and neutral, or in simple say negative, ...
0
votes
1answer
15 views

Learning a threshold from a matrix

I have a 2D matrix, and I want to learn a threshold from this matrix. All items in the matrix bigger than the threshold will become 1s, and all smaller than the threshold will become 0s. Then, I have ...
0
votes
0answers
49 views

None of the known overfitting prevention techniques works for me, according to learning curves

I am working on HTRU2 dataset to evaluate classification models. Even though I obtain good results in terms of accuracy-MSE: I have an overfitting problem according to the learning curves below. In ...
-1
votes
1answer
35 views

What is evaluation metric for two sets? [closed]

I've two sets one is ground truth and other is output of my machine learning models. Assume my groundtruth set is A={1,2,3,4,5} and output of machine learning model is B={3,4,5,6,7,8}. One way I can ...
0
votes
0answers
19 views

How to improve cross validation score? [closed]

I am working on regression problem with around 600 observations , train / test split = 0.2 and CV= 5, I have tired all the possible techniques to improve cross validation score : Removed outliers ...
0
votes
0answers
9 views

Dataset containing news URL,headline and label as Finacial/Non Financial crime [closed]

I need help in getting a dataset which contains News URl,NEWS headline/text and label as if they belong financial or non financial related crime.I need it specifically to extract out details using NER ...
0
votes
1answer
12 views

Dealing with missing data in SVD

I am a newbie to machine learning and I am trying to apply the SVD on the movielens dataset for movie recommendation. I have a movie-user matrix where the row is the user id, the column is the movie ...
0
votes
0answers
14 views

Which algorithm and/or libraries to use for prediction and probability matching of event (per second) data having numerical values?

Suggest algorithm to match input data with existing records and calculate matching probability. Based on matching probability predict the output similar to a matching record. Data contains per second ...
0
votes
1answer
36 views

Machine Learning classification problem

I'm trying to do this classification problem, depicted in the following Figure. The task is to separate the blue elements from the red elements in a cartesian (x,y) coordinate system. I have to: ● ...
0
votes
0answers
8 views

Database augmentation using Sobel filter (aka gradient)

I'm working in PyTorch and to improve a ProGAN training I'm augmenting a database by calculating, for each image, the horizontal and vertical gradient via the Sobel operator. Do you have any reading ...
6
votes
1answer
195 views

A the end of a big DS project, should I make trained models available on GitHub?

I almost completed two big Data Science personal projects based on Deep Learning. They are the fanciest models I've implemented up to now, and I'm pushing all my code on GitHub. Do you advice to ...
0
votes
0answers
13 views

Machine Learning File Validator

I want to write ML code for file validation. Issue : There will be many csv / excel files coming from teams and Our BOTs processes those files. Our BOTs goes into error because of data format could ...
1
vote
1answer
31 views

How to remove background (watermark) logo from image

I have been scratching my head for a while. What I have is a scanned PDF document with text and water marked logo at the back as in the below image. I want to do OCR over this, which becomes very ...

1
2 3 4 5
149