Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

How can i understand multiple lstm cells by unrolling?

I do not Unterstand the concept of multiple units in lstm. If i have an lstm layer with 64 cells, how would be the cells applied to each time step by unrolling. My understanding is that each time step ...
WannabeMathMaster's user avatar
1 vote
0 answers
9 views

How to properly visualize high-dimensional embeddings along with the decision boundary in 2-D?

I have a number of embeddings (300-dimensional FastText vectors for each instance of each class) that I apply a classifier to (Logistic Regression for now). I want to visualize the embeddings as well ...
Metrician's user avatar
0 votes
0 answers
6 views

Exploring the Concept of Gradient Flow

Understanding the concept of "Gradient Flow" can be quite difficult as there is a lack of widely recognized and clearly defined resources that provide a comprehensive explanation. Although ...
StudentV's user avatar
0 votes
2 answers
11 views

How to rank relatedness of two feature in dataset by their distribution?

Let's say we are given a dataset and want to rank them by similarity of distributions. I don't want to use visualization. Is there any sufficient way that you can share with me? I have an idea like, ...
Ibrahim Rustamov's user avatar
0 votes
0 answers
6 views

pytorch: implementing logistic regression: input dimension of torch.nn.Linear is input.flatten(start_dim=1)

I tried to implement a logistic regression class using pytorch. The following implementation worked. ...
thingy's user avatar
  • 1
2 votes
0 answers
70 views
+50

How to remove the hotspots from given image by using Python and opencv?

In the picture below there are some regions which are very bright (i.e. more white). Some bright regions are wide and some are narrow or thin. The red box covers one such wide bright spot, and blue ...
Alok Maity's user avatar
0 votes
1 answer
16 views

What exactly is Gradient norm?

I found that there is no common resource and well defined definition for "Gradient norm", most search results are based on ML experts providing answers which involves gradient norm or papers ...
StudentV's user avatar
1 vote
1 answer
20 views

Different validation sets give very different results. What can be the reason?

I have ~78k microscopy images of single cells, where the task is to classify for cancer (binary classifier). The images are labeled according to which patient the data came from. I do the train-val ...
Emil Edvardsson's user avatar
2 votes
2 answers
183 views

Some simple questions about confusion matrix and metrics in general

I will first tell you about the context then ask my questions. The model detects hate speech and the training and testing datasets are imbalanced (NLP). My questions: Is this considered a good model? ...
FjkgB's user avatar
  • 89
0 votes
0 answers
13 views

Confusion regarding what constitutes a feature in a LSTM?

I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$. Let's just look at one observation for simplicity. ...
the man's user avatar
  • 101
1 vote
1 answer
18 views

How do I exploit partial labels for classification?

How does one learn a classifier from data that isn't always fully labelled? For example, say one has corrupted data from the CIFAR-10 dataset (which has labels like bird/automobile/ship/truck). Now ...
mister_mole's user avatar
0 votes
0 answers
22 views

What statistical test should I use to compare two random forest models, where each model has a different set of variables available? [closed]

What statistical test or methodology should I use for comparing two random forest models, where a different set of variables is made available for each model? I need a test where a power analysis can ...
Kyle's user avatar
  • 111
0 votes
1 answer
19 views

Struggling with understanding RandomForest model with SMOTE

From what I understand my code is telling me that my base model is performing at 96% on it's training data, 55% on it's test data. And my SMOTE model is performing at ~96% on both. From my ...
GroupTheory14's user avatar
0 votes
0 answers
14 views

Model returns near perfect PR-AUC score but other metrics seem fine. Is my model overfitting?

I am currently working on a very imbalanced dataset: 24 million transactions (rows of data) 30,000 fraudulent transactions (0.1% of total transactions) The dataset is split via Year, into three sets ...
Hai Nguyen's user avatar
-2 votes
0 answers
17 views

how do to divide the datasets into 80/20 per cent of training and test sets and 10 foldcross-validation during model training to minimize bias [closed]

Classification: Diabetes Expert System Sediba MedResearch is a research company that focuses on the study of diseases aiming to provide accurate expert systems that can help medical practitioners ...
Hipsonia Pitso's user avatar
0 votes
1 answer
21 views

Anomaly Detection: Large number of categories

Looking for some advice. I am working on an Anomaly detection problem, I am looking at parcels being transported from A-B and want to identify which parcels are considered anomalies for given routes. ...
darren's user avatar
  • 1
1 vote
1 answer
20 views

Extract phrases/keywords that are SIMILAR to a python list of keyword/phrases, from a document

EDIT : If I had to match single worded phrases, I could first tokenize the text from the document and then calculate the cosine similarity of all the tokens with all the keywords from the ...
spectre's user avatar
  • 1,566
0 votes
0 answers
12 views

In recommendation systems, for methods that use backproporgation to get user feature, do they need to retrain the whole model when a new user is added

I'm tying to learn about recommendation systems recently. I have some deeplearning background so I focused more on machine learning based methods for recommendation systems. I see that a lot of paper ...
meng lin's user avatar
  • 101
0 votes
1 answer
25 views

What are the approaches for extracting an injury and its description from a paragraph?

Suppose I have a paragraph which explains the injuries and its descriptions. I want to extract the injuries and its corresponding descriptions from the text. How can I do that? For example, the ...
SRJ577's user avatar
  • 197
1 vote
0 answers
40 views

Questions about time series and overfitting and underfitting

When we want to compare between loss and val_loss, do we only care about the last epochs? Do the following charts show that the model is not overfitted nor underfitted? Can you please give me comments ...
FjkgB's user avatar
  • 89
1 vote
2 answers
37 views

High accuracy on test and validation data but still can not predict on real data

Hello i am having a classification between two classes A and B and i have trained CNN model. I have high accuracy on all three set of data i.e training (98.7%) validation (99.3%) and test(98%) but ...
Ejaz's user avatar
  • 11
0 votes
1 answer
47 views

How to intrepret low F1 score and high AUC on training set?

I am currently working on a very imbalanced dataset: 24 million transactions (rows of data) 30,000 fraudulent transactions (0.1% of total transactions) and I am using XGBoost as the model to predict ...
Hai Nguyen's user avatar
0 votes
1 answer
20 views

What is the minimum ratio to consider the data set as balanced for the classification algorithm?

Can I consider 20% / 80% as a balanced dataset? My target variable ratio is 62% and 37%. I hope It is a balanced dataset. Please let me know if I am wrong. However, I would like to know, what is the ...
Iynga Iyngaran Iyathurai's user avatar
1 vote
0 answers
29 views

Can you do a power analysis to determine the sample size for a virtual species simulation which is modeled using random forests?

I am simulating virtual species on a 100x100 grid (the size for now). Each grid layer represents one environmental variable. The "suitability function" defines the probability of a presence ...
Kyle's user avatar
  • 111
1 vote
2 answers
32 views

guys, i am new to ML. i was trying to make linear regression model on fmri dataset. while passing x_train,y_train code , it shows me following error

...
brijesh's user avatar
  • 11
1 vote
0 answers
9 views

confusion with Xavier Initiliazation definition

When researching online, I keep finding that Xavier/Glorot initialization is: however, the original paper by Glorot said that this was a common initialization strategy that they soon found did not ...
tom394's user avatar
  • 11
1 vote
0 answers
32 views

Convolution neural network loss increasing instead of decreasing

I am working on a binary image classification task in which I have greyscale images of size (1, 224, 224) (all normalized between 0 and 1) and a set of labels (0 or 1). I have around 2.6k images with ...
Akshit Sharma's user avatar
0 votes
0 answers
9 views

Validation Loss not decreasing for RESNet model

I have been trying to train a Resnet model to classify Diabetic Retinopathy images into binary classes. The dataset consists of around 35k images. The val loss and accuracy does seem to behave weirdly ...
SarveshSC's user avatar
1 vote
2 answers
56 views

Looking for a couple of ideas please

I’ve got some data by postal zone that includes: Postal zone code Average rent value per square foot Brand affinity 1 Brand affinity 2 Brand affinity 3 Brand affinity 4 …and so on The brand affinity ...
Jake's user avatar
  • 11
0 votes
1 answer
32 views

corr() is giving an error. please help out of this problem and tell me what is this error about

when I am trying to run sns.heatmap(df.corr(),annot=True) this code in my jupyter notebook. this error is occuring. I cannot understand this problem. please help me.
Subhajit Sarkar's user avatar
0 votes
0 answers
11 views

what happens when weghts in perceptron algorithm is first initialized with some random values which is very distant from the correct values?

take this example of a small dataset so here there was a question that instead of initializing weight vector as zeroes what if we initialize to [1000 , -1000] (there is no offset i.e classifiers ...
Anshul Patel's user avatar
0 votes
1 answer
21 views

How complex can I make a classifier's loss function in Scikit-Learn?

I want to customize the loss vanilla loss function being used by scikit-learn classifiers like the Logistic Regression classifier, etc. For example, if the vanilla empirical risk minimization ...
Inferno Dynast's user avatar
1 vote
1 answer
34 views

Best classifier for ordinal categorical variables

I'm getting different results between R and Python for a classification problem using ordinal categorical features. I would like to ask you what do you think is the best classification algorithm in ...
user140259's user avatar
1 vote
1 answer
46 views

What should I Improve from my Neural Network Model (Logistic Regression)

Initial Information I built a Neural Network Model (Logistic Regression) to classify Lung Cancer based on the patient's (user) symptoms My dataset is kind of small (only about 276 data) Here is the ...
Jonathan's user avatar
0 votes
0 answers
19 views

LSTM can accept inputs of different shapes in some cases

I thought that with an LSTM you could use sequences of any length as input, but with shape fixed for each time step, but I encountered an anomalous behavior. The following code gives the error that I ...
stopper's user avatar
  • 111
0 votes
2 answers
37 views

Where can I find the applied data science research papers?

I'm trying to find conferences that have applied data science papers published. I'm only interested in top ranked conferences. And I notice quite a number of them are quite theoretical, e.g. IJAI, ...
Student's user avatar
  • 399
2 votes
2 answers
71 views

Random Forest Classification model performing much better with 70:30 TEST:TRAIN rather than the opposite

I'm working on a Classification problem as a side project and I'm receiving results contrary to what I'd expect. With 100,000 records, each with 7 components for X, the model is performing much better ...
GroupTheory14's user avatar
1 vote
0 answers
25 views

MobileNet validation loss not decreasing over time

I am trying to train a MobileNetV2 on a custom dataset, to image Classification task. Cardinality is 864 images, split in 70%/20%/10%, balanced between the 3 different classes. Weights are pre-loaded ...
elbarto's user avatar
  • 11
0 votes
1 answer
31 views

Does a classifier based on optimal bayes classifier equation classify every new instance the same way?

I'm trying to understand how optimal bayes classifier works and I was wondering if, given that the function we try to maximize when making a new prediction does not depend on the instance we are ...
Niccolò Zanieri's user avatar
0 votes
0 answers
13 views

Embedding vector of MaskRCNN (Resnet with FPN)

I have a MaskRCNN model for instance segmentation with Resnet 50 - FPN backbone trained in detectron2. And I want to extract the embedding/feature vectors for visualizing input and hopefully detecting ...
Sushil Khadka's user avatar
0 votes
0 answers
10 views

Can the addition of a non-support vector change the SVM solution?

If I understand the math behind the classic SVM for non-separable data correctly, the addition of a non-support vector (non-SV) should theoretically not alter the solution. My reasoning is that since ...
kate allerton's user avatar
0 votes
1 answer
22 views

Do I need to standardize time series data in change point detection?

I have process data in time series data(0min, 1min, ... 999min). I don't know what does the variables mean. They are just written in X1, X2, ... X52. Each row means the data at the time. At certain ...
PLl's user avatar
  • 1
0 votes
0 answers
23 views

dolly 2.3b machine laerning using trainer.train

first time poster here so please forgive me and correct me on my posting mistakes... Im trying to teach the databricks/dolly-v2-3b llm, some data, which is just one sentence. In the future I would ...
cep's user avatar
  • 1
0 votes
1 answer
23 views

How to make an RNN model in PyTorch that has a custom hidden layer(s) and that is compatible with PackedSequence

I want to make an RNN that has for example more hidden layers or layer normalization. I know that is it possible to make a custom RNN by subclassing nn.module, but with this approach is it not ...
Philip T 2007's user avatar
1 vote
2 answers
104 views

what qualifies as a data leakage?

I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
The Great's user avatar
  • 2,507
0 votes
1 answer
14 views

How do I use a column with data of different layers for AI?

I am working with real estate data for an ML/DL project. In the csv file there is a column in which each cell contains data like the examples below: ...
Muhammad Usman's user avatar
0 votes
0 answers
15 views

FTT Features to use after time-domain is transformed to frequency-domain

Please forgive the question if it sounds trivial/naive, I am from computer science background, not electrical/computer engineering. I work with GPS trajectory dataset for classification. Data was ...
Amina Umar's user avatar
0 votes
2 answers
38 views

can training for too long lead to overfitting? I am not sure about the specifics of this

does training for a large number of epochs lead to overfitting? I am concerned about this as I am getting an accuracy of nearly 1 on val and training dataset when I am training for 50 epochs
Priyanshu's user avatar
4 votes
1 answer
92 views

Are imbalanced data problems solvable? [closed]

I am working as a data scientist for the past 2 years where I have worked on problems related to binary classification, revenue prediction etc. In the past two years, I have had 2 problems that ...
The Great's user avatar
  • 2,507
0 votes
0 answers
7 views

Image classification approach for float outputs

I have an image input and the model should be able to predict its 15 feature values as output. I am being told that i should use an image classification model to solve this. can somebody suggest me a ...
RAVI's user avatar
  • 1

1
2 3 4 5
224