Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

predict if news article belong to specific category or not?

I am still new to machine learning. I am trying to build an ML model to predict if an article belongs to a category or not. for example, I have three categories : [war, politics, and crime]. I choose ...
user158789's user avatar
1 vote
1 answer
85 views

Is this the correct way to calculate word embeddings using Roberta?

I'm trying to write a program that using Roberta to calculate word embeddings: ...
user avatar
0 votes
1 answer
29 views

Recommendation system NLP ideas

The problem: If we have a clustering problem with lets say x groups. And each group has a document describing it, lets say 3 pages. Then we have n observations each with a smaller piece of text ...
Dylan Dijk's user avatar
0 votes
0 answers
11 views

Modify data from generator before sending to discriminator

I am trying to build a GAN, which will take in datasets of size (1, 1800) and generate me similar looking 1-D arrays. But, I want the generator to output an array of size (1, 400), which will be sent ...
Lucifer Williams's user avatar
1 vote
1 answer
60 views

Should I remove outliers before combining the dataset or after combining the dataset?

I am doing some exploratory data analysis on a custom dataset. I have 3 different data-frames: df1 belonging to class 0. df2 belonging to class 1. df3 belonging to class 2. I am doing k-means ...
Harshvardhan Uppaluru's user avatar
0 votes
1 answer
66 views

Extraction of label names from images

I have images of forms & I need to extract label names from the images like Name, Email, Address, Phone Number, etc. that require user input. I'm looking for approaches & tools that can help ...
Apoorva's user avatar
  • 287
0 votes
1 answer
26 views

Where can I get 5000+ classified images of zoo animals? [closed]

please help! We are college students doing this for a project. The project is using neural networks and want to build a model that takes in an input of a colored image of an animal and outputs the ...
user90061's user avatar
13 votes
2 answers
598 views

Why does data science see class imbalance as a problem for supervised learning when statistics does not?

Why does data science see class imbalance as a problem in supervised learning when statistics says it is not? Data science seems to seem class imbalance as problematic and needing special techniques ...
Dave's user avatar
  • 3,833
0 votes
0 answers
19 views

NLP Project on Multiclassification

I am fresher and i am working on multi classification project in my organization i am unable to get good accuracy… the project is basically a email classification into 60+ teams… But the input data is ...
Abhishek Khasre's user avatar
1 vote
0 answers
10 views

Spatio"temporal" Processing to extract Peak Positions in Data

I am working on an experimental setup that produces 2D widefield image data for each frequency in a frequency sweep. The resulting data has the Form MxNxFxC with M and N being the image pixels, F ...
Swoopoo's user avatar
  • 11
1 vote
1 answer
48 views

How to normalize the features without the knowledge of the min and max values in online learning?

I am developing an online learning platform where input features are gathered from various sensors. However, these features may have vastly different ranges. For example, displacement values may be ...
JJbow's user avatar
  • 15
0 votes
0 answers
25 views

Looking for a Statistical Modelling Technique for a Credibility Scoring Model

I’m in the process of developing a model that assigns a credibility score to fatigue reports within an organization. Employees can report feeling “tired” an unlimited number of times throughout the ...
Shibaprasad's user avatar
1 vote
1 answer
153 views

Understanding the concepts of word embedding in GPT-2

I have a program that calculate the word embedding using GPT-2 specifically the GPT2Model class: ...
user avatar
0 votes
0 answers
28 views

PR AUC curve with drop in precision

I have this PR AUC plot, with both PCA and autoencoder related curves having a huge drop of precision in the beginning and then increasing again, with PCA hitting 0 as you can see in the zoomed in ...
GabrielPast's user avatar
0 votes
1 answer
28 views

Binary Classification Dataset Distribution

I have a question regarding the distribution of my dataset. I have a binary classification task which I need to label. I now want to know how I should distribute the labels for optimal training / ...
Apatus's user avatar
  • 1
2 votes
2 answers
52 views

Handling Month-over-Month data in Regression Model

I have data similar to what you see in the picture. I want to use a RandomForest Regression model where I can use fields (excluding MONTH_END_DT and LOCATION_ID) to predict REVENUE_PER_UNIT. The idea/...
Larry Burholme's user avatar
3 votes
0 answers
37 views

How to predict multi-variate time-series from different samples [closed]

I'm having issues seeing the best way to predict a time-series when training on a dataset with different samples. I have a dataset that shows the weight of 10 rabbits from their first day to their ...
scootjow's user avatar
1 vote
0 answers
26 views

Learning curve dip after plateau when adding more samples

I have a questions regarding my keras machine learning model. Context: I am working on elementary particle physics, specifically LHC related data. I am training a regression model of 4 Dense layers ...
helton_arruda's user avatar
0 votes
1 answer
59 views

Effect on regression coefficients by multiplying a constant to a feature

I was solving one quiz question on Coursera and I found an interesting question. If you double the value of a given feature (i.e. a specific column of the feature matrix), what happens to the least-...
teddcp's user avatar
  • 165
0 votes
0 answers
12 views

Apriori algorithm formula: difference between support and frequency

What is the correct way to calculate support? I have seen two different ways and I'm confused as a result. One way (say first approach) is explained https://en.wikipedia.org/wiki/...
Srishti M's user avatar
  • 471
0 votes
2 answers
43 views

How can I use a model trained with batches to make predictions with single sample?

I'm training a PyTorch model with batches of 128 images, and after going through multiple convolutions, they're flattened (with .flatten) before being passed to a ...
Jake's user avatar
  • 1
2 votes
0 answers
47 views

Discretization of Multiple Time Series

I'm working on discretizing multiple time series for a project. Here's what I've done so far: I concatenated the train signals like this: [1,2,3,5] and [7,3,6,7] into [1,2,3,5,7,3,6,7]. Then, I ...
Nathaldien's user avatar
1 vote
1 answer
85 views

Are Energy-Based Models actually used in real data science jobs?

I have seen several algorithms used in data science job, but I haven’t heard so much about Energy-based models. Do they actually find application in companies and in real jobs in data science besides ...
J. Drawman's user avatar
0 votes
1 answer
80 views

Negative Log Likelihood of a Gaussian Model can be negative?

I am confused about the negative log likelihood of a guassian model: Can the negative likelihood of a gaussian model be negative ? lets suppose that the variance is going to 0 faster than the MSE ...
yoshcn's user avatar
  • 1
0 votes
0 answers
59 views

Decision tree for selecting tests to monitor ML drift?

I'm exploring concepts for monitoring drift and I have come up with the following decision tree: Criteria Decision Nature of Data Distribution KS test is suitable for comparing continuous ...
Chris Snow's user avatar
0 votes
2 answers
147 views

Email Parsing using Machine Learning

I am new to machine learning. I have a project in which i need to extract some data from email by parsing email using machine learning and would really appreciate if you could guide me with that. I ...
Cupcake's user avatar
0 votes
0 answers
55 views

How to find proper context in open book question answering?

I want to make an Open Book Question Answering / Retrieval Augmented Generation system. The major concern here is the proper context selection. There are some fundamental issues related to this. For ...
hafiz031's user avatar
  • 121
0 votes
0 answers
50 views

In MLP, multiple classes, using batches, For update weights, Would I have to calculate the accumulated error(of all samples) of each output neuron?

In Multilayer Perceptron neural networks, I know that there are two types of training: online training, and batch training, which consists of dividing the samples and updating the weights using the ...
will The J's user avatar
0 votes
0 answers
27 views

How decision trees handle category type features when splitting nodes?

If the tree is a binary tree, it will consider all possible combinations of ways to divide the dataset into two subsets based on that categorical feature. Or it simply selects the features with the ...
Truman's user avatar
  • 1
0 votes
0 answers
8 views

can't get higher Training Accuracy with Semi-Supervised Learning

I am currently training my model using a combination of labeled and unlabeled samples in the realm of semi-supervised learning. However, as I evaluate the model's training accuracy on labeled samples ...
phantrang's user avatar
0 votes
2 answers
116 views

How to detect and predict sensor faults and failures (for weather stations to be specific)?

Need help. Especially those knowledgeable in weather systems/meteorology. Best approach in detecting and predicting faulty weather sensors and their failures based on their readings alone? I'm doing a ...
noob101's user avatar
0 votes
0 answers
62 views

Calculating correlation between embedding features

I am looking for a technique to calculate correlation for embedding features (array of floats). I'm interested in the correlation between features (embedding-embedding) as well as between feature and ...
Drew Serles's user avatar
0 votes
0 answers
28 views

Forecast proportions through time

I want to predict/forecast the proportion of positives (sample testing positive) over a 1-3-6 month period. My data has a lot of negative sample tests, therefore, it's aggregated by month. Let's ...
Alejandro L's user avatar
1 vote
0 answers
25 views

Time-series Forecasting model for License monitoring

I am trying to build a forecasting model to predict the number of used licenses for an application-feature combination in the future. The data (time-series data) in which at a point in time, the '...
Sherwin R's user avatar
1 vote
2 answers
89 views

Why does undersampling before cross-validation lead to leakage?

I came across the paper "Leakage and the Reproducibility Crisis in ML-based Science" by Sayash Kapoor and Arvind Narayanan, wherein the authors argue that both over- and under-sampling the ...
Viades's user avatar
  • 13
1 vote
1 answer
22 views

TensorFlow LSTM model with lower epoch loss, but higher average RMSE. How/why?

I am very perplexed by the lower loss but higher RMSE: Here's a newer model with better loss scores on the same dataset and many predictors: ...
user2205916's user avatar
0 votes
1 answer
42 views

Is Machine Reading Comprehension (MRC) outdated?

I recently went through some litterature about knowledge-enhanced language models and found connections with the Machine Reading Comprehension (MRC) task. However, I couldn't find papers more recent ...
Barbara Gendron's user avatar
1 vote
0 answers
20 views

How to force my model heads to learn different things?

I have an Seq2Seq model that has 2 generative LM heads. I want the two heads to focus on different features/styles while decoding. The approach that I was thinking of is adding a distance cost to the ...
Tathagato Roy's user avatar
1 vote
1 answer
52 views

Whats the advantage of single target neural network over multi target neural network?

So right now I have made 2 neural networks to predict x and y coordinates separately. One for x, and one for y. I'm looking for a reason to backup my assignment. I have search for this and most of the ...
Secondary Juggernaut's user avatar
0 votes
0 answers
43 views

Unsupervised Learning with Time Series data?

I'm working with data which has daily aggregated user actvity over the course of several months (a user doesn't have to make an activity every day). I'm looking for a way to cluster similar users ...
user6132211's user avatar
0 votes
1 answer
30 views

TSNE plots of random data subsets are vastly different but labels are still clearly separated - what conclusions can we draw about the dataset?

I scraped a dataset of match data in a video game and labeled them according to their outcome (0 for loss, 1 for win). I wanted to see if there was actually any inherent relationship between the ...
Lilian Shi's user avatar
1 vote
0 answers
17 views

Averaging Weights of Identical LSTM Models for a Unified Global Model

I'm currently working on a project where I have several pre-trained LSTM models, all with the same architecture. My goal is to combine these models into a single global model by averaging their ...
albi_z8's user avatar
  • 11
0 votes
0 answers
71 views

How to do feature selection correctly in xgboost for time series forecasting after obtaining a good predictive model?

I have a very large dataset (~7 million rows) for which I have extracted ~500 features during feature engineering phase. I have trained an XGBoost which has a fairly good predictive capability (based ...
guestar's user avatar
0 votes
0 answers
9 views

I'm trying to build image search like Google Photo-Image with face is given to model & it'll get all the images in database in which he/she is present

When a user upload a selfie, the model search same person in dataset of images of multiple persons and get back all the images in which that person is present. Step 1: From dataset of images I detect ...
BKP's user avatar
  • 1
3 votes
1 answer
201 views

Pandas Profiling Not Working

I have been working on EDA lately and i got to know about Pandas Profiling , but i am unable to import the module Pandas_Profiling , I could import the module on Google Collab but couldnt import the ...
Sahil Rathout's user avatar
0 votes
0 answers
21 views

LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values

I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
dfried's user avatar
  • 101
0 votes
0 answers
47 views

R [Warning] No further splits with positive gain, best gain: -inf in lightgbm training

I read through some answers, it seems many people face this message before. The answer seems there is no further need to split the tree, so you need to adjust the super parameter to make new splitting....
cloudscomputes's user avatar
1 vote
0 answers
42 views

Occam's factor and the VC dimension

I was watching this lecture by Prof. Dr. Philipp Hennig (Probabilistic ML) and when he reached this formula which is the type two maximum log likelihood I had the following question: The Occam's ...
HAMDI ABDERRAHMENE's user avatar
1 vote
1 answer
128 views

How can I leverage machine learning for log analysis?

I am new to data science and trying to find possibilities of using datascience in tasks. I have a set of logs which I want to convert to json. The logs are more or less of same format and I can write ...
SUNITA GUPTA's user avatar
0 votes
0 answers
48 views

Improving performance of anomaly detection using dataset?

I am leveraging an isolation forest model from the scikit-learn library for anomaly detection in a time series dataset where each point in the dataset is a data frame. However, I possess additional ...
SUNITA GUPTA's user avatar

1
3 4
5
6 7
229