Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
11,115
questions
0
votes
0
answers
6
views
Alpaca Lora finetune
I'm playing with a pretrained Alpaca repo and would like to know what is the correct way to inform or update the model.
For example, if I ask it:
Can you see
it responds with:
Yes, I can see
. However,...
0
votes
0
answers
15
views
Existence of a "three-point" machine learning model?
I may want to ask if there are studies that exist which utilize a "three-point machine learning model. What I mean by "three-point machine learning model is that it may use several ...
0
votes
0
answers
4
views
How to estimate this variable in an MILP formulation
This is my first question being asked here. I've thought about different methods to do it, but to no avail. I want to estimate a variable that is either 0 or a positive number. Then I want to use this ...
0
votes
0
answers
7
views
Where can I find genuine attempts at Artificial General Intelligence?
I'm a curious researcher attempting to dive into Artificial General Intelligence, which I acknowledge is really poorly defined. After some searching, all I can really find about Artificial General ...
0
votes
0
answers
7
views
NLP: Infer intent of finalising a transaction in a dialogue/chat system
I have been tasked with tacking the following problem and I wanted to ask for different approaches on how to best approach it.
Problem
I am looking to infer the intent of finalising the transaction ...
- 1
0
votes
0
answers
7
views
Time Series with observations to every level of categorical variable
I am trying to predict a time series for each rg (generic data), and since I have many rg, then obviously I can't create a model ...
- 1
0
votes
0
answers
4
views
Is there a way for CTC to output different types of blanks?
I am using a CTC loss for math handwriting recognition in Tensorflow/Keras. The output consists of a sequence of symbol ids, with a spatial relationship between every pair of consecutive symbols. For ...
0
votes
0
answers
15
views
Which ML model should I use to reorder a schedule
I have a list of performers (up to 800 elements) and some requirements about distance, age and other stuff, like:
Performer should have 30 min to change outfit before appearing again in another group
...
- 101
1
vote
0
answers
10
views
Generating artificial training data with encoder and classical algorithm
I would like to know if this idea has been tried before, and if so, where I can find more information about it.
This is an approach to generating artificial training data for segmentation tasks using ...
- 11
0
votes
0
answers
15
views
Selecting an element in a sequence with self-attention networks
I have a doubt on I should set up the following problem:
Data:
My data is a tensor with shape (N, J, F) where N is the batch size, J is the sequence length, and F is the number of features of each ...
0
votes
0
answers
11
views
the accuracy of a random baseline
Hi everyone I am new in machine learning and deep learning field can someone explaining to me, What is the accuracy of a random baseline ?
0
votes
1
answer
32
views
Is there a machine learning model I should look into to predict the effect land topology will have on prevailing wind direction near bodies of water
I'd like to predict the change in wind velocity due to land near bodies of water. Warmer or colder land should change the wind velocity of nearshore breeze. I'd also like to predict wind shadows that ...
- 101
0
votes
1
answer
32
views
How do GPT models go from token probabilities to textual outputs?
Suppose GPT-2 or GPT-3 is trying to generate the next token, and it has a probability distribution (after applying softmax to some output logits) for the different possible next tokens. How does it ...
- 103
0
votes
1
answer
30
views
Single model or multiple models for predicting at each level in a multi-level classification problem
Given a flat structured data with features that can be considered hierarchical, where each feature is at a different level (e.g., Brand at the top level, Product, Color, and Size at different levels), ...
0
votes
0
answers
13
views
Accelerated learning when wrapping layers in a class
I am implementing a VGG-like network using Pytorch 1.13.1 (python=3.7.12) for image classification on the CINIC-10 dataset. The following two implementations turn out to have very different training ...
- 1
0
votes
1
answer
20
views
How many parameters does the vanilla Transformer have?
The original Transformer paper (Vaswani et al; 2017 NeurIPS) describes the model architecture and the hyperparameters in quite some detail, but it misses to provide the exact (or even rough) model ...
- 163
1
vote
2
answers
41
views
How to boost the performance of a single decision tree by adding additional trees?
I have a binary classification task and the data has imbalance issue (99% is negative and 1% is positive). I am able to build a decision tree that is carefully tuned, weighted, and post-pruned. Take ...
- 11
0
votes
0
answers
17
views
how each tree in random forest structured/built?
I'm new to machine learning and I want to use random forest for the problem I have. What I have done so far is I did the 80/20 split of the original data set.
I need to understand what will happen ...
- 1
0
votes
0
answers
7
views
fine tuning open ai model with historical data
i'm trying to understand more about training models and unsure how to approach this problem.
I have a bunch of historical financial data that I would like open ai to use as additional context when ...
- 101
0
votes
1
answer
12
views
moments of weight vectors in Adam
When performing backpropagation with Adam algorithm, are the moment and the second moment of the weight vectors calculated also for the weights in hidden layers?
- 120
0
votes
0
answers
7
views
Feature selection in high-dimensional datasets with sparse features
What are the most effective techniques for feature selection in high-dimensional datasets with sparse features in the field of natural language processing?
- 101
0
votes
0
answers
9
views
Word2Vec Data Leak
I want to train a machine learning model that can determine the sentiment of tweets about different stocks.
To do this I have a dataset, lets call it A. For dataset A about 30% of the data is labelled....
0
votes
1
answer
11
views
Integrating time context in a machine learning model
Basically, what I'm curious about, are there any methods in machine learning to make the model take into account events that happen in real time that affect the data points during that time period. ...
0
votes
0
answers
20
views
General question on the test and train data
Currently I'm working on a Kaggle problem. I have to predict an outcome by the given information. There are few metafiles for training a model with lots of features (>30). However, in the test file ...
- 1
1
vote
1
answer
20
views
Why the label is not explicitly involved in the loss function of skip-gram?
I am recently learning word embedding myself. When learning skip-gram from the paper https://arxiv.org/pdf/1310.4546.pdf[Distributed Representations of Words and Phrases and their Compositionality], I ...
- 11
0
votes
0
answers
12
views
What are the state-of-the-art knowledge distillation methods?
I need to implement some state-of-the-art knowledge distillation (KD) methods to distill dark knowledge of the teacher network to the student network with Pytorch.
I would really appreciated to any ...
- 1
0
votes
0
answers
9
views
Difference between Validation Error on Learning Curve and Validation Error Calculation in Machine Learning Model
I am encountering a problem where the validation error I see on the learning curve of my machine learning model is different from the validation error I calculate using the mean squared error function....
0
votes
0
answers
7
views
How to add query filter to the Nearest Neighbors algorithm?
I have Nearest Neighbors model, built with sklearn sklearn.neighbors.NearestNeighbors, which I use to make content based recommendations.
Sometimes I need to ...
- 101
0
votes
1
answer
20
views
GPT-2 architecture question
I am currently working on a NLP model that compares two comments and determines which one would be more popular. I have already came up with an architecture - it will be based on GPT-2. But now I am ...
0
votes
1
answer
28
views
Understanding correlation - Machine Learning
I am experimenting a project on identifying cancer or not - Binary classification
The dataset has many columns. Here, I added correlation values between few input columns and the target column[cancer/...
- 21
0
votes
1
answer
12
views
Do models of social systems suffer from prediction drift?
Background
I've created a binary classification model that predicts the probability of fraud for a given sample.
The choice of threshold allows me to set how many frauds are captured in the training ...
- 523
0
votes
1
answer
31
views
Why does Logistic Regression perform better than machine learning models in clinical prediction studies
I am developing binary classification models to predict a medical condition in my dataset. My results show that both Logistic Regression and Linear SVM consistently outperformed other ML algorithms (...
- 365
0
votes
0
answers
28
views
How to combine two datasets to create training data
I am working on a problem where I have say dataset 1 (D1) and dataset 2 (D2), now D1 has three features F1, F2, F3, and D2 has four features F1, F2, F3, and F4. The number of samples in D1 is 7000 and ...
3
votes
1
answer
35
views
How can someone build a dataset for a "propensity to purchase" model?
Ok, this might seem a trivial question for some and it's not even a question, more like a discussion. I read the rules and I believe it's everything fine, so I'm gonna take my chances...
Here's the ...
- 31
0
votes
0
answers
9
views
Practical limits of backpropagation
I have been performing a few neural network algorithms to perform linear regression.
First, I tried SGD. It takes around 20000 epochs to converge, but loss (MSE) is around 0.001.
(To note that perfect ...
- 120
0
votes
0
answers
10
views
Why is ReLU so effective?
The question title is pretty self-explicative.
I'm pretty new to ML and I can't figure out how a function that returns x if x > 0 else 0 can be so good as activation function
- 120
1
vote
3
answers
83
views
+50
mapping similar strings to same number values
I need a way to map strings to a numeric space, where the mapping moves similar strings to the same number.
For example:
in
...
- 61
0
votes
0
answers
9
views
Problem of constant shift in prediction for neural network regression model with gradient-domain loss function
I'm training a regression model using neural network which is trained on MSE of both output and spatial gradient of output. With some simplification, the model is:
$$
y = f(\mathbf{x};\theta)
$$
where ...
- 1
1
vote
1
answer
12
views
Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?
I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see.
We have the following kind of ...
0
votes
0
answers
24
views
Differential equations, real time measurements of variables, and ML
TL;DR: We measure variable $x$ every $10$ minutes, solve a differential equation $\frac{\mathrm{d}y}{\mathrm{d}t}$ where $y=f(x)$. We are interested in the time it takes for the cumulative value of $y$...
1
vote
2
answers
21
views
How can i implement an confucion matrix?
im trying to do a research but i need to make a confusion matrix how can i do that on this model?
https://www.kaggle.com/code/stpeteishii/race-classify-densenet201
Sorry im so so new to everything.
- 11
1
vote
1
answer
31
views
Best ML models for long term time series forecast
I have a project to make a long term prediction (like 5 years) of electricity production by types of power plants (solor, wind, coal, nuclear etc.).
I have access to time series data in MW [megawatts] ...
- 11
0
votes
0
answers
8
views
How to generate synthetic feature instead of synthetic image using Diffusion-based generative Model?
Is it possible to pass some extracted features from the pre-trained ResNet model to a diffusion model for training and further generate synthetic features instead of images like GAN or VAE?
P.S. I ...
- 301
0
votes
0
answers
12
views
Siamese Neural network inputs
A currently task involves the classification of bacteria as antibiotic susceptible and antibiotic resistant.
I have 4 data sets: treated resistant, treated susceptible, untreated resistant and ...
0
votes
1
answer
17
views
what is static objective function
Paper on Adam mentions Stationary Objective function. I am not able to find its definition on Internet (or may be it's there with some other name and I am not able to figure it out). I will be ...
- 203
0
votes
0
answers
5
views
Which is the best choice for evaluating models on small and unbalanced textual datasets?
We are dealing with a small multilabel dataset (around 15k samples) of texts that is imbalanced. Some classes have more than 4k samples and others have around 700 samples. We are using a classifier ...
- 139
0
votes
1
answer
29
views
Are some weight gradients equal?
I want to create a 3 layers neural network from scratch to perform linear regression.
The first and the second layer have 2 neurons, and the last layer has one neuron.
Feature vector x is divided into ...
- 120
0
votes
1
answer
16
views
How to split a single feature vector into a layer of 2 neurons
Given an array x = [1, 2, 3, ...] , I want to split each sample x[i] into 2 neurons.
My idea was to initialize a variable ...
- 120
1
vote
0
answers
23
views
SVM, Is the slack value always equal to the alpha value for points within the margin?
I'm preparing for an exam and I got stuck on this question.
I understand that the alpha values 'affects' how much influence corresponding data point has on the position of the decision boundary, and ...
- 11
0
votes
0
answers
23
views
Workflow for improving and comparing deep learning models
Say I have the most basic neural network that performs some task (eg Keras sequential model with one hidden layer, used to binary classification) and a list of ideas how one could improve it (like: ...
- 103