Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
11,490
questions
0
votes
0
answers
15
views
Whats a suitable feature selection method for Time series data across multiple files?
My problem is basically a higher dimensional regression, where my input is (100 levels, 300 timesteps, 23 features)
My goal is to build a deep learning LSTM model that finds which level the data ...
0
votes
0
answers
11
views
Can transfer learning on VGG better align with Across the Spider-Verse styles for improved style transfer?
Our project is working on implementing style transfer using styles from Across the Spider-verse. We've gotten some good outputs so far, but were wondering if we could better align the VGG model to the ...
- 1
0
votes
0
answers
10
views
Remove feature from kernel matrix
I am pretty new to machine learning so please bear with me :)
I am trying to do a binary classification task using an SVM with precomuted kernel (in python using sklearn).
I created my train kernel ...
- 1
0
votes
0
answers
18
views
The machine learning of identifying the inner workings of a lift
There are a few lifts that go up and down vertically. This system has its own inner workings so that it decides which lift to use at a specific level according to which lifts are being operated and ...
- 237
0
votes
0
answers
38
views
Determining VCdim for union of subspaces $H_i$ - short question
Consider $\mathcal{H} = \mathcal{H}_1 \cup \mathcal{H}_2 \cup \mathcal{H}_3$, where:
$\mathcal{H_1} = \{h_{a} : \mathbb{R} \rightarrow \{0,1\} \ | \ h_{a}(x) = 1_{[x \geq a]}(x) = 1_{[a, +\infty)}(x), ...
0
votes
0
answers
9
views
Help with Memory Usage Estimation of Java Threads
I am a data science beginner and I am looking for some directions regarding my problem. I hope it is ok if I ask my question here.
My data set consists of the overall memory usage of a JVM every 30s ...
- 101
1
vote
1
answer
42
views
Fitting users' reports with joint time-semantic model
I have a historical list of reports, made by users, containing what happened (taken from a list) and the time when the report was filed. And I would like to fit the data with some joint time-semantics ...
0
votes
0
answers
11
views
PyTorch YoloV5 model does not recognize objects
I am trying to build a yolov5 object detection model using parts of a model car. I took 200 pictures of each part, annotated them and trained the model for 50 epochs. Now, the model does not recognize ...
- 1
1
vote
0
answers
45
views
How to deal with a dataset with low feature importance?
I have a dataset whose feature importance lists descendingly as 0.25, 0.16, 0.11, 0.08, 0.05, followed by many features with 0 importance, and the importance drops dramatically. Although the total ...
- 41
1
vote
1
answer
27
views
Should I standardise time series data for deep learning classification?
Say I have time series data for classifying stars using deep learning based on stellar variability, with each time series data measuring the flux of the star overtime. For each star, I have the data ...
0
votes
1
answer
41
views
What type of machine learning am I looking for with these column types?
I have been learning a bit about machine learning and have used a few model types (xgboost, LogisticRegression) with some test data. The more I use these models the more I realize there is a specific ...
- 103
0
votes
0
answers
17
views
How do we set daterange in Sailpoint IIQ
I'm trying to set the daterange of a report dynamically. But it seems as if the package I'm using doesn't implement the interface Serializable as it is supposed to. Do you have any idea how I can ...
0
votes
0
answers
15
views
Right Cross Validation Implementation (Regression)
I am very new to machine learning and i am starting to work my way up. I have made an implementation for cross validation which will be used with ensemble models later. I have made a pipeline in ...
- 1
0
votes
0
answers
37
views
Forecasting Resource Depletion in a Distributed System
I manage a distributed system where each node contains six interchangeable resource slots, sourced from a diverse pool of resource types. Each type has a finite number of units, which get consumed ...
- 101
1
vote
1
answer
82
views
What do the terms in this equation $$θ_{1}x_{1}+θ_{2}x_{2}+θ_{0}=0$$ represent?
I am currently learning ML from a course by MIT. I am from a non tech background and unable to understand some equations shown in this lecture of Linear Classifiers:
https://youtu.be/yOKDzd73KgM?t=57
...
- 11
1
vote
1
answer
14
views
Model evaluation approach allowing manual experimentation without data leakage
In supervised machine learning, are there any evaluation approaches beside using a fixed holdout test dataset, which allow me as a scientist to manually compare preprocessing approaches, without ...
- 111
1
vote
1
answer
43
views
Data binning for interval data
I am trying to create a ML model for salary classification into 5 categories (0-90k, 90-120k, 120-180k and so on).
The problem is that in my dataset almost all salary data is presented in intervals. ...
- 13
2
votes
1
answer
60
views
Custom loss function in python
I am trying to implement a custom loss function inspired by https://arxiv.org/pdf/2305.10464.pdf. That is:
$ L(\mathbf{x}) = (1-y) \left\lVert \mathbf{x_{true} - \mathbf{x_{pred}}} \right\rVert^2 + y \...
- 21
0
votes
0
answers
34
views
How can i retrieve incorrect labelled predictions on new unseen data on image classification when i want to retrain my model later on?
I've recently made a binary image classification model with transfer learning. The model is used on an api and the predictions gets saved into a database. The problem is that it predicts images ...
1
vote
0
answers
20
views
How to derive at the expectation equation given in the paper "Video Diffusion Models"?
In the paper Video Diffusion Models, Section 3.1 mentions the following equation:
$$ E_q[x^b|,z_t,x^a] = E_q[x^b|z_t] + (\frac{{\sigma}_t^2}{{\alpha}_t})\nabla_{z_t^b}\log q(x^a|z_t)$$, where $x^a, x^...
- 21
0
votes
0
answers
20
views
training data of stable cascade vs stable diffusion
So according to this: https://waxy.org/2022/08/exploring-12-million-of-the-images-used-to-train-stable-diffusions-image-generator/
Stable Diffusion was trained on data from CommonCrawler. I believe ...
- 191
0
votes
0
answers
23
views
Question regarding model inference consistency
I am doing some research on gaze estimation model L2CS provided in here.
When you look at the inference predictions on provided image, when face/eye direction barely changes, inference predictions ...
- 1
0
votes
1
answer
27
views
How to train a recommender based on user_features, item_features and likes?
It is my first time dealing with recommender systems, so I don't understand which algorithm should I use. I am given dataset with columns "user_id", "item_id", "like" (0 ...
- 1
2
votes
1
answer
26
views
Text Classification with unlimited labels, Text Extraction?
I'm looking to use ML to read in a blob of text, and extract a name from that text blob. (The blob is from an OCR result from an iPhone)
The text blob varies in size, but the name is always present in ...
- 121
1
vote
0
answers
17
views
saving ML models with pickle to be deployed using Flask
I trained some ensemble Ml to predict, I needed to save with pickle so as to be able to deploy using Flask. To save with pickle I have tried several methods and read several articles but could not get ...
1
vote
0
answers
27
views
Recreating results from Research Paper
so I have been trying to recreate the results from this particular paper (Neural Collaborative Filtering).
The dataset I use closely resembles this .
I understand that I should my data into train and ...
- 11
0
votes
0
answers
31
views
Why the f1 score on validation dataset significantly higher than f1 score on testing dataset?
I'm using a TensorFlow model that look likes this:
...
- 1
0
votes
1
answer
17
views
Struggling with normalization/Standardisation for machine learning dataset
Sorry for what is probably a very obvious/rookie question. I'm currently doing a data science module for my degree and making very slow progress with the work.
The case study i'm doing is around HR ...
0
votes
1
answer
66
views
Build a Neural Network for Multi-output Regression
I have a network model that accepts about 25 inputs and outputs 3 actions.
The outputs are: delta X and delta Y of the robot and the angle of the robot.
After I enter the data into the model, I get ...
- 1
0
votes
0
answers
22
views
stabilityai/stable-cascade vs runwayml/stable-diffusion-v1-5
What are the major differences in these text-to-image AI models:
stabilityai/stable-cascade
runwayml/stable-diffusion-v1-5
in terms of architecture and performance?
- 191
0
votes
0
answers
19
views
Model improvement
Whenever I try to solve some ML problem I get stuck on the first model I choose. I understand the bias-variance tradeoff, but I think it is not the only way to debug a model. Are there any tools to ...
- 101
0
votes
0
answers
10
views
Patterns in weights of trained model?
Apologies for a naive question. Let's say I am training a simple feed-forward neural network using stochastic gradient descent with a fixed architecture, learning rate, number of training epochs, and ...
- 101
0
votes
0
answers
7
views
Variational AutoEncoder using a matrix as dataset with a time component or ...?
The project is to repair data/smoothing.
I would like to use a matrix of shape (5,13,3922):
5 differents types
13 features, one of them is the Times series, dates in format
yyyy-mm-dd that I turned ...
- 1
2
votes
2
answers
73
views
Autoencoders are fitting anomalies too good
I have a set of ~ 5000 greyscale images with resolution of 64x128. I want to do an unsupervised anomaly detection. As a first try, I chose convolutional autoencoders (AE) and trained an AE model. I ...
- 21
1
vote
0
answers
31
views
What are tools and lesson do I need to make an AI drone?
Recently, I've been exploring AI drones online, and I've conceptualized an exciting idea: creating an advanced automated AI drone. This drone would be capable of flight controlled through text ...
- 21
0
votes
0
answers
46
views
When is sampling bias acceptable?
Overview: Dataset is small and a bit messy and the task is to classify 5 classes wherein the targets are ordinal.
Feature Engineering and Selection, Model Tuning, etc. did not produce acceptable ...
1
vote
1
answer
28
views
How can a citation dataset (like Cora) have strongly connected components?
This website https://snap.stanford.edu/data/cit-HepPh.html shows that the High-energy physics citation network has strongly connected components and it's driving me crazy.
A SCC would mean that you ...
- 111
0
votes
0
answers
31
views
Extremely Imbalanced and Gapped Dataset in Regression Problem
Currently I am working with a biological dataset with a range of 0-to-1 to do a multi-task regression with Deep Learning. However, this dataset has an empty gap in the range 0 to 0.2 (however there ...
0
votes
0
answers
28
views
Is balancing imbalanced validation set for retraining model after hyperparameter tuning required?
The following are basic steps to modelling, but would like to ask in the case of imbalanced data, is balancing of train dataset required when retraining model on train + validation set after ...
1
vote
0
answers
7
views
Is GroupKFold needed if some samples have some of their feature values equal?
I am given a dataset $D$ of 10k enzyme-substrate complexes having a lock-key relationship, with each sample (complex) being characterized by enzyme features $x_e$ and substrate features $x_s$. That is,...
- 191
0
votes
0
answers
19
views
Clean CSV file using ML
I would like to build a Neural Network to clean a CSV file that contains wrong delimeters.
My approach is the following:
Identify rows that have more columns than the header. Define these rows as &...
1
vote
1
answer
24
views
Need inputs on logging Machine Learning models and their versions in logs for my application
So i have a web application that recommends movies based on the subject and question that the user puts in , it also uses NLP and ML models like Named Entity Recognition(NER) model to extract keywords ...
- 11
0
votes
0
answers
25
views
Adaptive Lasso Coefficient Weights
I'm trying to understand how the Adaptive part of Adaptive Lasso works. I understand that theoretically, the weights for zero coefficients are inflated to infinity. But can someone explain this ...
1
vote
1
answer
18
views
Call volume prediction using LSTM and GRU
Machine Learning call volume prediction using LSTM and GRU
I am trying to predict the number of incoming calls using LSTM and GRU I have done all the data preprocessing but upon training the model I ...
1
vote
1
answer
30
views
Best modelling method when target is a distribution
I have a regression task where each data sample is annotated by multiple (5-10) experts. I observe that the annotated target of each data sample is a Gaussian distribution.
Usually, people will use ...
- 11
0
votes
0
answers
23
views
Question about the limitations of regularization
I am training a neural network which is overfitting. Even when I increase the number of parameters, the test lost plateaus while the training loss keeps decreasing. Can regularization (like an L1 or ...
0
votes
0
answers
33
views
Multilabel Classification - Flat Binary Classifiers vs Hierarchical Binary Classifiers
Was researching on multi label classification to solve the problem of tagging news articles with topics and countries, where tags follow the syntax <topic>-<country>, and would like to ...
0
votes
0
answers
14
views
0
votes
0
answers
6
views
How to Identify Equipment Churn from Laboratory Service Records Without Direct Churn Labels?
I'm analyzing a dataset encompassing 20 years of laboratory equipment service records, which includes the equipment ID, service dates, types of equipment (HOOD_TYPE), and descriptions of performed ...
- 1
0
votes
0
answers
8
views
Resources for writing CNN for semantic segmentation
I am intermediate/advanced in Python and new to machine learning. Most of what I know about deep learning I learned through Deep Learning with Python by François Chollet. I am trying to do image ...
