Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
11,311
questions
0
votes
0
answers
9
views
Looking for datasets on automobile parts information for machine learning
I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
- 1
0
votes
0
answers
6
views
Hassle-free platform for a small data science team?
our team of 4 data scientists has been exploring options to automate our ETL, data storage and model updates and are looking for a hassle-free platform that can help with this.
We've had several ...
0
votes
0
answers
12
views
Seeking datasets for training a Language Model on U.S. mortgage loan processes
I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
- 1
1
vote
1
answer
31
views
How does oversampling or undersampling approch is going to help during the testing on real time data?
We have a dataset with class A as 10% only and Class B as 90% . Let say we did undersampling or oversampling on training data and we made 50% of class A and 50% of class B. But in reality the data ...
- 13
0
votes
1
answer
24
views
Test set won't have all the features present in the training set
I have to analyze a cardboard packaging system. To simplify, it consists of three parts, each of which needs to be set up (this depends on the pre-order): the printer, the slotter, and the cutter. The ...
- 1
0
votes
0
answers
7
views
How to selectively train a deep model based on the unavailability of a subset of the feature set
I am creating a deep learning binary classification model. Each sample in the dataset contains two mutually exclusive feature sets X and Y.
Feature set X is present in all samples; however, there are ...
1
vote
1
answer
19
views
Out-of-Range Target Variable in Sequence-based Machine Learning Model
I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the ...
- 11
2
votes
1
answer
1k
views
Why images, audio/video clips, text are regarded as unstructured data?
Tabular data is regarded as structured data, while other data types such as images, audio, video, text are regarded as unstructured data.
I am confused that, taking images as an example, they are just ...
- 23
0
votes
0
answers
6
views
SVM kernel for detecting if a substring appears in some given string
I'm trying to do the exercise in 16.1 in the book Understanding Machine Learning, Ben-David, et al. formulated as follows:
Consider the task of learning to find a sequence of characters
("...
- 155
0
votes
0
answers
15
views
How to estimate the weights for a weighted average if not every weight is used in every example?
Say that you have a data set that contains $N$ examples. For each example, there are three main sections of important information. The first most important piece of information is the name of the ...
0
votes
0
answers
29
views
Some questions from a noob - Chat bot
I'm a T-SQL and .NET developer but I need to understand some basis about a data science / ml project.
I would like to know the guidelines and be directed to the right topics about the steps that need ...
0
votes
0
answers
19
views
Using LLMs for structured data?
I've been trying to work with structured data in language models, and it's proving to be quite challenging. I'm confident that with Langchain, I should be able to solve the problem, but I'm not ...
- 21
0
votes
0
answers
31
views
Mislabeled problem with hospital data
I am writing this post to ask or see if someone can help me with this problem in case you may have faced a similar situation.
My problem has to do with a ML tool that I am trying to develop in a ...
- 1
0
votes
0
answers
56
views
Design a CNN that can detect the face in the given input image
a. What strategy will you use? How many layers? How many filters?
b. Describe the filters for each layer in detail.
c. Show how the convolutions will work and demonstrate that your CNN will be able to ...
- 1
0
votes
1
answer
43
views
Proper way to build time series dataset
I have daily time series but need to predict weekly total.
Here is example:
date
col0
col1
col2
col3
col4
target
0
2023-01-01 00:00:00
-1.021
0.708
-0.564
-0.502
0.244
-106.538
1
2023-01-02 00:00:...
- 1
0
votes
1
answer
34
views
Semantic Search on numeric data
I have a dataset in csv format. Most columns have numbers in them. The data is mostly number based, with little text.
A sample of my dataset:
I want to build a semantic search engine that can answer ...
- 275
0
votes
1
answer
40
views
How to create a bot for a real-time PvP game using machine learning?
Unfortunately, I am not well-versed in machine learning. However, I'm trying to understand if it's possible to create a bot for a real-time PvP game like, for example, Clash Royale or Random Dice: ...
- 101
0
votes
1
answer
64
views
Employee Attrition - Binary Classification - Predicting leavers
I'm trying to wrap my head around what the standard or best approach is for identifying which employees in a business would be most at risk of leaving, given their features and an indicator column ...
0
votes
0
answers
5
views
Clustering high-dimensional data
Every sample in my dataset consists of two components: its $x$ component is a $n\times m$ matrix, and its $y$ component is a $p\times q$ matrix (Feature selection and PCA have been done before). Apart ...
- 1
0
votes
2
answers
36
views
What methods do top Kagglers employ for score gain?
I’m currently taking a course on ML and part of my final grade is my position on a Kaggle competition (private one) regarding a classification task. The majority of groups tend to have a similar ...
0
votes
0
answers
28
views
Feature selection: ANOVA between features vs within a feature
I am currently performing feature selection on a dataset containing continuous and categorical features. The target is a continuous variable.
If I understand properly, ANOVA can be used between ...
- 1
0
votes
0
answers
23
views
Loss values seem to fluctuate, yet the weights are correct
I'm taking my first steps with tensorflow (and in ML in general), and using this piece of code to train a very simple model that tries to find the underlying linear relation: f(x,y) = 4x +7y -2 (+ ...
- 1
0
votes
0
answers
14
views
Data Transformation Pipeline Issue on Flight Fare Prediction End-to-End Machine Learning Project
I am following the process shown on Wine Quality Prediction End-to-End ML Project on Krish Naik's YouTube channel to do a Flight Fare Prediction Project.
I am facing an issue with Data Transformation ...
1
vote
1
answer
45
views
How do I use ML models to estimate current stress level based on past data?
I am new to machine learning and I cannot understand the difference between estimating current stress level and predicting future stress levels based on historical data. I have been told these are two ...
1
vote
1
answer
37
views
Should I apply the same data transformations in production for my classification model's inference steps
I am now moving my best classification model to production and doing tests currently.
Should I use the same scaler() I used in training during my inference in ...
0
votes
0
answers
16
views
How to make a dataset for Audio Speech Recognition of Arabic Language
I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
- 1
0
votes
0
answers
15
views
Best accurate face recognition library
I need to use an accurate library for face recognition in my attendance python project But I used the face_recognition but I could no get the desired result.is there any way to get the 99% accuracy?
I ...
- 1
1
vote
1
answer
42
views
Beginner basic clustering model and one-hot encoding?
I have a dataframe of natural disaster incidents in Afghanistan from 2016 - 2023.
Column names:
REGION (Northern, Eastern etc)
PROV_CODE (province)
PROV_NAME
DIST_CODE (district)
DIST_NAME
INC_DATE (...
- 13
0
votes
0
answers
23
views
Target variable is discrete ranging from 1 to 14, with each value having same proportion in the dataset, ML models fail miserably
I have a dataset of shape (55314,23). The target variable is league_rank. There are exactly 3951 leagues in this dataset, with each club having a ranking from 1 to 14. The variable is discrete, and ...
- 1
0
votes
0
answers
11
views
Open Source Model or dataset for attention map visualization
Helloo (:
I'm trying to develop a python application where users can upload an image e.g. of their website and receive a heatmap or attention map in return. This map should indicate where users are ...
- 1
0
votes
1
answer
28
views
Beginner clustering project, what are the input features and how do I analyze the data?
I am a beginner to data science. I have this dataset on natural disaster events in Afghanistan from 2016 - 2017. Columns:
REGION (ex. North, North West, etc)
PROVINCE_NAME (kind of like US 50 states)
...
- 13
0
votes
0
answers
14
views
What do I need to take into account if I want to test one model on different test datasets?
I'm just new to machine learning, I want to test a generative model on two different test datasets to compare its performance on different datasets. What do I need to take into account about the ...
- 1
0
votes
0
answers
5
views
Oblique Decision Tree HHCART algorithm explaination
I am now studying a paper about implementing oblique decision tree algorithm
https://arxiv.org/abs/1504.03415
I was wondering if any code implementation and alternative explaination, as a support ...
- 149
1
vote
1
answer
16
views
Is it possible to run ML models on SBC boards?
Can I run models like Tortoise TTS or OpenAI Whisper on SBCs like Orange Pi 5 or Odroid N2+?
- 111
0
votes
0
answers
5
views
Data Tracking For Process Improvement
I work in a manufacturing plant with 6 production lines. As part of the maintenance crew, it is my job to record all maintenance work performed on the equipment inside of our shared OneNote. This ...
- 1
0
votes
0
answers
8
views
LGBM handle categorical variable with categories which have quantities less than that of min_data_in_leaf
I am building a LGBM model where categorical features have been encoded using ordinal encoding. The categories get values from number from 1 to a max number that are all consecutive. How does LGBM ...
1
vote
1
answer
30
views
activation=tf.keras.activations.relu vs activation='relu'
Both models are for binary classification problems
Model 1
...
0
votes
1
answer
26
views
Confusion with tensorflow's Sequential Dense Layers
I'm working on a regression probem using Tensorflow, and have created two models with slight differences in their first Dense layer.
The Models
...
0
votes
0
answers
30
views
How to build a regression model with same targets for different observations?
I'm working on a project involving around 90 3D-printed cubic samples with different structures. After conducting a compression test, I obtained stress-strain curves with 700 data points for each ...
- 1
0
votes
0
answers
18
views
What are typical hyperparameter ranges and significance for SB3's offline algorithm?
I'm trying to perform an hyperparameter tuning on a SAC algorithm (Stable Baselines 3) with optuna lib.
I guess hyperparameters have different significance and typical ranges depending of the selected ...
- 1
6
votes
1
answer
544
views
How high of a correlation coefficient of a feature with a target variable is considered too high?
Currently my classification model is doing too well on all of the train, validation, and test datasets. I'm assuming there is a data leakage in the features, and therefore I've computed the ...
- 457
0
votes
0
answers
6
views
Measuring Product Search effectiveness
I want to measure the effectiveness of my search engine, one of the ways i can do that is by measuring the rate at which a customer reformulates the previous query. Hence, I need to quantify inter-...
- 23
1
vote
1
answer
24
views
Which machine learning models are rational to use on NP-hard and NP-complete "theoretical" problems?
Time and time again I run into "surprising" NP-hard problems that seem naturally simpler than they are. I recently worked on a weighted graph theoretical problem where the point is to ...
- 13
0
votes
0
answers
30
views
Optimal ML classification approach
Background: I have an app data (impressions, user activities) that I can use as features for a multiclass classifier (5 classes).
I just want to discuss about some things that our team is having a ...
1
vote
0
answers
16
views
0
votes
1
answer
21
views
One class classification using Exclusively Positive Examples
I have a dataset consisting only of the positive class, and I want to train a model to identify this data. Is it possible to use one-class classification models for this task? Additionally, how can I ...
0
votes
0
answers
9
views
Features' confidence interval
I have some input feature representing a physical quantity that's measured with a given incertitude.
Is there a way to feed the information about the incertitude to, let's say, a regressor using that ...
- 111
0
votes
0
answers
30
views
Machine learning can be broken down into supervised, unsupervised, and reinforcement learning. Is there anything else?
Is it even logically possible to have a type of machine learning that doesn't fall into those three paradigms?
Supervised: a dataset of inputs and outputs are fed to an algorithm which learns a ...
0
votes
3
answers
58
views
Why the one validation score is lower than the other sections of cross validation
I was working on RandomForestClassifier and doing hyperparameter tuning. But something caught my attention. I always get a lower validation value in the 2nd part of Cross Validation. Here is the code:
...
0
votes
0
answers
32
views
Winter Holt's Time series model
I am confused with the Winter Holt's Time series model usage. I use 2 years of data to train and want to predict 3rd-year data.
Note1: I have partial 3rd year's data, but I want to use it to check my ...
- 1