Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

how to tune hyperparameters inn regression neural network

hope you are enjoying good health,i am trying to built a simple neural network which has to predict a shear wave well log values from other well logs,but my model's is stuck in mean absolute error of ...
user avatar
0 votes
0 answers
5 views

seasonality in classification model

I am building a classification model to predict customer status a year from a given time. There seems to be some seasonality, for example, more changes occur in Summer than in Winter etc. so my ...
user avatar
0 votes
1 answer
10 views

Why does hyperparameter tuning occur on validation dataset and not at the very beginning?

Despite doing/using it a few times, I'm still slightly confused by the use of a validation set for hyper parameter tuning. As far as I can tell, I choose a model, train it on training data, assess ...
user avatar
  • 21
0 votes
0 answers
10 views

Optimal representative participation in the age of AI/ML?

Just answered the great questions of the 2022 Stack Overflow Developer Survey and earned the Census - Badge, here I see the number of filled surveys and as an enthusiastic beginner in data science I ...
user avatar
  • 101
0 votes
0 answers
5 views

Finding the tightest (smallest) triangle that fits all points

I'm supposed to find an algorithm that, given a bunch of points on the Euclidean plane, I have to return the tightest (smallest) origin centered upright equilateral triangle that fits all the given ...
user avatar
0 votes
0 answers
8 views

splitting point in LightGBM?

I am not able to understand how the first root node is selected in LightGBM and how the splitting at nodes happens further. I read blogs and related documents and I understand that in this histogram-...
user avatar
  • 27
1 vote
1 answer
19 views

Interpreting cluster variables - raw vs scaled

I already referred these posts here and here. I also posted here but since there is no response, am posting here. Currently, I am working on customer segmentation using their purchase data. So, my ...
user avatar
  • 2,141
0 votes
1 answer
25 views

Dealing with observation with arbitrary number of categories with arbitary number of values

Suppose to have a set of elements $X = \{x_1, x_2, ..., x_n\}$. Each element is characterised by a set of features. The features characterising a particular element $x_i$ can belong to one of $q$ ...
user avatar
  • 111
1 vote
1 answer
21 views

Need term or method name for evaluation of CNN without ground truth using e.g. a regression model

I have the following problem, I have trained a CNN and I can evaluate the network in-sample. I want to use the trained model for the class prediction of images for which I have no ground truth. ...
user avatar
0 votes
0 answers
25 views

Is test data required to be transformed by training data statistics?

I am using a dataset (from literature) to build an MLP and classify real-world samples (from wetlab experiment) using this MLP. The performance of MLP on the literature dataset are well enough. I am ...
user avatar
1 vote
1 answer
19 views

My Linear Regression Model Mean Absolute Error(MAE) is 0.29 and R2 0.20 , Is this a acceptable Model?

My Linear Regression Model Mean Absolute Error(MAE) is 0.29 and R2 0.20 , Is this a acceptable Model ? How can increase the r2 score ?
user avatar
0 votes
0 answers
7 views

Is it beneficial to use a batch size > 1 even when all computing power can be used?

In regards to training a neural network, it is often said that increasing the batch size decreases the network's ability to generalize, as alluded to here. This is due to the fact that training on ...
user avatar
  • 101
0 votes
0 answers
14 views

Create features for each row or only for a specific value

I have a problem. I want to predict when the customer will place another order in how many days if an order comes in. I have already created my target variable ...
user avatar
  • 77
1 vote
1 answer
30 views

What is the Purpose of Feature Selection

I have a small medical dataset (200 samples) that contains only 6 cases of the condition I am trying to predict using machine learning. So far, the dataset is not proving useful for predicting the ...
user avatar
  • 333
0 votes
0 answers
9 views

Recommendations for modelling panel data

sending positive wishes to y'all. I have about 10 years of growth rates in real estate prices and some other macroeconomic variables such as inflation, unemployment rates, fuel prices, growth in ...
user avatar
0 votes
0 answers
18 views

how to reduce overfitting and improve confusion matrix

I am trying to apply the following model on my data which is consists of (4030 samples as 5 classes) each sample is MFCC features which is extracted from an audio clip consisting of (20 second) and I ...
user avatar
  • 27
0 votes
0 answers
8 views

How to assign costs to the confusion matrix

I am trying to assign costs to the confusion matrix. That is, in my problem, a FP does not have the same cost as a FN, so I want to assign to these cases a cost "x" so that the algorithm ...
user avatar
  • 320
0 votes
1 answer
17 views

multi dimensional time series and matrix profile method

I have a time series of the following format: ...
user avatar
0 votes
1 answer
32 views

What are the requirements for a word list to be used for Bayesian inference?

Intro I need an input file of 5 letter English words to train my Bayesian model to infer the stochastic dependency between each position. For instance, is the probability of a letter at the position 5 ...
user avatar
  • 3
1 vote
1 answer
15 views

How to analyse the accuracy and loss graphs of model history?

I want to understand how to analyse the loss and accuracy (any metric) graphs that we plot from the model training history. Here's my graph, What can we say from the slope of graph? Does it matter? ...
user avatar
3 votes
1 answer
104 views

Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?

I'm using an XGBoost model for multi-class classification and is looking at feature importance by using SHAP values. I'm curious if multicollinarity is a problem for the interpretation of the SHAP ...
user avatar
0 votes
0 answers
10 views

Unable to generate useful insights on a highly cardinal data

I'm working on CRM data, did some cleaning, encoding and ran a decision tree classifier from which i plotted a feature_importance graph From that I found that Sales person column is one of the ...
user avatar
1 vote
0 answers
32 views

Reverse engineer PII sensitive data from Inceptionv3 pre-trained model generated features

I'm using the pre-trained Inceptionv3 to build out features from proprietary documents. Some of these documents contain sensitive PII data. I use the 2K output from the second last layer as the ...
user avatar
  • 11
0 votes
1 answer
20 views

Feature creation ideas for propensity models?

I'm working on a propensity model, predicting whether customers would buy or not. While doing exploratory data analysis, I found that customers have a buying pattern. Most customers repeat the ...
user avatar
  • 109
0 votes
0 answers
13 views

matrix profile distance measure characterization

If there are various types of distances measures for time series, such as Euclidean, DTW, and shape-based ones, how can we characterize the matrix profile distance measure? Profiling one?
user avatar
0 votes
0 answers
6 views

SHAP values interpretation for clasification

I'm trying to understand how SHAP values are calculated for Classification. As far as I understand for each feature the SHAP values are calculated by: $$ \phi_i = \sum_{S \subseteq F \setminus {i}} \...
user avatar
  • 1
-1 votes
0 answers
7 views

Check weather the vehicle is moving forward or backward using yolo

I want to detect weather the car is moving forward or backward in the video, i am able to detect the bounding box on the car, now i want to do the post processing and tell that this car is moving ...
user avatar
1 vote
1 answer
12 views

reuse of LDA model for new data

I am working with the LDA (Latent Dirichlet Allocation) model from sklearn and I have a question about reusing the model I have. After training my model with data how do I use it to make a prediction ...
user avatar
0 votes
0 answers
11 views

A toolbox with code for context free grammar verificiation for logical formulas

I work with Linear Temporal Logic (LTL) formulas. I am looking for a toolbox to verify if an LTL formula can satisfy a given Context-Free Grammar (CFG). For example, we have the LTL formula G(x_1>0)...
user avatar
  • 1
0 votes
0 answers
6 views

Applying filters to custom objects in an image

I would like to create an application that adds image filters (Snapchat-style) to photos of cats or chairs (just for the sake of this question). In order to do that properly, I thought of using Active ...
user avatar
  • 101
0 votes
0 answers
14 views

Imbalanced classification

I've tried all kind of oversampling undersampling techniques and I've tried also weighted Xgboost ( the model I'm trying to improve) I couldn't surpass a very Bad F1 score : 0.09 What should I do
user avatar
0 votes
0 answers
30 views

Find how the properties of an entity affect a certain property of its surrounding

We have a set of things (physical entities): ($A_1$, $A_2$, $A_3$, $A_4$,...). Each of those has certain attributes that we can measure at time $t = 0$. Let $B_i$ represent those attributes, so each ...
user avatar
0 votes
0 answers
12 views

ValueError: continuous is not supported

I am working on a regression problem and building a model using Random Forest Regressor but while trying to get the accuracy I am getting ValueError: continuous is not supported. ...
user avatar
1 vote
1 answer
34 views

Time Series Clustering on sales data -- any ideas?

I have a retail store dataset, and I am interested to do some time series clustering on this data, what idea you find interesting for this purpose? I have so far: What sales trends there are across ...
user avatar
0 votes
0 answers
15 views

Make fitted xgboost or linear regression model predicts values in thé future

I have a machine learning model that I fitted with xgboost and linear regression. My dataset has thirteen features and has price as the target. Is there any way to ...
user avatar
0 votes
0 answers
11 views

Batch normalization

Part 1 Im going through this article and wanted to try and calculate a forward and backward pass with batch normalization. When doing the steps after the first layer I get a batch norm output that are ...
user avatar
  • 1
1 vote
1 answer
29 views

Understanding SGD for Binary Cross-Entropy loss

I'm trying to describe mathematically how stochastic gradient descent could be used to minimize the binary cross entropy loss. The typical description of SGD is that I can find online is: $\theta = \...
user avatar
  • 13
3 votes
1 answer
75 views

How are parameters selected in cross-validation?

Suppose I'm training a linear regression model using k-fold cross-validation. I'm training K times each time with a different training and test data set. So each time I train, I get different ...
user avatar
  • 109
0 votes
1 answer
21 views

Feature importance of a linear regression

What is the easiest and easy to explain feature importance calculation for linear regression? I know I can use Shap to compute feature importance, but I find it difficult to explain it to stakeholders,...
user avatar
  • 109
1 vote
0 answers
10 views

In which way GAN generator transforms the data(for transforming a noise to the data)?

I have the problem: I understood how GAN works in general, but I need information how it work detailed. The part I don't understand is how the random noise at input is transformed to data on the ...
user avatar
  • 11
0 votes
0 answers
12 views

Predicting a signal based on other signals

I want to predict a signal based on other related signals, how would I go about doing this? My current approach is to do some feature extraction (in the time and frequency domain) on both the ground ...
user avatar
1 vote
1 answer
15 views

Combining two separate confusion matrix results from two seperate machine learning model to overall increase the True Positive accuracy

What are the steps involved if it is possible to add two confusion matrix results together to get a better final prediction. we have calculated two confusion matrixs as follows from naive bayes and ...
user avatar
0 votes
1 answer
15 views

For feature selection, do we use Chi-squared with Mutual Information together?

Or do we only choose one out of two for categorical data.
user avatar
  • 1
0 votes
1 answer
22 views

Using Sci-Kit Learn Clustering and/or Random-Forest Classification on String Data with Multiple Sub-Classifications

I have a set of data with some numerical features and some string data. The string data is essentially a set of classes that are not inherently related. For example: ...
user avatar
0 votes
0 answers
13 views

I need to plot only training curve in the fastai library using the learner.recorder.plot_losses() function . FASTAI devs pls help

I have a task where I need to only plot the training loss and not the validation loss of the plot_losses function in the fastai library with learner object having ...
user avatar
1 vote
1 answer
44 views

Derivative of MSE Cost Function

The gradient descent: $\theta_{t+1}=\theta_t-a\frac{\partial}{\partial \theta_j}J(\theta)$ But specifically about $J$ cost function (Mean Squared Error) partial derivative: Consider that: $h_\theta(x)=...
user avatar
2 votes
3 answers
63 views

Entity Embeddings of email address

I have a set of email address e.g. guptamols@gmail.com, neharaghav@yahoo.com, rkart@gmail.com, squareyards321@ymail.com..... Is it possible to apply ML/Mathematics to generate category (like NER) from ...
user avatar
  • 73
0 votes
1 answer
38 views

How can I export the best classifier from my code to a model for real future usage?

...
user avatar
  • 1
0 votes
2 answers
40 views

Exploratory data analysis (EDA) on large dataset

I am working with lots of data (we have a table that produces 30 million rows daily). What is the best way to explore it (do on EDA)? Take a frictional slicing of the data randomly (100000 rows) or ...
user avatar
  • 23
0 votes
0 answers
9 views

Searching for object detection PyTorch library (like Segmentation models)

I'm searching for something like SegmentationModels https://github.com/qubvel/segmentation_models.pytorch for the purpose of the classification (single packet that includes various models like Yolo, ...
user avatar
  • 101

1
2 3 4 5
208