Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
1
vote
1answer
51 views

Sales person performance prediction

I am trying to predict/forecast salesperson performance weekly, monthly, quarterly, and yearly based on the products that they sold over 3 years. As part of this effort, I grouped their number of ...
2
votes
1answer
679 views

Linear regression : ValueError: operands could not be broadcast together with shapes (3,) (1338,)

I try to use linear regression for insurance data . But had error on the when try to call a function with features parameter. Here is my code: ...
2
votes
2answers
52 views

Modeling social media post scheduling optimization

Problem: I want to maximize performance for social media posts by optimizing the time when they are published. Current model: ...
1
vote
1answer
10 views

Predictive Recency-Frequency-Monetary (RFM) through Classification of Customer Charateristics

I have an RFM model that I use to segment customers based on RFM score. What I would like to do is: Understand more about the charateristics of my customers than just their RFM score; Be able to ...
1
vote
1answer
19 views

How to approach this: Percentage change in one KPI leading to change in other KPIs?

I want to know how can I approach or model this problem. I have 7 KPIs (3 of them dependent on each other) and one main KPI (total 8 KPIs). I want to understand effect of these 7 KPIs on the main KPIs....
0
votes
1answer
20 views

Interpreting evaluation metrics with threshold/cutoff

I was doing churn prediction for a company. I've got the following results by applying 3 classifier. Model Accuracy AUC Logistic Regression 0.671 0.736 Decision Tree (pruned) 0.681 0.665 Decision ...
0
votes
0answers
6 views

Confidence intervals for evaluation on test set

I'm wondering what the "best practise" approach is for finding confidence intervals when evaluation the performance of a classifier on the test set. As far as I can see, there are two ...
0
votes
0answers
39 views

Image segmentation with cityscapes dataset

I am a beginner in image segmentation and am trying to use the code: from dilatednet import DilatedNet from multiclassunet import Unet but get the error: ...
5
votes
1answer
675 views

Time Series pattern recognition and classification problem

I have some labeled sensor data. Now, I would like to know how to extract features from time series using DFT, DWT, and HAAR transforms. I know that the transformations above transform a signal to ...
0
votes
0answers
12 views

How to apply two input and one output with LR and SVM

Q1: how to feed 2 input to LR and SVM? My dataset consist of three columns which are: sentence1 , sentence 2, and label (1 if the sentence2 is a paraphrased of sentence1) I prepare my data and convert ...
0
votes
0answers
7 views

validation loss early increase (during warm-up)

Several questions have been asked about validation loss behavior during training of a DNN. It's clear to me that validation loss and accuracy are somehow correlated, but their curves can differ from ...
1
vote
1answer
203 views

Derivation of dz[1] for backpropagation

Can anyone mathematically prove this equation given the values of $dz^{[2]}$, $W^{[2]}$, $z^{[1]}$ and the activation function $g^{[1]}$ $dz^{[1]} = w^{[2]T}dz^{[2]} * g^{[1]'}(z^{[1]})$
0
votes
0answers
51 views

How to split Algorithms into multiple machines? [closed]

I have NLP big dataset (millions of records) and I am trying to find best algorithm that can do predictive modeling / classifications I use these algorithms mainly (Logistic Regression, Random Forest, ...
1
vote
1answer
218 views

Machine learning for missing data in time series

We have two time series columns - column A is the reference column ( source of truth) and column B is a ''cousin'' of column A, in the sense that it exhibits ( or should exhibit) the same patterns, ...
2
votes
0answers
27 views

Multi-arm bandits

I would like to model a problem as a multi-armed bandit problem. In the data we have contextual information (user demographics, preferences, etc.) but this contextual information of each user is not ...
0
votes
0answers
21 views

How does XGBoost perform in Parallel

So what I know about boosting technique, Like we train the data and update the weights of falsely predicted values or try to minimize the loss in the next model. So basically, it's the sequential ...
1
vote
1answer
630 views

Read back a saved LGBMClassifier model

I trained a LGBMClassifier model and saved in a file in this way: ...
2
votes
2answers
42 views

Do we need to pre-process both the test and train data set?

I've been given 2 datasets , and there are missing values in both the test and training data set. Do I need to pre-process test.csv also or is it only for train.csv?
1
vote
1answer
44 views

multiclass classification

I want to build an ml model, which can when given a text input, can predict predefined tags or labels for the text. I already built one such model, but the problem with that is that it only predicts ...
2
votes
1answer
161 views

Loss function for age classification

I am building a CNN model for age classification. Assuming age of a person is between 1-100, my last Linear Layer contains 100 output neuron. Now i want to find an appropriate loss function for this ...
1
vote
2answers
624 views

Predict Best Branch Locations For a Company

I have a dataset of existing 156 branches of my company with the longitude and latitude of each branch. Now we want to open 10 more branches. How can we predict the best locations for opening the new ...
1
vote
2answers
30 views

log(odds) to p formulation

$$Log(Odds) = log({p \over (1-p)}) $$ $${p \over (1-p)} = e^{b+b_1x_1+....}$$ I understand up to here, however how does this: $$p = (1-p) e^{b+b_1x_1+...}$$ become: $$ p = {1 \over {1+e^{-(b+b_1x_1+......
0
votes
1answer
39 views

which type of machine learning algorithms perform better at extrapolation (in general)

Assuming that: the problem lies in the field of natural science, i.e. relationships between variables are physics-based and does not change depending on context its a regression based model Would it ...
1
vote
1answer
56 views

What is the meaning of the bubbles / spikes in the shap values ​plot?

Here are an example of shap values plot from here. How to interpret the 'bubble' or 'spikes' on this shap values plot I highlighted in yellow color?
2
votes
1answer
33 views

Why do Transformers need positional encodings?

At least in the first self-attention layer in the encoder, inputs have a correspondence with outputs, I have the following questions. Isn't ordering already implicitly captured by the query vectors, ...
1
vote
3answers
47 views

Test Binary Classifier on a Test-set that includes only one class

I'm working on a disease binary classification problem. 0 = healthy , 1 = not healthy The disease is a movement disorder that appears on the patient while moving a specific movement. I applied leave-...
1
vote
1answer
68 views

How to measure model success in production

I have a model running on a productive system. The model predicts if some lead will become a sale. How would you develop a check, which checks the success and the accuracy of the model? There is a ...
2
votes
2answers
32 views

How to use External Data Sets in test set

I have a doubt regarding usage of external datasets like gdp rate, unemployment rate... etc., in test set for time series prediction. These datasets are historical and can be used along with train set,...
0
votes
2answers
178 views

Can we make two separate models vs one for classification?

Suppose I have a binary classification problem and my data is imbalanced, I can build a classification model using any of the algorithms and use an oversampling or undersampling technique to handle ...
2
votes
4answers
84 views

Are there ML Libs in Python robust to missing data?

So I was searching on how to handle missing data and came across this post from Machine Learning Mastery. This article states that some algorithms can be made robust to missing data, such as Naive ...
0
votes
1answer
2k views

Multiple-input multiple-output CNN with custom loss function

I have a set of 2D input arrays $(n\times m)$ namely $A,B,C$ and I would like to predict two 2D output arrays namely $d,e$ for which I have the expected values. You can think of the inputs/outputs as ...
1
vote
1answer
106 views

Training a Variational Autoencoder (VAE) for Random Number Generation

I have a complicated 20-dimensional multi-modal distribution and consider training a VAE to learn an approximation of it using 2000 samples. But particularly, with the aim to subsequently generate ...
0
votes
1answer
18 views

Working of Dense Layer

What kind of operation does Dense Layer perform to reduce dimemsion. So basically I have used Dense layer to compress the dimension all the time like from 10000 neurons to direct 2000 neurons or even ...
2
votes
1answer
251 views

Target Variable Encoding for Time Series Change point detection

I am working on a time series data for which I intend to impliment machine learning model for detecting change point in time series data. This data is recorded fom machinary and we have to predict ...
0
votes
1answer
25 views

How to write a reward function that optimizes for profit and revenue?

So I want to write a reward function for a reinforcement learning model which picks products to display to a customer. Each product has a profit margin %. Higher price products will have a higher ...
1
vote
0answers
67 views

Vector Arithmetic using WGAN-GP (Wasserstein GANs with Gradient Penalty)

Vector arithmetic in the latent space has been demonstrated to produce meaningful output image samples from a trained DC-GAN in the paper by Chintala et al. In fact, the vector arithmetic they ...
0
votes
0answers
9 views

Convert a simple NN model to LSTM model (made using Tensorflow.compat.v1)

I have a simple neural net which I would like to convert to an LSTM model. Can someone please help me with the code? The following code contains the neural net part: ...
0
votes
0answers
16 views

combining multiple models to get a clustering

Let's say I want to combine multiple neural networks to solve a supervised task. At training time I would ask every neural network and only improve the response of the one that is already the best. ...
0
votes
2answers
27 views

Can identifiers be used to train a model?

I recently participated in some Machine Learning competition where we were asked to decide whether a rider should accept or not a course (~2k riders and ~140k courses). It came up that some of the ...
3
votes
2answers
66 views

Keras deep learning speaker identification model excels during training and then fails predictions

I am attempting to create a 1:N speaker identification model with Keras using a TensorFlow backend. I used the LibriSpeech corpus for training data, and preprocessed the data by first converting each ...
3
votes
3answers
152 views

CART algorithm (Classification and regression trees) question

We fit a full classification tree model $T_k$ of given depth $k$ to data using the CART algorithm, and prune the tree by finding $E(k, \alpha) = min_{T\subset Tk} Err(T) + \alpha |T|$. Here, $Err(T)$ ...
1
vote
1answer
53 views

How to use CNN to deal with a 2D regression problem?

I have seven measurements (Obs1-7), each measurement has the dimension of [x,y,t] where x and y are coordinates and t is time. Now I want to build a model that uses the first 6 measurements to predict ...
1
vote
2answers
34 views

What effect repetitive data will have on the performance of the model

I understand that my question is very broad and that the correct answer may depend on various things. I want to get an idea in general what we may expect if we have repetitive data in our dataset. ...
0
votes
0answers
6 views

Test data for time sequential data

If I am trying to predict: the weather, the stock market, coffee sales per city, etc. there is no good way I can see to break out the data for training vs test data. For the weather case, training ...
1
vote
1answer
32 views

Will repeatedly fine-tuning on new data cause overfitting?

I have a binary classification model which I have trained on a training set. On the validation set its accuracy is ~85%. I set up early stopping which ended training when validation loss increased. ...
0
votes
0answers
3 views

Which tool do you use for creating continuous training pipelines for MLOPS?

One key component of MLOPS is continuous training. Which means the end to end training is put in a pipeline which can be triggered, versioned and metadata of the pipeline can be tracked. Thus enabling ...
3
votes
3answers
5k views

What are the disadvantages of Azure's ML vs a pure code approach (R/SKlearn)

Good Day, Microsoft offers their Azure Machine Learning Platform: https://azure.microsoft.com/en-ca/services/machine-learning/ Azure Machine Learning is designed for applied machine learning. Use ...
1
vote
1answer
19 views

Clustering with hierarchical data dependencies

I am currently looking into how to cluster data with hierarchical dependencies. An example of a problem that I want to cluster: we would like to cluster cities to identify similar characteristics with ...
2
votes
2answers
56 views

How to deal with Missing Not at Random Data for k-means clustering?

I am running k-means clustering on a customer dataset. One of the available demographic fields is inferred homevalue, represented as an integer. This field has value 0 when it's inferred that the ...
0
votes
1answer
3k views

GridSearchCV() to fine tune outputs ValueError and FitFailedWarning

I want to fine tune some parameters for my linear SVM. This is the code: ...

1
2 3 4 5
194