Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
11,314
questions
1
vote
1
answer
358
views
Derivation of dz[1] for backpropagation
Can anyone mathematically prove this equation given the values of $dz^{[2]}$, $W^{[2]}$, $z^{[1]}$ and the activation function $g^{[1]}$
$dz^{[1]} = w^{[2]T}dz^{[2]} * g^{[1]'}(z^{[1]})$
CommunityBot
- 1
11
votes
2
answers
6k
views
Why neural networks do not perform well on structured data?
I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
- 11.2k
9
votes
1
answer
5k
views
When does decision tree perform better than the neural network?
I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
2
votes
2
answers
314
views
How to train a machine learning algorithm with multiple labels
I have the following challenge and I very much hope that there is a solution to it. I also suspect that there is a simple approach to it. I just don't see it at the moment. Any help or advice is ...
CommunityBot
- 1
0
votes
3
answers
2k
views
NLP to detect duplicates for very technical language
I have the following scenario, to detect duplicate products based on the description fields. The Description Field contains product technical name, dimensions, characteristics. My model needs to ...
CommunityBot
- 1
0
votes
1
answer
12
views
Hassle-free platform for a small data science team?
our team of 4 data scientists has been exploring options to automate our ETL, data storage and model updates and are looking for a hassle-free platform that can help with this.
We've had several ...
1
vote
2
answers
83
views
Model Selection using Bias Variance Trade Off
I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%.
This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...
CommunityBot
- 1
2
votes
1
answer
1k
views
Normal vs Uniform Distribution for machine learning
I have a dataset that follows Zipf's law such that the majority of the values are concentrated at one end, with the remaining items containing a very small percentage. Training on the dataset as is ...
CommunityBot
- 1
1
vote
1
answer
1k
views
Column header in XTS and Data frame in R
I used quandl function extract stock data with object type as xts.
...
CommunityBot
- 1
1
vote
2
answers
48
views
How to create a bot for a real-time PvP game using machine learning?
Unfortunately, I am not well-versed in machine learning. However, I'm trying to understand if it's possible to create a bot for a real-time PvP game like, for example, Clash Royale or Random Dice: ...
- 111
1
vote
1
answer
40
views
Which type of Machine Learning is used in robots?
Which type of Machine Learning is used in robots? Is it supervised learning or unsupervised learning or Reinforcement learning?Especially the robots that were sent in space?
- 111
2
votes
1
answer
677
views
Can LSTM be used for non time series data?
I have a dataset -
This is a TOR network traffic dataset with labels added as TOR/ Non TOR.
I want to run an LSTM on it and classify it as Tor/Non Tor. Is that possible since this is not a time ...
CommunityBot
- 1
0
votes
1
answer
17
views
Does learning rate depend on input and output range?
I watched hours of videos on gradient descent and still feel pretty confused. Let's say I have a "model":
y = x * w
I use 2 as my target ...
- 291
0
votes
1
answer
1k
views
Embedding layer before LSTM layer
I am toying around with a clustering and churn prediction framework, cluschurn which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and ...
CommunityBot
- 1
0
votes
0
answers
19
views
Minimizing error in cosine similarity
Presume I have a vector space, and I am attempting to compress it into a latent vector space, while minimizing error in cosine similarity between entries. Suppose that I know the actual cosine ...
0
votes
3
answers
360
views
How to find the driver features towards a particular result in Classification problems
In a classifier model, we can predict the outcome class, but here I need to find out the features that drive towards a particular result in a classification problem, that are a strong indicator of a ...
CommunityBot
- 1
0
votes
1
answer
25
views
Aggregating decision criteria of different scales
Let's say I have a framework that performs a detection task on some dataset. In order to do so I use three different metrics (A, B, and C) as decision makers. A and B are probabilities, i.e., $ 0 \le ...
- 473
1
vote
2
answers
355
views
How do transformers differ from feature selection and regular machine learning?
This is perhaps a simplistic way of thinking, but to me transformers (attention based neural networks) focus on a subset of the input, learning what is important for the problem/prediction as the ...
CommunityBot
- 1
1
vote
1
answer
1k
views
Reverse of Yeo Johnson variable transformation done by preprocess function in caret package to get original Target and predicted values
I have done a Yeo Johnson transformation by using preprocess from caret package. I have predicted the target variable using linear regression. Now, I would like to reverse the transformation for both ...
CommunityBot
- 1
1
vote
0
answers
13
views
Two-part ML classification model on panel data - is it viable?
I have a dataset of medical encounters, and I aim to predict whether a patient will return to the hospital within 30 days after being discharged. Each row in my dataset corresponds to a specific ...
0
votes
1
answer
17
views
Logistic Regression Import error
Hi Im using SKlearn for a school project. I cannot importy Logistics regression without getting a numpy float error.
Have not been able to find any solutions online.
- 198
1
vote
1
answer
79
views
Recommendations based on other products seen
I am trying to develop a basic book recommender system to get in touch with the field and start learning methods and how to prepare the data.
The Dataframe I am using is pretty plain, it has the ...
CommunityBot
- 1
1
vote
3
answers
2k
views
Non-mutually exclusive classification sum of probabilities
So I have the following problem: I realized (while writing my master thesis) that I am still not sure/have vague descriptions of some of the machine learning principles. I already asked one question ...
CommunityBot
- 1
0
votes
2
answers
36
views
Counting Number of Holes in an Image of Cheese
I've been assigned a project that involves writing a script to detect the number of holes in an image of cheese. My background in AI is quite limited, so I was wondering if anyone could give me a good ...
- 306
0
votes
1
answer
66
views
Relationship between size of of data and number of classes
Hi we have a problem on image classification where data is quite less. However we have some flexibility on the classification.We can change the number of classes in our problem by clubbing and ...
CommunityBot
- 1
0
votes
1
answer
110
views
scikit-learn: feature analysis differs heavily from model coefficients
I am trying to perform linear regression and I want to analyse the available features beforehand. The task is to predict the value of a house.
Some of them might have a high impact on the label, ...
CommunityBot
- 1
4
votes
2
answers
162
views
ML/NN as Function Evaluator for further Optimization (maximization) - Practical Example
I am working on a production optimization problem; a very similar idea to what is described by Vegard Flovik How to use machine learning for production optimization. The following image, taken from ...
- 123
0
votes
1
answer
66
views
I have data with customer personal information and customer transaction. I cannot figure out how to use the data for training my model?
Customer information attributes:
ID
Age
Gender
State etc
Customer transaction
ID
Store ID
No of items bought
State etc
Store info
Store ID
State
Daily revenue
Store size etc
I want to predict if ...
CommunityBot
- 1
0
votes
1
answer
1k
views
Improving precision and recall for imbalanced large data set
I have a data set of 1 million points and 30 features. The output variable has multiple classes (1 to $n$) but the problem I'm interested in is only concerned whether the output belongs to class 1 or ...
CommunityBot
- 1
2
votes
1
answer
126
views
Impose similar metric on segments to model
I am training a binary classifier in a dataset using AUC as a score. The dataset has two main groups (we will refer to them as good and bad population). A property that this dataset has is having a ...
CommunityBot
- 1
1
vote
2
answers
82
views
How to do Feature clustering?
I have different datasets and I want to find out the features that are similar among the datasets. The datasets are of varying sizes.
example:
dataset1 has columns a,b,c,d,e
dataset2 has columns m,n,o,...
CommunityBot
- 1
4
votes
1
answer
855
views
Predicting app usage on mobile phone
I'm currently building an app that strives to predict how the users uses different apps and give the user a suggestion based on which apps it think the user will currently use (a ranked list based on ...
1
vote
1
answer
27
views
Ordering of Train/Val/Test set use in hyperparameter tuning
The way I read almost lots of ML advice on these datasets sounds like "You train a model that's randomly chosen hyperparameters first on the training set, then you ignore this bit of the work, ...
- 473
1
vote
2
answers
349
views
Plotting Polynomial Regression?
I'm reading through Hands-On Machine Learning with Scikit-learn and Tensorflow by Geron. I am creating a simple polynomial regression using sklearn's ...
CommunityBot
- 1
3
votes
3
answers
245
views
CART algorithm (Classification and regression trees) question
We fit a full classification tree model $T_k$ of given depth $k$ to data using the CART algorithm, and prune the tree by finding $E(k, \alpha) = min_{T\subset
Tk} Err(T) + \alpha |T|$. Here, $Err(T)$ ...
CommunityBot
- 1
3
votes
1
answer
91
views
Balance data using different criteria
I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID.
For each ID, I have a ...
CommunityBot
- 1
1
vote
1
answer
84
views
I Have Issues Installing Basemap
I tried to install Basemap and it gives me this:
preparing transaction: done
verifying transaction: done
executing transaction: failed
ERROR conda.core.link:_execute(507): An error occurred while ...
CommunityBot
- 1
1
vote
2
answers
503
views
Hellinger Distance in Gensim
I have set of documents as follows where each document has set of words that represents the content of it.
...
CommunityBot
- 1
0
votes
0
answers
8
views
ML approach for quantifying building quality perception
I'm working on a project to model public perceptions of buildings in a tourism context, focusing on attributes like beauty and mystery. The data I have is a labeled dataset of building photos, each ...
- 101
0
votes
4
answers
94
views
Bias and variance in the model o in the predictions?
This topic confuses me. In the literature or articles, when talking about bias and variance in automatic learning, specifically in cross-validation, do they refer to the high bias (underfitting) and ...
CommunityBot
- 1
1
vote
2
answers
773
views
What is preferred upsampling or zero padding?
When training a CNN one option is either to zero pad an image to make it bigger or upsample it. When should I choose each one?
What criteria is leveraged for choosing a method?
CommunityBot
- 1
0
votes
1
answer
853
views
Merging two datasets with different features for machine learning prediction
I'm trying to create a model which predicts Real estate prices with xgboost in machine learning, my question is : Can i combine two datasets to do it ?
First dataset : 13 features
Second dataset : ...
CommunityBot
- 1
0
votes
0
answers
15
views
Can I modify my training datasets (X_train and Y_train) while fitting the model?
I am new to ML and I am trying to train a forecasting model. The target variable (Y_train) has multiple columns, all of Boolean type. The features table (X_train), according to my approach, in the ...
2
votes
1
answer
306
views
Building Timeseries models for stock trading having multiple stocks
I have gone through some of the tutorials on the timeseries and all of them have taken one stock for the timeseries and tried to forecast it. My dataset contains many stocks for the time period(each ...
CommunityBot
- 1
1
vote
1
answer
425
views
Need help with prolog program
I am new to prolog and need help on how to write a prolog program. Here is what i am trying to do. I have downloaded the dataset from this link - https://archive.ics.uci.edu/ml/datasets/Blood+...
CommunityBot
- 1
0
votes
1
answer
67
views
How to calculate the evaluation metrics (i.e., F1 score) in leave one subject out cv when a subject belongs to single class only
I have dataset of 10 subjects. the dataset has 4 classess. 0,1,2 and 3. The distribution of classes are not same. For example subject 1 does not have 1,2 and 3. It belongs to zeros class. currently ...
CommunityBot
- 1
0
votes
1
answer
622
views
Visualizing the equation for separating hyperplane
I was wondering if I can visualize with the example the fact that for all points $x$ on the separating hyperplane, the following equation holds true:
$$w^T.x+w_0=0\quad\quad\quad \text{... equation (1)...
CommunityBot
- 1
1
vote
3
answers
135
views
Logistics Demand Forecasting with 20k Different Time Series
I'm trying to tackle a very challenging problem and I would appreciate your help.
My organization has a lot of different items which can be demanded by our clients. Those items can also be returned ...
CommunityBot
- 1
1
vote
1
answer
127
views
Data quality improvement as a part of preprocessing: Imputation
I have a python pandas dataframe representing a superset. The data contains a lot of nulls which I want to overwrite with real values.
the superset has:
both numerical and categorical data
some ...
CommunityBot
- 1
-2
votes
0
answers
21
views
Creating a data visualisation using matplotlib and unsupervised learning
I have to write a python code on a kmeans algorithm and heres the requirements.
You should submit a single file (not zipped) named myVisualiser.py containing a function with signature:
def ...
