Questions tagged [machine-learning]

Methods and principles of building "computer systems that automatically improve with experience."

Filter by
Sorted by
Tagged with
24
votes
5answers
3k views

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
17
votes
4answers
17k views

Does gradient descent always converge to an optimum?

I am wondering whether there is any scenario in which gradient descent does not converge to a minimum. I am aware that gradient descent is not always guaranteed to converge to a global optimum. I am ...
21
votes
6answers
6k views

Why do convolutional neural networks work?

I have often heard people saying that why convolutional neural networks are still poorly understood. Is it known why convolutional neural networks always end up learning increasingly sophisticated ...
114
votes
5answers
61k views

What is the “dying ReLU” problem in neural networks?

Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says: "Unfortunately, ReLU units can be fragile during training and can "die". For ...
38
votes
4answers
19k views

Adding Features To Time Series Model LSTM

have been reading up a bit on LSTM's and their use for time series and its been interesting but difficult at the same time. One thing I have had difficulties with understanding is the approach to ...
107
votes
5answers
126k views

The cross-entropy error function in neural networks

In the MNIST For ML Beginners they define cross-entropy as $$H_{y'} (y) := - \sum_{i} y_{i}' \log (y_i)$$ $y_i$ is the predicted probability value for class $i$ and $y_i'$ is the true probability ...
47
votes
5answers
10k views

Should I go for a 'balanced' dataset or a 'representative' dataset?

My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
29
votes
4answers
11k views

Quick guide into training highly imbalanced data sets

I have a classification problem with approximately 1000 positive and 10000 negative samples in training set. So this data set is quite unbalanced. Plain random forest is just trying to mark all test ...
61
votes
8answers
72k views

Open source Anomaly Detection in Python

Problem Background: I am working on a project that involves log files similar to those found in the IT monitoring space (to my best understanding of IT space). These log files are time-series data, ...
16
votes
4answers
4k views

Role derivative of sigmoid function in neural networks

I try to understand role of derivative of sigmoid function in neural networks. First I plot sigmoid function, and derivative of all points from definition using python. What is the role of this ...
11
votes
1answer
8k views

How to Predict the future values of time horizon with Keras?

I just built this LSTM neural network with Keras ...
4
votes
3answers
3k views

How to give name to topics created using LDA?

I have categorized 800,000 documents into 500 categories using the Mahout topic modelling. Instead of representing the topic using the top 5/10 words for each topics, I want to infer a generic name ...
43
votes
10answers
34k views

Machine learning - features engineering from date/time data

What are the common/best practices to handle time data for machine learning application? For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
40
votes
4answers
34k views

Why mini batch size is better than one single “batch” with all training data?

I often read that in case of Deep Learning models the usual practice is to apply mini batches (generally a small one, 32/64) over several training epochs. I cannot really fathom the reason behind this....
35
votes
2answers
27k views

How to interpret the output of XGBoost importance?

I ran a xgboost model. I don't exactly know how to interpret the output of xgb.importance. What is the meaning of Gain, Cover, and Frequency and how do we ...
23
votes
3answers
42k views

Data Science Project Ideas [closed]

I don't know if this is a right place to ask this question, but a community dedicated to Data Science should be the most appropriate place in my opinion. I have just started with Data Science and ...
17
votes
2answers
13k views

Doc2Vec - How to label the paragraphs (gensim)

I am wondering how to label (tag) sentences / paragraphs / documents with doc2vec in gensim - from a practical standpoint. Do you need to have each sentence / paragraph / document with its own ...
13
votes
2answers
4k views

Linear regression with non-symmetric cost function?

I want to predict some value $Y(x)$ and I am trying to get some prediction $\hat Y(x)$ that optimizes between being as low as possible, but still being larger than $Y(x)$. In other words: $$\text{cost}...
7
votes
2answers
934 views

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
3
votes
1answer
3k views

What are useful evaluation metrics used in machine learning

I am using CNN in order to predict codes after analyzing text. As an example, I will write "I am crazy" .. the model will predict some code " X321". All this based on CNN. I want to evaluate my ...
4
votes
1answer
9k views

Why should softmax be used in CNN

In the last layer of CNNs and MLPs it is common to use softmax layer or units with sigmoid activation functions for multi-class ...
8
votes
1answer
1k views

Can The linearly non-separable data be learned using polynomial features with logistic regression?

I know that Polynomial Logistic Regression can easily learn a typical data like the following image: I was wondering whether the following two data also can be ...
131
votes
17answers
106k views

Best python library for neural networks

I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?
77
votes
15answers
57k views

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common ...
22
votes
6answers
8k views

Deep learning basics

I am looking for a paper detailing the very basics of deep learning. Ideally like the Andrew Ng course for deep learning. Do you know where I can find this ?
13
votes
2answers
8k views

High-dimensional data: What are useful techniques to know?

Due to various curses of dimensionality, the accuracy and speed of many of the common predictive techniques degrade on high dimensional data. What are some of the most useful techniques/tricks/...
27
votes
7answers
25k views

Why should the data be shuffled for machine learning tasks

In machine learning tasks it is common to shuffle data and normalize it. The purpose of normalization is clear (for having same range of feature values). But, after struggling a lot, I did not find ...
15
votes
1answer
16k views

Why ReLU is better than the other activation functions

Here the answer refers to vanishing and exploding gradients that has been in sigmoid-like activation functions but, I guess, Relu...
13
votes
5answers
3k views

Beginner math books for Machine Learning

I'm a Computer Science engineer with no background in statistics or advanced math. I'm studying the book Python Machine Learning by Raschka and Mirjalili, but when I tried to understand the math of ...
6
votes
2answers
12k views

Python or R for implementing machine learning algorithms for fraud detection [closed]

I was wondering which language can I use: R or Python, for my internship in fraud detection in an online banking system: I have to build machine learning algorithms (NN, etc.) that predict transaction ...
10
votes
4answers
752 views

What initial steps should I use to make sense of large data sets, and what tools should I use?

Caveat: I am a complete beginner when it comes to machine learning, but eager to learn. I have a large dataset and I'm trying to find pattern in it. There may / may not be correlation across the data,...
6
votes
1answer
570 views

data science / machine learning resources? [closed]

In a few weeks I'm starting a new job that will be involved in machine learning and data science. I have a masters degree in probability / mathematics but I have no knowledge of machine learning and ...
11
votes
2answers
2k views

Solving a system of equations with sparse data

I am attempting to solve a set of equations which has 40 independent variables (x1, ..., x40) and one dependent variable (y). The total number of equations (number of rows) is ~300, and I want to ...
8
votes
2answers
2k views

Machine Learning Steps

Which of the below set of steps options is the correct one when creating a predictive model? Option 1: First eliminate the most obviously bad predictors, and preprocess the remaining if needed, then ...
4
votes
1answer
18k views

How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as: np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data) On second file, i'm trying to ...
12
votes
1answer
8k views

back propagation in CNN

I have the following CNN: I start with an input image of size 5x5 Then I apply convolution using 2x2 kernel and stride = 1, that produces feature map of size 4x4. Then I apply 2x2 max-pooling with ...
10
votes
3answers
2k views

Statistics + Computer Science = Data Science? [closed]

i want to become a data scientist. I studied applied statistics (actuarial science), so i have a great statistical background (regression, stochastic process, time series, just for mention a few). But ...
20
votes
6answers
9k views

Encoding features like month and hour as categorial or numeric?

Is it better to encode features like month and hour as factor or numeric in a machine learning model? On the one hand, I feel numeric encoding might be reasonable, because time is a forward ...
16
votes
2answers
3k views

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
6
votes
2answers
5k views

How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
5
votes
2answers
5k views

Why large weights are prohibited in neural networks?

Why weights with large values cause neural networks to be overfitted, and consequently we use approaches like regularization to neutralize weights with large values?
1
vote
4answers
1k views

Exceptionally high accuracy with Random Forest, is it possible?

I need your help to find a flaw in my model, since it's accuracy (95%) is not realistic. I'm working on a classification problem using Randomforest, with around 2500 positive case and 15000 negative ...
5
votes
1answer
966 views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
4
votes
1answer
2k views

Correlation between specific columns of a data set

I have a CSV file which has 150 columns belonging to 7 categories but I want a correlation between 2 categories. The categories are movies and music, 12 and 19 columns respectively. Is there a way ...
2
votes
1answer
737 views

Cluster documents and identify the prominent document in the cluster?

I have a set of documents as given in the example below. ...
1
vote
2answers
573 views

Explain Binary Classification with output 0.5 (True)

What is the interpretation of output 0.5 of a typical classifier? I made a prediction and the probability of that data point being from the True class is 0.5.
2
votes
2answers
152 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
70
votes
5answers
41k views

Why do cost functions use the square error?

I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable. I have learnt that there is a hypothesis, which is: $h_\theta(x)=\...
64
votes
8answers
84k views

When should I use Gini Impurity as opposed to Information Gain?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
62
votes
3answers
45k views

Advantages of AUC vs standard accuracy

I was starting to look into area under curve(AUC) and am a little confused about its usefulness. When first explained to me, AUC seemed to be a great measure of performance but in my research I've ...