Hot answers tagged

7

Welcome to Data Science SE! Well, we say that most of our jobs is to wrangle with data, and that is because data is usually trying to deceive us... jokes aside: You have a missing data problem that means your have to clean your data and fill those missing values. To perform this cleaning process your must take the most classic statistician inside of you ...


5

Some things to take into account: Try to apply appropriate input space transformations, e.g. convert to polar coordinates. Despite the fact that a single hidden layer feedforward network can be a universal approximator, there is no guarantees about the number of neurons needed to approximate an arbitrarily complex function. Instead of having a single hidden ...


5

You basically need a t-SNE plot, the t-SNE will convert the high dimensional feature vector (several features in your case) to a 2d point and then you can use matplotlib to plot, while plotting you need to send in the class of the corresponding feature to get different colour for different classes of data points. once you do this, you will be able to judge ...


5

When class labels are known, you can use Linear Discriminant Analysis (LDA) for visualization to see whether classes are linearly separable. LDA is similar to PCA but supervised. It tries to project the data in a way that maximizes the distance between classes (here is a how-to post for Matlab, R, Python. Here is a mult-class LDA for Matlab). Also, we can ...


4

It is not possible to tell whether a machine learning algorithm is overfitting based purely on the training set accuracy. You could be right, that using more features with a small data set increases sampling error and reduces the generalisation of the SVM model you are building. It is a valid concern, but you cannot say that for sure with only this worry ...


4

If it's important for your use cases, you could try switching to Apache Zeppelin. As all Spark notebooks there share the same Spark context, same Python running environment. https://zeppelin.apache.org/ So what you're asking happens natively in Zeppelin. Or to be complete, it is an option to share the same Spark context / same Python envrionment between ...


4

The accuracy is different because there are k-classifiers made for each number of k-folds, and a new accuracy is found. You don't select a fold yourself. K-Fold cross-validation is used to test the general accuracy of your model based on how you setup the parameters and hyper-parameters of your model fitting function. What you do select is the number of ...


4

I would highly recommend doing some research into the architecture of random forests. There are many sites that provide in depth tutorials on RFs (Implementation in Python). Quick explanation: take your dataset, bootstrap the samples and apply a decision tree. Within your trees, you want to randomly sample the features at each split. You should not have ...


4

Adding to the answer above, The labeling totally depends on how you define it. You can define 0 as negative or as positive. However, for the sake of understanding and ease of readability, keep it meaningful. The instances that are correctly predicted are given by the diagonal. Here, '1' is True Negative or for the class labelled as 0 and '5' is True ...


3

If you want to combine the results from three different Neural Networks to "boost" the performance :) , you might want to look at the different Ensemble Learning Methods as I mentioned earlier. Which method you should use, depends on how you share or divide the training data between the three NNs. For example if the NNs are trained on same data but have ...


3

SVC Parameters: What prompted you to use a polynomial kernel? There are cases where this might make sense, but it certainly wouldn't be my first choice. A radial basis function is likely a better fit, which is why it is the default in SKLearn. Step back a minute and take in the big picture perspective. Does your human mind want to create a decision ...


3

You have to be sure that the algorithm is the same and the kernel functions are really the same. If you look at this python documentation page for kernels in scikit learn you will see there a description of poly kernel. Notice that you have a gamma and a degree. Gamma is by default 'auto' which is evaluated at 1/n_samples. For the same kernel you have '...


3

Cosine is not a commonly used activation function. Looking at the Wikipedia page describing common activation functions, it is not listed. And one of the desirable properties of activation functions described on that page is: Approximates identity near the origin: When activation functions have this property, the neural network will learn efficiently ...


3

What about doing cross validation on your training set? Once you have the different train/test splits I would start by printing the accuracy (number of correct predictions / total predictions) and the confussion matrix for each method. If you are using python sklearn.metrics offers a wide variety of useful functions.


3

For an unsupervised technique, if you have some metric of "goodness of fit", it makes sense to have a train-test split. In your case, it seems as though you'd want to split the data so that you can see if your SVM classifies the test data as outliers. However, your goal is unclear. It might help if you clarify what you mean.


3

The easiest way to tune a single hyperparameter is to use what is called the elbow method. Do the following: Define a range of C you want to try, i.e C = [1.0, 1.5, 2.0, ...] Loop over all values of C in your range Train a new model with the current value of C Evaluate each model on the validation set and store the results Plot your metric over over your ...


3

PCA doesn't remove any specific feature. What PCA does it to calculate linear combinations of your variables in such way that they get "summarized" in fewer variables. You don't eliminate variables, you reexpress them.


2

Please allow me to paraphrase your question to make sure I get your question right. Suppose you have 1 million data entries. Each entry consists of three inputs X1, X2, X3 and one output Y. One of the three inputs, X1, is a noise. You want to find the relation between X1 and Y. So you can remove the impact of X1 on Y. Let's use a simplified example, Y = ...


2

In general, there is no guarantee that ANNs such as a multi-layer Perceptron network will converge to the global minimum squared error (MSE) solution. The final state of the network can be heavily dependent on how the network weights are initialized. Since most initialization schemes (including Nguyen-Widrow) use random numbers to generate the initial ...


2

The trainbr mode uses the Bayesian regularization backpropagation. This method was presented in 1, which presents a regression problem with the loss function $$ E_D = \sum_{i=1}^n (t_i - a_i)^2 $$ where $t_i$ is the target and $a_i$ is the network's response. The paper proposes to add a regularization term, leading to a loss function $F$ of the form $$ F = \...


2

In gereral, there are four ways one can "connect" neural networks (depending on you application at hand) as described in Combining Artificial Neural Networks, Sharkey et al.: In the cooperative mode, there are various ways in which one can combine the decisions made by different models. One common way is to take the average of the predictions. Other ways ...


2

Some flavor of evolutionary algorithm may suit your problem nicely, since: The gradient of the objective function is unavailable or cannot be computed. The objective function itself is somewhat expensive to compute. The objective function is may have a large number of local maxima. In short, evolutionary algorithms do optimization by generating candidate ...


2

I think it might be a relatively trivial bug in your cost function for softmax: J = -(sum(sum((Y).*log(h))) + lambda*p/(2*m)) should be J = -sum(sum((Y).*log(h)))/m + lambda*p/(2*m) I.e. for softmax only, you have effectively subtracted the regularisation term from the cost function instead of adding it. Also, you forgot to divide the error term by ...


2

if yHat are your predictions and yval are your y true then tp = sum((yHaT == 1) & (yval == 1)); fp = sum((yHaT == 1) & (yval == 0)); fn = sum((yHaT == 0) & (yval == 1)); precision = tp / (tp + fp); recall = tp / (tp + fn); F1 = (2 * precision * recall) / (precision + recall);


2

to prevent overfitting in a model the training curve in a loss graph should be similar to the validation curve. That's not always the case, maybe the train set has slightly a different distribution from the validation set, therefore, our focus is just when training loss decreases while validation loss slightly or never decreases. In the former graph,...


2

to prevent overfitting in a model the training curve in a loss graph should be similar to the validation curve Similar in what way? If your training error keeps decreasing and the validation error started growing, you are overfitting. If you have a binary problem, why is the confusion matrix 3x3? Validation set accuracy after 33 epochs looks better than ...


2

1) It depends in what you define as positive and negative. Generally, and in particular in medicine, people tend to label $0$ as negatives and $1$ as positives, thus being $1$ the abnormal case. But this is completely arbitraty, you can do as you wish. 2) 0 are always displayed in the first row and column. That is, your model has classified one 0 correctly ...


2

Ensemble learning combines predictions from multiple learners. Boosting methods are one way to form an ensemble. Stacking is another. The important difference between boosting and stacking (and other ensemble methods) is that boosting applies a number of weak learners sequentially and then produces a final result via a weighted majority vote. The learners ...


2

Boosting is a type of Ensemble Learning, but it is not the only one. Apart from stacking, bagging is also another type of Ensemble Learning. Ensemble Learning is the combination of individual models together trying to obtain better predictive performance that could be obtained from any of the constituent learning algorithms alone. Boosting involves ...


2

This I can say because I ran the trained model on the entire dataset again (re-trained it) an predicted on the same dataset. You seem to be making a fundamental mistake here. If you train and test on the same data, your performance will not be representative of how the model can perform on unseen data points. Make sure that you train and test on ...


Only top voted, non community-wiki answers of a minimum length are eligible