New answers tagged

0

I'm taking a more "applied" view here: Normal (OLS) regression is linear and can take on any value for the predicted dependent variable $\hat{y}$. In contrast, Logit (via the logistic link function) restricts the outcome to $\hat{y} \in [0,1]$. This is a desireable property as you can interpret the predicted values directly as a probability. In fact, ...


0

I think one of the most significant issues is the loss measurement. For a point with true value 0, a predicted value of -1 or 1 contribute the same to the loss, but these are not equally bad predictions!


0

It would work, afterall ML is a lot about engineering and hacking the things together. However, it would perform not so well, as for example logistic regression. If you compare the a linear line and logistic regression you will notice that the gradients of their respective loss functions for points near the decision boundary ("threshold") and far away from ...


1

The cost function is the judge for your model. It judges how well your model perfoms. By choosing a loss function you choose which properties of your model outputs the loss function will judge. Mathematical convenience usually is desired for the loss function to be applicable The MSE will punish outputs that are further away from the desired value more ...


0

You are correct to approach this as a regression problem, mostly because you are interested in the order of your outputs. For example if there are 1000 people present and you predict 1005, it's a better prediction than 7005. If you were treating this as a classification problem, both of these would be interpreted as missclassifications. The most practical ...


3

It is common in applied machine learning to have the model with the lowest generalization error, as measured by score on validation data, also have the biggest delta from the score on the training data. There is nothing inherently wrong with overfitting, it depends on the goal of the project. The typical goal of applied machine learning is high predictive ...


0

What you are referring to is called a multi-input model and can be esaily built in most deel learning frameworks. The idea is to have both types of data as separate inputs, then use specific layers depending their types (recurrent layers to sequence data, CNN to images, and so on...) to later on concatenate them together. If you can use Keras, there is the ...


3

Your question is, what model is better between one that seems more overfitted (larger difference between train and eval set) but it has also higher scores or one that has less variance between train and eval set but at the same time it has worst results. Everything assuming that you have done a correct train test split and there is no data leakage and ...


2

You are dealing with noisy labels. I would not switch the labelling according to a trained model that learned on that particular data set, since probably you don't know which patterns lead to your models decision. Otherwise if you know the reason for the wrong labelling, you could try to build methods yourself that run a sanity check on your data. ...


4

There is only one answer to this question, which is no, it is not acceptable. Whatever transformation you apply to the train data (PCA, scaling, encoding, etc.) you have to also apply to the test data.


5

No, it does not make sense to do this. You model has learned how to map one input space to another, that is to say it is itself function approximation, and will likely not know what to for the unseen data. By not performing the same scaling on the test data, you are introducing systematic errors in the model. This was pointed out in the comments by nanoman ...


1

If every data point has a ground-truth label (i.e., one of the six labels), than any supervised learning technique can work, including Random Forest. If the labels come in batches, then the parameters of the model can be updated with each new batch. Either completely retrain the model with data up to the current time point or incrementally update the model ...


0

@(something) is used to call a function in MATLAB or octave. Suppose you create a function within a code. And you set a keyword for that function. we have a sum(x,y) function which takes two inputs and returns the sum. Now you fix the value of y, say y = 3; And you want to change the value of x every time. You can design the inline function by following: y=...


0

Main formula for SVM is - $y_i(wx_i +b) \geq d$ In the derivation process, it is changed to 1 to make it standardized for all hyper-plane. If it has to be described, it will be - "Greater than" "per unit of minimum margin distance" Let's suppose, If a hyper-plane has the minimum margin point at 4 Eucledien distance Another one has it at 4.5 Eucledien ...


0

the value can be larger than 1 but can a probability be larger 1? Isn't that against its definition? Speaking in a very simple language how a model(NN) works - It doesn't know if it is a probability Or a number. It only knows that it has to minimise the Loss to match the output. I see no reason why can't an output become > 1 if we don't use Sigmoid/Softmax ...


0

Loss function For most optimization algorithms, it is desirable to have a loss function that is globally continuous and differentiable. Two very commonly used loss functions are the squared loss and absolute loss. However, the absolute loss has the disadvantage that it is not differentiable at 0. The squared loss has the disadvantage that it has the ...


0

The concept of a loss function comes from decision theory. Although there are some 'classic' loss functions the point is to be subjective, in the sense of being flexible enough to represent any particular problem conctext. So in that sense, yes, loss functions can be customised. One of the main ways this has been achieved is via Bayesian regression, as the ...


0

Every cepstral coefficients can be considered as one of the best features for defining a musical piece. Most famous being the Mel Scale, as I can see you are already extracting MFCCs, you are good to go. Although you should have mentioned, which MFCC are you extracting, from experience (a little bit) first 15 are usually the most useful cause they have a ...


0

If you intend to use a summary statistics you would engineer it so it is well suited for your task, meaning captures most of the relevant information. For these things there is usually no best universal solution but it is problem specific. You did not specify what your problem is about so I can't help you there much, maybe use the median value.


2

It is a unit of distance, I would usually assume euclidean distance. In more detail: The data point $x_i$ is projected onto the vector $w$, which defines the orientation of the discriminating linear hyperplane as it is orthogonal to $w$. Where the discriminating hyperplane is "fixed" along the orientation of $w$ is decided by the bias Term $b$. So for ...


4

It affects anything optimized by a form of gradient descent, because it affects the relative scale of the dimensions of the input. If A is generally 1000x larger than B, then changing B's coefficient by some amount is in a sense a 1000x bigger move. In theory this won't matter but in practice it can cause the gradient descent to have trouble landing in the ...


0

Actaully the 1 doesn't matter. It's just a random parameter. No real meaning. You just assume some positive distance. Because the hyperplane is scale invariant, we can fix the scale of w,b anyway we want. Let's be clever about it, and choose it such that


0

Since you want to save the training min/max and use those to replace inf's in the test set, you need a custom transformer. To build a robust transformer, you should use some of sklearn's validation functions. And it's best to work in numpy, since as you point out an earlier transformer in a pipeline will have already converted an input dataframe to an ...


0

Here is a para that I found by searching What are hybrid methods in Machine Learning, on google. "In general, it is based on combining two different machine learning techniques. For example, a hybrid classification model can be composed of one unsupervised learner (or cluster) to pre-process the training data and one supervised learner (or classifier) to ...


0

Some recommendations based on what I've done. Here is a useful tutorial, which explains how to implement a CNN for wav files. https://medium.com/gradientcrescent/urban-sound-classification-using-convolutional-neural-networks-with-keras-theory-and-486e92785df4. In my case, it was overfitting and I wasn't able to fix that. This simple NN model gave the best ...


1

There are some papers which tell us that lower batch size may generalize better than large batch size. and large batch size may cause regularization in the model too. maybe that is the reason Bayesian optimization is suggesting a lower batch size for your dataset. Please check below papers, https://openreview.net/pdf?id=B1eyO1BFPr https://openreview.net/...


0

You should pass X and Y collectively to the ImageDataGenerator.flow() method. Please refer to this answer in case you are looking for a multi-output classification model using ImageDataGenerator in Keras. https://datascience.stackexchange.com/a/75034/98109


0

This actually makes sense since the magnitude of the data is much smaller when you fit Log(y) = model(X). $log error = \frac{1}{n} \sum_{t=1}^{n} abs( \frac{log(y_{t})-model(X_{t})}{log(y_{t})})$ $error = \frac{1}{n} \sum_{t=1}^{n} abs(\frac{y_{t}- exp(model(X_{t}))}{y_{t}})$ also, MAPE, is Mean Absolute Percentage Error.


0

You need to shuffle the whole dataset together before separating the features (X) and target variable (y). This is the only reason I can think of for getting this error.


0

If you are doing k-fold cross-validation, that might happened. Otherwise, I think, it is not logical to have such a change. If you share your code, it would be easier to find the source of the problem.


0

If you have just then parameters and two of them are important. You can plot the trees and see the threshold for each of the parameters. from xgboost import XGBClassifier from xgboost import plot_tree import matplotlib.pyplot as plt # fit the model model = XGBClassifier().fit(X, y) # plot single tree plot_tree(model) plt.show() The above code just plots ...


0

To answer your question we need to understand what the aim of the clustering analysis that you are doing. Some of goal's of clustering analysis are: Outlier Detection, Pattern Detection, Grouping Data together, etc Now depending on the type of data, we can choose the algorithm that best fits the data at hand. If you have only numerical features, then you ...


0

For the first idea about PCA, you can not simply just use 2 components. You need to take a look at the explained variance by your principal components and based on that you should select the required number of components. If, for example, you found that the first two components explain a significant amount of variance (e.g., more than 95%), then, you can use ...


0

Finally, I find time to answer this question whose answer was found in a well-known online course provided by Pr. Boyd for convex optimisation. In that course, he refers to applications of optimisation. One of its applications is penalty function approximation. As a brief answer, just define your penalty for the parameters you want and add it to the cost ...


0

It looks like you're adding the delta to the weights instead of subtracting it. Gradient descent is given by the following calculation run for some iterations: $$ x^{current} = x^{previous} - \alpha \frac{dy}{dx} $$ where $\alpha$ is the learning rate. We subtract the value because derivatives go in the direction of steepest ascent. So the following lines: ...


1

Note that in some cases you could use adaptive filters, that do not need to be explicitly trained. Examples of adaptive filters includes Least Mean Squares, Recursive Least Squares, Kalman... The subtle distinction between adaptive filters and traditional ML algorithms (like the ones that can be found in scikit-learn) is that the former do not follow the ...


1

One reason to convert numerical data to categorical data is to improve the signal-to-noise ratio. Fitting a model to bins reduces the impact that small fluctuates in the data has on the model, often small fluctuates are just noise. Each bin "smooths" out the fluctuates/noises in sections of the data.


0

I used YOLO v3 darknet implementation https://github.com/AlexeyAB/darknet I extracted video frames into images , then i annotated the images then started training the model... Good luck!


0

It relies on which kind of task u want to perform at the End. As I understood from your your question is that u have email with same pattern occurring on beginning as well as at the end. You wanted to perform an classification tasks on email based on real sense of email excluding subject and conclusion. There are multiple way u can do as follows: You can ...


1

There are infinitely many solutions except in corner cases like x = 0 or something. In your case here, you could simply find a solution with $A = b x^+$ where $x^+$ is the Moore-Penrose pseudoinverse. In R that would be something like A = b %*% ginv(x), where ginv is from the MASS library.


1

One possible solution when you do not have enough data is to use Transfer learning. This helps you to improve the performance of your model on the test data set. So, you can easily use one of the available pre-trained models in technical literature and update its weights based on your data. Take a look at this video. It is very helpful and you get a lot of ...


1

$f(x)$ is convex when $f(a)<f(x)<f(b)$ for every $a<x<b$. Overall, a function with a positive second derivative is convex. The MSE objective is of the form $MSE = ∑(y_{true} − 𝑦_{pred})^2$ The second derivative is positive. So, MSE is convex. You can follow this procedure for the rest of the functions. Here is also another answer to this ...


0

If they are highly correlated, probably you can not easily tell which feature leads to a happy country. My suggestion is to perform multicollinearity test before fitting any model to remove highly correlated features. After that, there a chance that you be able to get more insights about the pattern in your data.


2

It depends exactly on which kind of patterns you are talking about. Are they deterministic? That is, they are all the same, so you want to get everything after Dear, or before Att / Best Regards, you can explore regular expression patterns. In python, you can use re library: https://docs.python.org/3/library/re.html There are books about regular ...


1

As Kashra said, your "system" has an infinite number of valid solutions. However, there is one "canonical" solution, that might make more sense than others, depending what you are after. A matrix is actually a way of writing down a linear operator. A linear operator transforms one vector into another, so when you say $$ A \cdot x = b $$ you are basically ...


0

I was able to find a solution. Thanks to this article which uses LSTM with binary classification modeling: https://www.analyticsvidhya.com/blog/2019/01/introduction-time-series-classification/


0

Regarding your comment The system will likely not have an answer and should be approximated. Do you see a way to do this with a method like Least square` Yes you can do that. The Linear Regression is done using this method. If the b is not in the column space of A, to get an approximated solution, the vector b is projected onto the column space of A. ...


-1

You could also inform the model of the imbalance itself (either a True/False or a "class weight") depending on which modelling method you are using.


1

Here you actually do not have a system of linear equations that needs to be seen at a whole and solved together. Here you have 3 independent equations, each of them with infinite valid answers. So: $\begin{bmatrix} a_{1} & a_{2} & a_{3}\\ a_{4} & a_{5} & a_{6}\\ a_{7} & a_{8} & a_{9} \end{bmatrix}\times \begin{bmatrix} x_{1}\\ x_{2} \...


1

This is a method for evaluation of two clusterings in the presence of class labels so it is not proper for real clustering problems in which class labels are not available. Imagine you have class labels and you want to evaluate a clustering or (compare two clusterings). The most natural idea is to use Purity score. It simply checks labels with clusters and ...


Top 50 recent answers are included