MachineLearner
  • Member for 2 years, 10 months
  • Last seen this week
Can a neural network compute $y = x^2$?
22 votes

I think the answer of @ShubhamPanchal is a little bit misleading. Yes, it is true that by Cybenko's universal approximation theorem we can approximate $f(x)=x^2$ with a single hidden layer containing ...

View answer
Do you actually need math for your data science job?
16 votes

Having a solid mathematical background is crucial for data science. Someone without solid mathematical background will always use the algorithms as black box models. Mathematical reasoning is needed ...

View answer
Why should I understand AI architectures?
14 votes

You are right that you actually do not need to know the architectures if you just want to apply them. But there are to reasons why it would be good to understand the architecture. Models often do not ...

View answer
How could I estimate slope of lines on a scatter plot?
Accepted answer
9 votes

The procedure that you can use is the following. First cluster your data with gaussian mixture models. This method should also work with multiple lines with different slopes. It should be able to deal ...

View answer
Using Cross Validation technique for a CNN model
Accepted answer
8 votes

Question 1: Why do most CNN models not apply the cross-validation technique? $k$-fold cross-validation is often used for simple models with few parameters, models with simple hyperparameters and ...

View answer
In softmax classifier, why use exp function to do normalization?
6 votes

This question is very interesting. I do not know the exact reason but I think the following reason could be used to explain the usage of the exponential function. This post is inspired by statistical ...

View answer
Estimating the value of $\pi$ with a Monte Carlo dartboard: $<$ or $\leq$?
Accepted answer
6 votes

Short answer: Both formulations lead to the same answer. Mathematical explanation: In order to understand that let us look at two similar problems. Imagine we want to integrate a function $f(x)=x^2$ ...

View answer
What is the minimum amount of data required for sales prediction with ML
4 votes

As @Ethan said there is no general answer to this question. There are multiple perspectives that you need to take into consideration. Amount of the data: The general tendency is more data will lead ...

View answer
What it means for a neural network to memorize data
Accepted answer
4 votes

Memorization is the same as overfitting. The memory is implicitly represented by your weights. If your network does have enough parameters it will be able to memorize/overfit. In order to understand ...

View answer
Constraining linear regressor parameters in scikit-learn?
3 votes

I am assuming a linear regression of the form $$y = w_0x_0 + w_1x_1+ \ldots w_px_p + \varepsilon.$$ If we combine all output observations into a single vector $\mathbf{y}$ and represent the data ...

View answer
Can machine learning learn a function like finding maximum from a list?
3 votes

I will exclude educated designs from my answer. No it is not possible to use an out of the box machine learning (ML) approach to fully represent the maximum function for arbitrary lists with arbitrary ...

View answer
Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?
Accepted answer
3 votes

Both formulations lead to the same solution if you correctly choose $C$ for both cost functions and if $C>0$. If we have the regularized loss $$J_1=\dfrac{1}{2}\sum_{n=1}^Ne_n^2+\dfrac{1}{2}C\...

View answer
Coefficients of Linear regression for minimizing MSE
Accepted answer
3 votes

For a linear regression we have the loss function $$J(a,b)=\sum_{n=1}^N(y_n-a-bx_n)^2.$$ The partial derivatives are $$\dfrac{\partial J}{\partial a}=2\sum_{n=1}^N(y_n-a-bx_n)(-1)$$ $$\dfrac{\...

View answer
Why does the logistic regression cost function need to be the negative of log?
3 votes

Normally we want to maximize the likelihood (consequently the log likelihood). This is the reason why we call this method maximum likelihood estimation. We want to determine the parameters in such a ...

View answer
How to label overlapping objects for deep learning model training
Accepted answer
3 votes

There is no common practice in labeling the bounding boxes. It is always problem dependent. For example, if you want to count the chickens then you should also label the whole chicken as one instance ...

View answer
What are features for state-action pairs in RL?
3 votes

A feature vector is a vector that is containing basis functions. These basis functions are combining states and actions. We can use a feature vector to approximate our action-value function $q(\...

View answer
Normalizing the data set
3 votes

Why doesn't normalization have any effect on linear regressor performance (mathematical approach is appreciated)? Theoretically, normalization does not influence the performance of the model. In ...

View answer
Which method to use to remove trend from time series?
Accepted answer
2 votes

Detrend does a least squares fit (linear or constant) and subtracts this from your data points. You can look this up in the docs. Simply taking the difference between consecutive data points will in ...

View answer
How and Why to rescale image range between [0,1] and [-1,1]
2 votes

You can use the following scalings $$x’=\dfrac{x}{255} \qquad (1)$$ $$x’=\dfrac{x-127.5}{127.5} = \dfrac{x}{127.5}-1 \qquad (2)$$ for rescaling to $[0,1]$ or $[-1,1]$. The rescaling of inputs tries ...

View answer
How does cost function change by choice of activation function (ReLU, Sigmoid, Softmax)?
Accepted answer
2 votes

You need to discriminate between two types of neural networks. If your output variable is continous you can use linear, ReLU, tanh, logistic-sigmoid,... as activation functions, because these ...

View answer
Neural network for linear regression?
Accepted answer
2 votes

Neural networks are very good function approximators. Hence, they can approximate a wide range of nonlinear functions. Remember that linear functions are easier to represent than nonlinear functions. ...

View answer
Linear Regression
2 votes

The example that you have given is a very trivial case of linear regression but it can still lead to computational problems. Imagine we have sensor inputs and we want to estimate the temperature with ...

View answer
Does Convolution kernel size affect number of channels?
2 votes

Normally a convolutional neural network will get flattened into a single column vector after the convolutions and then maybe be processed by dense layer. In this model, the convolution $1\times1$ is ...

View answer
Categorical vs continuous feature selection/engineering
2 votes

One possibility to deal with categorical inputs is to introduce the category input vector $\boldsymbol{t}$. The category input vector of the $n^{\text{th}}$ observation is given by $\boldsymbol{t}_n=[...

View answer
What is the possible range of SVR parameters range?
Accepted answer
2 votes

I support vector regression the inverse regularization parameter $C$ can be selected from the interval $[0,\infty)$. In which $C=0$ means that we are very heavily regularizing and $C\to \infty$ no ...

View answer
Logistic Regression mapping formula
Accepted answer
1 votes

You get the output of the logistic regression $\sigma$, which is between $0$ and $1$. Default option (is spit out out from most packages): In order to get class labels you simply map all values $\...

View answer
How to evaluate goodness of fit of a sinusoidal model using an F-Test?
1 votes

You could try to use the following method. $$y=a\sin \left[\dfrac{\pi}{12}x-b\right]+ c$$ $$=a\left[\sin \dfrac{\pi}{12}x \cos b - \sin b \cos \dfrac{\pi}{12}x \right] +c$$ $$=a\sin \dfrac{\pi}{12}x \...

View answer
how to choose between data normalization or standadization?
Accepted answer
1 votes

Both are linear transformations. In general you should try both and see which performs better. MinMaxScaler: Has the problem that your features will not have the same range of values after scaling. ...

View answer
Difference between a log scale and linear scale : np.random.rand()
1 votes

Let us assume you want to do hyperparameteroptimization with a hyperparameter $h\in[0,1]$. Let us additionally assume that we want to test hundred possible values for the parameter. If you choose a ...

View answer
Is the _error_ in the context of ML always just the difference of predictions and targets?
Accepted answer
1 votes

The error can have different forms depending on the application. For example for a simple regression we often use the sum of squared deviations between the actual output $y_n$ for the input $x_n$ and ...

View answer