35

Maybe, but note that this is one of those cases where machine learning is not the answer. There is a tendency to try and shoehorn machine learning into cases where really, bog standard rules-based solutions are faster, simpler and just generally the right choice :P Just because you can, doesn't mean you should Edit: I originally wrote this as "Yes, but ...


28

No. Gradient descent is used in optimization algorithms that use the gradient as the basis of its step movement. Adam, Adagrad, and RMSProp all use some form of gradient descent, however they do not make up every optimizer. Evolutionary algorithms such as Particle Swarm Optimization and Genetic Algorithms are inspired by natural phenomena do not use ...


26

Yes. Very importantly, YOU decide the architecture of a machine learning solution. Architectures and training procedures don't write themselves; they must be designed or templated and the training follows as a means of discovering a parameterization of the architecture fitting to a set of data points. You can construct a very simple architecture that ...


18

On Kaggle, LightGBM is indeed the "meta" base learner of almost all of the competitions that have structured datasets right now. This is mostly because of LightGBM's implementation; it doesn't do exact searches for optimal splits like XGBoost does in it's default setting (XGBoost now has this functionality as well but it's still not as fast as LightGBM) but ...


14

I think the answer of @ShubhamPanchal is a little bit misleading. Yes, it is true that by Cybenko's universal approximation theorem we can approximate $f(x)=x^2$ with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $\mathbb{R}^n$, under mild assumptions on the activation function. But ...


13

Occam’s razor principle: Having two hypotheses (here, decision boundaries) that has the same empirical risk (here, training error), a short explanation (here, a boundary with fewer parameters) tends to be more valid than a long explanation. In your example, both A and B have zero training error, thus B (shorter explanation) is preferred. What if ...


11

Since you are going to minimize later on the log likelihood, there is actually no big difference between $\log 2^x=x * \log2$ and $\log e^x=x$. You see the difference is simply a constant. Nevertheless one could argue to use $2^x$ instead of $e^x$ und also use $\log_2$ instead of $\log$ when it comes to the optimizing step. In fact it is possible to use $2^x$...


10

Yes, using one-hot encoding on 24k features requires 24k input nodes. However this should not be a problem for Keras (or any other deep learning library). Natural language processing often uses one-hot encoding on words with a vocabulary size in the same ballpark. If you are using a "deep" model, one of your hidden layers should take care of reducing the ...


9

Neural networks are also called as the universal function approximation which is based in the universal function approximation theorem. It states that: In the mathematical theory of artificial neural networks, the universal approximation theorem states that a feed-forward network with a single hidden layer containing a finite number of neurons can ...


9

You are correct. For $n > 1$, the multiplication of derivatives does not necessarily go to zero, because each derivative could be potentially larger than one (up to $n$). However, for practical purposes, we should ask ourselves how easy it is to maintain this situation (keeping the multiplication of derivatives away from zero)? Which turns out to be ...


8

There's nothing about a recommendation system that absolutely necessitates some kind of machine learning. Indeed, I've seen decision systems in use that were essentially just someone's idea about what the customer's preferences ought to be. A recommender can be based on anything from a few ad-hoc 'common sense' rules, to a logistic regression someone did on ...


8

According to the title: No. Only specific types of optimizers are based on Gradient Descent. A straightforward counterexample is when optimization is over a discrete space where gradient is undefined. According to the body: Yes. Adam, Adagrad, RMSProp and other similar optimizers (Nesterov, Nadam, etc.) are all trying to propose an adaptive step size (...


8

You cannot really use k-means clustering if your data contains categorical variables since k-means uses Euclidian distance which will not make a lot of sense with categorical variables. Check out the answers to this similar question. You can use the following rules for performing clustering with k-means or one of its derivates: If your data contains only ...


8

Given you have images stretched out as columns in a table with ~48,500 rows, I am assuming you have the raw images that are 220x220 in dimension. You can use a function available via OpenCV called inpaint, which will restore missing pixel values (for example black pixels of degraded photos). Here is an image example. Top-left shows the image with missing ...


8

If k-fold cross-validation is used to optimize the model parameters, the training set is split into k parts. Training happens k times, each time leaving out a different part of the training set. Typically, the error of these k-models is averaged. This is done for each of the model parameters to be tested, and the model with the lowest error is chosen. The ...


8

Data science jobs cover a wide range of different activities so any answer is likely to be subjective. I'm in academia so my knowledge of the job market is limited, but from what I can see: The current context is very favorable to data scientists looking for a job, so anybody with some basic knowledge of ML has a chance. You're already above this level so ...


8

A weak learner can be either a classification or a regression algorithm: Boosting (Schapire and Freund 2012) is a greedy algorithm for fitting adaptive basis-function models of the form in Equation 16.3, where the $\phi_m$ are generated by an algorithm called a weak learner or a base learner. The algorithm works by applying the weak learner sequentially ...


8

To decide which strategy is appropriate, it is important to investigate the mechanism that led to the missing values to find out whether the missing data is missing completely at random (MCAR), missing at random (MAR), or missing not at random (MNAR). MCAR means that there is no relationship between the missingness of the data and any of the values. MAR ...


7

Which two accuracies I compare to see if the model is overfitting or not? You should compare the training and test accuracies to identify over-fitting. A training accuracy that is subjectively far higher than test accuracy indicates over-fitting. Here, "accuracy" is used in a broad sense, it can be replaced with F1, AUC, error (increase becomes decrease, ...


7

No, your model isn't supposed to know about your test data, if you include clues in your training about what's in your test data , you will do something called Data Leakage. Data leakage would lead to Over-fitting which will give you good results on that particular test set, but won't generalize to other data. Lets say, you deploy this model in production ...


7

Welcome to Data Science SE! Well, we say that most of our jobs is to wrangle with data, and that is because data is usually trying to deceive us... jokes aside: You have a missing data problem that means your have to clean your data and fill those missing values. To perform this cleaning process your must take the most classic statistician inside of you ...


7

Your decision boundary is a surface in 3D as your points are in 2D. With Wolfram Language Create the data sets. mqtrue = 5; cqtrue = 30; With[{x = Subdivide[0, 3, 50]}, dat1 = Transpose@{x, mqtrue x + 5 RandomReal[1, Length@x]}; ]; With[{x = Subdivide[7, 10, 50]}, dat2 = Transpose@{x, mqtrue x + cqtrue + 5 RandomReal[1, Length@x]}; ]; View in 2D (...


7

Conv1D is used for input signals which are similar to the voice. By employing them you can find patterns across the signal. For instance, you have a voice signal and you have a convolutional layer. Each convolution traverses the voice to find meaningful patterns by employing a cost function. Conv2D is used for images. This use case is very popular. The ...


7

Your understandings are right. deriving the margin to be $\frac{2}{|w|}$ we know that $w \cdot x +b = 1$ If we move from point z in $w \cdot x +b = 1$ to the $w \cdot x +b = 0$ we land in a point $\lambda$. This line that we have passed or this margin between the two lines $w \cdot x +b = 1$ and $w \cdot x +b = 0$ is the margin between them which we ...


7

Yes - Machine learning can learn to find the maximum in a list of numbers. Here is a simple example of learning to find the index of the maximum: import numpy as np from sklearn.tree import DecisionTreeClassifier # Create training pairs where the input is a list of numbers and the output is the argmax training_data = np.random.rand(10_000, 5) # Each list ...


7

Not every seed is the same. Here is a definitive function that sets ALL of your seeds and you can expect complete reproducibility: def seed_everything(seed=42): """" Seed everything. """ random.seed(seed) os.environ['PYTHONHASHSEED'] = str(seed) np.random.seed(seed) torch.manual_seed(seed) torch.cuda.manual_seed(seed) ...


6

I've been working on a drag-and-drop neural network visualizer (and more). Here's an example of a visualization for a LeNet-like architecture. Models with fan-out and fan-in are also quite easily modeled. You can visit the website at https://math.mit.edu/ennui/ The open-source implementation is available at https://github.com/martinjm97/ENNUI.


6

Method 1 : zip the file Upload the zipped file, there is an Upload button under the Files Section. Unzip it using the command on colab : !unzip level_1_test.zip Method 2 : upload the zip file to the google drive account. The only difference is in step 2 where in place of the GUI upload option you can run the google code_snippets to upload download ...


6

You can do this to get rid of the deprecation messages from sklearn.preprocessing import OneHotEncoder from sklearn.compose import ColumnTransformer ct = ColumnTransformer( [('one_hot_encoder', OneHotEncoder(), [0])], # The column numbers to be transformed (here is [0] but can be [0, 1, 3]) remainder='passthrough' # Leave ...


Only top voted, non community-wiki answers of a minimum length are eligible