New answers tagged

0

It appear that I forgot one point of the ANN, at least forgot one of its effects : the activation function. It is true that for linear activation, multi-layer can be reduce to a single-one, but with a non-linear function, a two-layer neural network can be proven to be a universal function approximator. Sources However, it is true that I dont ...


1

1) It is difficult to interpret a given weight on a given node for a given instance, especially in hidden layer. 2) You can only deal with specifical cases : Weights being very low for a given variable for all instance on the first layer means that you can remove the variable (If the variable is relatively uniform, standardized). For some activation ...


0

As a general answer for hyperparameter tuning, you have to try both and see what works better for your problem. I suspect that some (if not most) of general tuning rule have been observed on a given problem / with a given architecture. (for exemple the He paper is about vision, including convolutional layers). As for keras choice, sometimes, for practical ...


1

I am afraid there is no real answer to your question. But to still answer it I can only advise you to look for practical articles relating to your specific problem. May I suggest you to start with this very general article by LeCun : http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf ? Citing the introduction directly : "Designing and training a network ...


0

Pass a dictionary in the following format to class_weight parameter in fit_generator: { 'output1': {0: ratio_1 , 1: ratio_2} , 'output2': {0: ratio_3 , 1: ratio_4}} You can use class_weight from sklearn.utils to calculate class weights from your data References: https://github.com/keras-team/keras/issues/4735#issuecomment-267473722 https://scikit-learn....


2

The main problem I see here is that OHE is almost never a good idea with that many categories. With neural networks you will usually get better performance by using embeddings. So instead of X1 -{OHE}-> 10,000 -> {..} -> 1,000 you could go straight to X1 -{embedding}-> 50, where the embedding dimension should probably a lot lower than 1,000. ...


0

@Sammy below are the 2 images which i have mentioned in comment where im unable to find yhat =Way+b since our goal is to find y ,how can Way(weights * y) come into picture for prediction of Y ?


0

Keep in mind how matrix multiplication works with regards to the dimensions: Multiplying a matrix with dimensions $n,m$ by a matrix with dimensions $m,k$ results in a matrix of size $n,k$. Therefore, you can add as many rows as you like to the second matrix with no change to the shape of the result of the matrix multiplication. But of course the first ...


1

o(t) is not the result of concatenation of h(t-1) and x(t), but a simple matrix multiplication. See wikipedia for further details: https://en.wikipedia.org/wiki/Long_short-term_memory


0

Don't know if this is what you need but I know of the Ampligraph library: Python library for Representation Learning on Knowledge Graphs https://docs.ampligraph.org


4

Don't get hung up on the word "govern" here. $W_{ax}$, $W_{ay}$ and $W_{aa}$ are simply the weights and they play in principle the same role weights play in feed forward network (except that feedforward networks do not have $W_{aa}$): $W_{ax}$ are the weights from your input layer to the first hidden layer (just as they are in feedforward networks) $W_{ay}$ ...


1

I'm not sure that I understand every part of the process but there is one clear issue with it: because the CV is applied in the inner loop, there is a serious risk of overfitting the model with respect to the other parameters (feature subset, model type, sampling technique). Depending on the goal, this is not necessarily wrong but it's important to interpret ...


0

It really seems like a problem of formulating your probkem. On your 2D exemple you can split your $X_i$ along your axes and juxtapose them to build a data set. Each $X_i$ has two components : along axis 1 ($X_{i1}$) and 2 ($X_{i2}$). Basically you would just build your data set by juxtaposing the 18 columns : $X_{11}$ , $X_{12}$ , ..., $X_{i1}$ , $X_{i2}$ ,...


0

My guess: I agree with your colleagues. I see no reason to do anything other than a single neural network with multiple outputs. If necessary, increase the capacity of that single neural network until you see no further improvement. An stacking ensemble where you have a few neural nets whose inputs are fed as input into another neural network is itself ...


2

My question is, when we calculate partial derivative with respect to one parameter (e.g. weight between input x1 and 1st hidden layer neuron) then we are treating all other weights and biases as constants and we are evaluating how will cost function change if we were to take a step in the direction that is represented by that particular weight. Is this ...


0

Another option that u can do is instead (1-yi) one can use (-1-yi) in cross-entropy formula. Hp(q)=−1/N ∑i=1N(yi)log(p(yi))+(-1-yi)log(1−p(yi)).


1

One thing you can do, is forcing your labels (${-1,1}$) to be ${0,1}$ using this simple linear transformation: \begin{equation*}\hat{y} = (y + 1) / 2\end{equation*} This way, -1 maps to 0, and 1 maps to 1. For practical purposes, you can either change the outputs and labels of your model directly (before applying the original BCE), or slightly change to ...


0

In order to understand why it is a good value to use when determining new weight, we have to understand the maths behind backpropagation and what happens by updating weights at each iteration. Backpropagation uses the chain rule method to calculate the new weights. At each iteration, the weights are updated with the hope that we are converging towards an ...


0

The objective of back-propagation is to isolate the effect of each weight on the total error. Once the effect is isolated, each weight can be changed individually such the total error is minimized. During back-propagation, the effects of other weights are automatically isolated. Take, for example, the function: $$ E_{total}(x) = w_1x + w_2x^2 \\ $$ Then $...


1

Are you aware of this article/tutorial? https://machinelearningmastery.com/how-to-control-the-speed-and-stability-of-training-neural-networks-with-gradient-descent-batch-size/ I did not study for this answer, but I would guess that when you assure that the batches are randomized, a smaller learning would give the same result. My guess would be that you ...


0

Your MAE_val is: MAE_val= np.mean(np.absolute(y_val - pred_val )) On the other side, you fit your model on: history = model.fit(x_train, y_train, ... So you are calculating them on different objects. Training data on one side, and validation set for a final evaluation.


0

I created a rule to achieve reproducibility: Works for python 3.6, not 3.7 First install Keras 2.2.4 After install tensorflow 1.9 And finally in the code: import numpy as np import random as rn import tensorflow as tf import keras from keras import backend as K #-----------------------------Keras reproducible------------------# SEED = 1234 tf....


5

I will try to answer your question as shortly as possible. Yes, if you define probability as a risk, then the probabilities are risk scores. But, there's a catch in these scenarios, you will have to include the prevalence of a class to calibrate them. If Person A has a risk score of 0.9 but you have observed that the positive class is only 20% of the data, ...


0

If NaNs are not too frequent you can try to drop the rows containing it. Otherwise if your algorithm can't handle NaNs (i.e. lightgbm that is also a tree based algorithm can) you have to fill those values somehow. Usually it's a good idea to use a value that is out of the range of normal values (like -99 for a variable with only positive values) so tree can ...


1

I don't think machine learning is the right tool. It's difficult (at least for me) to formulate this as a learning problem. You could consider genetic algorithms as an alternative approach. Based on your description of how you do the matching by hand, it seems like the following assumptions hold: You're looking for an assignment of payments to loans such ...


1

I got it. You define a new model, which has an input, the shared embedding layer and a flattened output. Pass the output of .predict() from that model to y parameter of your main model's .fit() call, in like fashion: NUMERIC_FEATURES = [ # Define the subset of features that need passing to the numeric input layer ] vocab_size = 10000 # number of items ...


0

There are few online tools available that give you ability to draw "canned" CNN diagrams like NN-SVG EXAMPLE : Another popular choice seems to be InkScape There is always PowerPoint


1

In a narrow sense backpropagation only refers to the calculation of the gradients. So it does, for example, not include the update of any weights. But usually it is used refering to the whole backward pass. Also see Wikipedia: https://en.m.wikipedia.org/wiki/Backpropagation


1

In brief, backpropagation references the idea of using the difference between prediction and actual values to fit the hyperparameters of the method used. But, for applying it, previous forward proagation is always required. So, we could say that backpropagation method applies forward and backward passes, sequentially and repeteadly. Your machine learning ...


0

Well the critical step for finetuning/transfer learning any model in tensorflow[1,2+]: use right preprocessing for images (find the preprocessing which requires mobilenetv2) start with "imagenet" weights change the BatchNormalization behavior, in tf the default momentum value is 0.99 which is not best, in pytorch it is set to 0.9 which is better when ...


0

As far as I know, most researchers use general drawing tools to visualizes neural network architectures. Although there are some libraries to do it automatically. However, the visualizations in your paper are quite important to get your ideas across and using custom hand-made visuals allows you to visually explain your work in the most appropriate way (not ...


0

Several answers have been provided to the role of equivariance or invariance question "What is the difference between “equivariant to translation” and “invariant to translation”. Depending on how local they are, scalar and Convolution operators tend to be equivariant, max/min or range are more invariant, and subsampling/pooling can somewhat link those ...


3

As far as I can tell, you use a "normal" Logit in one approach and a Logit with L1 penalty in the other case (penalty': ['l1']), which is called "Lasso". In Statsmodels L1 penalty would be implemented like stated in the docs. Lasso and "normal" Logit are two different approaches. In the first case, (some) parameters are shrunken and can be set to zero, ...


0

Plot the loss of both train amd test loss on single graph after every epoch and check that it should not overfit or underfit..most commenly overfit happens in deep learning so keep track on that. Once the your distance between both train and test loss increases on graph stop at that time and save the weights. You can also keep track of weights of every ...


0

If the distribution of the test and training sets are different, the metrics will be fairly different. And in cases of imbalanced classes, accuracy is not a great measure. Consider using precision, recall, or f-score. See if they improve over epochs.


2

Yes, we can use the Discriminator of the GAN to classify images. But we should make sure that the images produced by the Generator are real looking. If you have trained your GAN on a large number of images and it is performing pretty well on the dataset then I insist you to treat the Discriminator model as a pretrained model ( like we do in transfer ...


1

Is it possible to train new model with this kind of data Yes, you need Convolutional Neural Networks (CNN) for image classification. If you only have one image per product in your dataset, I suggest you to use a lot of image augmentation, a technique that is meant to artificially increase the size of an image dataset by applying combinations of distortions ...


5

I need a way to supress previous answers. The answer to the question depends on the particular network that you are using. What kind of generative model is it? GAN? Autoencoder? Seq2Seq RNN? Having said that, if your network keeps outputting the same result, this is usually referred to as "mode collapse", which is typical for GANs. If your network is a GAN,...


1

Three ideas come to my mind (from simple to complex) Include an additional category for anything which is not a number and train your network on these $k+1$ categories. Apply another predictor in the first place which has been trained to differentiate between "number" and "no number". Iff the input is classified as a number you then run your number ...


1

I think the best way would be to augment some data and have an additional output class "unknown". However, if that is not possible or the neural net can not be retrained I would compare the distribution of the outputs of a hidden layer. For the CNN architecture below, calculate the empirical distribution for the outputs of a hidden layer after the flatten ...


1

A manifold is some kind of low-dimensions structure that exists in a higher-dimensional space. The classic example of this is the Swiss Roll dataset, which simply looks like a spiral with values that vary monotonically along the curves (represented by colors here). . The overall idea is that there is a simple, 1-dimensional representation of the color ...


5

The results seem quite reasonable to me, but I cannot be sure based on your given information. In your table of results, you have shown 10 instances where the uncalibrated probability for the mixed class is between 0.97 and 0.99. It means that on average one would expect only about 2% of misclassifications on these instances. In the small sample that you are ...


1

I know what is cython and make (but I never use YOLO!) Cython is a C-extension for python. It allows you to write code C/C++ in a python script. (use for very fast program execution) Make is command which executes your makefile. You can consider makefile is a build script to create/tune the necessary things like environment/folders/.. etc.


Top 50 recent answers are included