New answers tagged neural-network
2
Even i am getting accustomed with concepts of causal inference and working on it's use case for telecom data. Sharing some of the resources which might be helpful to you:
Causalnex Package
Pyro Tutorials https://github.com/altdeep/causalML
https://github.com/DeekshaD/causalML-lecturenotes (Notes of the course)
Some courses though they may be stat heavy:
...
1
In general, the smaller the batch size the more noisy the gradient updates will be. That could lead to the network being unable to converge or take too long. There are also methods such as batchnorm which require a large enough batch size to effectively compute some statistics.
On the other hand, the larger the batch size the larger the generalisation error.
0
Yes you are mostly correct. A feedforward neural network with a single layer and a sigmoid activation is a logistic regression which belongs to GLM type of models. Your second statement is unclear (weights interact with outputs) so I will try to break this down below:
Non-linear transformations (e.g. polynomial regression, logistic unit etc.) is often ...
1
You probably misanderstood the video, it is not said that a linear layer with Bias set to False is equivalent to :
x = np.dot(weights, x) + biases
Because that is not true, a layer without Bias is equivalent to
x = np.dot(weights, x)
The way he recreates the layer without Bias is actually with the following function :
x = x.dot(l1) # X = W1.X First ...
0
If you want to convert the categories to embeddings using tf.feature_column the best option is tf.feature_column.embedding_column.
0
One option is to convert counts to rate. Rates are always bound between 0 and 1. For example instead of a count of 100 events, the data could be encoded as a rate of .10 (100 events out of 1,000 opportunities).
0
You can answer that question for yourself by profiling and benchmarking the code. Python has Standard library modules for profiling.
0
In the definition of the convolutional layer you defined input shape to be (77, 1), but then your actual input has shape (None, 1, 77). As you can see, the dimensionality of the axes are swapped. They should match.
0
Your dataset doesn't match with the expected input's or inner's shape. You have to reshape them accordingly.
0
Generally speaking, if the requirement is just to classify those sounds as "siren", you will want to group everything as a "siren" sound.
However, if some other sounds (example: a whale sound) could be confused with a specific siren, the generalisation could be a problem and it would be better to learn every siren sound separately to ...
2
They are two implementations of different algorithms.
SHAP offers two model-specific explainer DeepShap and GradientShap for explaining neural network models. The former combine the idea of DeepLift and Shapley values, the latter combines the idea of IntegratedGradients and Shapley values. SHAP also offers a model-agnostic algorithm named KernalShap, which ...
1
There is one obvious problem with this task: the result is not a real review, it's a generated text which looks like a review. Given that the point of a product review is usually to provide the reader with some information about the product, it's not clear to me how this task would be useful: if the review can be made without even testing the product, its ...
0
CBOW and Skipgram are alternative model choices, the only difference is the loss function.
Call it a neural network if you like, but both models are really just a pretty simple loss function involving two matrices of trainable parameters (each with one vector per word). Once trained, one matrix is usually dropped and the other gives the "word embeddings&...
0
As far as I know it's very unlikely that a categorical variable with billions of possible values could be a good predictor for a ML model, but there is certainly some underlying information related to the IP address which are good predictors. So it's a problem of feature engineering not in an technical sense but in a design sense, i.e. using expert knowledge ...
0
No need to split the dataset. You can do this on runtime and using a single data loader. Following is the code for the same and explanations included in comments -
# imports
import torch
from torch.utils.data import Dataset
from torchvision import datasets
from torch.utils.data import DataLoader
# This is how I am downloading MNIST dataset, I use the .pt ...
0
I am very surprised not to find many more questions related to this (now) few years old paper, apart from few blog post blindly copy pasting the content of the article.
I'd be happy to get a clearer answer from someone that actually understand where the blind spot is removed.
0
Are the weights the same for all three classes for training? I have a standard vgg16 modified for 3 classes (cancer images),but the training data was mostly of one class. Until I evened out the weight values (augmentation didn't seem to help as much), I could never get past ~80% accuracy. After about 200 epochs, it kind of just caught on and started showing ...
0
You can pass the weights as follows:
class_weight={'Output_1':None,'Output_2':[1,2,1]}
where Output_2 is a softmax with 3 classes.
0
Everything seems good but you are not taking any outputs from model1, model2 and model_star?
Here is how I would code this thing -
import torch
import torch.nn as nn
import torch.nn.functional as F
class VGGBlock(nn.Module):
def __init__(self, in_channels, out_channels,batch_norm=False):
super(VGGBlock,self).__init__()
conv2_params = {'...
0
Yes, there is a way with TF 2.0, which is to use Ragged Tensors as below:
# Task: predict whether each sentence is a question or not.
sentences = tf.constant(
['What makes you think she is a witch?',
'She turned me into a newt.',
'A newt?',
'Well, I got better.'])
is_question = tf.constant([True, False, True, False])
# Preprocess the ...
-1
It seems as both repositories are for 'network explainers'.
The SHAP repository looks like the official implementation of the algorithms they cite in their citation section.
While DeepExplain repo has a broader cohort of algorithms of method also implementation, which I assume are not the the official implementations.
0
It is important what the numbers in a model mean. Embeddings capture the semantic relationships between entities as distance. This useful for machine learning because algorithms can learn how predict a label based on distance, thus learn how semantic relationships between entities are predictive.
0
I would try to train a neural network with some sort of self-supervised approach, where you take all of your images and you change them in some ways (mess with colors a bit, rotate, rescale, etc.) and the task of the network is to create embeddings for these two to be close together and far from all the other images.
The network will probably have harder ...
1
There are 2 different levels of complexity in a network :
Number of parameters
Number of operations (FLOPs)
It is especially important to make a distinction when using CNN since a convolution kernel is applied on many different pixels, so a same weights will be used in different computations. The ratio $operations/parameters$ is approximately $1$ in a ...
2
In the afforementioned image, we can see that even if Resnet-34 has more number of Convolutional layers, it still has 7-8 times less parameters than VGG-19.
Clearly, Convolutional layers are not at fault. But fully connected layers are!! In VGG-19 there are 3 big fully connected layers after the backbone. On the other hand, Resnet has global average pooling ...
0
The basic premise of transfer learning is that similar data modalities will hold similar relationships.
If the original data has similar relationships between the data points, then, that can be utilized by the smaller data.
So, the question is whether both the data have similar kind of relationships in them.
Case in point, the vision problems which hold ...
0
You probably don't require the use of artificial intelligence for your task, it seems quite 'easy' to do, check out openCV library, it probably has a function that does the job without AI (here is one for example).
The use of IA network may get you a more accurate result tho, so I would do it using classic image segmentation (U-Net is one of the most common ...
0
What is the appropriate approach for training time series data against multiple, consecutive labels?
What you are describing is commonly called incremental learning. Bayesian methods work well with incremental learning. Each posterior estimate would be updated based on the information currently available. In this case, it would the previous model estimates and new features.
0
Neural Networks (NN) model the feature interaction through the non-linear weighting of hidden nodes.
Random Forest (RF) are tree-based models. Tree-based learn feature interaction through recursive, conditional splitting. In the example you mention, a tree-based model would learn to split on feature 1 and then split on feature 2.
0
The problem is dealing with multi-class classification. So, in output layer try of using "SoftMax" as the Activation layer.
2
Current Statistical Learning Theory treats a learning algorithm like a "black-box", analysing its input versus outputs. Besides, it is usually criticised for lack of non-vacuous bounds (Despite the non-vacuous bound proved by Diziugaite and Roy).
Information Bottleneck Theory brings an Information-Theoretic perspective to the learning problem that ...
1
General remark: your two new models give very different results in terms of precision and recall, I find this a bit surprising. I would probably try different learning methods (e.g. decision trees, SVM) in order to investigate if this is really due to the features or not.
a) Absolutely, the threshold should be specific to the model and features, it would be ...
0
There's no thumb rule for designing a NN architecture. Every architecture is designed to perform well on specific problems, so you need to choose one for yourself, which best suites your problem.
You may consider these points while implementing a NN architecture for your problem,
Check if other researchers have worked on similar problems. If yes, review ...
0
Just to make things clear, features of a model are raw model outputs before the fully connected layers, i.e. output of your backbone. So in your code, you get the features of the model as shown in comments -
from model.res50 import ResNet
self.encoder = ResNet()
self.fc = nn.Linear(hidden_dim, num_classes)
def forward(self, data):
out = self.encoder(data)...
0
A quite useful site is this https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/. Some other things that you could do in order to avoid overfitting is:
early stoping, plot the accuracy of validation and test data sets over the time and when the validation accuracy becomes much better stop training your model.
...
1
Batch normalization can be interpreted as an implicit regularization technique because it can be decomposed into a population normalization term and a gamma decay term, being the latter a form of regularization. This was described in the article Towards Understanding Regularization in Batch Normalization, which was presented at the ICLR'19 conference.
Batch ...
2
Is that, we have to crop all the objects in every image and do binary
classification as object vs background for classifying the anchor has
object or not
The RPN gets the input from backbone network(VGG, Resnet etc.) as feature maps.
Here the RPN itself a CNN layer so it will handle different shaped anchors to FC layer.
For the loss calculation, each ...
0
I did not find the ready solution for it. I solve this task by myself.
Increase speed.
from scipy.io.wavfile import read, write
Fs, data = read(filename)
write(destination, int(Fs*1.25), data)
I save the file and increase its frequency by 1.25.
Add noise.
import numpy as np
from scipy.io.wavfile import read, write
Fs, data = read(filename)
...
0
As well known, machine only identify 0 and 1. Therefore, we, for an instance, "encode" characters and symbols with ASCII codes. 0 & 1 can only code two characters. To make sure the code is unique for all characters, we have to use a series number to code them. The hard ware experienced 8, 16, 32, and 64 bits (now 64 bits is most popular).
...
0
In general,
when we discuss this topic, it is around the idea of Stochastic GD, Mini-batch GD, Batch GD. The idea of averaging is to move the gradient towards an average of the Batch.
So, "N" refers to the batch_size in general.
On aggregation at the Output layer, it is more of a design implementation I believe.
Check this example for Keras
import ...
2
In the original paper there are some clarifying statements:
The four inputs to a unit in S2 are added, then multiplied by a trainable coefficient, and added to a trainable bias. The result is passed through a sigmoidal function. (p.7, col.1)
Here, sigmoidal function is generic.
As in classical neural networks, units in layers up to F6 compute.. This ...
0
You can implement MLPClassifier with GridSearchCV in scikit-learn as follows (other parameters are also available):
GRID = [
{'scaler': [StandardScaler()],
'estimator': [MLPClassifier(random_state=RANDOM_SEED)],
'estimator__solver': ['adam'],
'estimator__learning_rate_init': [0.0001],
'estimator__max_iter': [300],
'...
0
As mentioned by Devashish, make use of numpy arrays instead of python lists as they allow for vectorized computations and are much faster. Instead of interating over the python lists and multiplying values you can simply use vectorized functions such as numpy.dot to multiply the weight matrices with the input.
0
If you're looking at how the MSE is implemented in, e.g. Python, you'll see that, in essence, the mean is taken across both dimensions, i.e. features and samples:
def mse(x, y):
diff = x - y
err = np.square(diff)
return np.mean(err)
(when no axis is given as a parameter to np.mean, NumPy takes the mean across all axes)
See, for example, also ...
1
As the answer from 10xAI notes, n in the loss function refers to the number of samples over which you are calculating the loss, meaning that you are basically calculating the average loss for a specific batch of data. Your error is that you are dividing by the number of output neurons, which is incorrect as the number of output neurons/number of classes has ...
1
A trained model is just a well-defined set of floats i.e parameters
If we go for prediction,
What it will do are a large number of multiplication and addition of input data and the parameters. This will be done by the ALU (Both CPU and GPU)
A CPU will have a few ALU and a GPU will have thousands of them. So it will complete its task very quickly.
So, unless ...
1
The results you get will be identical* for identical inputs. So for any practical purposes, the accuracy you get will not depend on whether you use a cpu or gpu.
*up to floating point precision. From this post on the pytorch forum:
Both are implementing the floating point number computation standard.
So they are both correct (even though [they may be] ...
0
I have some discussion about this problem with my friend and found a good solution. The answer is to using CNN not as a classifier but as feature generator (embedded) and store it as vector. The next is to using vector similarities to check whether the person is same or not or who is in the picture.
0
One option is to increase performance on the new domain dataset is regularization. Regularization is the process of adding additional constraints or information to reduce the chance of overfitting.
There are many ways to regularize a CNN - train on more and/or better data, dropout, L1 or L2 regularization, or max norm constraints.
1
Without having more context this is how I would approach this -
Assuming there is not meaning to being 'close' in the prediction. I.E, if real 1001 number was 42, then predicting 41 and 99 is just as wrong.
If we assume no stochastic structure to the series, I.E previous number dosnt affect current number I would use the top 5 most frequent numbers as ...
Top 50 recent answers are included
