New answers tagged deep-learning
0
As I understand them, Bayesian optimization approaches are already somewhat robust to this problem. The evaluated performance function is usually(?) considered noisy, so that the search would want to check nearby the "best solution" $h$ to improve certainty; if it then finds lots of poorly performing models, its surrogate function should start to ...
1
RNNs and CNNs are not mutually exclusive! It might seem that they are used to handle different problems, but it is important to note that some types of data can be processed by either architecture. For instance, RNNs uses the sequences as the input. It should be mentioned that sequences are not just limited to text or music. Sequences can also be videos, ...
0
When I think about RNNs applied to Computer Vision, two main research areas of Deep Learning come up to my mind:
Image Captioning: Neural Networks trained to produce descriptions of images. In that case, you have a Conv ecoder that processed pixel data, and an RNN decoder that produces a description.
Video processing (I don't have a better term). Anything ...
0
$n_X$ is the number of feature, $400$ is the number of data.
Each of the entry of $A$ is the output of the sigmoid layer, it is between $0$ and $1$. We can then decide a threshold (typically $0.5$) such that if it is at least the threshold, we map it to $1$, otherwise, we map it to $0$.
1
The fourth dimension is because it is referring to either the full dataset(train/test) Or an individual batch.
600 - Number of images in the dataset or batch
64 x 64 - Size of each image
3 - Number of channels
1
600 can be the number of images of shape 64x64x3 and not only one image.
0
try without list operator []:
predictor.predict('I am very happy to meet you!')
0
One option is not to measure the performance of the hyperparameters on the loss function of the training data but measure performance of the hyperparameters on the elevation metric on the validation data. The end goal of the most machine learning systems is the ability to predict on unseen data. Focusing on "best solution" as measured by loss ...
2
Most deep learning frameworks have APIs that are significantly similar to NumPy. I recommend you take a look at PyTorch as it will let you refactor your code reasonably intuitively to make use of your GPU via Cuda. Speaking as someone who has coded a neural network in NumPy, I would highly recommend learning a popular deep learning framework. It will be ...
1
So the question asks between the difference between an attention vector and a positional vector.
To answer this question, will give some context into how the transformer differs from a sequential model, such as RNNs and LSTMs.
In the case of RNNs and LSTMs, data is fed sequentially "one-by-one" into the model to predict the output (whether that is ...
0
A student of mine did eventually end up doing exactly this:
https://noisemix.github.io/
data generation for natural language
pip install noisemix
She showed that nosification brought significant improvements on tasks like classifications.
However, there is much more to do, and noise is often task- and domain-specific.
0
It's because you need to fine-tune BERT for your specific task anyway. You can train it to classify based on either cls token, or mean of token outputs, or whatever.
In essence, CLS token of the last layer has connections with all of the other tokens on the previous layer. So, does it make sense to average manually?
0
I found this picture helpful for a quick explanation of dilated pooling.
0
If you need to make complex annotations of images try to use labelstud.io.
1
The size of the document on which NER is run shouldn't be a problem at all, a standard NER system scans the document sequentially and just marks any entity it finds.
The size of the entities to find might be more of an issue, because typical NER systems rely on the previous few words to detect the boundaries of an entity. If the entity spans a large sequence ...
0
This cannot be answered easily and is not really a good question. Do you expect a better performance based on your knowledge or some example?
Some basic questions to ask yourself:
Is the data big enough to learn from?
Is the model complex enough to learn and generalise?
Is the training going well (overfitting, underfitting)?
I would advise to either use a ...
2
If you want a DL approach, I recommend substituting the tf-idf by some kind of word embeddings.
For instance, you can take a pre-trained word embedding model, like glove, and average its outputs both in resume and job description, and then compute cosine similarity. However, I recommend to use a contextual word embedding (BERT-like), as the terms in resumes ...
0
From the documentation:
inputs: A 3D tensor with shape [batch, timesteps, feature].
Are you saying that you want 1 input and 1 feature, but you want to output 100 neurons? I would consider adding more timesteps.
train_X = train_X.reshape(train_X.shape[0],10,1)
test_X = test_X.reshape(test_X.shape[0],10,1)
Also, I wouldn't add regularization to a ReLU ...
1
Theoretically, you could take many pictures and map these pictures to the score of each player. However, I would advise against it. First, you would need plenty of pictures and it might be infeasible to cover all possible game scenarios. Second, game scoring is discrete whereas a traditional neural net would approach it as a regression. This means that your ...
0
I think you should use a pre-trained neural network for image recognition and adjust the weights to detect the individual objects you need.
Afterwards, you will need to combine this with some good old-fashioned scripts to manually calculate the score.
Deep Learning doesn't do magic, even less with < 100 pictures of a game. If you managed to take a really ...
1
I'll refer to an answer for a similar topic. If you have enough data for classes 2 and 3, there's no reason to change your training scheme if you use standard metrics. The baseline should always be training without changing the weights, and if you see that the model does very bad on classes 2 and 3, you can change the training scheme. However, I have rarely ...
0
If your dataset is too small, it won't apply to other new data easily. In this case, you should either:
try to increase your training dataset
Find new images and classify them to increase the training data size, the model will improve as you add new images, but this can be time consuming
use transfer learning
Find a model that someone else built on a ...
1
It looks like the new data has a different distribution from the training data. It looks like the training data is just a single fruit, with white background, and the new image you've passed is a picture of bananas with blue background. The model has probably learned something like: if blue image, then blueberries, and for this reason it classifies the blue ...
1
Class weights make sense only in the context of a loss function. When you validate your model you are making predictions and comparing to ground truth using a metric - but in that phase you aren't propagating back any changes, so weights are useless.
0
Sorry that this isn't a concrete answer, but I can offer some advice.
It sounds like you have a problem of many weak relationships. In this case, I think xGBoost or RandomForest would yield better results than Logistic Regression.
Also remember that preprocessing your data and creating new features might help more than choosing a different algorithm. ...
0
It sounds like you have good data - 50 columns and 100,00 rows!
I would do exploratory data analysis (EDA) and look for variables (columns) that are correlated with the response variable (re-offending) but NOT correlated with each other. If you can find a handful (~10) of these then you can build an excellent regression model.
Other techniques to try could ...
0
One idea is to use a shallow (a handful of layers) CNN. Deep CNN's are good at detecting objects in images. Shallow networks focus on lower-level features, such as edges and textures and colors. So, my suggestion is to train a simpler network on a larger training set by training it on image patches rather than on full cards. If the majority of your patches ...
0
#Here image is your batch.
# Add "batch" and "channels" dimensions
image = image[tf.newaxis, ..., tf.newaxis]
image.shape.as_list() # [batch, height, width, channels]
tf.image.resize(image, [height,width])[0,...,0].numpy()
0
You are describing a problem of supervised learning with multiple inputs. That is not an uncommon task and you can find many tutorials about multiple inputs for neural networks out there. Using Tensorflow, I personally recommend Keras Functional API for this task, since it gives you more control on the layers while keeping the high-level simplicity.
0
Imagine you have a bunch of points that lie roughly on the line y=x. Although you could find a polynomial that passes through each and every single point you could argue that the line y=x is a better approximator because it doesn't fit to the noise of each point.
That is the point of regularization. When you have a network with smaller weights a few small ...
2
It is better to go with a good GPU than CPU. Most of the time you will train your model on a GPU which gives you around 100x speed boost compared to CPU. So i guess your first config is good enough for normal DL task.
3
The way most people gain an initial understanding of label smoothing (and what most common explanations have to say on the subject) plays a great role in how one would approach this question.
At first glance, label smoothing is exactly what the name suggests: we modify the labels or some portion of them in order to get a better, more general, more robust ...
0
See the docs of keras
import tensorflow as tf
model.compile( ..., metrics=[tf.keras.metrics.Precision(), tf.keras.metrics.Recall()])])
0
Label Smoothing is a regularizer technique that is applied to target value so that the model can learn the data well without overfitting. There is no need to do label smoothing for validation.But even if you do it, it won't be problem.
0
Changing the batch size will not change the overall training time too much. Since with any batch size you are passing almost 80K images.
One(and the best) approach will be to use transfer learning.
If you have a compelling reason to do full training, you will need a GPU powered bigger hardware. Google Colab can be an option. There are many other options ...
0
google has published a word embedding that has embedding dimension of 300. Following the rule you have given it should have trained on $300^4 = 8.1*10^9$ words. If google is using ngrams instead of just words, then it seems plausible.
0
When you use keras fit, pass the value for x as a generator function which will provide (perhaps using yield) the batch of data (x, y) tuple. Also in the generator function, you can use checkpoint.
https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit
0
AFAIK label smoothing comes into picture while calculating the loss while training. There is no loss computation during validation.
0
Theoretically speaking, there aren't any disadvantages to having too much or too few data. It will only reflect in the overall performance of your model. Based on the Sherlock paper, it seems that it's a choice they made for their preprocessing. This is their explanation:
Certain types occur more frequently in the VizNet corpus than
others. For example, ...
5
Optimizers evolved with small Fix/Improvement on the previous one. So, if you will read in sequence, you will have a better understanding. In this context, RMSProp was a fix on Adagrad and it was an improvement on Momentum.
Let's see this Loss surface which is like a Valley (Imagine a River)
$\hspace{2cm}$
$\hspace{5cm}$Image source - http://d2l.ai/
Momentum ...
0
The one that most suits your demands would probably be "Pattern Recognition and Machine Learning" by C. M. Bishop, since it's both a classic starting book in the field and also contains an algebraic approach.
Additionally, you might check out the "Deep Learning and Applied AI" course taught by Prof. Rodolà (in the MSc in Computer Science ...
0
Standard implementations of PCA calculates statistics across the entire dataset in order to find the projection that has the greatest variance. The entire dataset needs to be loaded into memory for that calculation.
You are using ImageDataGenerator to generate synthetic variations of the data, greatly increasing the size of the training set that has to be ...
0
Try the following code. As you want all first value of zeroth dimension.
test=test[:,0,0]
0
If you are using Jupyter Notebook you should run the following code to clear your GPU memory so that you train perfectly
import gc
gc.collect()
If the problem still persists ,use smaller batch size like 4.
3
To build on the previous answer:
In transfer learning, the goal is to use a pre-trained model and tweak the model to then specialise it to suit a certain task.
So, what we do is, as SrJ has eluded to, keep the main model's architecture in tact. So this would be the 6 CNN layers (and possibly the three linear layers, if they were also involved in pre-training)...
2
RmsProp is a adaptive Learning Algorithm while SGD with momentum uses constant learning rate. SGD with momentum is like a ball rolling down a hill. It will take large step if the gradient direction point to the same direction from previous. But will slow down if the direction changes. But it does not change it learning rate during training. But Rmsprop is a ...
1
The conditional distribution of $Y$ when $X=a$ is bimodal. The mean is in the middle, and reporting it as such is correct (the mean of $1,2,3,91,92,93$ is indeed $47$).
This will be reflected somewhat in the variance being extremely wide. If you want to model a full distribution, you could consider quantile regression at many quantiles. I found the following ...
1
That data can be modeled as a statistical process, where the distributions and parameters change as a function of x. This is in contrast to modeling it as a typical statistical distribution which assumes the same distribution and parameters throughout a range.
The "a" range could be modeled as a bimodal distribution and the "b" range ...
0
You should retrain the last linear layers and keep the CNN layers unchanged.
0
It will not
In a very simple language, the model learns the characteristics in terms of the feature and map to result that you provide as a class. That mapping can be very simple or very complex(a big neural network)
What ever data it will get, it will divide the full space and everything will be mapped to a Class.e.g. >10000 will be mapped to Class-A ...
Top 50 recent answers are included
