noe
  • Member for 6 years, 1 month
  • Last seen this week
Why are Machine Learning models called black boxes?
Accepted answer
55 votes

The black box thing has nothing to do with the level of expertise of the audience (as long as the audience is human), but with the explainability of the function modelled by the machine learning ...

View answer
Why does Keras need TensorFlow as backend?
51 votes

This makes more sense when understood in its historical context. These were the chronological events: April 2009 Theano 0.1 is released. It would dominate the deep learning framework scene for many ...

View answer
Can BERT do the next-word-predict task?
Accepted answer
19 votes

BERT can't be used for next word prediction, at least not with the current state of the research on masked language modeling. BERT is trained on a masked language modeling task and therefore you ...

View answer
GANs (generative adversarial networks) possible for text as well?
Accepted answer
19 votes

Yes, GANs can be used for text. However, there is a problem in the combination of how GANs work and how text is normally generated by neural networks: GANs work by propagating gradients through the ...

View answer
Why do convolutional neural networks work?
18 votes

ConvNets work because they exploit feature locality. They do it at different granularities, therefore being able to model hierarchically higher level features. They are translation invariant thanks to ...

View answer
Neural Network for Multiple Output Regression
17 votes

What you are describing is a normal multidimensional linear regression. This type of problem is normally addressed with a feed-forward network, either MLP or any other architecture that suits the ...

View answer
What is a channel in a CNN?
Accepted answer
17 votes

Let's assume that we are talking about 2D convolutions applied on images. In a grayscale image, the data is a matrix of dimensions $w \times h$, where $w$ is the width of the image and $h$ is its ...

View answer
Why are autoencoders for dimension reduction symmetrical?
Accepted answer
16 votes

There is no specific constraint on the symmetry of an autoencoder. At the beginning, people tended to enforce such symmetry to the maximum: not only the layers were symmetrical, but also the weights ...

View answer
Is BERT a language model?
Accepted answer
15 votes

No, BERT is not a traditional language model. It is a model trained on a masked language model loss, and it cannot be used to compute the probability of a sentence like a normal LM. A normal LM takes ...

View answer
Why is the decoder not a part of BERT architecture?
Accepted answer
14 votes

The need for an encoder depends on what your predictions are conditioned on, e.g.: In causal (traditional) language models (LMs), each token is predicted conditioning on the previous tokens. Given ...

View answer
Is (manual) feature extraction outdated?
Accepted answer
14 votes

In the general case, this is by no means true. Let's break down the case for different data scenarios: For discriminative image models (e.g. image classification/labeling) this is true for some ...

View answer
model.cuda() in pytorch
Accepted answer
13 votes

model.cuda() by default will send your model to the "current device", which can be set with torch.cuda.set_device(device). An alternative way to send the model to a specific device is model.to(torch....

View answer
Detecting anomalies with neural network
Accepted answer
13 votes

From the formulation of the question, I assume that there are no "examples" of anomalies (i.e. labels) whatsoever. With that assumption, a feasible approach would be to use autoencoders: neural ...

View answer
What is the difference between active learning and reinforcement learning?
Accepted answer
11 votes

Active learning is a technique that is applied to Supervised Learning settings. In the supervised learning paradigm, you train a system by providing inputs and expected outputs (labels). The system ...

View answer
what is the first input to the decoder in a transformer model?
Accepted answer
11 votes

At each decoding time step, the decoder receives 2 inputs: the encoder output: this is computed once and is fed to all layers of the decoder at each decoding time step as key ($K_{endec}$) and value (...

View answer
Transformer model: Why are word embeddings scaled before adding positional encodings?
Accepted answer
10 votes

This is specified in the original Transformer paper, at the end of section 3.4: Transcription: 3.4 Embeddings and Softmax Similarly to other sequence transduction models, we use learned embeddings ...

View answer
Monitoring machine learning models in production
Accepted answer
10 votes

The changes in distribution with respect to training time are sometimes referred to as concept drift. It seems to me that the amount of information available online about concept drift is not very ...

View answer
Why is it wrong to train and test a model on the same dataset?
8 votes

It can happen that the model you train learns "too much" or memorizes the training data, and then it performs poorly on unseen data. This is called "overfitting". The problem of ...

View answer
Is there a model-agnostic way to determine feature importance?
Accepted answer
8 votes

Ways to "determine feature importance" are normally called feature selection algorithms. There are 3 types of feature selection algorithms: Filter approaches: they choose variables without using a ...

View answer
Python vs R for machine learning
8 votes

An issue all other answers fail to address is licensing. Most of the aforementioned wonderful R libraries are GPL (e.g. ggplot2, data.table). This prevents you from distributing your software in a ...

View answer
Python stemmer for Georgian
Accepted answer
7 votes

I don't know any Georgian stemmer or lemmatizer. I think, however, that you have another option: to use unsupervised approaches to segment words into morphemes, and use your linguistic knowledge of ...

View answer
How pre-trained BERT model generates word embeddings for out of vocabulary words?
Accepted answer
7 votes

BERT does not provide word-level representations, but subword representations. You may want to combine the vectors of all subwords of the same word (e.g. by averaging them), but that is up to you, ...

View answer
Why does vanilla transformer has fixed-length input?
Accepted answer
7 votes

The restriction in the maximum length of the transformer input is due to the needed amount of memory to compute the self-attention over it. The amount of memory needed by the self-attention in the ...

View answer
Proper masking in the transformer model
Accepted answer
7 votes

I will take as reference fairseq's implementation of the Transformer model. With this assumption: In the transformer, masks are used for two purposes: Padding: in the multi-head attention, the ...

View answer
Variable input/output length for Transformer
Accepted answer
7 votes

Your understanding is not correct: in the encoder-decoder attention, the Keys and Values come from the encoder (i.e. source sequence length) while the Query comes from the decoder itself (i.e. target ...

View answer
What is the purpose of setting an initial weight on deep learning model?
Accepted answer
7 votes

This is greatly addressed in the Stanford CS class CS231n: Pitfall: all zero initialization. Lets start with what we should not do. Note that we do not know what the final value of every weight ...

View answer
What knowledge do I need in order to write a simple AI program to play a game?
7 votes

It highly depends on the type of game and the information about the state of the game that is available to your AI. Some of the most famous game playing AIs from last few years are based on deep ...

View answer
Why is averaging the vectors required in word2vec?
Accepted answer
6 votes

The reason to average the embedded vectors of the words in a paragraph or document is to obtain a single fixed-size vector that represents the whole text. Then, the document-level vector can be used ...

View answer
Why is deep learning used in recommender systems?
6 votes

"Recommender Systems" is a very broad area and can be approached from different optics: latent variable models, graph models, etc. "Deep learning" is an umbrella term for gradient-...

View answer
Bert for QuestionAnswering input exceeds 512
Accepted answer
6 votes

The maximum input length is a limitation of the model by construction. That number defines the length of the positional embedding table, so you cannot provide a longer input, because it is not ...

View answer
1
2 3 4 5
15