# Tag Info

### moments of weight vectors in Adam

It looks like it. The equations in the description of the algorithm in Hands-on Machine Learning as well as the original paper do not differentiate between parameters (weights) in different layers. ...
• 136

### Integrating time context in a machine learning model

You could add more data points for time periods that are most recent, or remove data from older timestamps. Using weights for data may not be a good idea: using a weight would imply increasing the ...
• 120

### Why the label is not explicitly involved in the loss function of skip-gram?

Skip-gram is self supervised, the model uses the current word to predict the surrounding window of context words. The skip-gram loss function is the negative log likelihood of the observed context ...
• 19.3k

### How do you visualize neural network architectures?

For a solution for PyTorch I'd add TorchView. It is as easy as: ...
• 147
1 vote
Accepted

### Are some weight gradients equal?

The problem was that $\frac{∂y'}{∂w_{111}} \neq \frac{∂z_{a1}}{∂w_{111}}$, but $\frac{∂y'}{∂w_{111}} = \frac{∂z_{b}}{∂w_{111}}$, where $z_b$ is the sum of the products of all the z-vectors of the last ...
• 120

### mapping similar strings to same number values

It might be useful to frame this problem using common terminology. Hashing is mapping an object, a string in this case, to an integer. What you want are collisions (i.e., similar objects are mapped to ...
• 19.3k

### How can i implement an confucion matrix?

Confusion matrices are supported by scikit-learn (see also lya Lees answer). In cell 1, you can import it via ...
• 508

### Why do we need XGBoost and Random Forest?

First Question: XGBoost converts weak learners to strong learners. What's the advantage of doing this? Combining many weak learners instead of just using a single tree? Just to get the vocabulary ...
• 191
Accepted

### GPT-2 architecture question

presents are the model hidden states. This return value is used during inference, to avoid recomputing the previous steps' hidden states at every inference step (...
• 18.9k
1 vote

### Understanding correlation - Machine Learning

Unfortunately, things are not as simple. About Correlation For some simple models (especially linear / logistic regression), the correlation between feature and target variable is a good indicator ...
• 508
1 vote

### How can someone build a dataset for a "propensity to purchase" model?

This is a long post with many questions, but I will try my best to answer. Let's start with the terminology: when we say "propensity model", usually we mean predicting future events (e.g. ...
• 587

### In-batch Random Negative Sampling

In the paper you mentioned, the authors are using In-batch Random Negative Sampling (IRNS) for training a recommendation model. IRNS is a technique for training recommender models using negative ...
• 2,846

### Why does Logistic Regression perform better than machine learning models in clinical prediction studies

Clinical trial data is typically collected from a sample population and often has a limited size and number of features. Complex models applied to such data are more likely to overfit, whereas simpler ...
• 68
1 vote
Accepted

### Do models of social systems suffer from prediction drift?

I wouldn't call it normal but it surely is possible. There are several reasons: Changes in fraud patterns: If the model was trained on historical data that is no longer representative of current ...

### In-batch Random Negative Sampling

The authors mean that in each batch, there are 600 pairs, where each pair consists of one positive example (selected randomly from the set of positive examples) and 3000 negative examples (selected ...

### Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?

Why does your distribution contains 14 classes? What about the 102 others? Quick but generic answers: The number of classes always affect performance, because it's always easier to predict the ...
• 24.5k
1 vote

### mapping similar strings to same number values

It seems that a string similarity metric might help. The Levenshtein distance could be useful and it has a python module: ...
• 1,335

### How to forecast a timeseries with geolocation data?

On first impressions, I'm inclined to segregate data for locations which have significance (need not just be home, gym, office), from insignificant ones (travel, traffic) using continuous time spent ...

### How can i implement an confucion matrix?

Without having read too much your code, a confusion matrix states how many elements from class $y_1, y_2, y_3..$ have been associated by the model to class $y_1, y_2, y_3..$ So, for n classes, the ...
• 120
1 vote

### In-batch Random Negative Sampling

From your words, I guess the authors mean that each sample is formed by 3000 negatives and 1 positive, and so each batch is formed by 600(3000+1) examples. Indeed, the authors write that positive ...
• 120

### Best ML models for long term time series forecast

I would probably do it by finding the best equation that fits your graph. If it's linear, I don't think you'll have many problems, it's just matter of performing a linear regression. Else, you can try ...
• 120

### what is static objective function

A stationary point of a function is typically a local minimum or maximum, i.e. the gradient is zero, think of a parabola’s minimum. A stationary function would be a function that is constant wrt. some ...

### NLP to calculate similarity ratio between sentences of max 5-6 words

For calculating similarity scores between 2 short sentences "Fuzz" would work good. String Similarity The simplest way to compare two strings is with a measurement of edit distance. For ...

### Why custom training a Spacy model runs only the Initializing pipeline but the Training pipeline is not running?

Same issue here. My output ends at [INFO] Added vectors: en_core_web_lg And then the cell stops executing. I do not even get Finished initializing or anything. ...
• 1
1 vote

### Can feature engineering avoid overfitting?

Yes, it can be one of the ways to avoid overfitting. Overfitting occurs when a model becomes too complex and learns to fit the training data too closely, which can lead to poor performance on unseen ...
• 68
1 vote
Accepted

### ReLu layer in CNN (RGB Image)

You should put ReLU as the activation of the convolution layers. ReLU is not applied to the RGB values, but to the matrix obtained by convolving the image, also called the filter.
• 120
1 vote
Accepted

### How to split a single feature vector into a layer of 2 neurons

Your idea sounds good to me. By initializing the split variable with a random number between 0 and 1, and then splitting each sample into two neurons using that split value, you'll be randomly ...
1 vote

### How to forecast a timeseries with geolocation data?

You are correct there are many ways to model that data, in particular predicting future latitude and longitude coordinates. It sounds like you want to model the geospatial data probabilistically. One ...
• 19.3k

### Using time serie to predict another variable

The type of task you are describing is sometimes called time series extrinsic regression. There's not a lot of literature about, this but a paper that provides a good introduction to and evaluation of ...
• 981

### Using time serie to predict another variable

If the second variable is a binary variable indicating whether the participant got dizzy or not, you need to consider other factors beyond the raw head rotation data to accurately predict whether the ...

### Feature engineering for interest-based age classification

Good afternoon @theodre7, looking at what you showed, it's a little difficult to get a precise answer. But if it helps I'm happy. Thinking about a simple model, with what I see in this table being a k-...
• 1

### Can I fit a model with the parameters found with RandomizedSearchCV?

You can but this is not absolutely necessary: the model with parameters best_params_ is stored in best_estimator_ as long as you ...
• 449
1 vote
Accepted

### How to forecast a timeseries with geolocation data?

I see multiple possibilities, here: In General Some general remarks first: When designing you model, you should take reoccurring patterns into account: There will probably be a 24h pattern (for ...
• 508

### 99% accuracy in train and 96% in test is too much overfitting?

A significantly higher accuracy on the training set than the test set is generally an indication of overfitting. In your case, the difference in accuracy between the train and test sets is relatively ...
• 2,361

### Laptop for machine learning jobs

For data science and web development tasks, both combinations are capable of handling most of the tasks with ease. However, the i9 (12900H) and NVIDIA T600 4GB combination would likely be better for ...
• 2,361
Accepted

### Why use sampling instead of the mean value for policy in Reinforcement Learning?

If the mean is used, the value is approximately the same over time. Thus the actions will be very similar over time, providing less opportunity for the model to learn what other actions could be ...
• 19.3k
1 vote

### Statistical test for comparing number of clusters in data

I am not exactly sure where this $F$-statistic comes from but it looks like an adopted Chow-Test. I will not dive deeper into this, since there is a general problem, here: In general you only try to ...
• 508
Accepted

### Random Forest on high correlated data

Welcome to the data science sector. Your three points seem to relate to different aspects. I will try to address all three: 1. Feature Importance To explain the effect, I would go the other way and ...
• 508

### Decision boundary of an neural network

What you assumed were thresholds appear to instead be weights on the edges from the passthrough nodes. At least, that makes the lines consistent with those in the plot. I gave some hints to the same ...
• 10.8k

### Is it valid changing the classification treshold of neural networks for improving the classification performance?

Yes it is a very common thing to do, for controlling tradeoff of objectives. One often encountered example is to precision-recall tradeoff where we move the threshold to strike a balance between ...
• 587

### Stratifed time series split with the same imbalance ratio

their implementation does not guarantee that both classes exist with a similar imbalance ratio over the training and data splits. Is there a way to do that? First I would check how the class ratio ...
• 449

### unbalanced data on train set and test set

It is often useful to balance a training dataset. For example, if the model learns a decision boundary, that decision boundary will then learn to separate different categories based more on features ...
• 19.3k

### How to implement a prediction model of machine learning when missing part of the data necessary for task required

You could group by day and any other sensible variable the count of the crashes and work it as a time series, so to predict the number of crashes in a future scenario. You should use predictors that ...

### unbalanced data on train set and test set

It's also possible to decrease the learning step when updating weights learned from the majority class, and/or increase the learning step when updating weights learned from the minority class. See ...
• 123

### unbalanced data on train set and test set

If your dataset is sufficiently large and you might want to reduce its size for performance reasons anyways, you could do undersampling of label 1. However, if you only have a limited amount of data ...
• 126
1 vote

### How to Manipulate a Dataset

From this dataset, you should prepare new variables namely, X and y; where X = feature matrix...

### Laptop for machine learning jobs

The laptops are similar enough, that I would go with the cheaper one, but since you are starting out (I am assuming smaller datasets) and you want to do webdev, I would rather go with the I9, it will ...

### I am getting all scores as 100% on my machine learning models. Is it okay to have this kind of result?

This could indicate one of two things. Either your model is overfitting to the test data, or your features fully explain your target variable. The only way to be sure would be if you had (or could ...
• 981
1 vote

### Robustness vs Generalization

My approach to answering the question is in light of the language industry leaders are using like Google and Facebook in their open-source and published papers. I also try to fill in the gaps when ...