All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
2 views

K-means++ cosine distance

I am wondering how to implement k-means++ with cosine distance, acording to quote below (wikipedia), which says, that distance needs to be squared. But with square is lost direction of distance. ...
0
votes
0answers
5 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
0
votes
0answers
3 views

TSNE parameters

Trying to tune the parameters of sklearn.manifold.TSNE(n_components=2, *, perplexity=30.0, early_exaggeration=12.0, learning_rate=200.0, n_iter=1000, n_iter_without_progress=300, min_grad_norm=1e-07, ...
0
votes
0answers
8 views

How to extract features insights to change classifier decision?

I don't know if my question is specific enough but there's what I mean. Suppose we have high school grades of students who attended a Computer Science degree and whether or not they succeeded (given a ...
0
votes
0answers
5 views

Phrase/Token labeling

Looking for suggestions on how to define the following NLP problem and different ways in which it can be modeled to leverage machine learning. I believe there are multiple ways to model this problem. ...
0
votes
0answers
8 views

Testing of hypothesis - Which algorithm is best?

In my case, I have run 2 Algorithms with lstm (rnn) but with different loss functions on the same sample. I have to test whether which algorithm (loss function) works better than the other. For both ...
0
votes
0answers
5 views

How to inference of time series with RNN(like LSTM, GRU etc)

Say I am doing a time series prediction which predict some value for next time step with past T inputs from historical inputs. Say I am using a RNN module like LSTM or GRU. In trainning/validation, I ...
0
votes
0answers
6 views

Why concatenating these layers, why applying masks over and over to partial convoluted image?

I have to ask some questions about one topic. In this sentence of Nvidia's article of : https://arxiv.org/abs/1804.07723 , they are saying:"The last partial convolution layer’s input will ...
1
vote
0answers
7 views

Having issues with DataSet

I just started learning data science and am having a problem when generating a dataset. Dataset: ...
1
vote
0answers
3 views

Error while calculating accuracy and matrix multiplication in tensor flow code for regression

I was writing a code for linear regression using tensor flow but I was getting errors while calculating matrix multiplication using tensor flow and while calculating accuracy. ...
0
votes
0answers
9 views

How to feed the model with a stack of images instead of one by one?

I built a 2D model, but the dataset contains a group of images from different viewpoints for each patient, so the input should be a stack of images for each patient. I have compressed each group of ...
1
vote
0answers
11 views

Deep Learning with Time Series Data (containing Log Returns)

I am curious about how I would begin to approach this problem. I am working with a time series multi-indexed data frame (consisting of precomputed log returns) of various stocks. In this dataframe, ...
0
votes
0answers
12 views

How to calculate accuracy for regression using tensor flow [closed]

I was trying to run this code but I was getting some errors. I looked at the code thoroughly, but it appears correct. but I was getting errors at matrix multiplication and accuracy. ...
0
votes
1answer
15 views

How much data augmentation is required on an imbalanced dataset?

Imagine I have a dataset with positive and negative sentences, and I need to train a transformer (Like BERT) to do the binary classification. The problem is that there are 100 negative sentences and ...
0
votes
2answers
17 views

Matrix multiplication using tensor flow

I am trying to run this code for linear regression using Tensor Flow. I have to use Tensor Flow matrix multiplication, but I am getting errors. My code: ...
0
votes
0answers
6 views

Clustering: How to find which point in a cluster in the closest to the cluster centroid while using kprototype

I have a dataset which contains both numeric and categorical data. In order to carry out clustering in python I have applied kprototype which is the mixed form of kmeans to be used in such cases. I ...
0
votes
1answer
21 views

Understanding Lagrangian for SVM

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 22, he says: Lagrangian for optimization problem: $$\mathcal{L}(w,b,\alpha)=\frac{1}{2}\...
0
votes
0answers
13 views

Can I retrain my best model on all available data?

I split data on Zillow single-unit properties into train-validation-test 70-15-15 and trained a few different sklearn linear models to predict selling price. I chose the best one based on validation ...
0
votes
0answers
11 views

Best way to optimize problem with additively separable fitness function?

I am using a genetic algorithm to optimize a few hundred thousand real-valued variables. Each of the variables, $x_i$, has its own independent boundary condition. The fitness function uses each of ...
1
vote
1answer
13 views

Comparison of classifier confusion matrices

I tried implementing Logistic regression, Linear Discriminant Analysis and KNN for the smarket dataset provided in "An Introduction to Statistical Learning" in python. Logistic Regression ...
0
votes
0answers
10 views

Group points to reduce data set such that the linear regression stays the same

I have a very long dataset and I'm trying to reduce it by grouping the data in periods of 24 hours. In this way, there will be a single data point that represents that day, but they must yield the ...
1
vote
2answers
24 views

OneVsRest Classification why do the probabilites sum to 1?

I am using OneVsRest Classifier in sklearn. So a multilabel model, 4 models for each class (i have 4 classes). When i called the predict_proba method i therefore get an array with 4 columns each one ...
0
votes
0answers
13 views

Improving accuracy of 2D CNN with time series classification

After somewhat extensive optimization of hyperparameters, my test accuracy remains at around 70 %. I have tried techniques to augment time series but they only make things worse. Unlike image ...
1
vote
0answers
10 views

Time Series Forecasting with LSTMs in keras - convergence problem

I am trying to forecast a time series with multivariate input and multi output (multi step forecast). Since some of my input features are known for future time steps, wheras others are not, naturally ...
0
votes
0answers
7 views

How to add new words into word embedding model?

Inspired by the this post, I am curious about how to add new words into trained existing word embedding without retraining the entire embedding? My guess is of the following: there is no such thing as ...
0
votes
0answers
8 views

Keras model's embedding weight get NaN value

I am working on 3 categorical and 19 numerical features in which I plan to use trained embedding weights (from categorical features). After training, and get weights from embedding layers, I got NaN ...
0
votes
0answers
10 views

Multiclass to Binary Classification

Before I start, I would like to let you know that I am a novice in deep learning. I have an image dataset which contains around 900K images. The dataset divided by 3 classes and 3 subclasses for each ...
1
vote
0answers
12 views

How are regression trees fit in gradient boosting for classification?

What I understood is that even gradient boosting for binary classification we use regression trees. The first value we calculate is constant = log(odds). For the rest of the trees we try to fit ...
0
votes
0answers
13 views

Comments Moderation/Profanity Filtering

Just a brief background. I am working on a project for a live-streaming app and we want to improve our live comments moderation using machine learning. The problem is that it over filter/block words ...
0
votes
1answer
19 views

How to improve regression neural network?

I am new to deep learning and data science and trying to increase my knowledge by working on some hackathons. Currently, the hackathon project I am working on has the task to predict the closing price ...
1
vote
0answers
22 views

Word embedding autoencoder

I'm trying to train a word embedding autoencoder, but it either doesn't train, or trains but doesn't make predictions. I know I'm doing something wrong, so any help is greatly appreciated. Here is my ...
0
votes
0answers
8 views

How can I use Ensemble learning of two models with different features as an input?

I have a fake news detection problem and it predicts the binary labels "1"&"0" by vectorizing the 'tweet' column, I use three different models for detection but I want to use ...
0
votes
0answers
11 views

How should systematic uncertainties (up and down) in training data be handled in classification neural networks?

I have a classification neural network and nominal input data on which it is trained, however the input data has for each feature a systematic (up and down) uncertainty. How should the accuracy of the ...
2
votes
0answers
32 views

How can I solve this overfitting problem?

I think I have an overfitting problem. My goal is image captioning datasets with training 8,000 and testing 1,000. This my model for 50 epochs: ...
1
vote
0answers
22 views

How to make the values in different columns in the correct order based on another data frame (mapping) in Python Pandas

I am pretty new to Python and Pandas and I struggle with combining a messy dataframe from excel with a mapping. I have tried to find some solutions on the Internet, however with no success. My first ...
1
vote
0answers
7 views

What does it mean (non) convex "constraint"?

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On page 16, he says: SVM optimization problem can be given as follows: $$\begin{align} \max_{\...
0
votes
0answers
9 views

Find if a string appear before another string [closed]

I have a string variable containing patients' addresses. My goal is to flag patients who live in "401 30th street". I would like to flags strings that contain the number "401" ...
0
votes
2answers
24 views

How to stay up to date in NLP and use the best approaches?

There are many fast advancements in NLP field, BERT, RoBERTa, ALBERT, and XLNe, and no one can check the news or papers daily. Is there any way or site that keeps track of all these new developments ...
1
vote
0answers
9 views

Classification of RGB images

What is the preferred way to specify the features for image classification when the input consists of RGB images? Is it a good approach to flatten the image into a single vector (where for instance '...
0
votes
0answers
5 views

Continuous Bag of Words loss function and training objective

CBOW from what I understand, obtains a probability distribution $P(w|c)$ for all words $w$ in the vocabulary, given context $c$. Th loss function is: $-logP(w|c)$, which means this would be maximised ...
1
vote
0answers
11 views

Trouble with anomaly/novelty detection (on microscale) - need easy practical guide with Keras

I am relatively new to the field of machine learning. However, I already have solved simple image classification tasks with Keras (for example building CNNs and classifying MNIST...). The rough deep ...
0
votes
1answer
35 views

Understanding SVM mathematics

I was referring SVM section of Andrew Ng's course notes for Stanford CS229 Machine Learning course. On pages 14 and 15, he says: Consider the picture below: How can we find the value of $\gamma^{(i)}...
0
votes
1answer
15 views

For sklearn ML algorithms, is it possible to use boolean data alongside continuous data for the predictive data, and if so how can the data be scaled?

I have a medium size data set (7K) of patient age, sex, and pre-existing conditions. Age of course is from 0-101, sex is 1 for male, 2 for female, and -1 for diverse. All the pre-conditions are ...
2
votes
0answers
24 views

Binary classification with imbalanced dataset, about lightgbm output probability distribution

I trained a binary classifier for an imbalanced dataset. I did two experiments: lightgbm classifier, boosting_type='gbdt', objective='cross_entropy', SMOTE upsample After training the lgbm model, I ...
0
votes
0answers
20 views

Reinforcement Learning, wont learn and bad in test set

I'm study and try to understand better the reinforcement learning branch; In this case I want to learn the agent to make a reward; I've tried with: A2C DQN PPO2 but the agent in test env make ever ...
4
votes
2answers
102 views

How to perform text classification on a dataset with many imbalanced classes

I am completely new to NLP and I have been tasked with performing text classification on a dataset containing 193k records. The number of classes is 107. The class with the highest number of records ...
1
vote
1answer
19 views

Developing a deep learning hybrid architecture for a particular problem is a highly complicated task [closed]

I am currently conducting research on application of deep learning (sensor signal recognition). I spent about a year and a half sifting through the literature and discovered some research patterns. To ...
1
vote
0answers
9 views

I need for malware binaries and benign as well

I am working on the behavior of the malicious functions. I need a very large dataset of malware and benign binaries. there exist some sites where the dataset is not available more than 5 to 6 thousand....
2
votes
0answers
16 views

How to reduce the size of a machine learning model?

I am a beginner in this field. I needed a summariser model to summarise the text content on my server. I found a few good models on Hugging Face, the models work pretty well. However the models are ...

15 30 50 per page
1
2 3 4 5
610