Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
0
votes
1answer
48 views

Should outliers be removed only from the target variable or from any variable where they are found?

What I often do is that I check boxplots and histograms for target/dependent variable and after much caution, treat/remove the outliers. But this is what I do only for the target variable. I.e., if ...
0
votes
0answers
10 views

Why one of the features is dominating all rest of the features in my trained SVM?

I have been given a task to train the SVM model on conll2003 dataset for Named Entity "Identification" (That is I have to tag all tokens in "Statue of Liberty" as named entities ...
1
vote
0answers
19 views

How to train time series model with multiple time series of slightly varying lengths?

I read this Train LSTM model with multiple time series post and acted on the answer by using the TimeSeriesGenerator function to make a window and used stride/batch lengths so each run was split up. ...
0
votes
0answers
29 views

Using sequences for multilabel classification

I have a sequential dataset of events, which looks like the following: ['some text here', 'more text here'] -> target Each datapoint is a true sequence ...
1
vote
1answer
25 views

Distribution Shift vs Transfer Learning

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1] ...
1
vote
0answers
44 views

How to prepere dataset for binary classification (anomaly detection?) on timestamped sensor data (multiple files)?

my goal is to make prediction (good or bad data) on sensor data. I tried a lot, but failed to shape my data to get the desired output. scenario: I have multiple timestamped (time as it self is not ...
1
vote
1answer
94 views

Classification of curve data with machine learning

I have tabular dataset each row represent the curve, so goal is to filter out curves that do not follow Sigmoid function. Obviously first I can label the curves. My question would be is machine ...
1
vote
1answer
26 views

What is the purpose of positive parameter in sklearn.linear_model.ElasticNet?

I saw this parameter in the sklearn.linear_model.ElasticNet. What is the purpose of this? What is the possible scenario where we want to force the coefficients to ...
2
votes
2answers
39 views

How to stop a text-classification model from depending on only couple of the words from input text instead of entire sentence?

I have a text classification deep-learning model, which takes in a text and outputs a softmax probability. I am using glove embeddings to represent my input text in numerical form for the DL model. ...
0
votes
1answer
49 views

How could you build a Neural Network with the same weights for different nodes in the same layer

The structure I'm imagining is like the one in the image bellow where the output is softmax. For the hidden layer we would have $$Z_1 = W_{11}^{[1]}x_2 + W_{12}^{[1]}x_2$$ $$Z_2 = W_{21}^{[1]}x_2 + ...
0
votes
0answers
32 views

Random predictions from a multi-label image classification model

I am currently trying to create a model for multi-label image classification using Keras. The model has 8 classes, with a slight imbalance in the dataset (some have 300~, whereas some others have 600~)...
3
votes
1answer
65 views

Machine learning roadmap not for beginners [closed]

To introduce myself: I know what is RL, know some RL algorithms such as PPO, A2C. Know about offline RL, online RL. I have read many papers about RL. Such as MuZero, AplhaZero, Decision Transformer ...
1
vote
1answer
39 views

test data is not a good representation of train data

I have predefined train and test sets. On generating some statistics like value_counts and checking the unique values, I feel that there is a 'lot' of difference between the distributions of the ...
1
vote
0answers
15 views

Memory issues for AalenAdditiveFitter in Lifelines packages in Python

We are working on a problem related to survival analysis. We have already implemented Cox Proportional-Hazard Model and Accelerated Failure Time algorithm. Now we want to see how the covariates change ...
0
votes
0answers
13 views

Time Series Forecasting for multiple value prediction for any date

In a given example like : Imagine there is historic data stored for a library, and you need to predict the number of books that the library will have, and the number of people who will borrow these ...
0
votes
0answers
15 views

What is the difference between dilated convolution and bilinear interpolation?

I started studying semantic segmentation, so I read some papers about image segmentation to clinical images using as main architecture, the u-net. Maybe because I am newer on the field, but I don't ...
0
votes
0answers
33 views

How can I balance sentence data for NLP tasks

I have been given a task to train the SVM model on conll2003 dataset for Named Entity "Identification" (That is I have to tag all tokens in "Statue of Liberty" as named entities ...
0
votes
1answer
43 views

XGboost predict

I am trying to understand this XGboost example. After training: ptrain = bst.predict(dtrain, output_margin=True) they make prediction on test data, but the problem ...
0
votes
0answers
15 views

Generation of Anomalous data points

I have a task to generate some anomalous points in a real world dataset with 15 features, and a synthetic dataset of 5 features. I was thinking of using correlation between features, but it'll be a ...
0
votes
0answers
13 views

How to implement the contribution analysis using PCA?

I have been looking into implementing the Q-Residual and Hotelling's T statistics calculation to the PCA components which is similar to the following article and website: Structural Health Monitoring ...
0
votes
0answers
27 views

How do I work with noisy real world text data for text classification?

I have a topic classification model built upon Bert, when I deploy my model people input strings of a random nature like : "aaaaaa" "aaa bbb" "ab ab ab" and so on. My ...
0
votes
0answers
28 views

Predict recovered amount of credit?

I would like to understand which is the best Machine Learning approach (regression, classification, ...) in the following scenario: I have a dataset with hundreds of people, each of them with a credit ...
0
votes
0answers
22 views

How to make a forecasting model using labelled time series data of x to predict y?

I have made regression models in the past for this using x to predict the instantaneous value of y, however, I am curious if a time series approach could be more suitable. I'm working in python and I ...
0
votes
0answers
37 views

How to measure the pairwise similarity between two textual data sets?

I have N textual data sets, and each one is composed of thousands of documents. I want to compare them to find which data sets are more similar (Similar to what it ...
1
vote
1answer
37 views

Pytorch: How to make sure that all labels are present in each batch

How to make sure that each batch will have samples with all the labels? For example, consider sentiment analysis problem with labels positive and negative. ...
0
votes
0answers
17 views

Deployment without IaaS and SaaS

I have always seen ML deployments either on SaaS like Heroku or IaaS like AWS What the courses don't cover are deployment on a company's own servers. That's exactly my case wherein the company for ...
1
vote
1answer
45 views

Which algorithm is best for predicting diseases if symptoms are given? [closed]

After Topic modelling through LDA, I get the following dataset as result. ...
1
vote
0answers
7 views

Approximation of a confidence scores from a neural network with a final softmax layer: Softmax vs other normalization methods

Say that there is a neural network for classification and the 2nd to last layer are 3 nodes, and the final layer is a softmax layer. During training the softmax layer is needed, but for inference it ...
1
vote
0answers
8 views

What are the metrics to evaluate Data Quality?

Data quality refers to the overall utility of a dataset(s) as a function of its ability to be easily processed and analyzed for other uses, usually by a database, data warehouse, or data analytics ...
0
votes
2answers
26 views

Obtain precision at a certain probability value [closed]

With scikit-learn, one is able to compute the precision values as well the predicted probability output. To compute the precision values, the sklearn precision/recall function takes the true target ...
0
votes
2answers
50 views

How do I go for NLP based on phrases instead of sentences? [closed]

I have a list of words in this format: chem, chemistry chemi, chemistry chm, chemistry chmstry, chemistry Here, the first column represents the actual word which ...
4
votes
1answer
132 views

What changes is the Neural Network back-propagation algorithm doing on the weights?

I have seen the formula for back-propagation algorithm for neural network error minimization, but I am not quite sure about what changes it is performing on the weights individually. Let us suppose a ...
0
votes
1answer
33 views

Can a ML model predict output vector based on input vector?

This is a conceptual question. I am aware that many ML models predict the value of a variable on a row-by-row basis. Are there models that do so for vectors? For example, if my data is this: ...
1
vote
0answers
34 views

What skills are required to work effectively with AutoML? [closed]

What is the required set of skills for engineers to work with AutoML services on GCP and AWS platforms? I was wondering is it required to have the same set of skills as ML engineers who work with ...
2
votes
0answers
21 views

Transparent Matching / Recommendation System [closed]

I am thinking of a matching/recommendation algorithm which matches students to the right teachers for their individual problems. The dataset would look like this: Student Name Age Gender Weak ...
1
vote
1answer
33 views

What is the appropriate machine learning algorithm for this problem?

I have a dataset in which each sample contains a user id, a date, and the status associated with that particular user (active, expired & deactivated). The dataset contains records for a full year, ...
1
vote
0answers
33 views

Can any data be learned using polynomial logistic regression?

We know that a Taylor polynomial can approximate any smooth function. In binary logistic regression we're trying to fit a decision boundary to our data. But this decision boundary is not necessarily a ...
2
votes
0answers
27 views

Predicted values scatter plot angle

Could you please come up with any possible explanation on why this happens when one uses a deep learning model to predict something? The points should be on a 45 degree angle but they seem to be on a ...
2
votes
1answer
30 views

how to classify highly overlapping data after PCA and t-SNE?

I'm working on a classification (3 classes) of unbalanced weather data having 22 features. Even after applying PCA and t-SNE the data is overlapping. The best classification score achieved so far is ...
0
votes
0answers
22 views

Good classification, poor separation with TSNE/UMAP

I have been working on a classification problem for which I have been able to achieve good results across various classification metrics. I have been careful to ensure that I am not leaking ...
1
vote
1answer
20 views

How to incorporate static variables into ML

I have to establish an ML-based model where I predict precipitation in a complex terrain using multi-year daily observations from 50 stations. Besides a dozen of continuous variables, predictors ...
2
votes
0answers
54 views

calculate the VC-dimension [closed]

I have a question about VC-dimension. I have this claim and I need to find out what its VC-dimension is $ H\subseteq\{0,1\}^n $ collection of Boolean functions over n In my opinion the answer should ...
0
votes
0answers
58 views
2
votes
1answer
90 views

Is my approach about the ML model correct?

First of all, I am a newbie here and it is my first question on this platform, so I apologize for the mistakes about the format if there are any. In my thesis study, I am trying to identify the non-...
1
vote
1answer
36 views

Numerical Range in Decision Tree Classifier

Does Decision Tree Classifier works with Numerical Range? In this example set of dataset. I am planning to test the data regarding the age range of : 18-25 26-30 31-40 41 and above Using the ...
1
vote
0answers
32 views

Is there any way to remove background of an image fully with the help of post-processor techniques(like edge detector) after deep learning based model

I'm using a deep learning model (deep lab v3+ with xception as the backbone) for image segmentation and removing the background. The subject of the image is a person. And my target is to extract the ...
1
vote
0answers
12 views

Is there a list of neural style transfer architectures?

I'm trying to choose what neural style transfer architecture to use but I can't find a centralized list of all possible architectures I could choose from. Is there a place I could find this or could ...
1
vote
1answer
42 views

Find feature categories associated with a specific target category

I have a Dataset with three columns. Products (up to 200). The quality checks that have not been conducted at the final quality check. (Up to 70 different Quality Control Measures) The result of the ...
1
vote
0answers
29 views

XGBoost: How to obtain scale_pos_weight for multi classes?

I know there is a similar Qn at Unbalanced multiclass data with XGBoost. But I don't understand the reply provided by @Esmailian. What is the actual formula to obtain 1, 0.333 and 0.167? ...
2
votes
1answer
34 views

Can I compare two models trained on different but similar datasets to help find differences between the two datasets?

I have a multivariate dataset the contains A and B. I want to see if there are differences between the A and B samples. I currently have two ideas on how to do this, but I am not sure if they are ...