Questions tagged [machine-learning]

Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.

Filter by
Sorted by
Tagged with
1
vote
1answer
46 views

Build text complexity model based on complex examples

I try to build the user specific model which predicts whether arbitrary English text is complex for particular user or not. Having the complex and easy text samples allows to build such model but what ...
1
vote
2answers
233 views

Adaboost - Show that adjusting weights brings error of current iteration to 0.5

I'm trying to solve the following problem but I've gotten sort of stuck. So for adaboost, $err_t = \frac{\sum_{i=1}^{N}w_i \Pi (h_t(x^{(i)}) \neq t^{(i)})}{\sum_{i=1}^{N}w_i}$ and $\alpha_t = \frac{...
2
votes
1answer
81 views

How to implement single Imputation from conditional distribution?

In [*] page 264, a method of drawing a missing value from a conditional distribution $P(\bf{x}_{mis}|\bf{x}_{obs};\theta)$ which is defined as: I did not find any code implementation of this ...
0
votes
0answers
16 views

showing this error ValueError: shapes (1,400) and (10000,10) not aligned: 400 (dim 1) != 10000 (dim 0)

I am trying to run a network and get this error .I am a noob,can someone help me. ...
2
votes
3answers
74 views

In which situation should we consider a dataset as imbalanced?

I'm facing a problem about making a classification on a dataset. The target variable is binary (with 2 classes, 0 and 1). I have 8,161 samples in the training dataset. And for each class, I have: ...
0
votes
0answers
11 views

Balanced target variable in KNN classification

In many places, I have seen it only mentioned that predict the label of the query point as the label with more than half of the labels of it's K nearest neighbours. However, I don't see it mentioned ...
0
votes
0answers
5 views

Best method for address book ranking

I would like to be able to ingest an address book and determine from the contacts who someone’s closest family and friends are likely to be. Any advice on an approach appreciated. I am planning to ...
0
votes
0answers
8 views

Additional Sklearn Acceleration Packages

As a data scientist, I am always looking for ways to improve my workflows. I am familiar with Intel's sklearn acceleration module, Intelex. While this has sped up my algorithms between 3-70x based on ...
0
votes
0answers
7 views

Potential models/methods to predict walking times in postal center

I have a dataset containing the walking times it takes to walk from one postal bin to another postal bin in a mail distribution center. The features I have, include the total mail workers on the floor ...
0
votes
0answers
12 views

LLE and ISOMAP, strange question and intrinsic low dimensional structure?

I prepare for PhD entrance exam on AI and one question is surprized me. which of the following techniques using intrinsic low-dimensional structure detection for dimension reduction? A) ISOMAP B) ...
0
votes
1answer
18 views

Features of the fourier transform for machine learning

i intend to extract features from time-domain measurement data. I feed the features to machine learning algorithms to detect anomalies. In the time-domain, i extract mean, RMS, skew and standard ...
0
votes
0answers
13 views

Custom layers in Keras -- custom weights

I am trying to understand how to build custom layers in Keras and I went through a couple examples: here and here. The syntax is, of course, similar, but in non of the cases it is addressed why ...
2
votes
2answers
619 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
0
votes
1answer
15 views

Data Anonymization for all domains?

I am using a dataset from Marketing and sales department. The dataset contains customer name (company name), company address, pincode, no of orders placed, revenue generated from that customer etc. My ...
1
vote
0answers
5 views

quereies related to autoencoder

i want to design an deep auto encoder after following keras tutorial. Input is a simple 2-dimensional image consists of 512 rows and 50 columns matrix My trial code is ...
2
votes
1answer
58 views

What can we learn from PCA on non linear data?

Suppose we have dataset with 10 features which are not linear: ...
0
votes
1answer
43 views

XGboost predict

I am trying to understand this XGboost example. After training: ptrain = bst.predict(dtrain, output_margin=True) they make prediction on test data, but the problem ...
4
votes
2answers
518 views

Fuzzy Cognitive Maps

I am working on a project for prediction, using Fuzzy Cognitive Maps. I am new to data mining and want to learn about Fuzzy Cognitive Maps and its implementation. I also want to know what tools are ...
5
votes
2answers
9k views

How to do feature selection for clustering and implement it in python?

I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
1
vote
2answers
71 views

How to restructure my dataset for interpretability without losing performance?

What I am doing: I am predicting product ratings using boosted trees (XGBoost) with a dataset in this format: What I want to do: I want to use SHAP TreeExplainer to interpret each prediction my model ...
2
votes
4answers
1k views

Which AI algorithm is best for chess?

I'm working on my chess bot, and I would like to implement simple artificial intelligence for it. I'm new in it, so I'm unsure how to do it specifically on chess. I heard about Q-learning, Supervised/...
1
vote
0answers
19 views

Why concatenating these layers, why applying masks over and over to partial convoluted image?

I'd like to ask some questions about a topic. In a sentence of Nvidia's article they are saying: "The last partial convolution layer’s input will contain the concatenation of the original input ...
3
votes
1answer
108 views

What is the most appropriate machine learning approach for this scenario?

The scenario is pretty simple, and I'm sure it's been done a million times. The problem is I don't know the terminology to find the correct resources on the web. Scenario: I have an environment that ...
0
votes
1answer
1k views

Comparing dataframe object with string value in django

I'm implementing machine learning model and using training dataset from MySQL table and all this is built upon Django. So basically all the calculations are done by converting entire data from MySQL ...
1
vote
0answers
17 views

Comparing machine learning algorithms on features selected by a neural network

I'm reading a paper where they use a neural network to select 9 features from tabular input data with 20 features. And then, this is what feels weird to me, they run several machine learning ...
1
vote
1answer
1k views
1
vote
1answer
63 views

Error while writing perceptron algorithm binary classifier

I am a beginner and I am designing an binary classifier using Perceptron algorithm using FASHION-MNIST dataset. While designing the same I have written the following code: ...
1
vote
1answer
16 views

AUC-PR but there is no recall or precision

Is it possible to have a Precision-Recall curve like this if your recall is zero and your precision is not defined? How do I interpret this? I have checked that all the scores are right and I still ...
0
votes
0answers
18 views

What algorithm to use to cluster data

I need to solve some business problem, but I am not 100% sure what algorithm to use as I did not master machine learning yet. I kind of think I has to be clustering, but I would like to get some ...
1
vote
2answers
822 views

Alternative approach for Q-Learning

I have a question related to an alterative Q-Learning approach. I'd like to know if this already exists and I am not aware of it, or it doesn't exist because there are theoretical problems behind it. ...
0
votes
1answer
78 views

What is the best way of combining audio and visual data to make predictions?

I am trying to predict the probability of a disease by using audio and images, the audio and the images do not come from the same source. I am thinking of combining the outputs (maybe average them) of ...
1
vote
0answers
8 views

pcap post request body string classification

I want to classify malware traffic. I am going to use the machine learning method to classify the post request body string in the pcap file. The conventional text classification algorithm may not work ...
0
votes
0answers
12 views

Which all good certification courses available for NLP? Please enlist both free and paid [closed]

I want to be skilled at Natural language Processing and work on independent projects. Guide me towards useful online content.
7
votes
6answers
1k views

Is there any way to explicitly measure the complexity of a Machine Learning Model in Python

I'm interested in model debugging and one of the points that it mentions is to compare your model with a "less complex" one in order to check if the performance is substantially better on ...
2
votes
1answer
35 views

Understanding experiments in Continual Learning

Via paper Continual Learning Through Synaptic Intelligence, I see this figure for Split MNIST benchmark, but there is a point I can get. Here there are 5 tasks, ...
1
vote
1answer
90 views

Question about balancing training data for sentiment analysis (machine learning)

My question is about when to balance training data for sentiment analysis. Upon evaluating my training dataset, which has 3 labels (good, bad, neutral), I noticed there were twice as many neutral ...
2
votes
1answer
46 views

Should bag of words in training set include test set data when doing text classification?

I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
3
votes
2answers
64 views

ConvNet with concatenated data

I have a basic question regarding convolutional neural network. Assume I have a set of 1000 RGB images and I train a CNN from this set. I can obviously split each of my RGB images into 3 different ...
1
vote
2answers
270 views

Will one hot encoding / unbalanced columns cause bias to Clustering Analysis?

I'm wondering if having too many columns about one certain feature is gonna cause bias to the clustering analysis. For example, if my dataset has columns = ['incoming calls', 'outgoing calls', '...
0
votes
1answer
199 views

How to use CNN for labelling images

I have theorical question that I couldnt decide how to approach. I have tons of grayscaled shape pictures and my goal is seperate these images to good printed and bad printed. For this, I look at ...
1
vote
1answer
147 views

What's the best way to train a model when having a very big dataset split in batches?

I was wondering what is the best way to train my Keras Model, in a binary classification context. My full dataset is composed of 2.3M rows and 120 columns, which is really big. As you would imagine, ...
4
votes
1answer
145 views

How to incorporate new features in an existing machine learning model?

Suppose we have trained a regression model $M$ on a fixed set of $n$ features, $F_1,F_2,…,F_n$ on a particular dataset $G$. Now assume that after model training, additional features ($F_{n+1},…$) ...
1
vote
1answer
134 views

Can LSTM be used for non time series data?

I have a dataset - This is a TOR network traffic dataset with labels added as TOR/ Non TOR. I want to run an LSTM on it and classify it as Tor/Non Tor. Is that possible since this is not a time ...
0
votes
1answer
599 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
0
votes
0answers
13 views

NaNs not being ignored despite program being told to do so; Linear Regression

I have gone through the process of filtering a set of data that I have by removing excess numbers and values that are invalid. However, there is a lot of data that occasionally has ' ' in it, and ...
3
votes
1answer
64 views

Balance data using different criteria

I have a dataset of audio and text files that I want to balance using different criteria to train a neural network. The text and matching audio file are grouped under one ID. For each ID, I have a ...
42
votes
4answers
18k views

Is it always better to use the whole dataset to train the final model?

A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, ...
0
votes
2answers
2k views

What does the classification report interpret? Class 1 indicates abnormal data

How to interpret the report and How is precision, recall values are calculated for individual class labels. What is the significance of macro avg ? Does this report signify a good predictions by the ...
1
vote
1answer
14 views

Why do I get an ValueError for an SVR model with RFE, but only when using pipeline?

I am running five different regression models to find the best predicting model for one variable. I am using a Leave-One-Out approach and using RFE to find the best predicting features. Four of the ...

1
2 3 4 5
197