Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

0
votes
0answers
8 views

I have payments data and I need suggestions from experts what can I do

I have data for a local payment service that is not compatible with standards for electronic payment services such as Visa cards. The service depends on sending a message to the buyer from the ...
0
votes
1answer
20 views

Comparative Analysis of two sets of data

I have 2 sets of data which consist of marks of 60 students in a particular subject in year 1 and year 2 respectively. Year 1 : 86, 76, 87, 67, 89, 95.... so on (60 students) Year 2 : 82, 67, 99, 77,...
0
votes
4answers
50 views

What Unsupervised Machine Learning algorithms can be used to detect Fraud in insurance?

I'm about to start a project regarding Fraud Detection in Insurance but there is no dependant variable to train the model and classify it. Please provide the algorithms that I can use to detect Fraud ...
0
votes
1answer
19 views
0
votes
2answers
27 views

Statistical analysis vs Data mining

Why the A/B Testing, the Correlation and the Regression are considered methods of Statistical Analysis, while the method of "Association Rules" belong to the Data Mining field ? Which are the ...
0
votes
0answers
8 views

How to measure uncertainty and prediction errors for a naive Bayesian Classifier?

I have a small dataset of 30 rows and 5 columns (4 features and 1 class). The classifier is used to give the likelihood of occurrence of an incident. thus, the class variable gives the probability of ...
1
vote
1answer
41 views

Finding Patterns in Error Reports - How to Approach?

I am new to data science and have been doing research to familiarize myself and try to find a solution to my problem but I have not come across anything that seems to fit. I am looking to learn and ...
-2
votes
0answers
19 views

clustering categorical data(text)

I have a dataset in csv form for each user and I am trying to cluster the common behavior for users in multiple clusters (pages visited by ALL, MOST, FEW, UNIQUE users).Any recommendations on what ...
0
votes
0answers
10 views

What is confidence reflexivity in association rules?

I did not understand the meaning of reflexivity term in association rules. I could not find a basic explanation about it. All I found was just academical researches and they heavy for me. Can anybody ...
0
votes
0answers
15 views

Constructing a graph where any node can have the highest PageRank among all nodes

I'm trying to solve q5.1.4 in mmds chapter5, however, I'm not sure how I can even start. The question is: Construct, for any integer n, a Web such that, depending on β, any of the n nodes can have ...
1
vote
0answers
13 views

How to discretize numerice values with predfined ranges in weka?

I'v imported csv file into weka. one of the features have a value with minimum 0 and maximum 160. now, i want to discretize value into three range as you can see below: less than 6 > L more than 6 ...
1
vote
0answers
17 views

Remove attributes with missing values exceeding a given threshold in WEKA

I imported csv file into WEKA, i have features that have missing value that has missing value percentage of 70% or above, i want to remove these features by weka or also sort that features by missing ...
0
votes
2answers
25 views

Predicting missing data. Looking for good data predicting technique

I am analysing data for Countries Trade GDP. Some of the countries have missing GDP value for given a year. However, I have Grand Total for the entire region for that year. Is there a good data ...
0
votes
0answers
10 views

Target Encoding for test dataset

While training, i have used target encoding, to built some features, but i am wondering, how to encode features for test data-set? One way, i can recall, is to use training dataset, to encode ...
0
votes
0answers
13 views

Hotels data for recommendation engine

What are the ways get hotel data such as reviews, descriptions, ratings and etc? My company needs it for our internal system. I searched for some APIs and web scraping, but couldn't find anything good....
0
votes
2answers
21 views

Find frequent segments of GPS trajectories

I have a dataset containing GPS coordinates (latitude and longitude), a timestamp variable and a subject ID to identify different persons. What I want to explore is two things: Find frequent routes (...
0
votes
1answer
30 views

Excel Data to Venn Diagram

I don't really know if I'm posting in the right place but I hope so. I'm writing my bachelor degree, and I collected some data about fonts and language support. At the moment, all my data is stored ...
0
votes
1answer
17 views

What would you do in Knn specific case

I'd like to know what would you do in this specific and unrealistic case appliying Knn when k = 1, k = 2 and k = 3. Class 1 individuals: [1,1] [2,2] [2,4] [2,5] Class 2 individuals: [3,1] [4,1] [4,2] ...
0
votes
0answers
10 views

What is correct recognition rate?

In data mining what is correct recognition rate? and How to calculate it? Is it the accuracy or recall or precision or true positive rate?
0
votes
1answer
25 views

density of a synset

I am reading the paper Text Classification Using WordNet Hypernyms. In it, the author gives the definition of synset density as the number of occurrences of a synset in the WordNet output divided by ...
1
vote
3answers
53 views

How to create clusters based on sentence similarity?

I have data which looks like following. Data is a group of sentences which are similar, but have few unique words in between like TABLEA, TABLEB etc. ...
0
votes
0answers
13 views

Multilevel Association Rules: Dealing with redundancies

I am currently this online PDF I found on google about multilevel association rules and near the end of the article it mentions about getting rid of redundancies. It mentions the following: There ...
0
votes
1answer
26 views

How Does Weighted KNN Work?

I am reading notes on using weights for KNN and I came across an example that I don't really understand. Suppose we have K = 7 and we obtain the following: Decision set = {A, A, A, A, B, B, B} If ...
0
votes
0answers
14 views

What Is the difference between churn analysis, churn prediction and user retention analysis?

I have never learnt these concepts in machine learning study. When I apply for data scientist jobs, there are these concepts/fields: churn analysis churn prediction user retention analysis What ...
0
votes
1answer
22 views

How can i use Hellinger Distance on array of different length?

I have to use Hellinger distance to compare arrays that are not of the same length. Is there a way to do it correctly? Put zero in the missing fields of the shorter array doesn't sound that good to me ...
0
votes
1answer
45 views

When to stop collecting data?

I have developed an iterative process via which I can collect data in batches. The data are points in a predefined 3D space. I am trying to explore and locate clusters in that 3D space based on my ...
-1
votes
0answers
17 views

Choosing the right hyperparameter and score for building ensemble

I want to build an ensemble model from individual classifiers(e.g KNN,SVM etc) for classification purpose. Before building the ensemble mode, I want to select the best hyperparameter from the ...
1
vote
1answer
22 views

Wave form analysis ML algorithm [closed]

I am trying to determine a person's emotions from their speech. This immediately rings Machine Learning bells and the first step in any ML problem is getting and processing data. My first question is, ...
1
vote
2answers
50 views

Suggest an order to read data science books [closed]

I'm an undergraduate student that is currently studing data science. However, since we generally don't delve deeply onto things in class, I want to take some time to review the fundamentals and ...
1
vote
0answers
12 views

How to preprocessing data of a game [closed]

I have a table of users' scores like this: ...
0
votes
1answer
23 views

why do we have to calculate the entropy of parent node in Information Gain?

Why do we need the entropy of parent node in the Information Gain. Information Gain = entropy(parent) - w * entropy(children) We can compare the entropy of the children without the need for the ...
1
vote
1answer
26 views

How does KNN work if there are duplicates?

I am currently debating with my friend about how KNN handles duplicates. Suppose K = 2, and we have a 1-dimensional set of data points to illustrate my dilemma I = {1, 2, 2, 2, 2, 2, 6} Thus is it ...
1
vote
1answer
27 views

error length > 1 and only the first element will be used?

I'm new i data mining and R, and i'm working with a dataset and trying to remove the "white space" elements. i tried to create a function to do that. ...
1
vote
0answers
41 views

Robust linear regression using RLM

I have regression problem and I am using RLM provided by statmodel in python However, when I fit the data, it gives float division by zero. Any idea why? Here is the code ...
1
vote
1answer
74 views

SVM model classifying into one class only, after standardization

I'm trying to use SVM in R (e1071 package) to classify samples as normal or tumor. I have two separate data sets - Training (~50 samples, 100 features) and Test (~60 samples). These data sets are ...
1
vote
0answers
34 views

Forecasting/predicting techniques for qualitative data?

I have a food alert dataset composed of nominal qualitative variables, such as type of alert, country of origin, action taken, etc. as well as the date on which the alert was recorded. What ...
2
votes
1answer
31 views

Which data mining or machine learning algorithm would be appropriate for learning ordered frequent patterns?

I have a dataset as (var1, var2, out), where the ordered pair <var1, var2> gives out. ...
0
votes
1answer
16 views

Change values from nominal to numeric

I want to change the values of the class labels from nominal into numeric. e.g if the values of a class are {iris-setosa,iris-virginica,iris-versicolor} i want to make them {0,1,2} so the instances ...
-1
votes
0answers
23 views

Perceptron on Logistic Regression

I have already did the training for the data but I am not sure how to do the rest. I just need someone to explain how I should proceed? Implement a perceptron for logistric regression. For your ...
-1
votes
0answers
14 views

How do we do a spatiotemporal analysis using Maxent model in R?

I'm trying to understand the spatial distribution of a butterfly species with respect to climatic conditions (19 variables). I have a spatiotemporal dataset with 15 years of monthly count data of ...
0
votes
1answer
19 views

Attribute selection from dataset

I work on a dataset with numeric values.The class labels has also numeric values.I made the 6 numeric class labels into one witch contain values like e.g happy_pleased. I want to insert the new .arff ...
1
vote
0answers
10 views

How do I store/model data needed for my recommendation module?

I'm reading data from a store's product catalog, a 100mb xml file which contains product-wise attributes like prices, categories, etc. Given a product_id, my job ...
0
votes
1answer
32 views

Sentimental Analysis on Twitter Data [closed]

What are best ways to perform sentimental analysis on Twitter Data which I dont have labels for?
1
vote
0answers
19 views

How do I prepare data in which each output row depends on multiple input rows?

My goal is to predict the value of Y based on multiple values of X1 and X2 for each observation of Y. In my example, I want to predict whether a customer will file for bankruptcy (table 1) based on ...
3
votes
2answers
69 views

How can I use two different datasets as a training model for svm

I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one ...
-1
votes
0answers
10 views

How would you derive two association rules using Apriori algorithm from these attributes?

Using only the attributes Student, Credit Rating and Class and the apriori algorithm, I'm supposed to derive two association rules and give their support and confidence. I'm not sure how to go about ...
0
votes
1answer
14 views

How would you describe cluster 2 from this output of a run of the EM program?

My description: Cluster 2 consists of 9511 instances, the age is around 42 (ranges between 29.7207 and 54.5257). Considering Age, Cluster 2 is very well separated from Cluster 1, with a distance of ...
4
votes
1answer
74 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
0
votes
0answers
29 views

One sided time series alignment with dynamic time warping - reference constraint

I want to align multiple time series sequences $T:=\lbrace t_1,\ldots,t_N\rbrace$ of varied length in time using Dynamic Time Warping (DTW), such that all vectors in $T_{warped}$ will have the same ...
0
votes
1answer
51 views

Confusion with euclidean distance implementation

I have to implement the k means algorithm from scratch using python on this data set that has 29 columns and 476 rows. With all these different data points I am confused on how I can calculate the ...