Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

0
votes
0answers
21 views

Chain Rule and Gradient

I want to compute the gradient using the chain rule. But I do not know and I am hoping someone can explain how can I use the chain rule to compute the gradient?
1
vote
1answer
28 views

SVM model classifying into one class only, after standardization

I'm trying to use SVM in R (e1071 package) to classify samples as normal or tumor. I have two separate data sets - Training (~50 samples, 100 features) and Test (~60 samples). These data sets are ...
1
vote
0answers
25 views

Forecasting/predicting techniques for qualitative data?

I have a food alert dataset composed of nominal qualitative variables, such as type of alert, country of origin, action taken, etc. as well as the date on which the alert was recorded. What ...
2
votes
0answers
14 views

Which data mining or machine learning algorithm would be appropriate for learning ordered frequent patterns?

I have a dataset as (var1, var2, out), where the ordered pair <var1, var2> gives out. ...
0
votes
1answer
8 views

Change values from nominal to numeric

I want to change the values of the class labels from nominal into numeric. e.g if the values of a class are {iris-setosa,iris-virginica,iris-versicolor} i want to make them {0,1,2} so the instances ...
0
votes
0answers
12 views

Job-info classification [on hold]

I have a few .csv files including jobs and should classify them into 4 categories (in python) such as company description, expertise, experience and the end text. Is there any idea around? should I ...
0
votes
0answers
20 views

Perceptron on Logistic Regression

I have already did the training for the data but I am not sure how to do the rest. I just need someone to explain how I should proceed? Implement a perceptron for logistric regression. For your ...
0
votes
0answers
7 views

How to model channel allocation behavior of the wifi system

I am working on a problem for weeks without progress. Here it is: Inputs are the csv files about activities of many access points (AP), each row has this format [time: mac_address: ...
0
votes
0answers
12 views

How do we do a spatiotemporal analysis using Maxent model in R?

I'm trying to understand the spatial distribution of a butterfly species with respect to climatic conditions (19 variables). I have a spatiotemporal dataset with 15 years of monthly count data of ...
0
votes
1answer
11 views

Attribute selection from dataset

I work on a dataset with numeric values.The class labels has also numeric values.I made the 6 numeric class labels into one witch contain values like e.g happy_pleased. I want to insert the new .arff ...
1
vote
0answers
9 views

How do I store/model data needed for my recommendation module?

I'm reading data from a store's product catalog, a 100mb xml file which contains product-wise attributes like prices, categories, etc. Given a product_id, my job ...
0
votes
1answer
26 views

Sentimental Analysis on Twitter Data [closed]

What are best ways to perform sentimental analysis on Twitter Data which I dont have labels for?
1
vote
0answers
16 views

How do I prepare data in which each output row depends on multiple input rows?

My goal is to predict the value of Y based on multiple values of X1 and X2 for each observation of Y. In my example, I want to predict whether a customer will file for bankruptcy (table 1) based on ...
3
votes
2answers
35 views

How can I use two different datasets as a training model for svm

I know that you're supposed to scale your test data using the parameters (mean and stdev) from your training data. This is relatively simple; but what if the number of samples is limited in one ...
0
votes
0answers
9 views

How would you derive two association rules using Apriori algorithm from these attributes?

Using only the attributes Student, Credit Rating and Class and the apriori algorithm, I'm supposed to derive two association rules and give their support and confidence. I'm not sure how to go about ...
0
votes
1answer
12 views

How would you describe cluster 2 from this output of a run of the EM program?

My description: Cluster 2 consists of 9511 instances, the age is around 42 (ranges between 29.7207 and 54.5257). Considering Age, Cluster 2 is very well separated from Cluster 1, with a distance of ...
0
votes
0answers
16 views

How would you use the KDD process to identify patients with high risk of heart attacks?

For example: A hospital has accumulated many records over the years. They would like to identify patients who have a high risk of heart attacks. How would you use the KDD process to identify these ...
0
votes
0answers
8 views

pruned tree results in only terminal node

I am learning decision tree algorithm with binary classification. I have trained a decision tree and got pruned tree with just one feature as decision node. Now when I omit that particular feature ...
0
votes
0answers
13 views

What are the implications of different bin sizes and counts in RFM analysis?

I am building a library for customer segmentation which includes an RFM computation. For skewed datasets with e.g. 60% of frequency values being 1, there would be a disproportionate number of ...
4
votes
1answer
68 views

Can Expectation Maximization estimate truth and confusion matrix from multiple noisy sources?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
0
votes
0answers
19 views

One sided time series alignment with dynamic time warping - reference constraint

I want to align multiple time series sequences $T:=\lbrace t_1,\ldots,t_N\rbrace$ of varied length in time using Dynamic Time Warping (DTW), such that all vectors in $T_{warped}$ will have the same ...
0
votes
1answer
28 views

Confusion with euclidean distance implementation

I have to implement the k means algorithm from scratch using python on this data set that has 29 columns and 476 rows. With all these different data points I am confused on how I can calculate the ...
0
votes
0answers
10 views

create random forest using decision trees generated by different decision tree algorithms

We're planning to conduct a data mining study which will be using ID3, C4.5 and CART algorithms. Naturally, this would then generate three different decision trees. Would it be possible to create a ...
-1
votes
0answers
33 views

feature weighting and feature subset selection

I'm planning to work on feature selection and am looking for some useful documentations, tutorials, books or anything about this topic and especially about feature weighting. Thank you for your help!
0
votes
0answers
26 views

ID3, C4.5 and CART random forest

I'm just curious, since the building blocks of random forests are decision trees, is it possible to build a random forest for each algorithm? (ex. a random forest using the ID3 algorithm)? I'm ...
0
votes
0answers
11 views

How to get count “Number of people” gathered for a public event through social networking sites

I am currently working on a sales forecast prediction modeling. As part of it I need to find out an approximate count of people gathered for a public gathering based on "Date" and "Zipcode" or "Lat,...
0
votes
1answer
20 views

What is the quantity sold for a specific fruit & country combination?

What is the algorithm that generates these potential quantities that meet the given criteria? Essentially - there are number of quantities for a fruit and country combination. E.g: ...
1
vote
1answer
24 views

Calculation of distance between samples in data mining

I am confused about a little issue related to distance calculation. What I want to know is, while calculating the distance between samples in classification or regression, is the label or output class ...
3
votes
1answer
92 views

Regularization in simple math explained

I read a lot of articles online about how regularization works and most of them just show the equations with regularization terms but did not use example numbers to explain how the coefficient values ...
0
votes
0answers
19 views
0
votes
1answer
33 views

Is there an upper bound for k in nearest neighbors-based methods?

When applying a nearest neighbors-based method to a data of, for instance, 2000 points, what is the largest number of neighbors that can be considered ? I am using a nearest neighbors method in an ...
0
votes
0answers
13 views

I want to create a Data cleaning module for English language to clean certain Emails. How do i begin?

Here is an Example of an Email in my .xlsx file. End goal is to perform text summarization and Topic modelling of the various Email's present in the Excel file.
0
votes
1answer
57 views

Exploratory Data Analysis

I am working on this dataset. Dataset has missing values. What would be the best method to impute the missing values. Also values are missing from target feature as well. So far I have dropped those ...
0
votes
0answers
27 views

Intiative detection problem

I currently try to wrap my head around a problem of detecting an initiative in a company report document. The following are the types of initiatives a company can report: health, no_poverty, ...
0
votes
0answers
19 views

MondrianEvaluationException on a date dimension with more than 1000 entries

I'm having trouble handling large date ranges in a OLAP report, the error I get is the following: mondrian.olap.fun.MondrianEvaluationException: Aggregation is not supported over to list with ...
0
votes
1answer
24 views

We are using PostgreSQL to store big data and are concerned it may crash the on-board Neural Net. Any thoughts?

We have figured out how to write Big Data to our PostgreSQL database. We would like to run this (normalized) data into the on-board Neural Network in Orange. Our lead programmer said he thought Orange ...
0
votes
1answer
53 views

A dataset has skewness = 1 with missing data. Standard deviation around median is 1.5. How much data will be unaffected?

There is no other description about the data, if it is univariate, bivariate, etc. neither the type of distribution is given. I recently came across this question, I would like to know how skewness ...
1
vote
0answers
15 views

Unsupervized NER + sequence tagging to extract attributes from product descriptions

I am trying to extract attribute values from product descriptions in an unsupervised way. For example, given the product title "Variety pack fillet mignon and porterhouse steak dog food (12 count)", ...
1
vote
0answers
20 views

How to measure retention statistics?

I have a dataset with the ID, name, joining date, leaving date as features. I was asked to measure employee retention and health of it. What can I derive from these? What are some latest trends and ...
0
votes
0answers
12 views

What keywords should I be looking for when verifying web-mined data?

I am scraping websites using the scrapy tool. The websites are all for restaurants and similar places. I have the name, latitude, and longitude information for them from OpenStreetMap. Sometimes ...
1
vote
0answers
15 views

Cross entropy loss increase but precision get better

i am working on classification model. my test result shows precision is getting better despite loss is increase. is it just the nature of my data or is there some kind of theoretical explanation?
1
vote
2answers
56 views

huge doubt on anomaly detection

from the naked eye itself, we can tell in the region 5161 the network usage is high so that is the anomaly in my case, then why do we want to apply k-means and other machine learning algorithms to ...
2
votes
1answer
29 views

Convert a csv file so that all attributes will become columns with corresponding element [closed]

so I have this dataset in a csv file. I would like to convert it into a matrix form. The data currently looks like this: ...
1
vote
0answers
27 views

Can we use doc2vec to detect outlier documents?

I have a set of documents and I want to identify and remove the outlier documents. I am just wondering if doc2vec can be used for this task. Or are there any recently evolved, promising algorithms ...
2
votes
0answers
23 views

ACF (Autocorrelation) estimation

I have two signals "Signal A" and Signal B" and I plot the autocorrelation for each signal with itself as shown below. How I can determine which signal has a stronger correlation with itself. To ...
1
vote
0answers
13 views

Algorithm for finding the best traders

I have a data of 700,000 transactions of forex traders which use various tools for investments. For beginners i want to find the association patterns between following metrics performance / ROI ...
0
votes
0answers
98 views

Extract variable names of formula from a document

How can I extract all of the variable names from a given research paper? For example Some applications of algebraic curves to computational vision For example, below is the text and I need to ...
1
vote
1answer
22 views

How do I change the colours on scatter plots? [closed]

Probably an obvious one, but I just cannot figure it out. I know there's a drop down menu on the left that allows you to decide what to colour, but I cannot seem to find a way to change what colour ...
1
vote
0answers
9 views

Datasets for Weighted graph clustering with detailed ground truth

I want to compare a new algorithm for clustering nodes of weighted graphs (directed or undirected, both are fine) to some previously proposed algorithms but in order to do the comparison I need more ...
1
vote
0answers
11 views

calculate days from a column to today in r

I have data set with a Date column, now i am trying to calculate duration in days from today to dates in date column with a new column named duration. I tried ...