Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Live peak / trough detection (data provided)

At the bottom of this question is the data of three time series in CSV-format. All are of same length and they all contain measurements of the same event "A". But each time series is using a ...
0
votes
0answers
17 views

Algorithms or statistical methods for detecting trends based on multiple time series

First of all, I have little experience with data science, statistics, and working with time series, so please excuse my perhaps foolish and amateurish question. I have a data set with many different ...
0
votes
1answer
16 views

Relationship of Bias and size of dataset

I was reading the following book: http://www.feat.engineering/resampling.html where the author mentioned the below: Generally speaking, as the amount of data in the analysis set shrinks, the ...
0
votes
0answers
12 views

Association Mining / rules are the statements appears to be true

I have a problem. I don't know how I cloud explain which of the three statements appears to be true and if my calculation below is invalid?` The following table summarizes the results of a medical ...
1
vote
3answers
69 views

Association Mining - is buying Independent?

I have a problem. I can't not solve this exerciese. What is the best way to solve this exerciese? What are the approaches for this kind of exerciese? The following table summarizes transactions in a ...
2
votes
1answer
39 views

exclude variables with no variation during prediction?

I am working on a binary classification problem. I do have certain input categorical variables such as gender, ethnicity etc. ...
0
votes
1answer
21 views

How to use rule-based labelling intelligently?

I have a dataset like below The outcome column is labelled as positive if the % difference between target final Qty and ...
0
votes
1answer
22 views

Associationg Rules/ Mining is this association rule strong

I have a problem. I can't not solve this exerciese. What is the best way to solve this exerciese? What are the approaches for this kind of exerciese? The following table summarizes transactions in a ...
0
votes
2answers
26 views

How to show prototype of output before building model

Currently in my work, we are working on a POC for a AI project. We intend to do a binary classification using traditional classification algorithms. However, my boss wants me to show a feel of the ...
0
votes
1answer
18 views

Estimate timeline for a ML Project

I am a novice data scientist and have been asked to provide an estimate for a data science project in our organization. From the problem stmt description, i am able to understand that it is a ...
0
votes
0answers
7 views

Given a job viewed or applied

I am very new to analyzing text data and extracting information out of it. I need some suggestion help from the communnity. The dataset is from the job website where 'id' denotes the job id, 'abstract'...
0
votes
0answers
34 views

How to classify a point using Bayes classifier?

The following 2 classes dataset is available: $$ C_{1}=\left \{ 17,18,19,20,21 \right \},C_{2}=\left \{ 12,14 \right \} $$ If we assume each class distribution is gaussian, To which class does x=16 ...
0
votes
1answer
31 views

Data Preparation for dates

I have two columns, a birth year and the first year of being a customer. There is a large number of samples that have a birth year less than the first year of being a customer which does not make ...
0
votes
1answer
20 views

Keyword extraction with Orange

I’m very new to using Orange Data Mining software and I’m having a hard time finding what I am looking for. I have 30,000 text files. I wish to use Orange to help me extract key words and phrases, ...
0
votes
0answers
8 views

sampling fall on which category of data mining

I have a question regrading the steps of data mining. After searched on Google I came to know that Data mining have 7 key steps ...
0
votes
0answers
37 views

Nonparametric Outlier Detection

Which Nonparametric outlier detection do you suggest to detect outliers (red points) in these plots? I have tested std, IQR, etc., but no good result. It is just one vector including normal and ...
1
vote
0answers
32 views

How to combine data from multiple Google Trends queries effectively?

As you might know, Google Trends works by normalising a random sample of the search term data, with the sample changing at least once per day, from my experience. This is not an issue for western ...
1
vote
0answers
14 views

Appropriate objective for a data science project [closed]

I've done a couple of DA projects found on the internet and I want to start one myself. I'm interested in baseball data and I think I can exploit a lot of that data but also want to delimit the topic. ...
0
votes
0answers
18 views

How to read multiple JSON files to a dataframe in R? can I get the code with the comments on it to understand it better?

I have multiple json files in a folder called "app users". These json files has NAN values as well in it.
0
votes
0answers
10 views

How to extract rules for decision tree from ID3 classification

I am working on a program to implement the ID3 algorithm. The program takes in user input for setting a threshold and creating a decision tree. ...
1
vote
0answers
14 views

Is python & tableau enough to become data analyst? [closed]

Currently, I working as SEO Analyst. I want to become a data analyst and so I have started learning python with data science from udemy course. Can Anyone help me with guidance to learn data analytics?...
0
votes
1answer
31 views

How to convert duration column from 1 hr. 17 min to 77 min in pandas? [closed]

I am trying to convert all the hr., min, and sec into just mins. For example, 1 hr. 17 min to 77 min, and 34 sec to 0.56 min, not 0.34 min. So I have used this code: ...
1
vote
0answers
25 views

Association rules - Find 100% confidence rules [closed]

Suppose there are 100 items, numbered 1 to 100, and also 100 baskets, also numbered 1 to 100. Item i is in basket b if and only ...
1
vote
1answer
23 views

How to calculate the information conveyed in a message for a given dataset

Given the data sets. Test Set ...
0
votes
0answers
31 views

Is it possible to derive anything useful from this piece of data?

Let's say you have online Profile A. Profile A is present on 3 websites: X, Y, Z. ...
1
vote
0answers
42 views

Apriori algorithm with tags

In apriori algorithm, we can create association rules with respect to the frequencies of the corresponding data set. My question is, what if we have tags data in addition to the transaction data. For ...
0
votes
0answers
28 views

Identify the missing value in a table

How to compute the missing values x in the below table. which formula can be used to obtain the values of x. The values in the last row and last column are the sums of the rows and columns. What is ...
1
vote
0answers
23 views

Unstructured data not template based to structured data

I am working on a project where my goal is to extract data from a large diversity of pdf that does not follow a template (i.e unstructured data not template based). My ORC part works well and now I am ...
0
votes
0answers
32 views

What are the metrics to evaluate Data Quality?

Data quality refers to the overall utility of a dataset(s) as a function of its ability to be easily processed and analyzed for other uses, usually by a database, data warehouse, or data analytics ...
0
votes
0answers
5 views

Analyzing Microscopic Residue

My goal is to analyze a set microscopic residue. I'd like to apply entropy discretization to attribute A1, segmentation by natural partitioning to A2, then correlate A1 and A3. The first attribute of ...
0
votes
1answer
16 views

How to apply entropy discretization to a dataset

I have a simple dataset that I'd like to apply entropy discretization to. The program needs to discretize an attribute based on the following criteria When either the condition “a” or condition “b” is ...
1
vote
1answer
34 views

How to approach this: Percentage change in one KPI leading to change in other KPIs?

I want to know how can I approach or model this problem. I have 7 KPIs (3 of them dependent on each other) and one main KPI (total 8 KPIs). I want to understand effect of these 7 KPIs on the main KPIs....
2
votes
0answers
26 views

Is there a way to scrape tweets in realtime from a list of specified users? [closed]

I am trying to build a scraper that will run continuously and save the tweets from a list of users instantaneously or within seconds of the user tweeting it. It could save the tweet details to a ...
0
votes
1answer
32 views

Interpreting evaluation metrics with threshold/cutoff

I was doing churn prediction for a company. I've got the following results by applying 3 classifier. Model Accuracy AUC Logistic Regression 0.671 0.736 Decision Tree (pruned) 0.681 0.665 Decision ...
4
votes
1answer
61 views

difference between novelty, concept drift and anomaly

Concept drift is when the relation between the input data and the target variable changes over time. like changes in the conditional distribution. is novelty an outlier? what should I think of? what ...
0
votes
0answers
34 views

KNN search of high-dimensional embedding vectors

I have a large database of n-dimensional numerical vectors. Each vector is the embedding of a vertex into n-dimensional vector space. Vertices belong to a graph, and an algorithm such as node2vec or ...
2
votes
3answers
69 views

Does One-Hot encoding increase the dimensionality and sparsity of dataset?

There are two ways to convert object datatype into numeric datatype, first is One-Hot encoding and second is simply map the numerical tags to different values. For example for column Age containing ...
0
votes
0answers
27 views

Does this make data leakage in time series? # need help for understanding time series data

Does this make data leakage in time series? I already read this, data leakage when scaling time series Data leakage is when information from outside the training dataset is used to create the model. ...
0
votes
1answer
15 views

Can you explain how these precision, recall and purity are calculated in this context?

I am studying to get my first job as data scientist and I stumbled upon this. I have been banging my head on the wall for hrs to understand this concept. But couldn't get concrete answer. Just 1 ...
0
votes
0answers
40 views

Is there any difference between data mining architecture and Knowledge discovery from data architecture?

I am looking at architectures of these two and they seem too similar to me. this is KDD architecture-: This is data mining architecture. I see no difference between them. in data mining architecture ...
0
votes
0answers
9 views

Differential Gene Expression Analysis problem

I have my own gene expression data that I am trying to perform differential gene expression analysis on. When I load it I can see all the columns. When I run Differential Gene Expression Analysis I ...
0
votes
1answer
23 views

How to determine the correlation using $X^2$ values

I am trying to understand how to calculate the correlation between two variables using p values. I am familiar with using the classic $X^2$ formula from the The Morgan Kaufmann series in data mining. ...
0
votes
1answer
53 views

MaxDiff Histogram usage

I would like to understand how the MaxDiff algorithm works. The only example that I could find online was a set of slides by Chris Crifton. Given An attribute C 12, 5, 5, 5, 12, 1, 1, 20, 20, 2, 2, 12,...
1
vote
0answers
9 views

Identifying the best group that are most productive

can you please give some idea to solve this problem, Below is my data, I want to identify the best inventory based on the CPM(target value) .. For example: I want to be able to say - Business section, ...
0
votes
2answers
97 views

Calculating the dissimilarity between term frequency vectors

Given that a document is an object represented by what is called a term frequency vector. How can we calculate the dissimilarity between term frequency vectors?
0
votes
1answer
17 views

Calculating the dissimilarity between two numeric attributes

I was wondering if someone could explain how to calculate the dissimilarity between two numeric attributes. Given that a numeric attribute is a a measurable quantity, represented in integer or real ...
1
vote
2answers
49 views

Selecting Training Set from Model CV Error

What I would like to do is recursively: Train the model on all data Remove the sample(s) with highest error Repeat until the remaining samples have an acceptable error The hypothesis is: "To ...
0
votes
1answer
908 views

Calculating the supremum distance

In this example I would like to figure out how to calculate the supremum distance. Given the example shown below I would like to know what f and h represent. Any help would be greatly appreciated.
0
votes
1answer
13 views

Calculating Dissimilarity between attributes

I would like to figure out how to calculate the dissimilarity between attributes Jack and Jim. Given the attributes table shown below Given the Relational table. and the example calculations I would ...
0
votes
1answer
34 views

Confusions about fp tree growth numerical-:

If fcam=3,Would conditional fp tree still be c=4 or what would it be? My guess is that it should be f=3,c=4,a=3,m=3. Am I right? Or what else would it be? Please guide. Also, in this below figure(same ...

1
2 3 4 5
23