Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

1
vote
0answers
8 views

Algorithm for finding the best traders

I have a data of 700,000 transactions of forex traders which use various tools for investments. For beginners i want to find the association patterns between following metrics performance / ROI ...
0
votes
0answers
14 views

Extract variable names of formula from a document

How can I extract all og the variable names from a given research paper? For example Some applications of algebraic curves to computational vision For example, below is the text and I need to ...
-1
votes
0answers
9 views

trends and researches on Association rules

I want to have a in-depth knowledge on association rules mining. what are the trends on Mining Association rules ?
1
vote
1answer
15 views

How do I change the colours on scatter plots?

Probably an obvious one, but I just cannot figure it out. I know there's a drop down menu on the left that allows you to decide what to colour, but I cannot seem to find a way to change what colour ...
1
vote
0answers
5 views

Datasets for Weighted graph clustering with detailed ground truth

I want to compare a new algorithm for clustering nodes of weighted graphs (directed or undirected, both are fine) to some previously proposed algorithms but in order to do the comparison I need more ...
1
vote
0answers
10 views

calculate days from a column to today in r

I have data set with a Date column, now i am trying to calculate duration in days from today to dates in date column with a new column named duration. I tried ...
0
votes
1answer
16 views

What is the difference between Missing at Random and Missing not at Random data?

I have been working with a dataset where the missing data seem to following a few particular patterns. I have gone through a lot websites and articles related to missing data but I haven't been able ...
0
votes
0answers
12 views

Why SVM with zero number of support vector is yielding good results?

I have implemented a multi-kernel SVDD for one-class classification. It is yielding good results in the case of the number of support vector is zero. I have tested on various datasets like Iris, Germa-...
4
votes
2answers
48 views

Pandas vs Linux Datascience [closed]

I joined a data science learning community in my college and we are using linux terminal commands and awk commands to practice gathering some information from big datasets stored in csv files. About ...
0
votes
1answer
20 views

what to do if the missing data in one column is based on some value/condition in another column in r?

I have a dataset with 20,000 observations and 19 variables. To start off with I have a gender column which has three levels namely 'M', 'F' and 'U' where U can be taken as not disclosed. Whenever ...
0
votes
0answers
4 views

Aggregate Titles to Autosuggest-Keywords

Let's say you have a bunch of movie titles and their search-frequency, box-office or whatever rankings. Now you want to have an Autosuggest that works like that: you type in "Har" and you get ranked ...
0
votes
0answers
7 views

Where can i find Public Datasets related to Astronomy and other fields? [migrated]

I'm planning to do some research. Therefore, Appriciate if you could kindly help me to find out Astronomical datasets.
0
votes
0answers
8 views

Securing elearning using data mining techniques

This is my first question here. I was given the topic above as my school project topic and I have no idea what that is. I need an explanation of what it means to secure elearning using data mining ...
0
votes
0answers
42 views

Mathematical Representation of a High Dimensional Data Set

I have a data set which is composed of 240 observations/signals and every observation/signal have 101 features/attributes. The data set is given in below Image: This is just a single Matrix, y shows ...
0
votes
0answers
13 views

Is it possible to perform transfer learning using kernel ridge regression or any non-iterative approach of machine leaning techniques?

kernel ridge regression can be used for regression or classification task. How can we perform transfer learning for kernel ridge regression?
0
votes
1answer
13 views

Dwell Time computation from Real-Time user location

We have a collection of real time user locations from our application. Each record contains longitude, latitude and the time at which the geo location was captured. How can I compute the dwell time at ...
1
vote
0answers
37 views

Counting similarity between two GIT commits

I am writing as part of my master thesis work an analysis of developers' work. I am looking for a new parameters to be consumed in Topological Data Analysis process. I would like to compare a ...
1
vote
0answers
13 views

Time Series Autocorrelation Estimation

How do I find joint PDF f(xi,xj) needed for estimating autocorrelation from 1-D time series data X, using the kernel windowing technique? I am well aware of kernel windowing technique for estimation ...
0
votes
1answer
17 views

How to recommend items after finding similar users in recommendation system

As the title explains my problem, I'm done with creating a recommendation system that can give me similar users for any given new user. The problem I face is, If I extract the list of products that ...
0
votes
1answer
27 views

Impact of sparse features on tree-based models

Say you have a highly imbalanced binary classification problem. Some of the features are binary features, where they're false most of the time, but when they're true they tend to be highly predictive (...
2
votes
0answers
6 views

Sacle lines and GOF stats for Metric MDS?

Sirs I am new to Orange3 and am getting to grips with Metric MDS - I think the paired links is inovative Usually MDS plots have scales at the bottom and left side but there do not appear to be present ...
1
vote
0answers
16 views

Determining nature of content in an article [closed]

What is the general idea (in layman's terms without technical details) of knowing the nature of content of an article? For example: Whether an article is spam or promotional content, or is a fake news,...
1
vote
1answer
23 views
1
vote
1answer
26 views

Partitioning data into features/labels and train/test after reading from csv file

I need to read data from csv file and then first partition that data into features and labels and then into training and testing set. However, there are several issues cropping up again and again. ...
1
vote
0answers
7 views

Where to find Facial Visual Object Class

I am trying to make a face/head detection model using YOLO and darkflow. In order to do this I need a set of images peoples face annotated in Visual Object Class, VOC format. I would prefer data set ...
1
vote
0answers
11 views

Learning to parse elements from text and assigning them to categories

I am trying to generate a table with values parsed from unstructured text. Below are a couple of examples of possibly thousands of entries. For each entry, I would like to identify the title, assign ...
1
vote
0answers
21 views

How can spatiotemporal population data be used for modeling migration?

I have a dataset that contains the population of butterflies(5 species) for 15 years for different locations. I want to model it against the climate index collected for same time period and location. ...
1
vote
0answers
11 views

Seminal papers about analysis and application based on publicly released data [closed]

The US government releases a lot of varied data here. Data science and data analytics have greatly matured over the last decade varying from Cambridge Analytica on one extreme to Google/Amazon Ads ...
0
votes
0answers
8 views

How to find correlation among multiple attributes in group by dataframe object?

I have a data frame with following attributes : CP - Counting point of vehicles A-Junction - Starting node of a road ...
1
vote
0answers
24 views

Good dataset for sentiment Analysis in Tickets for IT Support

I am working on Sentiment Analysis in Tickets for IT Support, But I have only seen the movie reviews and Twitter sentiment analysis which are not related to my work. I found a research paper related ...
1
vote
0answers
33 views

what type of data mining techniques are employed to identify interesting patterns in data?

I would like to know what data mining technique can be employed to identify the pattern in data. I have a dataset with 35 factors that can be a good indicator of drought onset. I want to identify ...
1
vote
1answer
27 views

How to give names/labels to topics in LDA

I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused. How can I use the information produced ...
2
votes
0answers
16 views

personalize the output with Rstudio

I'am trying to mine some data with Rstudio, but i have problem in the output results. Trying to search big number from 100000 to 900000 the function works well but ...
0
votes
0answers
7 views

Sequential frequent pattern finding from patients dataset along with demographic information

I have patient events data along with their demographic and personal information. I successfully applied Sequential Frequent Patterns in R using SPADE package. But I would like to associate ...
1
vote
1answer
57 views

Implementation of a distributed data mining paper

I have a project about distributed data mining and I need to do some implementations, So I've searched and found this paper. The address of dataset is mentioned in the paper and I've downloaded it. ...
1
vote
2answers
28 views

Is there any logic to adding a threshold to see if two variables are related?

I have two variables $X$ and $Y$ given as tuples of $(x, y)$, and I want to see if there is a relationship between the two variables. I can do so by finding the correlation coefficient. However, I ...
2
votes
1answer
89 views

Why is sampling useful in machine learning?

I have met that question online and I wanted to know where sampling can simulate complex processes and why? Why is sampling useful in machine learning? Sampling can increase the accuracy of the ...
1
vote
0answers
36 views

Frequent itemset vs. frequent sequence mining algorithms on variables with an inherent order

I've been using Pyspark's PrefixSpan algorithm to mine sequences of variables with an inherent order (bus stops). I know that PrefixSpan takes into consideration the order of the items it is fed, ...
1
vote
0answers
13 views

What are the ways to merge data from different sources? Is there any software framework?

Is there any software library or framework that allows merging data from a different source on regular basis? As I understand it should store and support a different kind of rules for cleaning and ...
0
votes
0answers
8 views

What is a good standard dataset to compare LDA text mining technique to another one I’ve developed?

I developed a topic modeling technique which uses text network analysis to detect topics within a document (or a corpus). I want to compare it to Latent Dirichlet Allocation (LDA). What is a good ...
0
votes
0answers
56 views

What really does a Data Scientist? [duplicate]

I'm asking that, because after going to a presentation I understood, that Data Scientist is practically someone who only makes calculations and statistics, neither program or very little. Can someone ...
0
votes
0answers
26 views

Selecting representative data

I have a data set of over 10000 datapoints and over 500 variables. Running computations on this is taking forever!! i want to choose a smaller subset of data to run models on but im unsure on how to ...
0
votes
0answers
10 views

Looking for correlation between a big set of variables: divided in groups or all together?

I have a tiny classified dataset (2 classes and over 20 samples for each class) where I would like to find the highest pair-correlated variables. I doubt if I should mix both groups of samples and run ...
1
vote
0answers
16 views

What are the possibilities with userdata from a vast panel?

I have a panel of a few thousand people that I regularly send surveys to. Throughout the years, I have gathered for every person (legally, of course) their address, age, gender, line of work, family ...
6
votes
2answers
109 views

Computational aspects are typically ignored by statisticians

In the introductory chapter of "Process Mining: Data Science in Action" (2016 - Van der Aalst, pag 11) the author says that : Although data science can be seen as a continuation of statistics, the ...
0
votes
2answers
23 views

How to standarize feature vector with data in different scales?

Let's suppose I have a dataset with numerical attributes of different types. Let's suppose I want to employ a Neural Network for supervised classification with that dataset. For that, I need to ...
0
votes
0answers
10 views

Preprocessing during training and scoring of LSH

I am using LSH algorithm given in Spark. The dataset I am using has both numeric and categorical columns. Hence before training ...
1
vote
1answer
18 views

scalable tools to build kNN graph over sparse data

I'm looking for scalable tools to build kNN graph over sparse data points. The dimension and number of data points can be both up to millions. What I have tried already: sklearn.neighbors....
0
votes
1answer
19 views

Conversion of Interval data to ordinal data

I recently learned about Level of measurements and I am really confused in this MCQ where I think the right answer is a and d both. Lets say the three participants finished the race in 45 seconds,35 ...
1
vote
0answers
97 views

Extracting keywords from pdf file with python

I have a pdf file(link below). I have to extract keywords from it and also need have there frequency in pdf file. E.g. 'Java':42, 'inheritance':3. I have to store ...