Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Which one to choose to identify patterns of user activity: Sequence analysis or process mining?

I have the following user activity data that where for each user the activity type they were engaged are recorded along with the phase: ...
0
votes
1answer
14 views

Python : Approaches to mine for time ordered sequences

I have a dataset that looks like this : ...
-2
votes
0answers
7 views

What are the major differences between rule based classifiers and association rule mining? [closed]

I was looking through some exam questions on Data Mining and came across this question in of the question papers.
0
votes
0answers
18 views

Can we use feature selection for 2 target variables?

My dataset has 2 predicted variables since I am doing multioutput-multiclass classification. The two predicted variables are PurchasedItem1 and PurchasedItem2. I was using BORUTA for feature selection....
0
votes
0answers
12 views

Best Data Science Method to group similar data [closed]

I have collected two years worth of NFL data for the upcoming season. I have every team's Money_Line, Score, Opponent Score, Total, Spread, Team Total, the team percentage increase or decrease in ...
0
votes
1answer
17 views

Which tool is good to collect tweets on 50 keywords over the last 5 years and then analyze them with the LDA algorithm or sentiment analysis?

I want to find tweets from the last 5 years to a topic. For this I decide for 50 Keywords (related to the main topic), where I want to find data on Twitter. I want to find out how the trend on the ...
0
votes
0answers
12 views

How many items does apriori algorithm consider while working on data set of say 100 product?

For finding supermarket product association rules it is said that apriori is suited to identify frequent itemsets. From here then we can mine association rules. Given 100 products, and their ...
1
vote
2answers
30 views

What is the purpose of the 'train model' step in data mining?

My understanding is that training a model is something done in machine learning using training data so that the model can predict values when new data is given to it. Data mining is the process to ...
1
vote
1answer
20 views

How Adaboost calculates error for each weak learner in training?

I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed. I will ...
1
vote
0answers
32 views

How to Fit S shape (Sigmoid Function) in my scatterplot [closed]

How to interpret my chart? I want to get the maximum likelihood in logistic regression with this result (I'm really not sure if this is how it looks like): I am currently using logistic regression to ...
1
vote
0answers
19 views

how to deal with columns that has different value in only 1 or 2 rows?

I have very high dimensional data. Almost 20% of the columns has different value in less than 1% of rows. All of these are binary columns and many columns has 0s filled in more than almost 98% of rows....
0
votes
0answers
21 views

Temporal data types: time-series vs temporal sequences

Regarding the types of temporal data, I saw on a temporal data mining book (Mitsa, Theophano. Temporal data mining. CRC Press, 2010), that there are 3 types of temporal data: time series, temporal ...
0
votes
1answer
13 views

Compare two tends with big difference in absolute value

I'm studying the Spotify streams trend for different nations in order to show the impact of the pandemic on music listeners. So far i retrieved my data, and plotted it. Obviously, since the various ...
0
votes
1answer
28 views

regarding lasso.score in lasso modeling using scikit-learn

I once saw the following code segment of using lasso model based on scikit-learn ...
-1
votes
0answers
19 views

What's going on with my “triplet optimization” algorithm?

I've tried StackOverflow and Comp Sci SE, and everyone points here. I wrote an algorithm for data analysis using the CERN ROOT framework. It takes in three files of sorted UNIX timestamps from a ...
0
votes
1answer
18 views

Algorithms for Anomaly Detection of Event Sequence Data [Python/R]

I am building an anomaly detection system of event sequence data (transactions). For each timestep, a transaction can be in any of 76 different stages. My dataset is therefore a 3D array of size(m,t,N)...
1
vote
0answers
30 views

Association Rules

I'm using association rules for a project and noticed that there is a dearth of papers in the last 10-15 years on the topic although it seemed really popular 15-25 years ago. Is there any reason why?
0
votes
0answers
10 views

Are there any methods for detecting binary time series periodicity?

I am struggling with finding methods which can be used to detect the periodicity of binary time series.(the binary time series look like 0,0,1,0,0,1,0,0,1... or 1,0,0,0,1,1,0,1,0,0,0,0...)
0
votes
0answers
21 views

does someone know a DCore (Density Core Clustering) implementation?

Hello everyone i'm a student of computer science. I'm writing a thesis about Clustering in which we analyze some types of clustering methods. My professor told me to research about an algorithm called ...
2
votes
3answers
35 views

Combining Two CSV's in Jupyter Notebook

I want to combine both CSV files based on Column1, also when combined each element of Column1 of both csv should match and also each row or Please suggest how to reorder Column1 according to another ...
0
votes
0answers
11 views

Data-mining/web-scraping scientific articles and return the journal's impact factor of that year

Is there a way to search for scientific articles about something like "Stimulated Raman Histology" and find the distribution of Impact Factors of the corresponding journals that published ...
3
votes
1answer
69 views

When to One-Hot encode categorical data when following Crisp-DM

I have a dataset that contains 15 categorical features (2 and 3 level factors which are non-ordinal) and 3 continuous numeric features. Seeing as most machine learning algorithms require numerical ...
-1
votes
1answer
23 views

How can I predict the true label for data with incomplete features based on the model learned by data with complete features? [closed]

for example, the model was learned by training data with complete features (f1,f2,f3,f4,f5,f6) but, I wonder the model can test data with incomplete features (f1,f2,f3) to attach true label into these ...
1
vote
1answer
28 views

Selecting threshold for F1 Score

When selecting a probability threshold to maximize the F1 score prior to deploying a model (based on the precision-recall curve), should the threshold be selected based on the training or holdout ...
2
votes
1answer
57 views

KNN Imputation utilize mean or mode?

In my current project, I am doing KNN imputation with K = 5 and I am using sklearn.impute.KNNImputer. I have a mix of continuous and nominal variables(encoded as 0/1 or ordinal ones that have been ...
1
vote
1answer
44 views

PCA & Clustering Confusion

I have a question related to K-Means clustering and PCA. In my project, I have two target classes - 0 and 1- and I am trying to group the records that were predicted as 0 into 5 clusters. I am using ...
2
votes
1answer
22 views

How to visualize categorical data with numeric columns

I have data like as shown below I would like to represent the above tabular data in a visual form. However, the below graph may not work because my real data as 50K unique drug names. So, is there ...
0
votes
0answers
18 views

What scenarios are best to capture the behavior of a single person performing a regular task ? Not yet answered

I'm doing research on Predicting actions using CCTV feeds to control appliances. For example, it can be something as simple as turning the light off in your room when you go to bed at night. For my ...
0
votes
1answer
23 views

What to choose for null hypothesis and alternate hypothesis?

In this article which explains types of t-tests for “Independent Two-Sample t-test” example: We can confirm that the t-statistic is again less than the t-critical value so we fail to reject the null ...
0
votes
1answer
40 views

Association Rules with Python (coded dataset)

I have this dataset which I really need to use association rules techniques on. The dataset has like 90 variables, many of which are ordinal. Thing is, the data is already coded using numbers instead ...
0
votes
1answer
17 views

Best Approach to Forecasting Numerical Value Based on time series and categorical data?

Consider a dataset of thousands of car repairs that have been performed. In simplest of terms, the columns to consider are the time of year when it was broken (seasonal changes in demand for car ...
1
vote
2answers
31 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
0
votes
1answer
61 views

Is this attribute numeric or categorical (ordinal)? Help!

So I have this dataset I need to perform several techniques on as part of a data mining/machine learning project of some sort in PYTHON. There are a couple of features however, that have me very ...
1
vote
1answer
29 views

How to do data augmentation for Machine Learning on statistical data?

I am training machine learning classification model. I have data in csv format with lets say 5 features(or columns) and 100 such observations(or rows). I want to add more similar data to improve my ...
4
votes
2answers
220 views

a data point - too little to spend resources during data preparation?

I am learning analytics online and have some quick questions. Usually when we do analysis, why is that we usually ignore the items/data points that are less frequent? Let's say for ex: we have is ...
0
votes
0answers
11 views

Space partitioning to estimate product weight

Looking for some vocabulary to help me refine my research so I can tackle this problem. Here's an overview of the problem statement I'm working on. At my company we manufacture various products, ...
0
votes
0answers
10 views

How to validate Association Role Mining

I have used apriori to mine roles on a training set and the result is similar to the below I have been looking around for a kind of validation or measure of such ...
0
votes
2answers
46 views

Need help understanding data leakage

I am a newbie to this stuff so I am sorry if my question is stupid~ I need help understanding what data leakage between X_train and X_test is and when exactly it happens. I am currently working on a ...
0
votes
1answer
31 views

kMean clustering for recommendation

I have a file with 50000 rows from a library platform. Each individual row saves a user, and shows the order in which the user, has selected. The books could be from various categories (e.g. roman, ...
1
vote
1answer
42 views

How to get two boxplots on same axis?

I have two dataframes (both of different lengths, etc) but they have exact same columns and I would like to explore the distribution of one of these columns by plotting a boxplot. However, how do I ...
1
vote
2answers
26 views

Will one hot encoding / unbalanced columns cause bias to Clustering Analysis?

I'm wondering if having too many columns about one certain feature is gonna cause bias to the clustering analysis. For example, if my dataset has columns = ['incoming calls', 'outgoing calls', '...
1
vote
0answers
31 views

Genetic algorithm - Feature selection packages in Python

Can you share some packages in Python which are implemented that I can use for selecting features based on a genetic algorithm? I did refer to this AUTO-ML post and found out that it is useful but ...
0
votes
0answers
17 views

Python Pandas, how to select cases for which timestamps should be rounded?

I need to only select the cases for sensor type == air to be rounded to the nearest 5 seconds but do not how I should use a function to make this happen. I do have the following lines: ...
3
votes
1answer
54 views

Replace human judgements with mathematical approach/theories

I would like to give a context of what I did. 1) Let's say there are two dictionaries (dict A and dict B) each containing a list of words/terms as shown below. 2) Now my task is to find matching ...
0
votes
0answers
22 views

k-means clustering

In my data, I have zero trade flows between some countries. I have latitude and longitude information and by using python, I converted them into the addresses. To handle zero trade flows problem, I ...
2
votes
1answer
19 views

Topology of Data

Do every data resembles some form of space like manifold? Meaning, do geometry of data (especially the big data) is a manifold? Is every data embedded to some sphere? If that is so, can we also say ...
1
vote
1answer
55 views

Automatically Detect Valid/Interesting pivot/drill downs of a dataset

Imagine you are given a tabular data set with a limited set of columns and rows and you are asked to find the valid/interesting pivot configuration by exploring the data. The brute-force option is to ...
0
votes
0answers
10 views

Hypothesis testing with a historical time-series dataset

For example, what are some ways to prove/disprove viewing rates go down on the weekends (compared to weekdays) in a historical time series data set?
0
votes
1answer
56 views

Why do i get error when I use corpus in Orange v 3.25.0 text mining widget and no import document option?

I am experimenting with Orange data mining tool. When i use 'Corpus' from text mining. It gives me error. . I tried many things, but still unable to resolve this issue. Besides that, In text mining, ...
1
vote
0answers
18 views

Is it necessary to transform data to normal distribution when removing outliers for xgboost?

sorry if this is statistics 101 but i cannot find a similar question. I am wanting to use xgboost to classify my data in two classifications. my data is numerical (financial statement data) and i can ...

1
2 3 4 5
21