Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

The tag preprocessing says it is a data-mining technique ! Is it a correct definition?

This site appears to give Wrong definition of preprocessing of data.Data-mining has no connection with preprocessing of data. The data preprocessing focuses on data preparation for developing software ...
1
vote
0answers
18 views

How to process categorical variable having lots of unique values in linear regression?

I have House Price dataset and I am using linear regression to predict the house price. while data preprocessing I found a variable called "Location" and it have around 342 unique value. For ...
-1
votes
2answers
66 views

How to identify patterns in dataset [closed]

I have a dataset that pertains to calls received to a hospital emergency helpline. My task is to identify patterns in the data. How do I approach examining the dataset to identify patterns? Extract ...
0
votes
0answers
16 views

How to find the relationship between one dataset and two others?

From a STEM problem, I vary a variable x within a range and calculate quantities $U(x)$, $V(x)$ and $W(x)$. I want to figure out an analytical relationship between unknown $U(x)$ and the other two ...
1
vote
1answer
24 views

Question about information leakage

I am well aware that to avoid information leakage, it is recommended to fit any transformation (e.g., standardization or imputation based on the median value) on the training dataset and applying it ...
1
vote
1answer
28 views

How to predict when and why of hospitalization?

I have an EHR data source which has info on a) Patient visit records (Inpatient, outpatient, Emergency etc) and why did he visit hospital (diagnosis codes attached to each visit) b) Patients drugs ...
1
vote
0answers
10 views

Comments vs. Files (& how to import w/o packages) for Data Science [closed]

I am trying to settle on an efficient & pythonic workflow for developing scripts in data science work that are scalable and don't turn to spaghetti once things get complicated and parameters are ...
0
votes
0answers
13 views

How to pick the most optimal set of association rules?

I'm currently running association rules on a dataset that has about 22000 rows, but there is no obvious threshold that the support value must have. So, I ran the rules three times with the following ...
0
votes
0answers
25 views

How to Integrate/map Population Data with/over Sample Data

Problem Definition: Our organization is conducting different type of surveys and Census in our country. The basic difference between Census and Survey is that, the target of Census is the Complete ...
0
votes
0answers
21 views

Invent first, find its use later

The typical pipeline in ML is Find a data-related problem that you want to solve Build a model or algorithm that feeds on data related to the problem to try to solve the problem Check if the solution ...
0
votes
0answers
15 views

regression by grouping the dependent variable

I have a large dataset exploring the effects of the independent variables on the dependent variable using Poisson regression since the dependent variable is a count variable. However, the range of the ...
0
votes
0answers
9 views

Trouble understanding k means distance calculation with high dimension

If I have a dataset with 10 features, 100 samples, and set I k = 3, three rows will become the 'centroid'. However, if I have 10 features how is the centroid calculated i.e. what exactly is it? ...
-1
votes
1answer
37 views

Self Organizing Map (SOM)

How do you use SOM as a supervised learning technique? Which approach can be added to SOM to turn it to supervised learning?
1
vote
1answer
30 views

How to implement a POS tagger using Word co-occurence and Clustering concepts?

I am thinking of implementing a POS tagger by myself. a POS tagger, extracts the syntax role of a word in a sentence. According to my studies, word co-occurrence is a technique to analyze word ...
-1
votes
2answers
38 views

Machine Learning for medical researchers [closed]

My friend is a medical researcher and he want to use machine learning for prediction. Is there any one who is not a computer science person and he learnt programming and machine learning in a very ...
1
vote
1answer
31 views

group similar subjects and train only using them

I have a dataset with 5k subjects. It's a binary classification problem where I have 3000 positive and 2000 negative subjects. Now to build a model, I don't like to train the usual way (where we build ...
0
votes
0answers
18 views

how do i preprocess percentage data?

I am analyzing a problem where i have 5 diseases and the measure of effectiveness of a remedy in its 7 first applications. The data is organized as an Excel spreadsheet as follows: (The spreadsheet ...
2
votes
0answers
12 views

Keep retweets during topic-modelling [duplicate]

I got a dataset made out of tweets and I need to classify them into topics. For topic modelling with LDA I have cleaned out the dataset (removing stopwords, mentions, symbols, etc). Do I need to ...
1
vote
1answer
40 views

Is there a standardized way to do data analysis?

Is there a standard way to do data analysis? So for example, something like this: 1. Data mining 2. Data cleaning 3. xx 4. Data and result interpretations I ...
0
votes
2answers
76 views

Demonstrating and visualizing cluster centroid updates by using the first four iterations in Python?

I managed to create a dataset with 6 clusters and visualize it with the code below, now I would like to visualize demonstration of update of the cluster centroids in KMeans algorithm. This ...
0
votes
0answers
11 views

Bayesian Network Modeling on time series data with constant discrete features

I'm trying to model a dynamic bayesian network that can infer relationships between traffic sate of different road links over time. The main theme of mystudy is to model the relationship of change in ...
0
votes
1answer
45 views

precision recall curve is like stairs [closed]

I am training an ensemble model using a 400 data set sample this led to a precision recall curve that looks like stairs ? what would be the reason beside the low number of samples ?
0
votes
0answers
14 views

Direct Discriminative Pattern Mining for Effective Classification - implementation

I'm looking for any actual working code implementation of the DDPMiner algorithm mentioned in the Direct Discriminative Pattern Mining for Effective Classification article form 2008 I'm having real ...
1
vote
1answer
20 views

Help with DDP Mining algorithm for Effective Classification of data sets from 2 groups

I'm trying to implement the DDPmine algorithm from this article as part of some project, and I do not understand where in the algorithm we use the Class Label of each transaction? We have transactions ...
1
vote
1answer
60 views

Machine Learning - Precision and Recall - differences in interpretation and preferring one over other

I have summarising this from lot of blogs about Precision and Recall. Precision is: Proportion of actual positives that classifier has predicted as positive. meaning out of the sample identified as ...
0
votes
0answers
11 views

query category wise dara

I have stored the some products and there respective category in sql. Like this. Now I want fetch all product from table with respective category. output should be=>
0
votes
1answer
13 views

Scaling the data iteratively one by one vs batch scaling

I have 2000 signals in a dataset of shape (2000, 400000) where each signal is recorded within the range -127, 128. I want to downscale each signal from (-127, 128) ...
2
votes
0answers
18 views

What branch of data science to use for revealing preceding conditions of desired outcome

On financial data series, I would like to determine if some conditions precede a desired outcome. For example, let's take as an example the daily data of a stock. On some days, the stock experience ...
1
vote
1answer
35 views

Recommender System for mostly unique user and items

I am trying to develop a recommender system for a job matching problem. My data consists of past matched candidate profiles and job profiles as well as if there was a success such that both, candidate ...
0
votes
0answers
27 views
4
votes
4answers
969 views

What is the difference between AI, ML, NN and DL? [closed]

What is the difference between the following four categories: Artificial Intelligence (AI) Machine Learning (ML) Neural Network (NN) Deep Learning (DL) Data Science My current understanding is that ...
1
vote
1answer
30 views

Mining timelines in a long text

I am trying to detect timeline of brands histories. For my specific case, I believe it is easy because data is already clustered. For each Wikipedia article I can spot sentences surrounding dates. ...
0
votes
0answers
13 views

Where can I find Business Intelligence Case Studies?

I am teaching an undergraduate course on business intelligence and data mining. I was wondering if anyone has any resources where students can search for BI case studies? I've tried google myself but ...
0
votes
0answers
18 views

What kind of techniques that I can use to see which type of of entries in the dataset became more popular lately(over a time-span)?

This is for my school project. I've been digging the internet in to find a solution that is backed-up really well but I couldn't find a metric that makes sense. I have a dataset, that has objects say: ...
1
vote
1answer
39 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
0
votes
0answers
18 views

Prediction using OSM and Airbnb Listing dataset

I am building a model to predict the price of an Airbnb listing. Using OSM("https://wiki.openstreetmap.org/wiki/Downloading_data") and Airbnb listing("http://insideairbnb.com/get-the-...
1
vote
1answer
75 views

Macro averaged in binary classification

Is macro averaged the precision, recall and F1 score recommended for binary classification data? The calculation of precision is calculated for both class and averaging it makes it seems better. Or ...
0
votes
1answer
96 views

How to convert a pytorch geometric graph to Networkx Multigraph?

I have a weighted graph stored in Data object, and I want to convert this graph to Networkx MultiGraph. Here is what I have tried: ...
1
vote
1answer
18 views

what is criterion in flcuster of scipy package?

Could some one explain what does criterion of fcluster indicate? I tried to read the documentation but I am unable to understand. What does maxclust criterion indicate?
4
votes
5answers
2k views

Why does my model produce too good to be true output?

I am trying to run a binary classification problem on people with diabetes and non-diabetes. For labeling my datasets, I followed a simple rule. If a person has T2DM...
2
votes
1answer
37 views

Why Continous Variable Buckets Overfitting model

I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
1
vote
1answer
19 views

Can a Box plot be used for finding the useful features from the dataset? [closed]

I am reading a book by professor Trevor Hastie and professor Robert Tibshirani called "Introduction to Statistical Learning". In the applied section of the chapter 4, there is a question 11(...
1
vote
1answer
35 views

unit-testing Machine Learning models

I have been asked to unit-test my machine learning model(not the code that made the model). Since we wouldn't actually know what predictions models make, how to carry out the unit-testing to check the ...
0
votes
1answer
29 views

GridSearchCV Acting Weird

I am using GridSearchCV to find the best combination of parameters for SVM. However, the parameters chosen by GridSeasrchCV do not seem to be the best ones. I tried some parameters randomly and they ...
0
votes
0answers
8 views

Get latest Item by Date for a Recommender System

I am building a Recommender System where I am giving the User 3 Recommendations depending upon for the Webpage he is on. Let's say My model gives me 3 Recommendations from 2020, 2019, 2015. I would ...
-2
votes
1answer
63 views

Which machine learning model to choose? [closed]

I am a beginner in data science. I am facing the problem of choosing the most appropriate algorithm for my specific problem. I am building a recommendation system that gives students insight into ...
2
votes
0answers
71 views

Which one to choose to identify patterns of user activity: Sequence analysis or process mining?

I have the following user activity data that where for each user the activity type they were engaged are recorded along with the phase: ...
0
votes
1answer
50 views

Python : Approaches to mine for time ordered sequences

I have a dataset that looks like this : ...
0
votes
1answer
24 views

Which tool is good to collect tweets on 50 keywords over the last 5 years and then analyze them with the LDA algorithm or sentiment analysis?

I want to find tweets from the last 5 years to a topic. For this I decide for 50 Keywords (related to the main topic), where I want to find data on Twitter. I want to find out how the trend on the ...
0
votes
0answers
12 views

How many items does apriori algorithm consider while working on data set of say 100 product?

For finding supermarket product association rules it is said that apriori is suited to identify frequent itemsets. From here then we can mine association rules. Given 100 products, and their ...

1
2 3 4 5
21