Questions tagged [data-mining]
An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.
1,176
questions
0
votes
0
answers
14
views
GitHub Archived Repositories
I'm trying to build a model that observes patterns of source control usage, from how many files are changed per commit, how many contributors there are, even semantic analysis on the commit messages.
...
- 101
1
vote
1
answer
26
views
Making a netcdf data using xarray
I am very very new to the world of data science as I only started using it in my new job so I would really appreciate help from the community experts (maybe also in simple words :)).
I am trying to ...
- 113
1
vote
0
answers
23
views
Mining association rules between time series
I have a pandas dataframe that represents a time series. My time series is segmented over the phase type that the robot is performing (i.e. I have a column with the phase type per timestamp and the ...
- 45
0
votes
0
answers
3
views
Can support be accounted multiple time for same sequence in sequential pattern mining?
I want to find top-k frequent sequential patterns from a list of sequences. Order of occurrence matters here (subsequence (1,2) is not same as (2,1)). Suppose I have 2 sequences:
S1=[1,2,3,1,5,2]
S2=[...
- 1
2
votes
3
answers
136
views
How to forecast a timeseries with geolocation data?
I have created a dataset with my geolocations from the last three months. The data set contains longitude, latitude, and timestamp, with a frequency of every 5 minutes. Based on this data, I want to ...
- 93
0
votes
0
answers
15
views
Decent cloud provider to run python algorithmic trading [closed]
I'm searching for stable safe intuitive solutions to deploy my algo trading online, I have no experience in cloud computing, and I want to train my models weekly locally and update cloud.
I't will ...
0
votes
0
answers
16
views
Finding a relation between three variables
I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
- 1
0
votes
1
answer
30
views
Classification of a noisy data
What method can be used to classify data in the following example?
There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
- 1
0
votes
0
answers
8
views
What models/techniques can I use to generalize industry specific datasets?
I have a few dictionaries pertaining to different industries (ie. tech, manufacturing, education, etc.). These dictionaries map phrases and keywords to a sentiment score. I'd like to create a ...
0
votes
0
answers
30
views
Refining AI problem statement - suggestions
I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
- 2,449
0
votes
0
answers
20
views
0
votes
0
answers
16
views
Creating song vectors from playlists
Given a large number of playlists, I would like to create song vectors using this data. Furthermore, I would like to measure the performance of the model, so that it can be optimized for some ...
- 111
1
vote
1
answer
32
views
How to increase retention?
As you might already know there is a concept of retention. Let's say I have created a game and today hundred people have downloaded my game. Let's say tomorrow 47 out of yesterday's hundred people are ...
- 121
0
votes
0
answers
5
views
Can Data Mining or traditional EDW also do Process Mining?
Can Data Mining also do Process Mining? Can frameworks and tools used for Data Mining or EDW be used for Process Mining?
Summarize the problem
I'm investigating Process Mining with a view to create ...
- 101
0
votes
0
answers
126
views
AttributeError: 'NoneType' object has no attribute 'longitude'
I'm trying to get the location longitude of a address using geopy.
normally this works completely fine.
...
0
votes
0
answers
21
views
Any ideas on deciding a hotel competitive group?
Suppose for each hotel, I need to find a group of hotels which are competitive to this hotel. Competitive hotels mean they probably share the same group of customers, and the customers will make a ...
1
vote
1
answer
59
views
revenue forecast using regression - what is the input for future?
I have a dataset with quarter wise revenue for past 3 years from Jan 2020 to Dec 2022. I have 4642 customers.
Each customer has 1 row of data which includes features based on his purchase frequency, ...
- 2,449
1
vote
0
answers
13
views
How to do unsupervised clustering on sentences to find intents
I am working on chatbot for students.
So, I have chatlogs on conversation between student and tutor, which is on mathematical problems (no labels).
First thing I want to find is intents, such as ...
- 13
0
votes
0
answers
8
views
User Actions in Browser mining
I'm looking for a way to collect user actions: opening new tabs, switching between tabs, entering text, following links in browser by user. it should log user actions as well as context: what was the ...
0
votes
0
answers
13
views
Timeseries Task Enhancement using ExponentialSmoothing
I'm working on a time-series problem, and after running ExponentialSmoothing with different configurations I got the best two models shown in the screenshot below where first one is based on MAE ...
1
vote
1
answer
25
views
Empirical indications regarding demanded skills and tasks of data science jobs?
I am wondering if there are is any information about the current (and prospected) shares in skills required for advertised/existing data science jobs. This includes of course also the concrete tasks ...
- 111
0
votes
0
answers
69
views
Is it possible to turn the gmail inbox into a dataset for AI fine tuning?
My idea is to have a dataset of my gmail emails and replies. The purpose is to create a bot that can reply new emails based on all past correspondence in my inbox. How do i prepare such dataset from ...
- 1
1
vote
1
answer
20
views
Data Analytics Documantaions
I am working as a data analyst in a company. Me and my colleagues use different tools and software to analyze the data and make the reports (e.g., Excel, Python, R, Alteryx, SQL, Tableau).
Each one ...
- 1,905
1
vote
1
answer
29
views
g-mean affected by threshold moving?
Is g-mean (geometric mean of sensitivity and specificity) a better evaluation metric than other metrics (that are derived from the confusion matrix) in situations where decision thresholds are not the ...
- 68
0
votes
1
answer
24
views
Orange: Predictive Model for Future Wifi Speed
I would like to use Orange to create a model that will allow me to predict future WiFi speeds using average quarterly WiFi speed from the last few years. The data sets I am using include the average ...
- 1
0
votes
0
answers
62
views
Orange data mining: instances with missing targets are ignored while scoring
In predictions warning showing "instances with missing targets are ignored while scoring" is coming how to resolve
0
votes
0
answers
11
views
word2vec, attention for predicting the next state in business processes
For this repo and paper: https://github.com/diogoff/unlabelled-event-logs
Business processes are modeled as Markov and Expectation Maximization is used to find the model.
So suppose a business process ...
- 562
0
votes
1
answer
24
views
What is the name for 'cheating' with data, i.e. letting machine learning models use out-of-sample data?
What is the term for abusing data and machine learning methods to get better results than normally (with proper training and test set)? Some of my co-workers call it data-mining but I do not believe ...
- 143
1
vote
1
answer
34
views
What is the fastest way to detect periodicity in a binary time series?
Example,
T = array([0,1,1,1,0,0,1,0,1,1,1,0,0,1,1,1,1,1,0,0,1,0,1,1,1,0,0,1,0,1,1,0,0,0,1,0,1,1,1,0,0,1])
( T is almost a repeat of the array([0,1,1,1,0,0,1]) six ...
- 11
0
votes
0
answers
19
views
how to predict most frequent purchased item in e-commerce?
i have the following dataset:
witch its market transaction dataset and i need to predict the most frequent purchased items based on transaction history, for example: if sara bought milk and cookies ...
0
votes
0
answers
38
views
linear regression - at future time points
I have a dataset of customer transactions containing revenue, customer id, region, product category, product id, support team, date of transaction etc. The data ranges from Jan 2017 to Nov 2nd 2022.
...
- 2,449
0
votes
0
answers
77
views
How to scrape stock moment data from Online web Trading Platform
I recently started a Data Analysis project that involves data relating to stock market prices (i.e currency pair information). An example is AUD/CAD trade. I wish to scrape 1-minute price details from ...
0
votes
1
answer
23
views
In between data partition and feature selection which one need to perform 1st
I need to perform a feature selection on my dataset. My dataset is an imbalanced dataset where the class of interest is the minority class. Therefore, recall and F2 measures are two important metrics ...
- 359
0
votes
0
answers
15
views
Attribute selection using Weka produce different arrangement for training and test set
My question is related to attribute selection using Weka. The method that I used to select the attributes is Infogain filter which is available in Weka. I selected top 30 attributes from my dataset ...
- 359
0
votes
1
answer
31
views
Why an already trained model is not generalizable to another related dataset?
A model is trained to predict the median temperature of Boston. The resulting model works well according to their validation data. However, this model performs poorly when used to predict the ...
- 1,267
0
votes
1
answer
26
views
Classification for choice data
It is essentially a choice modelling problem, but hopefully can be addressed by classification.
Suppose one needs to choose a route to drive to work among many candidates in his mind. These candidates ...
- 101
0
votes
1
answer
48
views
Likert Scale Target Variable
I have a case study where the target variable (a single factor) gauged through multiple items. the items are measured using 5-Likert scale (Never, Seldom, Sometimes, Often, Very often, Always)
since ...
0
votes
1
answer
24
views
Fraud Pattern Detection Dataset Creation
I have simulated a dataset for clicks on YouTube videos that records each click using dummy data. In the dataset, I collect information such as timestamp, ...
- 195
0
votes
0
answers
16
views
What is two-phase learning with random under sampling?
I came across this new technique of sampling which is using two-phase learning from amazon paper on Reducing Amazon’s packaging waste using multimodal deep learning
Can someone help in understanding ...
- 141
1
vote
0
answers
57
views
Jenks goodness of variance fit - Interpretation
I am working on clustering/grouping 1D data. I am trying to find bins of multiple variables seperately.
So, I tried the jenks natural breaks algorithm.
Based on the ...
- 2,449
2
votes
0
answers
38
views
Should credit be given to AI model - low data scenario [closed]
In my office, we recently built an AI model for project success prediction using binary classification.
Though the dataset size was small (977 records), my boss still wanted to go ahead with the POC ...
- 2,449
1
vote
1
answer
30
views
Building a simulation model for IoT device
I have an IoT device that takes inputs in bytes and returns the output in bytes; My goal is to build a software simulator for the IoT device.
Some example inputs & outputs:
...
- 11
0
votes
0
answers
119
views
How to generate the best combinations through machine learning model?
I want to select the best combinations as the input to my task, but the possible combinations are too many, cannot really try each one of them, so I am wondering if there is any machine learning model ...
1
vote
0
answers
21
views
Working with massive data what is the right approach
let's say I have database with massive data (millions of rows)
additionally Let's say 26 Million rows are entered every day
I want to build a fraud model to check these 26 Million rows every day..
as ...
- 35
0
votes
0
answers
11
views
Recommend low cost channel based on customers demographics
I am new to data science.
I am trying to implement recommendation system which only uses customer demographics data and activities like gender, age, customer transaction frequency in last 1,3 and 6 ...
- 121
0
votes
0
answers
41
views
Good and Bad claims classification with time associated events using Machine Learning
I am working on a problem where I have some insurance settlement data with CaseId where customer a went through some events {E1,E2,E3...En} to get their insurance ...
- 101
0
votes
0
answers
26
views
From meter's timeseries data, know if it associated with electric vehicle
We currently got millions meter data, table would like below
id
timeseries
consumption_wh
1
timestamp
xx.xx
1
timestamp+30m
xx.xx
1
timestamp+60m
xx.xx
...
...
...
1
timestamp+30m*n
xx.xx
2
...
- 1
1
vote
2
answers
49
views
Automate Clustering predictions and RFM metrics
We did a POC for customer segmentation and followed the below approach
a) extract data from source system (SAP business objects)
b) Use python jupyter notebook to manipulate, merge and group data (...
- 2,449
0
votes
0
answers
20
views
Productionizing ML model and result distribution
We completed POCs for our project involving classification algorithm (Random Forests)
Background
This project is about predicting project success rate (predict_proba). Data will be refreshed every 45 ...
- 2,449
0
votes
1
answer
16
views
Category under which smote falls into
So, I have a tree diagram where different algorithms are present. For example XGBoost falls under gradient boosting, decision tree and random forest comes under classification and many more. I would ...
