Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

GitHub Archived Repositories

I'm trying to build a model that observes patterns of source control usage, from how many files are changed per commit, how many contributors there are, even semantic analysis on the commit messages. ...
1 vote
1 answer
26 views

Making a netcdf data using xarray

I am very very new to the world of data science as I only started using it in my new job so I would really appreciate help from the community experts (maybe also in simple words :)). I am trying to ...
  • 113
1 vote
0 answers
23 views

Mining association rules between time series

I have a pandas dataframe that represents a time series. My time series is segmented over the phase type that the robot is performing (i.e. I have a column with the phase type per timestamp and the ...
  • 45
0 votes
0 answers
3 views

Can support be accounted multiple time for same sequence in sequential pattern mining?

I want to find top-k frequent sequential patterns from a list of sequences. Order of occurrence matters here (subsequence (1,2) is not same as (2,1)). Suppose I have 2 sequences: S1=[1,2,3,1,5,2] S2=[...
2 votes
3 answers
136 views

How to forecast a timeseries with geolocation data?

I have created a dataset with my geolocations from the last three months. The data set contains longitude, latitude, and timestamp, with a frequency of every 5 minutes. Based on this data, I want to ...
  • 93
0 votes
0 answers
15 views

Decent cloud provider to run python algorithmic trading [closed]

I'm searching for stable safe intuitive solutions to deploy my algo trading online, I have no experience in cloud computing, and I want to train my models weekly locally and update cloud. I't will ...
0 votes
0 answers
16 views

Finding a relation between three variables

I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
0 votes
1 answer
30 views

Classification of a noisy data

What method can be used to classify data in the following example? There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
  • 1
0 votes
0 answers
8 views

What models/techniques can I use to generalize industry specific datasets?

I have a few dictionaries pertaining to different industries (ie. tech, manufacturing, education, etc.). These dictionaries map phrases and keywords to a sentiment score. I'd like to create a ...
0 votes
0 answers
30 views

Refining AI problem statement - suggestions

I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
  • 2,449
0 votes
0 answers
20 views

ı am writing data process pipeline with luigi but ı get error

...
0 votes
0 answers
16 views

Creating song vectors from playlists

Given a large number of playlists, I would like to create song vectors using this data. Furthermore, I would like to measure the performance of the model, so that it can be optimized for some ...
1 vote
1 answer
32 views

How to increase retention?

As you might already know there is a concept of retention. Let's say I have created a game and today hundred people have downloaded my game. Let's say tomorrow 47 out of yesterday's hundred people are ...
  • 121
0 votes
0 answers
5 views

Can Data Mining or traditional EDW also do Process Mining?

Can Data Mining also do Process Mining? Can frameworks and tools used for Data Mining or EDW be used for Process Mining? Summarize the problem I'm investigating Process Mining with a view to create ...
  • 101
0 votes
0 answers
126 views

AttributeError: 'NoneType' object has no attribute 'longitude'

I'm trying to get the location longitude of a address using geopy. normally this works completely fine. ...
0 votes
0 answers
21 views

Any ideas on deciding a hotel competitive group?

Suppose for each hotel, I need to find a group of hotels which are competitive to this hotel. Competitive hotels mean they probably share the same group of customers, and the customers will make a ...
1 vote
1 answer
59 views

revenue forecast using regression - what is the input for future?

I have a dataset with quarter wise revenue for past 3 years from Jan 2020 to Dec 2022. I have 4642 customers. Each customer has 1 row of data which includes features based on his purchase frequency, ...
  • 2,449
1 vote
0 answers
13 views

How to do unsupervised clustering on sentences to find intents

I am working on chatbot for students. So, I have chatlogs on conversation between student and tutor, which is on mathematical problems (no labels). First thing I want to find is intents, such as ...
  • 13
0 votes
0 answers
8 views

User Actions in Browser mining

I'm looking for a way to collect user actions: opening new tabs, switching between tabs, entering text, following links in browser by user. it should log user actions as well as context: what was the ...
0 votes
0 answers
13 views

Timeseries Task Enhancement using ExponentialSmoothing

I'm working on a time-series problem, and after running ExponentialSmoothing with different configurations I got the best two models shown in the screenshot below where first one is based on MAE ...
1 vote
1 answer
25 views

Empirical indications regarding demanded skills and tasks of data science jobs?

I am wondering if there are is any information about the current (and prospected) shares in skills required for advertised/existing data science jobs. This includes of course also the concrete tasks ...
0 votes
0 answers
69 views

Is it possible to turn the gmail inbox into a dataset for AI fine tuning?

My idea is to have a dataset of my gmail emails and replies. The purpose is to create a bot that can reply new emails based on all past correspondence in my inbox. How do i prepare such dataset from ...
  • 1
1 vote
1 answer
20 views

Data Analytics Documantaions

I am working as a data analyst in a company. Me and my colleagues use different tools and software to analyze the data and make the reports (e.g., Excel, Python, R, Alteryx, SQL, Tableau). Each one ...
  • 1,905
1 vote
1 answer
29 views

g-mean affected by threshold moving?

Is g-mean (geometric mean of sensitivity and specificity) a better evaluation metric than other metrics (that are derived from the confusion matrix) in situations where decision thresholds are not the ...
  • 68
0 votes
1 answer
24 views

Orange: Predictive Model for Future Wifi Speed

I would like to use Orange to create a model that will allow me to predict future WiFi speeds using average quarterly WiFi speed from the last few years. The data sets I am using include the average ...
0 votes
0 answers
62 views

Orange data mining: instances with missing targets are ignored while scoring

In predictions warning showing "instances with missing targets are ignored while scoring" is coming how to resolve
0 votes
0 answers
11 views

word2vec, attention for predicting the next state in business processes

For this repo and paper: https://github.com/diogoff/unlabelled-event-logs Business processes are modeled as Markov and Expectation Maximization is used to find the model. So suppose a business process ...
0 votes
1 answer
24 views

What is the name for 'cheating' with data, i.e. letting machine learning models use out-of-sample data?

What is the term for abusing data and machine learning methods to get better results than normally (with proper training and test set)? Some of my co-workers call it data-mining but I do not believe ...
1 vote
1 answer
34 views

What is the fastest way to detect periodicity in a binary time series?

Example, T = array([0,1,1,1,0,0,1,0,1,1,1,0,0,1,1,1,1,1,0,0,1,0,1,1,1,0,0,1,0,1,1,0,0,0,1,0,1,1,1,0,0,1]) ( T is almost a repeat of the array([0,1,1,1,0,0,1]) six ...
  • 11
0 votes
0 answers
19 views

how to predict most frequent purchased item in e-commerce?

i have the following dataset: witch its market transaction dataset and i need to predict the most frequent purchased items based on transaction history, for example: if sara bought milk and cookies ...
0 votes
0 answers
38 views

linear regression - at future time points

I have a dataset of customer transactions containing revenue, customer id, region, product category, product id, support team, date of transaction etc. The data ranges from Jan 2017 to Nov 2nd 2022. ...
  • 2,449
0 votes
0 answers
77 views

How to scrape stock moment data from Online web Trading Platform

I recently started a Data Analysis project that involves data relating to stock market prices (i.e currency pair information). An example is AUD/CAD trade. I wish to scrape 1-minute price details from ...
0 votes
1 answer
23 views

In between data partition and feature selection which one need to perform 1st

I need to perform a feature selection on my dataset. My dataset is an imbalanced dataset where the class of interest is the minority class. Therefore, recall and F2 measures are two important metrics ...
  • 359
0 votes
0 answers
15 views

Attribute selection using Weka produce different arrangement for training and test set

My question is related to attribute selection using Weka. The method that I used to select the attributes is Infogain filter which is available in Weka. I selected top 30 attributes from my dataset ...
  • 359
0 votes
1 answer
31 views

Why an already trained model is not generalizable to another related dataset?

A model is trained to predict the median temperature of Boston. The resulting model works well according to their validation data. However, this model performs poorly when used to predict the ...
  • 1,267
0 votes
1 answer
26 views

Classification for choice data

It is essentially a choice modelling problem, but hopefully can be addressed by classification. Suppose one needs to choose a route to drive to work among many candidates in his mind. These candidates ...
  • 101
0 votes
1 answer
48 views

Likert Scale Target Variable

I have a case study where the target variable (a single factor) gauged through multiple items. the items are measured using 5-Likert scale (Never, Seldom, Sometimes, Often, Very often, Always) since ...
0 votes
1 answer
24 views

Fraud Pattern Detection Dataset Creation

I have simulated a dataset for clicks on YouTube videos that records each click using dummy data. In the dataset, I collect information such as timestamp, ...
0 votes
0 answers
16 views

What is two-phase learning with random under sampling?

I came across this new technique of sampling which is using two-phase learning from amazon paper on Reducing Amazon’s packaging waste using multimodal deep learning Can someone help in understanding ...
1 vote
0 answers
57 views

Jenks goodness of variance fit - Interpretation

I am working on clustering/grouping 1D data. I am trying to find bins of multiple variables seperately. So, I tried the jenks natural breaks algorithm. Based on the ...
  • 2,449
2 votes
0 answers
38 views

Should credit be given to AI model - low data scenario [closed]

In my office, we recently built an AI model for project success prediction using binary classification. Though the dataset size was small (977 records), my boss still wanted to go ahead with the POC ...
  • 2,449
1 vote
1 answer
30 views

Building a simulation model for IoT device

I have an IoT device that takes inputs in bytes and returns the output in bytes; My goal is to build a software simulator for the IoT device. Some example inputs & outputs: ...
0 votes
0 answers
119 views

How to generate the best combinations through machine learning model?

I want to select the best combinations as the input to my task, but the possible combinations are too many, cannot really try each one of them, so I am wondering if there is any machine learning model ...
1 vote
0 answers
21 views

Working with massive data what is the right approach

let's say I have database with massive data (millions of rows) additionally Let's say 26 Million rows are entered every day I want to build a fraud model to check these 26 Million rows every day.. as ...
  • 35
0 votes
0 answers
11 views

Recommend low cost channel based on customers demographics

I am new to data science. I am trying to implement recommendation system which only uses customer demographics data and activities like gender, age, customer transaction frequency in last 1,3 and 6 ...
0 votes
0 answers
41 views

Good and Bad claims classification with time associated events using Machine Learning

I am working on a problem where I have some insurance settlement data with CaseId where customer a went through some events {E1,E2,E3...En} to get their insurance ...
0 votes
0 answers
26 views

From meter's timeseries data, know if it associated with electric vehicle

We currently got millions meter data, table would like below id timeseries consumption_wh 1 timestamp xx.xx 1 timestamp+30m xx.xx 1 timestamp+60m xx.xx ... ... ... 1 timestamp+30m*n xx.xx 2 ...
1 vote
2 answers
49 views

Automate Clustering predictions and RFM metrics

We did a POC for customer segmentation and followed the below approach a) extract data from source system (SAP business objects) b) Use python jupyter notebook to manipulate, merge and group data (...
  • 2,449
0 votes
0 answers
20 views

Productionizing ML model and result distribution

We completed POCs for our project involving classification algorithm (Random Forests) Background This project is about predicting project success rate (predict_proba). Data will be refreshed every 45 ...
  • 2,449
0 votes
1 answer
16 views

Category under which smote falls into

So, I have a tree diagram where different algorithms are present. For example XGBoost falls under gradient boosting, decision tree and random forest comes under classification and many more. I would ...

1
2 3 4 5
24