Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Looking for datasets on automobile parts information for machine learning

I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
Anand 's user avatar
0 votes
0 answers
12 views

Seeking datasets for training a Language Model on U.S. mortgage loan processes

I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
Anand 's user avatar
0 votes
0 answers
16 views

Pytrends package in python is not working correctly

I would like to download data for the keyword "Investment" for all countries from google trends. I wrote the following code: ...
entropy's user avatar
0 votes
0 answers
13 views

Get accurate alias list of people without information on aliases

I'm working on a project where I have a large dataframe of paintings (from the Art500k dataset), each row corresponds with a painting, containing the author's name in the ...
me9hanics's user avatar
0 votes
0 answers
10 views

Help with creating a panel dataset

I have some medical information of individuals spanning ten years which includes first name, last name, dob, and other variables (two/three that could help match observations i.e. state living, ...
GreenArrow's user avatar
0 votes
0 answers
14 views

How to choose the right typt of ANN architecture for a regression model

So, im working on a project where i am leveraging ai to get accurate price predictions in terms of houses and real estate properties. I would like to use an artificial neural network so now i have to ...
maizen's user avatar
  • 1
0 votes
1 answer
22 views

Clustering words with similar meanings

What methods are there to cluster words/word phrases with similar meanings together from a list of words/word phrases?
ros's user avatar
  • 1
0 votes
0 answers
10 views

What is the best sampling strategy for correlation analysis?

I have a big dataset, and i want to finds subspaces with high correlation among features. I want to take only samples of data. So, what is the best sampling strategy in this context. Thanks
Imen F's user avatar
  • 21
0 votes
0 answers
34 views

Does clustering belong to the domain of data mining or to the domain of machine learning?

Question 1. Does clustering belong to the domain of data mining or to the domain of machine learning? Or to both domains? Question 2. Depending on the answer to Question 1, could you please suggest a ...
Ommo's user avatar
  • 101
0 votes
0 answers
27 views

Unable to create associations using Apriori association rule algorithm for my dataset where the support is 0.01, confidence=0.8 and maxlen=10

Here's the code that I'm using: library(arules) library(arulesViz) library(tidyverse) library(readxl) library(knitr) library(ggplot2) library(lubridate) library(plyr) library(dplyr) tr <- read....
Abhik Bagchi's user avatar
0 votes
0 answers
34 views

Where to find the statement "time series discord is the best overall technique among all techniques" in Chandola et al.(2009)?

Senin et al. (2015) "Time series anomaly discovery with grammar-based compression" claim exactly the following (copied and pasted): in a recent extensive empirical study by Chandola et al., ...
Ommo's user avatar
  • 101
0 votes
0 answers
34 views

Using Bayesian statistics in time series forecasting

I would like to forecast demand count time series of taxi fleets at different locations on the map at different points in time. I.e. multivariate demand Time series forecasting. Given hierarchinal ...
Jose_Peeterson's user avatar
0 votes
2 answers
67 views

Been stuck on a DS problem. Just need to know whether this problem statement is solvable or not

I've been stuck on the following problem for weeks now. To be clear I'm not asking the community to provide a full solution. Just a few ideas or at least confirmation on whether this problem statement ...
Aakash Dusane's user avatar
0 votes
0 answers
55 views

Implicit feedback based recommendation - User - User recommendation

My dataset involves customer_id, item_id and Count of purchases. My datset size is small...we have 15 users and 317 items.. Currently, am trying to build a recommendation system based on user based ...
The Great's user avatar
  • 2,555
0 votes
0 answers
8 views

I want to import my states legislation into a database. I need advice in parsing large HTML, TXT into an importable form. It is Large PDF initialy

My data consists of large pdf files containing state legislation. I can convert the pdf to xml, txt or html, and I can manually go through, identify the tables and write a script to extract only the ...
RBB's user avatar
  • 1
0 votes
2 answers
60 views

How do I separate periodic data from time series data?

I am currently working on a classification task of gym exercises based on accelerometer data. I am trying modularize window extraction so I can train my model based on metrics within a window (which ...
David Chen's user avatar
0 votes
0 answers
24 views

Please give me suggestions to identify scammers from communication data

I have some communication behavior data of carrier subscribers, containing information on calls made, duration and base stations. What do I need to do with this kind of data or what kind of machine ...
WnagoiYy's user avatar
-1 votes
2 answers
139 views

How to generate synthetic dataset with dependent and independent features?

I want to generate a synthetic dataset, in python, composed of 10 features. 5 are dependent among each others and 5 are independent. Generate dependent features. ...
I Sui's user avatar
  • 57
1 vote
1 answer
164 views

How to predict what someone will order?

Suppose Prof. X goes to a road side tea-coffee shop everyday at 5pm just after his office. After reaching there he tosses a coin, and places his order tea or coffee. The shop owner Y has been ...
S. M.'s user avatar
  • 173
0 votes
0 answers
29 views

SMOTE-Tomek links with imbalanced dataset

I have a question: if I want to use SMOTE-Tomek links for my imbalanced dataset, should I now apply the method before or after the split?
Hind's user avatar
  • 1
0 votes
2 answers
39 views

Data Analysis Process

How do you feed data from a data warehouse to Python for ad-hoc analysis? My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
Bunny Eat Data's user avatar
1 vote
2 answers
163 views

what qualifies as a data leakage?

I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
The Great's user avatar
  • 2,555
4 votes
1 answer
147 views

Are imbalanced data problems solvable? [closed]

I am working as a data scientist for the past 2 years where I have worked on problems related to binary classification, revenue prediction etc. In the past two years, I have had 2 problems that ...
The Great's user avatar
  • 2,555
0 votes
0 answers
21 views

Can TSNE and other visualisation methods separate multivariate normal blobs?

Consider we have two classes of points. Both of them come from a multivariate normal distribution with an unrestricted covariance matrix. Let's assume, that the densities of those distributions do not ...
Karol Szustakowski's user avatar
0 votes
0 answers
20 views

How to assign a group to new data using k means?

can we assign a group or cluster to a new data(test data) using unsupervised learning techniques? For expamle k means clustering.
Kurati Sakyarshi's user avatar
0 votes
1 answer
43 views

Hadoop, Spark and Cloud

It seems Hadoop, Spark, and different versions of Clouds offer facilities to store and analyze big data. There are some articles comparing Hadoop and Spark (for example, this article). There are also ...
Tara's user avatar
  • 1
0 votes
0 answers
25 views

Anomaly detection on subset of categorical data

Imagine that I'm a health department, and I have a data stream that describes whether someone gets sick after eating food from a restaurant. Each data point in the data stream arrives 24 hours after ...
leontp587's user avatar
  • 101
0 votes
1 answer
42 views

Study data science at university [closed]

Is it a waste to study data science in university, despite the advancement of artificial intelligence and automation of analysis? regards
Amirhossein Qaem's user avatar
0 votes
0 answers
96 views

Orange Data Mining creating a target variable on a table at run time

In Orange Data Mining I have created a table at run time and I need to declare one of the columns as a target. I have seen all widgets, but with no solution. I think to use a Python Script but I don't ...
Mario Capurso's user avatar
0 votes
0 answers
13 views

Gaining insights into commonly co-bought categories of products using association rules (frequent itemset mining)

Let's say I have a small database of supermarket transactions. Each records consists of a tx with several bought products, each having a unique identifier (eg, 1,2,3,4,5 etc.). Running A-priori on ...
user148445's user avatar
2 votes
1 answer
1k views

Can I use LLM to explain codebase?

I am a Data Engineer, and I am currently assigned a task to refactor an outdated code and rectify any bugs present. However, I am unable to comprehend the code written in the existing codebase. ...
Yiffany's user avatar
  • 158
0 votes
0 answers
20 views

GridSearchCV using best paramters

i have got a question, i want to use GridSearch on a MLP and my question is after finding out best parameters do i have to train my model with the data again with the best parameters ?
heyoka955's user avatar
0 votes
0 answers
31 views

GitHub Archived Repositories

I'm trying to build a model that observes patterns of source control usage, from how many files are changed per commit, how many contributors there are, even semantic analysis on the commit messages. ...
Paul Hazen's user avatar
1 vote
1 answer
45 views

Making a netcdf data using xarray

I am very very new to the world of data science as I only started using it in my new job so I would really appreciate help from the community experts (maybe also in simple words :)). I am trying to ...
Shaz's user avatar
  • 125
1 vote
0 answers
47 views

Mining association rules between time series

I have a pandas dataframe that represents a time series. My time series is segmented over the phase type that the robot is performing (i.e. I have a column with the phase type per timestamp and the ...
Andrea's user avatar
  • 45
0 votes
0 answers
22 views

Can support be accounted multiple time for same sequence in sequential pattern mining?

I want to find top-k frequent sequential patterns from a list of sequences. Order of occurrence matters here (subsequence (1,2) is not same as (2,1)). Suppose I have 2 sequences: S1=[1,2,3,1,5,2] S2=[...
Linh H.'s user avatar
2 votes
3 answers
455 views

How to forecast a timeseries with geolocation data?

I have created a dataset with my geolocations from the last three months. The data set contains longitude, latitude, and timestamp, with a frequency of every 5 minutes. Based on this data, I want to ...
sander's user avatar
  • 93
0 votes
0 answers
30 views

Finding a relation between three variables

I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
Scholar7's user avatar
0 votes
1 answer
48 views

Classification of a noisy data

What method can be used to classify data in the following example? There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
Mic's user avatar
  • 1
0 votes
0 answers
8 views

What models/techniques can I use to generalize industry specific datasets?

I have a few dictionaries pertaining to different industries (ie. tech, manufacturing, education, etc.). These dictionaries map phrases and keywords to a sentiment score. I'd like to create a ...
Thetafinity's user avatar
0 votes
0 answers
34 views

Refining AI problem statement - suggestions

I am looking for some guidance. My company is a electronic goods manufacturing company. We work with multiple distributors (around 7 distributors) across specific regions to sell our products. But ...
The Great's user avatar
  • 2,555
0 votes
0 answers
79 views

ı am writing data process pipeline with luigi but ı get error

...
Tunahan Deniz's user avatar
0 votes
0 answers
21 views

Creating song vectors from playlists

Given a large number of playlists, I would like to create song vectors using this data. Furthermore, I would like to measure the performance of the model, so that it can be optimized for some ...
ThePortakal's user avatar
1 vote
1 answer
40 views

How to increase retention?

As you might already know there is a concept of retention. Let's say I have created a game and today hundred people have downloaded my game. Let's say tomorrow 47 out of yesterday's hundred people are ...
Narek's user avatar
  • 121
0 votes
0 answers
7 views

Can Data Mining or traditional EDW also do Process Mining?

Can Data Mining also do Process Mining? Can frameworks and tools used for Data Mining or EDW be used for Process Mining? Summarize the problem I'm investigating Process Mining with a view to create ...
Nimbocrux's user avatar
  • 101
0 votes
0 answers
374 views

AttributeError: 'NoneType' object has no attribute 'longitude'

I'm trying to get the location longitude of a address using geopy. normally this works completely fine. ...
Istiak Ahmed Khan's user avatar
0 votes
0 answers
21 views

Any ideas on deciding a hotel competitive group?

Suppose for each hotel, I need to find a group of hotels which are competitive to this hotel. Competitive hotels mean they probably share the same group of customers, and the customers will make a ...
Xinru Luo's user avatar
1 vote
1 answer
75 views

revenue forecast using regression - what is the input for future?

I have a dataset with quarter wise revenue for past 3 years from Jan 2020 to Dec 2022. I have 4642 customers. Each customer has 1 row of data which includes features based on his purchase frequency, ...
The Great's user avatar
  • 2,555
1 vote
0 answers
22 views

How to do unsupervised clustering on sentences to find intents

I am working on chatbot for students. So, I have chatlogs on conversation between student and tutor, which is on mathematical problems (no labels). First thing I want to find is intents, such as ...
Glinty's user avatar
  • 23
0 votes
0 answers
8 views

User Actions in Browser mining

I'm looking for a way to collect user actions: opening new tabs, switching between tabs, entering text, following links in browser by user. it should log user actions as well as context: what was the ...
Standard Template Construct's user avatar

1
2 3 4 5
24