Skip to main content

Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Feedback on this plan to make a data pipeline

I am picturing something like the following: List internet sources you want to acquire data from. Study the source code structure for each site. Try to build an “extractor” for each individual site ...
Julius Hamilton's user avatar
0 votes
0 answers
20 views

How to visualize a decision tree classifier?

I'm using ID3 algorithm to build a classifier and was wondering if there is any way to visualize the decision tree that the algorithm builds. This is my code for a decision tree in Python: ...
Xx_22's user avatar
  • 11
1 vote
1 answer
27 views

How to match the two datasets on Electoral bonds given by SBI?

Just stumbled across the Electoral bond dataset https://www.kaggle.com/datasets/shaundanielll/electoral-bond-data-state-bank-of-india There are two csvs- One contains Encashment data(Date of ...
dirtynerdy's user avatar
0 votes
0 answers
7 views

comparing class length of multilabelbinarizer

...
Luna Sandler's user avatar
0 votes
0 answers
24 views

How to Detect and Identify German Compound Words in a Text using Python

I would like to know if there is any python libraries or packages available which can help me detect and extract German compound words. I took a look at nltk and spacy packages and they don't seem to ...
Ali Mazhar Sadat's user avatar
0 votes
1 answer
21 views

How to plot PCA loadings vs spectral region

I am trying to plot the PCA loadings vs spectra. So far I can retrieve the PCA coordinates per observation. In a manual, I saw there was a widget called curves to do this. It does exactly what I need ...
Microbiologie13's user avatar
0 votes
0 answers
21 views

How to find optimized combinations of values in input columns that produces the values in output column

I have multiple input columns that produce an output column. I want to find optimized combinations of the values in the input columns that produce the values in the output column. For eg: So, the ...
BitastaB's user avatar
0 votes
0 answers
17 views

How do you actually train a model to make predictions on new inputed data?

I created a system for my thesis, this system can predict whether something will succeed or fail. I use k-fold cross validation to see the performance, then the model that will be used to predict new ...
Agus Krisna Perdana's user avatar
0 votes
0 answers
8 views

SubGroup Discovery on unseen data

The technique of subgroup discovery in data mining is known for using Minimum Description Length (MDL) and Rule-Based Induction to calculate the complexity of a learner and divide a dataset into ...
Emiliano Rosso's user avatar
0 votes
1 answer
24 views

Working with functions [closed]

What are some good sources for learning how to deal with functions in Python and R. I have used them a lot of times and they can be a bit challenging. I like to write out a plan of what I want done ...
Salem Abdulkerim's user avatar
0 votes
0 answers
93 views

Looking for datasets on automobile parts information for machine learning

I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
Anand 's user avatar
0 votes
0 answers
89 views

Seeking datasets for training a Language Model on U.S. mortgage loan processes

I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
Anand 's user avatar
1 vote
0 answers
86 views

Pytrends package in python is not working correctly

I would like to download data for the keyword "Investment" for all countries from google trends. I wrote the following code: ...
entropy's user avatar
  • 11
0 votes
0 answers
23 views

Get accurate alias list of people without information on aliases

I'm working on a project where I have a large dataframe of paintings (from the Art500k dataset), each row corresponds with a painting, containing the author's name in the ...
me9hanics's user avatar
0 votes
0 answers
11 views

Help with creating a panel dataset

I have some medical information of individuals spanning ten years which includes first name, last name, dob, and other variables (two/three that could help match observations i.e. state living, ...
GreenArrow's user avatar
0 votes
0 answers
20 views

How to choose the right typt of ANN architecture for a regression model

So, im working on a project where i am leveraging ai to get accurate price predictions in terms of houses and real estate properties. I would like to use an artificial neural network so now i have to ...
maizen's user avatar
  • 1
0 votes
1 answer
55 views

Clustering words with similar meanings

What methods are there to cluster words/word phrases with similar meanings together from a list of words/word phrases?
ros's user avatar
  • 1
0 votes
0 answers
10 views

What is the best sampling strategy for correlation analysis?

I have a big dataset, and i want to finds subspaces with high correlation among features. I want to take only samples of data. So, what is the best sampling strategy in this context. Thanks
Imen F's user avatar
  • 21
0 votes
0 answers
36 views

Does clustering belong to the domain of data mining or to the domain of machine learning?

Question 1. Does clustering belong to the domain of data mining or to the domain of machine learning? Or to both domains? Question 2. Depending on the answer to Question 1, could you please suggest a ...
Ommo's user avatar
  • 103
0 votes
0 answers
50 views

Unable to create associations using Apriori association rule algorithm for my dataset where the support is 0.01, confidence=0.8 and maxlen=10

Here's the code that I'm using: library(arules) library(arulesViz) library(tidyverse) library(readxl) library(knitr) library(ggplot2) library(lubridate) library(plyr) library(dplyr) tr <- read....
Abhik Bagchi's user avatar
0 votes
1 answer
47 views

Where to find the statement "time series discord is the best overall technique among all techniques" in Chandola et al.(2009)?

Senin et al. (2015) "Time series anomaly discovery with grammar-based compression" claim exactly the following (copied and pasted): in a recent extensive empirical study by Chandola et al., ...
Ommo's user avatar
  • 103
0 votes
0 answers
37 views

Using Bayesian statistics in time series forecasting

I would like to forecast demand count time series of taxi fleets at different locations on the map at different points in time. I.e. multivariate demand Time series forecasting. Given hierarchinal ...
Jose_Peeterson's user avatar
0 votes
2 answers
73 views

Been stuck on a DS problem. Just need to know whether this problem statement is solvable or not

I've been stuck on the following problem for weeks now. To be clear I'm not asking the community to provide a full solution. Just a few ideas or at least confirmation on whether this problem statement ...
Aakash Dusane's user avatar
0 votes
0 answers
76 views

Implicit feedback based recommendation - User - User recommendation

My dataset involves customer_id, item_id and Count of purchases. My datset size is small...we have 15 users and 317 items.. Currently, am trying to build a recommendation system based on user based ...
The Great's user avatar
  • 2,565
0 votes
0 answers
14 views

I want to import my states legislation into a database. I need advice in parsing large HTML, TXT into an importable form. It is Large PDF initialy

My data consists of large pdf files containing state legislation. I can convert the pdf to xml, txt or html, and I can manually go through, identify the tables and write a script to extract only the ...
RBB's user avatar
  • 1
0 votes
2 answers
87 views

How do I separate periodic data from time series data?

I am currently working on a classification task of gym exercises based on accelerometer data. I am trying modularize window extraction so I can train my model based on metrics within a window (which ...
David Chen's user avatar
0 votes
0 answers
24 views

Please give me suggestions to identify scammers from communication data

I have some communication behavior data of carrier subscribers, containing information on calls made, duration and base stations. What do I need to do with this kind of data or what kind of machine ...
WnagoiYy's user avatar
-1 votes
2 answers
199 views

How to generate synthetic dataset with dependent and independent features?

I want to generate a synthetic dataset, in python, composed of 10 features. 5 are dependent among each others and 5 are independent. Generate dependent features. ...
I Sui's user avatar
  • 57
1 vote
1 answer
174 views

How to predict what someone will order?

Suppose Prof. X goes to a road side tea-coffee shop everyday at 5pm just after his office. After reaching there he tosses a coin, and places his order tea or coffee. The shop owner Y has been ...
S. M.'s user avatar
  • 163
0 votes
0 answers
32 views

SMOTE-Tomek links with imbalanced dataset

I have a question: if I want to use SMOTE-Tomek links for my imbalanced dataset, should I now apply the method before or after the split?
Hind's user avatar
  • 1
0 votes
2 answers
42 views

Data Analysis Process

How do you feed data from a data warehouse to Python for ad-hoc analysis? My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
Bunny Eat Data's user avatar
1 vote
2 answers
434 views

what qualifies as a data leakage?

I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
The Great's user avatar
  • 2,565
5 votes
1 answer
249 views

Are imbalanced data problems solvable? [closed]

I am working as a data scientist for the past 2 years where I have worked on problems related to binary classification, revenue prediction etc. In the past two years, I have had 2 problems that ...
The Great's user avatar
  • 2,565
0 votes
0 answers
26 views

Can TSNE and other visualisation methods separate multivariate normal blobs?

Consider we have two classes of points. Both of them come from a multivariate normal distribution with an unrestricted covariance matrix. Let's assume, that the densities of those distributions do not ...
Karol Szustakowski's user avatar
0 votes
0 answers
23 views

How to assign a group to new data using k means?

can we assign a group or cluster to a new data(test data) using unsupervised learning techniques? For expamle k means clustering.
Kurati Sakyarshi's user avatar
0 votes
1 answer
64 views

Hadoop, Spark and Cloud

It seems Hadoop, Spark, and different versions of Clouds offer facilities to store and analyze big data. There are some articles comparing Hadoop and Spark (for example, this article). There are also ...
Tara's user avatar
  • 1
0 votes
1 answer
44 views

Study data science at university [closed]

Is it a waste to study data science in university, despite the advancement of artificial intelligence and automation of analysis? regards
Amirhossein Qaem's user avatar
2 votes
1 answer
2k views

Can I use LLM to explain codebase?

I am a Data Engineer, and I am currently assigned a task to refactor an outdated code and rectify any bugs present. However, I am unable to comprehend the code written in the existing codebase. ...
Yiffany's user avatar
  • 158
0 votes
0 answers
37 views

GitHub Archived Repositories

I'm trying to build a model that observes patterns of source control usage, from how many files are changed per commit, how many contributors there are, even semantic analysis on the commit messages. ...
Paul Hazen's user avatar
1 vote
1 answer
48 views

Making a netcdf data using xarray

I am very very new to the world of data science as I only started using it in my new job so I would really appreciate help from the community experts (maybe also in simple words :)). I am trying to ...
Shaz's user avatar
  • 125
1 vote
0 answers
50 views

Mining association rules between time series

I have a pandas dataframe that represents a time series. My time series is segmented over the phase type that the robot is performing (i.e. I have a column with the phase type per timestamp and the ...
Andrea's user avatar
  • 45
2 votes
3 answers
805 views

How to forecast a timeseries with geolocation data?

I have created a dataset with my geolocations from the last three months. The data set contains longitude, latitude, and timestamp, with a frequency of every 5 minutes. Based on this data, I want to ...
sander's user avatar
  • 93
0 votes
1 answer
67 views

Classification of a noisy data

What method can be used to classify data in the following example? There is a table (hundreds of strings and hundreds of columns). Several columns in this table uniquely allow you to classify each row:...
Mic's user avatar
  • 1
1 vote
1 answer
48 views

How to increase retention?

As you might already know there is a concept of retention. Let's say I have created a game and today hundred people have downloaded my game. Let's say tomorrow 47 out of yesterday's hundred people are ...
Narek's user avatar
  • 121
0 votes
0 answers
467 views

AttributeError: 'NoneType' object has no attribute 'longitude'

I'm trying to get the location longitude of a address using geopy. normally this works completely fine. ...
Istiak Ahmed Khan's user avatar
1 vote
1 answer
77 views

revenue forecast using regression - what is the input for future?

I have a dataset with quarter wise revenue for past 3 years from Jan 2020 to Dec 2022. I have 4642 customers. Each customer has 1 row of data which includes features based on his purchase frequency, ...
The Great's user avatar
  • 2,565
1 vote
0 answers
31 views

How to do unsupervised clustering on sentences to find intents

I am working on chatbot for students. So, I have chatlogs on conversation between student and tutor, which is on mathematical problems (no labels). First thing I want to find is intents, such as ...
Glinty's user avatar
  • 33
1 vote
1 answer
29 views

Empirical indications regarding demanded skills and tasks of data science jobs?

I am wondering if there are is any information about the current (and prospected) shares in skills required for advertised/existing data science jobs. This includes of course also the concrete tasks ...
Radio Controlled's user avatar
1 vote
1 answer
33 views

Data Analytics Documantaions

I am working as a data analyst in a company. Me and my colleagues use different tools and software to analyze the data and make the reports (e.g., Excel, Python, R, Alteryx, SQL, Tableau). Each one ...
N.IT's user avatar
  • 1,995
1 vote
1 answer
48 views

g-mean affected by threshold moving?

Is g-mean (geometric mean of sensitivity and specificity) a better evaluation metric than other metrics (that are derived from the confusion matrix) in situations where decision thresholds are not the ...
pparker's user avatar
  • 392

1
2 3 4 5
24