Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-mining]

An activity that seeks patterns in large, complex data sets. It usually emphasizes algorithmic techniques, but may also involve any set of related skills, applications, or methodologies with that goal.

0
votes
1answer
14 views

What kind of storage media should I use for large data storage [on hold]

In my hands there is a data of about 60k × 1k. This data is stored in the txt file. Because the data is large, I would like to use the hard disk capacity efficiently for what kind of data file or ...
0
votes
2answers
33 views

Loops in R programming,

I want to update this Remaining column in this table below: How do I do this in R programming? I tried while (n.Remaining>0) { n.remaining <- n.total-n.expense} Desired Output:
1
vote
0answers
35 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my LightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
0
votes
0answers
7 views

Model and Type for Medication Impact

I have following 3 datasets from various patients: Diagnosis Dataset: Patient_Key, Diagnosis Code,Diagnosis Date Medication Dataset: Patient_Key, Medication Name, Dose, Fill Date Lab Result Dataset:...
0
votes
0answers
23 views

Filtering unique row values in SQL, Advanced

My data looks like this: Number(String), Number2(String), TransactionType(String), Cost(Integer) For number 1, Cost 10 and -10 cancel out so the remaining cost is 100 For number 2, Cost 50 and -...
0
votes
0answers
12 views

Data Mining in Practice: Analysing Real Estate

I need to analyse a data set of around 30,000 properties listed or sold over 2 years The main aim is to: Minimize risk and boost profit investments in a short-term market and a long-term strategy. ...
0
votes
0answers
27 views

How to find connectivity pattern to make bipartite Graph / clusters? [on hold]

I have to find relationship between two entities. I am able to analyse both independently but unable to find common interest/ relation between both entities. I have done EDA I have made clusters of ...
1
vote
1answer
36 views

What kind of data visualization should I use?

I'm going to program a customized phone keyboard where some letters are larger than others, depending on how often I misstype them. For example, if I often pressed "w" instead of "e", I'd make the "e" ...
-2
votes
0answers
18 views

linear mixed plot [closed]

what is the factor to consider linearlity of plots.
0
votes
0answers
9 views

which machine learning algorithm we can use for comparing more than two groups(numeric data) [closed]

which machine learning algorithm we can use for the comparison of more than two groups of technologies
1
vote
0answers
24 views

What are some good fields to research in data science? [closed]

I am recently working on my thesis. I had started with recommender systems, but figured out that there is nothing more to optimize in this field since researchers have found around 8.5 RMSE for ...
1
vote
1answer
13 views

does index of my data which is of type “Date time index” plays a part in reggression?

I'm new to data science and I'm working on a regression problem. My question is the index of my data which is of type "Date time index" plays a part in regression? I mean is it Okay if i drop the ...
1
vote
0answers
14 views

Mining Association rules from a data warehouse and a transactional database [closed]

I wonder if it is possible to perform market basket analysis to extract the association rules from a data warehouse and a transactional database in the same time to predict the future purchases of a ...
2
votes
3answers
74 views

How to select between models when AUC scores are similar?

I use two machine learning algorithms for binary classification and I get this result : Algo 1 : ...
2
votes
1answer
24 views

Are there any activation functions which on inputting integer data will produce the output as integers?

Idea is to create the model for ethereum mining which deals with only integer data.
0
votes
0answers
22 views

Mixed linear Model

I have a dataset with G1 (Grade of first semester)and G2(Grade of second semester)and G3 the final grade. I need to model it using linear mixed model with repeated measure to study the variation ...
2
votes
1answer
20 views

Machine Learning Models for Non-Imaging Medical Clinical Data

I am a beginner in machine learning. Since machine learning is a wide area, I would like keep my focus on clinical data. When I search for machine learning samples in healthcare, most of the examples ...
0
votes
1answer
28 views

What to do after training a classifier?

In terms of application, what happens after we train a classifier? What can we learn from it? For example, if I trained a classifier to predict the success of a Kickstarter campaign with 80% ...
0
votes
1answer
20 views

Using iGraph to build a Distribution Model

I would like to analyze the distribution of the Customers from a Shop, if the Shop is closed or terminated. Consider the following sample data-set; ...
0
votes
1answer
8 views

Is there any data set containing list of windows software and the category each one of them belongs to?

I wanted a dataset which lists many windows softwares according to their categories for a project. eg: anaconda: software development, roblox: gaming, VLC: video player etc. I searched but could not ...
1
vote
0answers
15 views

How to obtain causal inference using double machine learning?

How can Double machine learning be used for causal inference in treatment effects? What is the intuitive working of Double machine learning?
0
votes
1answer
25 views

How to convert binary classifier to multiclass classifier?

I am a biggener student in Machine learning, and I want to ask if is it possible to convert a binary classifier label (y) by applying some condition on column1 to get a third situation. I.e. ...
0
votes
0answers
10 views

Advice on advanced analysis for end term Project

I have an end of term project for a class in Applied Data Analytics. I am a software engineer of 18 years and I have learned a lot so far using R with little problems. This question is less ...
0
votes
0answers
11 views

How panel data analysis can be used and what are the uses of panel data analysis/regression?

I have a dataset with Cross-section and Time-Series Data. After doing some reading I came accross an area named as "Panel data analysis/regression". However, I am still not clear how and why we use "...
0
votes
0answers
9 views

How to impute opt out data

Consider the following problem: You have been given some survey data on job satisfaction and have to create a model for the employees with the highest risk of leaving the company. The questions have ...
1
vote
0answers
17 views

How to prepare data for time series RNN

Our goal is to predict different mixtures of gases in case of an air pollution or in space shuttles.We have measurements that are taken from medical devices and we are taking measurements from ...
0
votes
0answers
19 views

Create a machine learning model classification using my data

I'm working in gdpr project and i wanna to detect personals informations, the idea is to create a mechine learning able to understand the name of the column and classifiy in category for exemple : ...
2
votes
0answers
27 views

Using datasets to predict the results for other devices

I have a datasets that contain results from a series of physical tests. It has about a dozen features and the outcome of each test is distinguished by 3 different classes. The dataset includes ...
1
vote
1answer
17 views

Is dimension reduction helpful to select features for a classification problem?

Let's say I have a data set but I don't know what features are relevant to solve a classification/regression problem. In this case, is it worth/good to use a dimension reduction algorithm and then ...
1
vote
1answer
42 views

K-Means visualisation problem 8 numerical feutres [closed]

Hello im looking for exemple with python for K-Means clustering when i have data set with more than 6 feutres. thanks
2
votes
2answers
51 views

MAE,MSE and MAPE aren't comparable?

I'm a newbie in data science. I'm working on a regression problem. I'm getting 2.5 MAPE. 400 MAE 437000 MSE. As my MAPE is quite low but why I'm getting high MSE and MAE? This is the link to my data ...
1
vote
0answers
11 views

Visulazing a specific value of a column with the corresponding values

I have recently started using data studio to make visualizations before diving into analysis and data prep! I have a column named NAME OF STORE that good different names and other columns like Date, ...
1
vote
0answers
16 views

is there any package in python similar to tidyrin R? Except Pandas [closed]

is there any package in python similar to tidyr in R? Except Pandas
1
vote
0answers
65 views

R NbClust for Python?

Is somebody know a Python equivalent to R NbClust? I'm searching for a way to determine the optimal clusters number for many 'similar datasets' and NbClust sounds good but it's for R and i'm a Python ...
1
vote
0answers
4 views

Building a minimal encoding of nominal labels from numerical features

I have a set of ~500 objects, only a single instance of each kind, and each has ~500 numerical features ... I need to find the minimal encoding which allows to identify objects. I am interested in ...
2
votes
0answers
15 views

Sub-sampling so that sample statistics match population statistics

I want to investigate the impact of various testing strategies on a product. Let's say chairs. I start with 500 random chairs that I've picked up from garage/yard sales. They come in all shapes and ...
1
vote
0answers
25 views

preparing time series data for building a rnn

I am preparing time series data for to build an RNN model (LSTM). The data is collected from sensors installed in a mechanical plant. Consider I have data for input and output temperature of a ...
3
votes
0answers
32 views

How to filter this signal to get a heartbeat signal?

I am giving my first steps in data analysis, gathering/cleaning. To learn, I am trying to create a simple code that can detect heartbeats from color variations from the image coming from the camera ...
1
vote
0answers
5 views

How to a object type data which contains a string of unique id for feeding in my model?

I am having a dataset in which data type of unique id of a user is in object form. I need to convert it into Int for feeding this data into my model. here is first rows of my dataset. ...
1
vote
0answers
15 views

Assessing performance of an agent based on commission rate, market share and revenue

I have a set of data for agents selling properties (apartments) for a company in different states. The company would like to assess the performance of the different agents given the following: Number ...
3
votes
1answer
102 views

How can i find trend time for my articles?

our article is time-based, that means is my article search more in a specific time. as you can see in under chart this article search more in specific period time. if my dataset looks like this(it ...
1
vote
1answer
38 views

Document parsing modeling and approach?

I'm relatively new to data science / machine learning (yes, I know) and am experimenting with text analysis. I only want a relatively naive approach and am looking to know whether my approach is valid ...
1
vote
1answer
15 views

Display a subset of classes in axes in ggplot

I have the following plot, is there any way in ggplot to display just the numbers 1 to 10 instead of all of them? Numbers from 10 and after are not so important, but I need to display the ones before. ...
2
votes
0answers
17 views

What is the interest to implement a statistics model in a groupby object instead of use the whole groups in a sample?

I saw this topic and also this one in dask documentation. I wonder what is the real interest to make a model for each group instead of considering a whole sample where all groups would appear? As far ...
2
votes
2answers
94 views

Is there any similarity function to compare two strings and give them a score like scipy cosine similarity for comparing arrays?

I want to compare strings and give them score based on how similar the content is in them just like comparing two arrays in scipy cosine similarity. For example : string one : 'Pair of women's ...
1
vote
1answer
12 views

How can I create a new column of binary values from my TfidfVectorizer sparse matrix?

I currently have a sparse matrix object of TfidfVectorizer which is of 1000 length. Right now it is displayed like this : ...
1
vote
1answer
84 views

Weighted mean with summarise_at dplyr

I strictly need to use the summarise_at to compute a weighted mean, with weights based on the values of another column ...
1
vote
1answer
12 views

How can I extract the top words from a string in my dataframe column?

I have a column in my dataframe which is in a string providing description of a product. For example : This is a shirt. It is blue in color. The sizes available are large, small. The shirt is tight ...
1
vote
1answer
15 views

How can I see a long string in my dataframe?

I have a column in my dataframe in which there are sentences which are too long. I want to see them as a whole but every time I perform even a simple iloc operation i get output like 'i am going to...'...
1
vote
2answers
21 views

Time of trainig vs time of prediction, which one is used during classification algorithms comparison?

I need to use many algorithms for making a binary classification, such as Logistic regression, SVM, XGBoost, CatBoost, ... I get an interesting improvement but All of those algorithms (except LR) take ...