Questions tagged [data-cleaning]

Data cleaning is a preliminary step to statistical analysis in which the data-set is edited to correct errors and to put it into a form suitable for processing by statistical software.

161 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0
votes
1answer
189 views

Comparing data sets with different measurements

I'm currently writing a thesis based on Cyber Crime, however I'm unsure of the proper to compare/analyse my data sets to talk about them in my thesis. One piece of data (https://www.pandasecurity....
0
votes
1answer
291 views

Tool vs Python Script for Transforming Data in Mongo

We have a bunch of Mongo collections (data collected from APIs, web scraping, etc) that we need to transform to a cleaner data structure (standardized schema) on a monthly basis. Are there any good ...
0
votes
1answer
35 views

Data Cleaning Techniques - What to drop?

I have been struggling with this problem for a few weeks but it seems to be out of my league. I have a dataset of product sales over the course of 55 weeks. It contains information on the store ID, ...
0
votes
1answer
43 views

Grouping similar rows to detect duplicates

I have a dataset containing all the real estate ads for sale in the process of publishing a city: ...
-1
votes
1answer
25 views

Increase accuracy in binary classification with ambiguous data

I am fairly new to Datascience and currently working on an assignment that requires me to do a binary classification on a set with about 9 parameters for X. I tried working on it using different ...
-1
votes
2answers
4k views

AttributeError: 'DataFrame' object has no attribute 'c'

project=pd.read_csv("train.csv",sep=',') this dataset train.csv has 200column, named as var_0,var_1 ....... var_199 ...
-1
votes
1answer
161 views
-1
votes
1answer
78 views

Text standardisation for manually entered data

I am working on a project that involves dealing with manually entered text data. I have a dataset of customs records where the customs officers manually enter the name and address of companies ...
-1
votes
1answer
122 views

Do I need equal number of bins for all attributes?

I want to change 8 attributes which are numeric into nominal. I used equal width binning to specify intervals. Does the bin for each attribute need to be equal? For example, when I discretize ...
-1
votes
2answers
72 views

Creating a dataset for benchmarking of timeseries preprocessing capabilities

I have been tasked with comparing the capabilities of different startups offering AI-assisted data preprocessing. Due to legal reasons I cannot offer company data for the benchmarking, not even ...
-2
votes
1answer
220 views

clean the data with sample selection?

I am using Python to do weather forecasting. Here is the original data. Thanks! The input has 180 features and the meanings are: Suppose we are doing forecast for hour k. We will use the historic ...

1 2 3
4