# data science python data cleaning

I am preparing a dataset for a model, but somehow the code just doesn't run well.

The major error is:

File "/Users/liangjulia/Desktop/UW DS Certificate Learning Material/untitled6.py", line 61
^
SyntaxError: can't assign to literal


### Code:

# import statement
import numpy as np
import pandas as pd
import csv
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder
# loading dataset, it is a combination of categorical and numerical data
hp.columns = ['age','workclass','income','education','education-num','marital-status','occupation','relationship','race','sex','capital-gain','capital-loss','hours-per-week','native-country','salary-range']
# dataset basics
hp.shape
hp.dtypes

# account for all value '?'
hp.replace('?','na')
hp.isnull().sum()

#Remove onsolete data point in income
hp('income').dropna()

# replace all aberrant values
hp.replace('nan', 0)
hp.replace('NULL', 0)

# change data type of certain data point to numerical
number = LabelEncoder()
hp['income'] = number.fit_transform(hp['income'.astype('str')])
hp['capital-gain'] = number.fit_transform(hp['capital-gain'.astype('str')])
hp['capital-loss'] = number.fit_transform(hp['capital-loss'.astype('str')])

# Choose the datapoint 'income' to perform the data cleaning and remove outliers
LimitHi=np.mean('income') + 2*np.std('income')
LimitLo=np.mean('income') + 2*np.std('income')
BadIncome = ('income' > LimitHi) & ('income' < LimitLo)

# Replace outliars

# normalize the Income Column using numpy
minmaxscaled =('income' - min('income'))/(max('income') - min('income'))

# bin age data into several ranges
hp['bin'] = pd.cut(hp['age'], [15,30,45,60,75,90])

# construct new categorical data point with existing data point
hp['EvalonInvestment'] = 'zzz'
hp.loc[(hp['capital-gain'] >= 50000), 'loc2'] = 'investmentking'
hp.loc[(hp['capital-gain'] > 10000) & (hp['capital-gain'] < 50000), 'loc2'] = 'good-investment'
hp.loc[(hp['capital-gain'] > 0) & (hp['capital-gain'] <= 10000), 'loc2'] = 'ok-investment'

print(hp)


It should be hp['income2'] because you can't assign a mutable object to an immutable object such as a string

What line 61 should read is

income2 = pd.to_numeric(hp['income'], errors = 'coerce')


Let's break this down.

You want to assign a new variable income2 to the output of a method in the pd class, to_numeric.

to_number takes two arguments: arg : list, tuple, 1-d array, or Series, a choice on how to treat errors, and an optional downcast operator. You find this out by

help(pd.to_numeric)


Line 61 originally provided the arg Adult.Income, which could be a list, tuple, 1-d array, or Series, but it hasn't been defined, so it can't be a valid arg.

Try

type(hp['income'])

and you get

pandas.core.series.Series


which is a valid argument.

This line is not the only problem in your code. I suggest that you use iPython so that you can see all of the other errors. Or just use the command line and copy in a few lines at a time and you'll find new error messages to resolve.

Errors in code are like cockroaches -- they seldom travel alone.

Good luck!