Firstly, filling a column which has most values as null might not make sense. You'll have to look at the nature of the business problem you face. You can try dropping those columns. You can drop the rows which contain null values if the number of null values for columns in that row fall below a particular threshold (based on your data, of course). Use: dataset.dropna(inplace=True) to achieve this

That being said, there are multiple ways you can fill (impute) null values:

1. You can try filling values based on the mean (or median or mode): dataset.fillna(dataset.mean(), inplace=True). However this approach has it's limitations if your column is non-numeric.

If you want to use an sklearn implementation, try using the Imputer method.

from sklearn.preprocessing import Imputer

impute = Imputer(missing_values='NaN', strategy='mean', axis=0)

impute.fit(dataset)

dataset = impute.transform(dataset)

2. For categorical columns, try using fillna with the most frequent value in a column:

dataset[column].fillna(value=df['column1'].value_counts().index[0], inplace =True)

more information on this method can be found here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.fillna.html

3. Try using a KNN to fill in missing values. There's a package called fancyimpute which will allow you to do this:

from fancyimpute import KNN

dataset = pd.DataFrame(KNN(number_of_neighbors).complete(dataset))

You'll need to choose the number of neighbors appropriately. Also, fancyimpute needs a numpy array as input.

There are many other methods, but these should do the trick.

Firstly, filling a column that has most values as null might not make sense. You'll have to look at the nature of the business problem you face. You can try dropping those columns. You can drop the rows which contain null values if the number of null values for columns in that row falls below a particular threshold (based on your data, of course). Use: dataset.dropna(inplace=True) to achieve this

That being said, there are multiple ways you can fill (impute) null values:

1. You can try filling values based on the mean (or median or mode): dataset.fillna(dataset.mean(), inplace=True). However, this approach has it's limitations if your column is non-numeric.

If you want to use a sklearn implementation, try using the Imputer method.

from sklearn.preprocessing import Imputer

impute = Imputer(missing_values='NaN', strategy='mean', axis=0)

impute.fit(dataset)

dataset = impute.transform(dataset)

2. For categorical columns, try using fillna with the most frequent value in a column:

dataset[column].fillna(value=df['column1'].value_counts().index[0], inplace =True)

more information on this method can be found here: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.fillna.html

3. Try using a KNN to fill in missing values. There's a package called fancyimpute that will allow you to do this:

from fancyimpute import KNN

dataset = pd.DataFrame(KNN(number_of_neighbors).complete(dataset))

You'll need to choose the number of neighbors appropriately. Also, fancyimpute needs a numpy array as input.

There are many other methods, but these should do the trick.