When I looked up different sources that talk about dropping the columns of a dataframe that contains missing values, I got answers that are starkly different from one another.
Some sources say, columns with missing values should be dropped when the percentage of missing values is more than 5-10%, other sources say the threshold is 25%, 50%, 80-85%, etc.
It is also said that null value columns should be only dropped when the number of records is in millions.
In general, under what circumstances should a column with missing values be dropped, with regards to the quantity of missing values & size of dataset? Is there a clear cut answer to my query, or should the problem purely be solved based on intuition & understanding of the dataset?
NOTE: I only need clarity on the dropping of columns with null values, not imputation of these missing values. Please answer accordingly.