I am a data-science rookie and I would like to use Python/ R to create a correlation matrix (something like this: http://www.marketcalls.in/python/quick-start-guide-compute-correlation-matrix-using-nsepy-pandas-python.html) and build a machine learning model. However, I have some questions and would really appreciate some guidance.
Question 1: Although the data file is pretty big and have more than 350,000 entries, some columns missed many values (i.e., 60%/ 70% of the values are missing). I am wondering should I abandon those columns/ delete those rows/ any other great recommendations? And what is a good threshold, is it okay to proceed with columns that 20%? 30%? 40% values are missing.
Thank you very much. Greatly appreciated your help!!