R basic workflow

I am absolutely new in R, and my problem is that I do not have any Real World experience in it. I mean, I have learnt a lot but I am always struggling when I get a new task to deal with. Generally speaking I am talking about, how to start to deal with a new task.

Sometimes the dataset is so big (surprisingly :)) that I am not able to get the panorama about it and the usually used functions such as str(), summarise(), head(), tail() maybe sample_n from package dplyr are not enough to fill me in satisfactorily.

Almost every example that I found on the net, were about datasets which were almost perfect. If we need to clean the data at all, we can relatively easily identify the basic problems because the problems are unambiguous and you can realize them when you check the head() or something.

What about the real world data? What if the columns shifted in the middle of your dataset, or there are some rows where the values consists an inappropriate symbol or space or something (salary, price, phone-number etc)?

In summary: - What is your general method to getting familiar with your dataset (lets assume that we are already know what is the meaning of the variables because we have a description about it)? - Do you have a general examining method?

I know that there are no two similar projects, I am really interested in YOUR basic workflow (with some examples or explanations) though.

• I highly recommend you goto Kaggle. Look at what people have been doing. Apr 17, 2018 at 2:22
• "columns shifted in the middle of your dataset" is indeed a significant problem, and one that occurs before serious "data-science" can begin. I often have unit-tests on the data to check certain assumptions, ranging from checking class to verifying ordinality and rank to comparing with expected ranges. But if you have doubts about the legitimacy of the CSV or XLS(x) data format, then the data-validation stage is critical to have any assurance in follow-on summarization/viewing/panoramic exploration. Apr 23, 2018 at 18:16
• ... and for data unit-tests, I literally use the testthat package, as it does a good job of summarizing which and how assumptions/tests fail. The trick is coming up with meaningful and robust "tests". Apr 23, 2018 at 18:17