10
$\begingroup$

I'm in the process of preparing to teach an introductory course on data science using the R programming language. My audience is undergraduate students majoring in business subjects. A typical business undergrad does not have any computer programming experience, but has taken a few classes which use Excel.

Personally, I am very comfortable with R (or other programming languages) because I majored in computer science. However, I have the feeling that many of my students will feel wary of learning a programming language because it may seem difficult to them.

I do have some familiarity with Excel, and it is my belief that while Excel can be useful for simple data science, it is necessary for students to learn a serious programming language for data science (e.g., R or Python). How do I convince myself and the students that Excel is insufficient for a serious business student studying data science, and that it is necessary for them to learn some programming?

Edited in response to comment

Here are some of the topics that I will be covering:

  • Data processing and data cleaning
  • How to manipulate a data table, e.g., select a subset of rows (filter), add new variables (mutate), sort rows by columns
  • SQL joins using the dplyr package
  • How to draw plots (scatter plots, bar plots, histograms etc.) using the ggplot2 package
  • How to estimate and interpret statistical models such as linear regression, logistic regression, classification trees, and k-nearest neighbors

Because I don't know Excel very well, I don't know whether all of these tasks can be done easily in Excel.

| improve this question | |
$\endgroup$
  • $\begingroup$ Without knowing what is on your syllabus, this question cannot be answered. Having said that, you should take a look at Power Pivot/Data Model in Excel. You can easily handle multi-gigabyte datasets with millions of rows in Excel these days, and it's fast. $\endgroup$ – Gaius Jul 6 '17 at 10:33
  • $\begingroup$ @Gaius I added some details of what I want to teach in the course $\endgroup$ – I Like to Code Jul 6 '17 at 14:09
  • $\begingroup$ Your points 1-4 are well supported by the Data Model support.office.com/en-us/article/… - for point 5 I would suggest the free tier of AzureML studio.azureml.net $\endgroup$ – Gaius Jul 6 '17 at 15:45
  • $\begingroup$ AzureML also works with R btw $\endgroup$ – Gaius Jul 6 '17 at 15:47
  • 4
    $\begingroup$ About your last point - take a look at the book "Data Smart" by John Foreman - amazon.com/Data-Smart-Science-Transform-Information/dp/… $\endgroup$ – Gregory Demin Jul 7 '17 at 0:19
8
$\begingroup$

First of all check out this post. It has many reasons why Excel is inferior to other solutions, regarding data science tasks. Excel also can't handle large datasets (hundreds of thousands of records - not to mention anything in the vicinity of Big Data), image and sound data.

Excel is good for simple tasks concerning spreadsheets; it emphasizes more on presentation and ease of use, while having minimal support for actually analysing the data. Unless all you want to do is to calculate simple statistical measures (mean, average, etc) or building a very simple model (e.g. linear regression), Excel is inefficient. That being said, 99% the work a company has to deal with concerning data is simple enough to be manageable through Excel.

However Data Science mainly deals with regression, classification and complex models that excel isn't equipped to handle! If your students want to have a look at data science you need to teach them a tool that will be useful to them (R, python, etc.). These languages also have libraries with tons of built in models to "play with".

Another really huge reason I would go with the latter options is that they are open source. I personally feel that open source software should be preferred from an educational standpoint to proprietary solutions (this is also why I suggest python and R over Matlab)!

| improve this answer | |
$\endgroup$
  • $\begingroup$ I agree with all of the above, but he did say they're business majors. Why not teach R but also make sure to demonstrate an R/Excel plugin? $\endgroup$ – CalZ Jul 6 '17 at 11:46
  • 1
    $\begingroup$ "Excel also can't handle large datasets (hundreds of thousands of records" <-- yes it can, easily. And it can act as a client to serious back-ends such as AzureML and PowerBI. I am not an Excel "fanboy" so much but it bemuses me to see supposedly "data driven" people who don't even know basic tools. $\endgroup$ – Gaius Jul 6 '17 at 15:41
  • 1
    $\begingroup$ What if it is a million rows dataset plus thousands of columns, on the same "basic" machine (16 gb rams, i7 ecc), which solution would open it faster? I'm not trying to denigrate Excel, just an honest curiosity. Too my knowledge I can't even open such dataset in Excel. RStudio reads it with no problem on the same PC. $\endgroup$ – RLave Jul 2 '18 at 13:27
7
$\begingroup$

I just got done with a Masters in Business Analytics and was faced with the same problem you are describing. Luckily I am a technical person and was able to teach myself R and Python, but I was stuck teaching the rest of the class how to use R and Python. The classes I had that used R/Python were handicapped by the lack of technical understanding by the students and so too much time was spent covering how to just open R/Python. The classes that went the other route were underwhelming and not very practical. I wanted to do for a class project something that ended up not being able to be done in Excel because of its limitations but the teacher wouldn't accept any other tools.

It may not be something you can do right away but I would highly recommend that you try and get the department to require a programming course prior to taking your course. Data Science and Business Analytics IMHO should be cross discipline degree paths that require a good bit of Computer Science, but until the programs mature and the university system gets better it might not happen for a while.

| improve this answer | |
$\endgroup$
  • $\begingroup$ You mentioned that you "wanted to do for a class project something that ended up not being able to be done in Excel because of its limitations." What were you trying to do which could not be done in Excel? $\endgroup$ – I Like to Code Jul 7 '17 at 2:34
3
$\begingroup$

I think you need to be teaching them a popular Data Science language like Python or R. Excel is not going to help them in a real job, and isn't practical for data science purposes. I would probably say Python would be most valuable to them in the long run, and with packages like scikit-learn your regressions and classifications can be demonstrated in very few lines of code which they can read and understand more easily. It is not always easy to understand what R is doing by just reading it.

Another word of advice: Don't waste time forcing your students to set up an IDE and download the necessary packages, if you use python create a virtual environment for them with all the necessary packages, and set up an IDE like pycharm(they can get this and most other IDEs under a student/academic license) where then can develop and run their code through UI rather than console which they may find daunting and confusing. If you go down the R route then make sure you have an IDE like RStudio set up for them and make sure all of the includes and package installs are either included in your example code or fully described.

| improve this answer | |
$\endgroup$
  • $\begingroup$ "Excel is not going to help them in a real job" it certainly is if that's what all their colleagues are using. What real jobs in your experience don't use Excel? $\endgroup$ – Gaius Jul 7 '17 at 11:29
  • 3
    $\begingroup$ Any Data Science role working with large amounts of data, mine included. Which DS jobs do you think would use Excel as their primary tool, out of interest? $\endgroup$ – Dan Carter Jul 7 '17 at 14:12
  • $\begingroup$ I see from your profile that you're a student? Oh. These are business students taking one course in DS. In their business jobs they absolutely will use Excel as their primary tool. $\endgroup$ – Gaius Jul 7 '17 at 14:20
  • 1
    $\begingroup$ Sure, you are right they will likely to use Excel in a business type role, however as OP put plainly: they have already taken courses which cover Excel. Couple this with the fact that Excel is not adequate for industry or academic Data Science and it is clear that teaching them 'Excel for Data Science' is not going to help them in a real job, as I said. You cannot teach a man(or woman) to fish, by teaching them to speak French. $\endgroup$ – Dan Carter Jul 7 '17 at 15:22
  • $\begingroup$ So what if they've already taken courses on Excel? Don't treat like dimwits incapable of learning R. We're not talking Haskell or LISP here! $\endgroup$ – Emre Jul 7 '17 at 16:40
2
$\begingroup$

How do I convince myself and the students that Excel is insufficient for a serious business student studying data science

Create in R a huge data.frame (couple mln rows and hundreds of columns), save it as .xlsx.

Show them the time difference in loading it with R, and in Excel on the same machine. Compare basic statistics operations between the two on the same dataset, even plots.

Point no. 2-4 on yout list can be done in Excel too, just A LOT more painfully, show them a couple of example of how much simple (and faster) is filtering with dplyr, compared to basic Excel, again on a huge dataset this would highlight the difference.

Bonus point if you can come up with a dataset that crashes your PC with Excel going.

Also, I'd enphatize the "free-to-use" part of R (or Python). For example, compared to SAS, if you simply want to try one solution (ie some kind of cluster), you load the library, and give it a try, no need to pay more, just for trying.

To me that's the beauty of it, you can try for free whatever you need, and often that's key in DS, imagine if you'd have to pay for each library you install.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Excel and Data Science - sounds really strange to me. Maybe Excel and 'Data Analysis'.

Anyways, I think a good compromise between Excel and R is: KNIME (http://www.knime.org/knime-analytics-platform). It's free on the desktop and much easier to get started. You can import / export to Excel but also use R, Python or Java if the ~ 1.000 nodes miss some functionality that you need. Since the workflows are visually created, it's also much easier to show them to someone who doesn't know any programming languages - which is quite an advantage in some companies.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I think the problem is that you are trying to convince your students that by taking your class, they can do data science similar to the level of modern data science, i.e., fancy stuff like image processing, face recognition. You hear this saying most of the time, "by taking this class, you will..." What you need to teach them is the love for data and the courage to look through a bunch of data, messing around with them to hopefully make some sense out of them. The moment they can do that, you can call them data scientists and you should feel proud of yourself for now having a new generation of data scientists. After that, if they are very serious about data science, they can go on taking other intense courses that deal with math, statistics, and computer science (programming experience like you said). I was in the situation similar to your students. I had no CS background but wanted to break into data science and AI by taking some online classes with fancy promises. I ended up wasting tons of money yet found myself in immense frustration (oh, I need to take this class to know this algorithm, oh they are talking about neural networks now so I have to sign up for the other class, etc.) TL;DR. Tools just account for 1% of the problem you have. With your background, you should have no problem in figuring out the above tasks in Excel in a week.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.