# Tag Info

Accepted

### Is Python a viable language to do statistical analysis in?

Python is more "general purpose" while R has a clear(er) focus on statistics. However, most (if not all) things you can do in R can be done in Python as well. The difference is that you need ...
• 7,536

### Is Python a viable language to do statistical analysis in?

Python being more widely used is an important consideration. This will especially become important when applying for a job. Also Python has as many if not more key statistical and ML/AI tools as R, ...
• 1,969

### Math PhD (Nonlinear Programming) switching to Data Science?

Data science jobs cover a wide range of different activities so any answer is likely to be subjective. I'm in academia so my knowledge of the job market is limited, but from what I can see: The ...
• 25.6k
Accepted

### Plot Two Categorical Variables

Well, there are a few ways to do the job. Here are some I thought of: Scatterplots with noise: Normally, if you try to use a scatter plot to plot two categorical features, you would just get a few ...
• 308
Accepted

### Math PhD (Nonlinear Programming) switching to Data Science?

I don't disagree with the other answers, but here's a different perspective you should bear in mind. Also, I can offer answers to your specific questions as someone who left academia (applied math/CS) ...
• 276

### Additive vs Multiplicative model in Time Series Data

Calculate one day returns. Plot histogram of daily returns. Calculate $log(\frac{price_{i+1}}{price_i})$. Plot histogram of above logarithm. If second plot is more likely to be normally distributed ...

### Additive vs Multiplicative model in Time Series Data

I want to know which model between additive and multiplicative best suits the above data. It is hard to tell just by looking at it. A multiplicative decomposition roughly corresponds to an additive ...
• 388
Accepted

### How to interpret standard deviation calculated by excel

You should interpret it as 12.5%. The percentage values that you are seeing in your spreadsheet (93.33%, ...) are only formats, internally Excel is dealing with ...
• 3,009

### Is Python a viable language to do statistical analysis in?

I'd like to add two points to the existing answers: There is excellent interaction between R and python, with various possibilities for either direction. To me, it's not that much of a decision ...
• 1,010

### Is Python a viable language to do statistical analysis in?

One thing that can be a gotcha coming from R to Python is that the Python stats ecosystem tends to be more machine learning-ey oriented rather than inferential stats-ey oriented. This can create some ...
• 181
Accepted

### What advantages does Data Visualization have in EDA?

First, visualization is just an easy and intuitive way to understand underlying patterns in your data. Everything that you can achieve through this, can also be achieved through painstakingly printing ...
• 636

### What advantages does Data Visualization have in EDA?

The simplest example is Anscombe's Quartet . These four data sets are quite different, which doesn't appear just by looking at the summary stats.
Accepted

### Is it Possible to plot Scatter Plot + Histogram + Correlation Values in a single plot (in python)?

Corrmorant is based on ggplot, but it seems that there is no equivalent in Python. However, you can redo it thanks to this code: ...
• 4,794

I suggest you take a look at the TidyTuesday repo, where every week they post a raw dataset, a chart or article related to that dataset, and ask you to explore the data. The repo also contains other ...
• 27.1k

### Solve tough clustering problem with overlapping clusters

The Problem is that many clustering algorithms focus on distances (between points, clusters and so on). Especially at the connection between the two desired clusters, distances between points are ...
• 1,574

### Pandas Profiling Not Working

Something to note is that the package name will soon change to ydata-profiling, so we should use the new name. This is the announcement on their Pypi site: ⚠️ ...
• 27.1k
Accepted

### ValueError: pos_label=1 is not a valid label: array(['N', 'Y'], dtype='<U1')

pos_label is an argument of scikit-learn's precision_score (docs); its purpose is, well, to indicate which label is the positive ...
• 2,018

### Math PhD (Nonlinear Programming) switching to Data Science?

Erwan nailed it (+1). But I think my addition is a little too long for a comment. You seem to be well ahead of where I was when I landed my DS job. I was in pure math, a couple of postdocs in, and ...
• 12k

### Math PhD (Nonlinear Programming) switching to Data Science?

Just Don't take this advice blindly: The subjects you have mentioned in mathematics are core to solving problems using machine learning/Deep learning, programming is a tool to implement all this ...
• 142
Accepted

### How to view entire 5788 rows and 7 columns.?

pandas has a max rows setting - https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html Though perhaps looking at a 5,000+ row csv in an editor, or a spreadsheet or some IDEs have a csv ...
• 994

### Is Python a viable language to do statistical analysis in?

I eventually do plan on moving more towards ML One aspect that I would like to add based on what I observed. Things are moving with more focus towards Deep Learning e.g. Neural Networks and in this ...
• 5,644
Accepted

### Compare two tends with big difference in absolute value

I think it would be better to use a standard scaler that removes the mean and divides by the standard deviation. See here for more info and an implementation using ...
• 1,415

### Labelling a Time series dataset

Some feedback/tips/tricks/opinions here: Problem setup Including requirement analysis. Gotta decide how the system/solution should work, how to know ho how well we are doing, and then how to get there....
• 1,527

### Data cleaning and data transformation before EDA?

Although not very helpful, the answer is probably "it depends". I like to do data cleaning and some EDA together since EDA can highlight appropriate treatments to clean the data - e.g. influencing ...
• 1,625
Accepted

### Are Hadoop and Python SciPy used for the same?

I think you're quite confused. Hadoop is a collection of software that contains a a distributed file system called HDFS. Essentially HDFS is a way to store data cross a cluster. You can access ...
• 2,430
Accepted

### Tools to explore various datasets

There is a very cool active Python package called pandas-profiling, is exactly what you want. With a simple pandas_profiling.ProfileReport(df) it returns a lot of ...
• 4,279
Accepted

### How far can one go with excel?

TL;DR If you have unlimited time and use a 64-bit version of Excel, you can get as far with Excel as any other data analysis tool. Time I mention time as my first factor, because Excel only has ...
• 15k

### Metrics/Methods for deciding duration of video retention for on-demand websites

Clustering can be an ideal choice here. From the question, seems the data will most probably be in continuous format. Essentially, clustering is a method of finding groups of similar objects. The ...
• 411