1
$\begingroup$

I am researching on pay-scales, and wish to receive advise to treat data of salaries.

Objective

My interest is to approximate the salary corresponding to different hierarchical levels in an organisation.

Methodology

How to bin

I thought about using quantiles : knowing that there are, say, 10 levels in an organisation (e.g. President, Director, ..., Worker) I would like to estimate the average pay for the corresponding level.

I thought to use quantiles; am looking at the doc of pandas :

Which choice would it be more appropriate ?

Describing observations

My distribution seems to be linear to a certain point, and then it seems to follow an exponential curve:

enter image description here

Could you advice best approach to approximate the quantiles ?

I am in doubt if calculating quantiles against the whole distribution, or if infer two distributions to best reflect observations.

I thought about extracting one distribution as a collection of data points lower than x standard deviations, and a collection of points greater than x standard deviations.

Respectively, I would have these distributions then:

enter image description here

(describing lower payscales, which seems to me linear)

and

enter image description here

(describing higher payscales, which seems to me exponential)

Advices for the analysis are much appreciated.

Considerations about the sample

Keep in mind each values are reported differently in their frequency (for lower payscales, there are more reports, which it makes sense because there is more turnover).

Should I keep into account to estimate an error ?

Improve this question
$\endgroup$
5
  • $\begingroup$ Your plots look very strange for salaries: are they supposed to be the distribution of the salaries, i.e. X axis is the salary value and Y is the frequency of this salary? Why are they not plotted as histograms? That would be more standard, I suspect there's a mistake in the plots: it depends on the population but i's very unusual that there would be very few people on the low end of the range and many people on the high end, it's almost always the opposite when looking at revenue. $\endgroup$
    – Erwan
    Jan 19, 2021 at 22:33
  • $\begingroup$ Actually I realize that these are definitely not distributions of salaries, unless some people are paid 0 as salary. $\endgroup$
    – Erwan
    Jan 19, 2021 at 22:43
  • $\begingroup$ @Erwan thank you your comments make sense, I have not added details :) Data points are reported salaries for each company role (actually, it is an average value reported ). The list of salaries is sorted: the Y axis reports the monetary value. The X axis, simply the id corresponding to each role. $\endgroup$
    – user305883
    Jan 20, 2021 at 19:37
  • $\begingroup$ So what I am trying to do, is to group different roles to approximate a hierarchical level of an organisation. Each data point is an average of reported salaries; the number of reported salaries for each role may vary (as well as the deviation from the average of each role). Hope this clarification can be of help to discuss how to bin the distribution (qcut or cut): I see your answer, would you like to comment ? Im still in doubt which may better approximate the hierarchical level in an organisation. Maybe cut would be more appropriate ? $\endgroup$
    – user305883
    Jan 20, 2021 at 19:41
  • $\begingroup$ Ok I see, ll try to add a few ideas in my answer to at least get you started. $\endgroup$
    – Erwan
    Jan 20, 2021 at 23:04

1 Answer 1

Reset to default
1
$\begingroup$

I'm going to just answer about the difference between the two functions qcut and cut because it's a very important difference:

  • the first qcut is indeed about quantiles, which means that it's about dividing the data into bins, each containing an equal number of points. For instance if you use deciles it means that there is the same number of people in the first decile as in the last (or any other), so the useful information is the range of values for each bin. For example if the first decile ranges from 0 to 500, then 10% of the people are paid less than 500.
  • the second cut is not about quantiles, it simply creates bins which all cover an interval of values of equal length. For instance the first interval would be 0-500, the second 500-1000, etc. In this case the number of points in every bin is usually different, and that's the useful information. Typically this is how a histogram is built: equal intervals but possibly different frequency (number of data points) in every bin.

[edit following OP's comments]

Data points are reported salaries for each company role (actually, it is an average value reported )

Ok so the data that you have contains a distribution of the salaries for every role.

The X axis, simply the id corresponding to each role

It's not a very good idea to transform an ordinal/categorical variable into a continuous one, in a case like this you could instead plot a bar for every role (with the name of the role on the X axis). Even more precise, you could plot a boxplot for every role, keeping the order of the roles by mean salary. This way it would be possible to visualize not only the mean (usually replaced by median in a boxplot) but how much variations there are for each role.

Another visualization idea would be to plot the full distribution of salaries with a difference color for every role. In this case X would usually be the salary, Y would be how many people are in this interval (bin) of salary, and stacked bars with different colors show the proportion by role. This would show not only the link between salary and role but also how many people are in each category.

how to bin the distribution (qcut or cut)

I suspect that this is the wrong question, because apparently you're not very familiar with these concepts and how to represent them. So I would suggest that you first play with the visualization ideas I mentioned: normally these don't require you to do any binning yourself, the library should take care of it (see examples in Python).

Knowing whether you need cut or qcut depends on what you want to do with the result. As I said above, quantiles are useful for solving questions such as "80% of the employees have a salary lower than X", whereas cut directly represents "X% of the employees have a salary between 100 and 150". Ultimately the two represent the same information, just under different perspectives. If the difference is still confusing, I'd suggest you start with regular bins with cut because quantiles are slightly more complex to interpret.

Improve this answer
$\endgroup$
15
  • $\begingroup$ thank you for elaborating your question and hints on visualisations. I don't have information on the number of employes for each role. My question though, is not about how to visualise information, but: how to estimate an average value for each organisational level ? Let's assume that a company has 10 levels; at higher levels corresponds higher salaries. Assume the sample of salaries offers at least a data point for each level: how to split the distribution to find the "turning-points" ? I want to estimate the average salary for each level. $\endgroup$
    – user305883
    Jan 21, 2021 at 13:59
  • $\begingroup$ Hope my question is clearer now. Please let me know if need to edited. I thought about splitting the curve in quantiles to group roles having similar salaries. Maybe another approach would be more appropriate ? Like fitting a curve and observe how the derivative may change, for 10 intervals ? $\endgroup$
    – user305883
    Jan 21, 2021 at 14:06
  • $\begingroup$ @user305883 I'm sorry but I'm even more confused now: your graphs show continuous values across the X axis, so these are not by role, right? Also if you have a sample for every role, I don't see what is the problem with obtaining the mean for every role? $\endgroup$
    – Erwan
    Jan 21, 2021 at 15:23
  • 1
    $\begingroup$ @user305883 so you don't know which role corresponds to which level? you don't even have a salary range for a level? And you don't have the real proportion by role either? In this case your only option is clustering (grouping) one-dimensional data, but it won't work very well since there are no clear separation between the clusters. It's going to be a guessing game if you don't have any relevant indication in the data. If you had the real distribution of points you could try to model a mixture of Gaussian distributions. You can use either intervals (cut) or quantiles (qcut), but the result... $\endgroup$
    – Erwan
    Jan 22, 2021 at 11:16
  • 1
    $\begingroup$ Let us continue this discussion in chat. $\endgroup$
    – Erwan
    Jan 22, 2021 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.