Gracefully removing observations with outliers in N fields

Question

I have a function.

remove_outliers <- function(x, na.rm = TRUE, ...) {

    #find position of 1st and 3rd quantile not including NA's
    qnt <- quantile(x, probs=c(.25, .75), na.rm = na.rm, ...)

    H <- 1.5 * IQR(x, na.rm = na.rm)

    y <- x
    y[x < (qnt[1] - H)] <- NA
    y[x > (qnt[2] + H)] <- NA
    x<-y

    #get rid of any NA's
    x[!is.na(x)]
}

Given a dataset(numbers) like this:

  x
  5
  9
  2
  99
  3
  4

The functioning is obvious

remove_outliers(numbers)

means I now have this:

  x
  5
  9
  2
  3
  4

However, what if I have an ID that I want to retain, such as:

number_id    numbers
12              5
23              9
34              2
45              99
56              3
67              4

How do I remove the outlier(99) with the remove_outliers function(or another, better suited function), to get this data:

number_id    numbers
12              5
23              9
34              2
56              3
67              4

(note the entire observation with the outlier has been removed)

And how can I scale this solution to handle n more variables?

I can do it very ungracefully by taking out each column separately and building a new data frame with loops, but it's hardly readable and a mess to debug. Is there a more graceful way?

Answer 1

This will achieve what you want. You can remove outliers from any column you wish, just pass that column number as an argument in the function.

id <- c(12,23,34,45,56,67)
num <- c(5,9,2,99,3,4)
prac <- data.frame(id, num)


remove_outliers <- function(x, col) {

  #find position of 1st and 3rd quantile not including NA's
  qnt <- quantile(x[ ,col], probs=c(.25, .75), na.rm = TRUE)

  H <- 1.5 * IQR(x[ ,col])

  x[ ,col] <- ifelse(x[ ,col] < (qnt[1] - H) | x[ ,col] > (qnt[2] + H), NA, x[ ,col])


  #get rid of any NA's
  x <- x[!is.na(x[ ,col]), ]
  x <- assign("dataset", x, envir = .GlobalEnv)
  return(x)
}

remove_outliers(prac, 2)