Data Science

Welcome to Data Science Stack Exchange

Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It's built and run by you as part of the Stack Exchange network of Q&A sites. With your help, we're working together to build a library of detailed answers to every question about data science.

We're a little bit different from other sites. Here's how:

Ask questions, get answers, no distractions

This site is all about getting answers. It's not a discussion forum. There's no chit-chat.

Just questions...

...and answers.

up vote

Good answers are voted up and rise to the top.

The best answers show up first so that they are always easy to find.

accept

The person who asked can mark one answer as "accepted".

Accepting doesn't mean it's the best answer, it just means that it worked for the person who asked.

What can be done with highly correlated variables (>.95 and <-.95)

up vote 14 down vote favorite

I hope we can remove the highly correlated variables based on the feature importance may be with PCA etc.

Is there anything we can do with highly correlated variables/

Thanks in advance !

2 Answers

up vote 4 down vote accept

You need to remove them. Redundant features only increase the computation time, increase model complexity (with no benefit) which means making interpretation of model/analysis more sophisticated and if they are many, removing them prunes your vector space by improving the density of information in dimensions of vector space (it helps e.g. in finding nearest neighbors).

up vote 3 down vote

I think merging such correlated features and create a new one, will also be a good idea. In that way we will not lose any information.

For example, sum up the values of different correlated features and take an average of it, will be the very basic option.

Get answers to practical, detailed questions

Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.

Ask about...

  • Specific issues with data science
  • Real problems or questions that you’ve encountered

Not all questions work well in our format. Avoid questions that are primarily opinion-based, or that are likely to generate discussion rather than answers.

Questions that need improvement may be closed until someone fixes them.

Don't ask about...

  • Anything not directly related to data science
  • Questions that are primarily opinion-based
  • Questions with too many possible answers or that would require an extremely long answer

Tags make it easy to find interesting questions

All questions are tagged with their subject areas. Each can have up to 5 tags, since a question might be related to several subjects.

Click any tag to see a list of questions with that tag, or go to the tag list to browse for topics that interest you.

What can be done with highly correlated variables (>.95 and <-.95)

up vote 14 down vote

I hope we can remove the highly correlated variables based on the feature importance may be with PCA etc.

Is there anything we can do with highly correlated variables/

Thanks in advance !

You earn reputation when people vote on your posts

Your reputation score goes up when others vote up your questions, answers and edits.

+10 question voted up
+10 answer voted up
+15 answer is accepted
+2 edit approved

As you earn reputation, you'll unlock new privileges like the ability to vote, comment, and even edit other people's posts.

Reputation Privilege
15 Vote up
50 Leave comments
125 Vote down (costs 1 rep on answers)

At the highest levels, you'll have access to special moderation tools. You'll be able to work alongside our community moderators to keep the site focused and helpful.

500 Vote to close, reopen, or migrate questions
1000 Edit other people's posts
2000 Access to moderation tools
see all privileges

Improve posts by editing or commenting

Our goal is to have the best answers to every question, so if you see questions or answers that can be improved, you can edit them.

Use edits to fix mistakes, improve formatting, or clarify the meaning of a post.

Use comments to ask for more information or clarify a question or answer.

You can always comment on your own questions and answers. Once you earn 50 reputation, you can comment on anybody's post.

Remember: we're all here to learn, so be friendly and helpful!

up vote 9 down vote

You need to remove them. Redundant features only increase the computation time, increase model complexity (with no benefit) which means making interpretation of model/analysis more sophisticated and if they are many, removing them prunes your vector space by improving the density of information in dimensions of vector space (it helps e.g. in finding nearest neighbors).

edit

In the situation of two highly correlated features, actually choosing one does the same job. Question mentioned .95 which means they are practically the same. Plus the fact that merging strategy needs to be chosen. What is ur idea for merging? - Kasra Manshaei Feb 7 at 17:44

add a comment

Unlock badges for special achievements

Badges are special achievements you earn for participating on the site. They come in three levels: bronze, silver, and gold.

In fact, you can earn a badge just for reading this page:

 Informed Read the entire tour page
 Student First question with score of 1 or more
 Editor First edit
 Good Answer Answer score of 25 or more
 Civic Duty Vote 300 or more times
 Famous Question Question with 10,000 views

see all badges

Sign up to get started

Signing up allows you to:

  • Earn reputation when you help others with questions, answers and edits.
  • Select favorite tags to customize your home page.
  • Claim your first badge:  Informed
Looking for more in-depth information on the site? Visit the Help Center

Data Science Stack Exchange is part of the Stack Exchange network

Like this site? Stack Exchange is a network of 174 Q&A sites just like it. Check out the full list of sites.

Stack Exchange