New to data Science. Which techniques best to use Large data set in insurance company? [closed]

Question

Dataset Features

Insurance underwriting dataset for 8 years.

• age
• location
• amount insured
• some other features...(medical evidence)

Not all feature will be available to all applicants.

Target Variable

• Decision on whether the applicant can be insured

Question

What techniques can be used and which ones would work best? Outline a high-level overview - I do not think I have to go into too much detail as I do not have any data.

Things I have considered

I am thinking to first slice the data and analyse it in parts and see can I find a pattern.

Regression analysis could be carried out. Possible Logistic regression?

I could take a sample and perform hypothesis tests?

I know that this is an ideal machine learning situation. I don't have any experience in this field and I think I am better to stick with methods I have some knowledge of.

I know this is very ambiguous, but a little nod in the right direction and I would be very appreciative.

Answer 1

I think you need to do couple of tests to see what all variables are important with respect to your Target Variable(client can be insured: Yes/No) - this kind of test is called Predictor Importance test.

As you you have mentioned this is sector is new for you, I would suggest you to take all the variable you think are useful. Convert categorical variables to factors using as.factor() numeric variable to numeric as as.numeric(). The reason for explicit transformation is sometimes algorithms cannot understand like:

test1 <- c(1,2,2,4,4,1,2)
summary(test1)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.500   2.000   2.286   3.000   4.000 
test1 <- as.factor(test1)
summary(test1)
  1 2 4 
  2 3 2 

Now once the data is ready, you can give them as an input to Boruta, by using this you get the predictor importance graph. For better understanding you can go through this Link or else if you want to learn why, how and when you can go through this Link for different tests for different variables.

Boruta does all the above by itself and the outcome is set of important features, With Respect to that you can feed those respective features to your model for getting better results/accuracy.

As your problem is Binary Classification, you can use the following Algorithms:

1. Decision Trees
2. Random Forest
3. Bayesian Network
4. Support Vector Machines(SVM)
5. Neural Network

For all the above algorithms I've attached a link for your reference in which you can find at-least one example for each.

Hope my answer is helpful, mark as answer if you got what you needed.