Are there any heuristics for deciding on where to start with the number of layers for a neural network? I built one with with 7 layers that has 40 input features and this took around 8 hours to train completely, but now I want to add a bunch of polynomial features which will increase the input size of my network to a few thousand features. Since there are so many features will my network benefit from having more layers? If so, what is a good starting point for the number of features?
Here is some information about my problem: suppose I have a set of usage data from a website. So if a user logs into the website one day their usage statistics during that day is calculated. I am using a few statistical tools to summarize the data and I want to make a prediction about whether or not the user will use the website in the next 60 days. My plan is to try and engineer all the possible monomial quadratic features from this summary and feed this information into a DNN. Without the feature engineering I have roughly 60 different features, but with it I will have at least an addition 3600 features.