It highly depends on your task, your data and your network. Basically, PCA is a linear transformation of the current features. Suppose your data are images or a kind of data that locality is important. If you use PCA you are throwing away those locality information. Consequently, it is clear that people usually do not use them in convolutional networks. For sequential tasks, again it highly depends on your agent whether is online or not. If it is online, you don't have the entire signal from the begining. Even if you have that for offline tasks, by doing such reduction transformations you are again throwing away sequential information, I have to say I've not seen the use of them. I guess there main use is in tasks where your problem can be solve using simple MLPs which you don't keep sequential or local information. In those tasks due to the fact that you can employ PCA which leads to the reduction of highly correlated features, the number of parameters of your training model can be reduced significantly.

It highly depends on your task, your data and your network. Basically, PCA is a linear transformation of the current features. Suppose your data are images or a kind of data that locality is important. If you use PCA you are throwing away those locality information. Consequently, it is clear that people usually do not use them in convolutional networks. For sequential tasks, again it highly depends on your agent whether is online or not. If it is online, you don't have the entire signal from the beginning. Even if you have that for offline tasks, by doing such diminishing transformations you are again throwing away sequential information, I have to say I've not seen the use of them. I guess their main use is in tasks where your problem can be solved using simple MLPs which you don't keep sequential or local information. In those tasks due to the fact that you can employ PCA which leads to the reduction of highly correlated features, the number of parameters of your training model can be reduced significantly.