Yes, there are control flow constructions in neural networks, for instance:
Variable-length sequence classification with recurrent units: in text classification tasks with recurrent units (vanilla RNNs, LSTMs, GRUs), both at training and inference time, you basically iterate over the time dimension of the input batch, passing the next token and the previous steps' state to the recurrent unit. This is a for
loop.
Variable-length sequence generation: in text-generation encoder-decoder networks, at inference time the decoder generates one discrete token at a time until a special "end-of-sequence" token is generated. This is pretty much a while
construction. The decoder can be based on any type of neural net units: LSTMs, CNNs, self-attention (Transformer), etc.
As you can imagine, this kind of control flow construction is typical of variable-size input/outputs. You won't see this kind of construction in most image processing architectures.
Control-flow stuff is more difficult to implement in declarative deep learning frameworks, like Theano or Tensorflow 1.x. In Tensorflow 1.x, there was actually a specific primitive to implemente while loops: tf.while_loop
. With imperative frameworks like Pytorch or Tensorflow 2.x, control-flow constructions are far easier to implement, because you can integrate normal logic between the computations.