Usually, the setup of having two conv. layer followed by one pooling layer works well for CIFAR. Since your batch size is relatively high and the learning rate relatively low these are probably not the source for the lack of convergence. For further analysis a plot of your train and validation loss (not accuracy) would be helpful.
Therefore, I suggest the following steps:
1. Plot train and valid loss curves to understand where you're standing in terms of convergence and overfitting/underfitting, e.g. see this article on how to read loss curves.
2. If your current model capacity is insufficient train for more epochs or add another stack of layers with the same structure (features should be increasing throughout the network by for example first having two conv. layers with depth 32 and the next two conv. layers with depth 64)
3. If overfitting becomes a problem add dropout
Moreover, I do not think you need multiple fully connected layers at the end but they might not do harm either. Adam might work well the standard learning rate here but if you have manually optimized yours that is fine too (the learning rate is, in fact, the most influential hyperparameter so it almost always makes sense to optimize it).
In deep learning it usually makes a lot of sense to check what networks have worked on the task you're working on (or similar tasks). For CIFAR you find a simple but well working network here.
