Doing back propagation is a step-wise procedure (you propagate back one step after another), which - if you do the math - comes from the chain rule. So basically, you have to do both the derivative of your costs as well as the derivative of the activation function.
Example
Here is a small example. We start with a small network:
Mathematical setting
Following your notion, the output of the network is $y'=f_B(z_B)$ where $f_B$ is the activation function of node $B$ and $z_B$ is the input of node $B$.
The input $z_B$ is then given by the weighted sum of the outputs from both nodes $A_1$ and $A_2$. Let's call the outputs $o_1,o_2$ and the weights $v_1$ and $v_2$, i.e.
$$z_B = v_1\cdot o_1 + v_2\cdot o_2$$
The rest of the network is similar (e.g. $o_1=f_{A_1}(z_{A_1})$).
Our loss for a single sample $i$ would then be $\mathcal{L}_i = \frac{1}{2}(y-y')^2$. And the costs would be the average loss $\mathcal{C} = \frac{1}{m}\sum_{i=1}^m \mathcal{L_i}$.
Note Formally, I would have to add the sample index $i$ also to $y'$, $z_B$, $o_1$, $o_2$, since they differ for each sample. I omit it here for the sake of a simple notion.
Backpropagation
The goal is to find weights $v_1$ and $v_2$ that minimize the costs.
Step 1: We start - as you heard before - with the derivative of the costs
$$\frac{\partial}{\partial v_1}\mathcal{C}=\frac{1}{m}\sum_{i=1}^m\frac{\partial}{\partial v_1}\mathcal{L}_i$$
Step 2: We compute the derivative of the sample loss. This is the point where the chain rule is used:
$$\frac{\partial}{\partial v_1}\mathcal{L}_i = (y-y')\cdot \frac{\partial}{\partial v_1}y'$$
Step 3: We need to compute derivative the output $y'=f_B(z_B)$ in order to compute Step 2. Again, we use the chain rule, to propagate the derivative out the cost further back:
$$\frac{\partial}{\partial v_1}y' = f_B'(z_B)\cdot \frac{\partial}{\partial v_1}z_B$$
Here, $f_B'$ is the derivative of the activation function $f_B$
Step 4: We need to compute the derivative of $z_B$ in order to compute Step 3. This is simple:
$$\frac{\partial}{\partial v_1}z_B = \frac{\partial}{\partial v_1}(v_1\cdot o_1 + v_2\cdot o_2) = o_1$$
If we now put everything together, we finally get the derivative of the costs. So the derivative of the activation function is needed to achieve the bigger goal of computing the derivative of the costs.
Further steps
You would also want to optimize the lower layers of the network, e.g. $z_{A_1}=w_{11}x_1 + w_{12}x_2$, so you would also compute $\frac{\partial}{\partial w_{11}}\mathcal{C}$, which would start in the same way and differ in step 4 and require more steps to propagate the derivative further back. Luckily, the backpropagation for all weights can be done at the same time, which leads to the backpropagation algorithms, you learn about.
