One argument is that we generally don't know, a priori, what will be the best solution to an RL problem. So by tweaking the reward function, we may bias the agent towards what we think is the best approach, while it is actually sub-optimal to solve the original problem.
This is the main issue.
Am I missing anything?
I think you are missing the main point of setting a reward function. It should be a value that is maximised by an agent achieving the goals it has been set. Each time you change the reward function you may be explicitly setting new and different goals.
Changing a reward function should not be compared to feature engineering in supervised learning. Instead a change to the reward function is more similar to changing the objective function (e.g. from cross-entropy to least squares, or perhaps by weighting records differently) or selection metric (e.g. from accuracy to f1 score). Those kinds of changes may be valid, but have different motivations to feature engineering.
Tweaking a reward function is also called reward shaping, and sometimes it can have good results. It does come with the same risks as above, but if done carefully it can improve learning rate. For example if to achieve the main goal A, it is absolutely necessary to achieve B and C as interim steps, then it should be OK to reward achieving B and C - typically the things you have to worry about is whether it is possible for the agent to repeatedly achieve B or C via some loop through states - so you may need to add to the state vector whether it has achieved B or C enough times and only grant reward for first visit to B and C.
Feature engineering does still exist in RL, and is about how you represent state and action spaces. For instance, in a chase scenario where an agent needs to get close to a moving target, you should find that representing the state as polar co-ordinates of the target from the current agent's position is far easier for RL to learn optimal policy than if you represent state as cartesian co-ordinates of agent and target.