I'm not aware about for which condition I have to give the rewards. Is my reward logic is correct or I have to change it?
The reward structure in reinforcement learning is a little bit flexible, provided it expresses the goal of your agent.
In your case, you want to avoid two bad scenarios in a continuous problem. There is also a good scenario - providing enough water - which overlaps with the negative consequences of not providing water on demand, but may be subtly different.
Your suggestion of granting a negative reward for spilled water, and a negative reward for unavailable water seems quite reasonable. You have a few choices within that that could make differences to the behaviour of an optimal agent. Given your problem description so far, I think these are a free choice for you:
You can decide on a fixed cost for an error e.g. $-1$ reward, or base the cost on the amount of water spilled or missing when required.
You can weight rewards for water spilled differently to water missing when required. This is the sort of thing that might be a business decision, based on different costs for the different kind of errors. If spillage costs are low, demand is high and impact of failed distribution is high, then the optimal policy may involve over-filling the tank and spilling water deliberately.
You can have a positive reward for the amount of water supplied. This is similar to having a negative reward for failed supply, but again might be weighted differently in a business scenario. If you are supplying water to paying clients, then you would care about profits here and have 3 types of reward covering the profits from selling water, the loss from wasted water and clean-up costs, and (the maybe more nebulous, but still estimated) loss from failing to meet customer demand when supply is not available.
You should bear in mind, that combined with the behaviour of the environment, these decisions may change what the optimal policy is. Until you define these values, there is no single optimal policy. As your problem is simple, there may be a lot of overlap between optimal policies, but choices you make in reward functions and how the environment behaves will result in differences perhaps in how willing the agent is to risk over-filling in some cases.
As the problem is continuous, the long-term reward that you want to optimise could become infinite. To avoid this, and make the problem tractable, you need to make one of the following choices:
Use a discount factor, $\gamma$, which can be from $0$ to $1$ (but not actually 1 for a continuous problem)
Formulate your goal as maximising average reward.
In your case I suggest using a highish discount factor, e.g. $\gamma = 0.99$, as most of the RL literature for value-based agents uses discount factors.
Finally, you may have an issue here:
The distribution of removal of water is not fixed. we can remove any amount of water from bottle, i.e. any continuous value between [0, 5]
RL depends on the environment behaving as a Markov Decision Process. Which means the theory works best if there is a fixed distribution here. That could be a flat distribution in $[0,5]$, or it could be drawn from a complex real-world scenario, but the assumption from the theory is that the distribution of changes to state is a function of most recent state and action plus a stochastic element.
When you simulate your problem in order to train the agent, you will have to use an actual distribution. The agent will then optimise against the scenarios it experiences. Depending on what you simulate, it may do poorly in other situations. However, provided you don't take this to extremes, the agent should learn to optimise over a broad range of states, and many reasonable distributions of the removal of water.
