3
$\begingroup$

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define

  1. State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the shipment, AVAILABILITY of capacity etc.

  2. Action Space (price itself, can range from 0 to inf) we need to determine the price itself.

  3. Reward Signal (Rewards can be based on a similar offer to other customers, seasonality, remaining capacity.

I am planning to use Multi-Layer Perceptron for getting inputs from the state space and the outputting the price.

I am not sure how to define a reward function. Please help me in defining the mathematical formula for the reward function based on the price as an action space?

-- UPDATE --

State space that evolves over the time is the remaining capacity (Logistics). Consider at the initial time step is 10,000 kgs capacity and at over a period of time the capacity decreases and when the capacity is full and it cannot take anymore shipments, then the episode completes.

The agent will have to find an optimal price based on the following rewards.

Improve this question
$\endgroup$
16
  • 1
    $\begingroup$ The way to define a reward is to start with your goals and how you measure success of the agent. Could you add those? Also, you don't seem to have a state space that needs reinforcement learning. It looks more like a contextual bandit problem. Could you please identify any state variables that evolve over time, and what the time steps are? If each time step is a new, unrelated customer etc, then this is not really RL, although repeats of same customer might be handled as RL. $\endgroup$
    – Neil Slater
    Mar 15, 2019 at 12:12
  • $\begingroup$ Hi, I have updated the question. Kindly take a look into it. $\endgroup$
    – Karthik Rajkumar
    Mar 15, 2019 at 12:56
  • $\begingroup$ Thanks, that explains well how this maps to RL. However, I am still not sure what the goals are. Will it simply be total price sold at, or profit? Profit seems more likely the true goal, presumably you need to account for the current mix of destinations and route plan if this is a single container which must tour all the destinations in its itinery? $\endgroup$
    – Neil Slater
    Mar 15, 2019 at 13:38
  • $\begingroup$ For example, Similar price offered for the same origin and destination and the type of the shipment is 2.5 $ per kilo, Then based on the similar offer we can increase or decrease so the customer will accept the offer we provide. Lets take Seasonality. Any festival time we can increase the price as there will be more demand Or if capacity decreases and only few kilos left for accommodation we can increase the price. $\endgroup$
    – Karthik Rajkumar
    Mar 15, 2019 at 13:41
  • $\begingroup$ As well as capacity filling being end of episode, is this time limited? If you have an infinite number of customers lined up, then you can just set very high price and wait to make a huge profit. But reality is not like that, and once you accept your first customer, you will have limited opportunities to fill the rest of the capacity or be in breach of contract etc $\endgroup$
    – Neil Slater
    Mar 15, 2019 at 13:42

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.