I will try to be concise and understandable. I really really need help.

Scenario: I have a network with few nodes and links. On each link there are some slots (#1 to #800). I generate traffic requests (come one by one) which wants to go from one node to another and needs certain slots to do so. So, my task is to allot the slots to each upcoming requests and finally achieve a low rejection probability i.e. able to allot slots to as many requeats as possible. The allotted slots are alos freed up depending on when the arrived requests leave the system. I use Poisson process to do but this is not imp here.

What I thought: There have been certain simple benchmarks to do this but I wanted to use Q learning so that in the long run the agent (a centralized controller) takes better decisions as to which slots to assign on the particular link i.e. which slots position (#1 to #800).

What I did: I decided to take the state space as the links (say 10 links in my case) and the action space as the #1 to #800 slots. I use binary notation 1 to say slot is occupied or 0 that is is free.

Problem encountered: But it is long later I realized that my state space is infinitely big. For E.g. For request 1 I give two slots on Link1 & state is 1 1 0 0 0 ....upto 800 zeros. Another request comes (say 3 slots) and say Link1 state is now 1 1 1 1 1 0 0 0...upto 800 zeros. This is when I relaized state space is unimaginable large as departures can also occur leading to freeing up and some 1s becoming 0s and so on.

What I am asking for: So, does anyone have any ideas on how can I still use Q learning in this case. The point being that someone already used deep Q on this. I was thinking I am approaching it in a different and simplified way of just using link state as state space that would enable me to have a small Q table. But it is later I realized that each link state will vary every time and lead to large state space thus putting me back to square one after investing lot of time on this. So, please give any suggestions as I don't want to leave it all together.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.