How can we make our own environments for reinforcement learning? I have a heavy doubt that a game engine is used.. And if a game engine is used, which game engine is used that can be downloaded for free?
Classical reinforcement learning (e.g. vanilla Q-learning) is not normally applied to games. There are some environments to play (and do research) on classical RL benchmarks (e.g. balancing pole), like OpenAI Gym. It can be extended to your own benchmark environments.
Games, nevertheless, are a standard testbed for deep reinforcement learning algorithms. Probably the most popular approach is to use Atari games. I believe they were popularized among the RL practitioners after the landmark article Playing Atari with Deep Reinforcement Learning by DeepMind. You can find multiple implementations of RL environments on top of the Atari emulator in github (e.g. this and this).
Another popular deep RL game-based testing environment is Starcraft. DeepMind open-sourced their Starcraft RL environment.