What does performance is not up to the mark mean?
To optimize performance you first need to understand it.
A typical workflow is this:
1. Define a baseline performanc
E.g. predict values with a constant (average, mode) or pick randomly from a distribution and measure the accuracy of that
2. Compare your 1st draft model (in your case your RandomForest) to that baseline
If your 1st draft model isn't better than the baseline, especially a random baseline, you have some errors in your code, data, etc. Try to find and eliminate those first.
If it is better but not by much, you have top optimize the model (see next step). If it is better by much you are either a) done or b) not satisfied with the absolute performance in which case you have to optimize.
3. Optimize your model parameters
Now you start grid searching your model parameters to get every last bit of performance out of it to make sure the problem is with the model and not the parameters.
4. Beauty contest
If your performance is still subpar now you can try other algorithms but do not expect wonders from this step. If you already tried RandomForest maybe go for:
- Boosted models like XGBoost
- Naive Gauss models
Fit them on the same data, do parameter optimization and compare results.
5. Back to the data
Most likely just picking another model did not help if the original performance was too far away from acceptable. Then you have to go back to the data, collect more, feature engineer, etc.