13
You are right. If someone is using regularization correctly and doing hyperparameter tuning to avoid overfitting, then it should not be a problem theoretically (ie multi-collinearity will not reduce model performance)
However it may matter in a number of practical circumstances. Here are two examples:
You want to limit the amount of data you need to store ...
9
There is definitely a lot of work to do on the NLP and knowledgebase side of things before you can realise your agent. However, as the question suggests, we can ignore those details and focus on: Can reinforcement learning (RL) be used to train a "deceptive" agent?
The short answer is yes, this is entirely possible. In principle this is ...
3
What is a great deception? It could be defined as a believable set of information aiming to a final deceitful objective.
Just like any RL model, you can maximize a score thanks to small rewards leading to bad directions, and a great reward if the final reward is reached (ex: great loss of money).
As a consequence, you have to make sure that steps could be ...
1
What type of information would you see as contact information? If it's just phone number and email address I would probably use a simple rule based classifier using (regex) pattern matching, which will probably already get you quite far. In addition you could use specific keywords that are often related to contacting someone such as call, send, mail, etc., ...
1
Few points to not here:
Multicollinearity effects linear model much more as compared to Random forest as it is picking up different set of features (read sampling with replacement) for every model and every model/tree see different data points. Feature importance may be impacted a little by multicollinearity
Multicollinearity does not impact model ...
1
The error tells you, you are trying to use the predict method on the model variable, but model is a string instead of a tensorflow/keras model which does not have this method. You therefore need to use load_model and pass it the location of the model file, which should return you a tensorflow model using which you can then use predict.
1
A few comments:
I think that you should first make a precise and realistic list of your goals, especially if you're going to learn on your own. Why? Because data science is a vast domain, nobody knows everything, not even the top contributors on CrossValidated or here on DSSE. Additionally most of them have a degree related to data science and years of ...
1
If the model is a soft classifier (i.e. it predicts a probability before converting it to a class), then the simple option is to use the underlying probability (e.g. with a predict_proba function).
Another option is to directly train a regression model: in the training data, any instance with class "match" is represented as 1 and any instance with ...
1
The feature selection step is there to guard against model overfitting. The feature selection step may decide that all the variables in the dataset are relevant, or it may decide to remove some. If no feature selection step is performed then no variables are removed and the resulting model may be well-fitted but may be (and likely is) overfitted.
The main ...
1
As far as I understand your two options, option 2 is certainly closer to the correct answer, because your first option appears equivalent to training two independent models. Note that in general incremental learning is not limited to the case of adding new classes to the training data though.
Incremental learning means that the training can be extended:
In ...
1
I found Superhuman AI for multiplayer poker a good paper on card games. Even though it is talking about a 6-player game, there is still much to use here. E.g. they reduce the huge number of betting moves to round numbers by action abstraction. (But note that this is mainly for training, and they let it be flexible when in actual play.)
Also this bit sounds ...
1
There have been success modeling card games with reinforcement learning (RL) using Deep Q-learning Network (DQN) with experience replay. Experience replay buffer is a large collection of tuples: (state (s), action (a), reward (r), next state (s′). An RL agent can learn which combinations lead to the highest reward. The representations are stored in a deep ...
Only top voted, non community-wiki answers of a minimum length are eligible