I am a newbie in Data Science, so please don't blame me for stupid questions.
Here is my problem. I've got a dataset (no empty cells, only numerical values) consisted of 15 columns (1st - user id, last one - response, 2-14 - different features such as eventtype, country, browser etc., actually this dataset describes users actions on some webpage). Response value could be 1 or 0, so it's definitely a classification problem.
In order to have some practice I would like to try main algorithms on this dataset (kNN, Random Forest, SVC). The problem is that in this dataset I've got around 5000 unique id's and the total number of lines is around 17000, so different users have several lines related to their actions on the webpage.
In this case, algorithms listed above gave too good predictions, but I suppose it's not correct to use them on such data, because they make predictions for the object based on this object's historic data. Which options do I have to improve the dataset?
I think I could try to leave only unique id's in dataset, but it will reduce the dataset size from 17k lines to 5k lines, which, I suppose, will have a negative effect on predictions. Which other strategis I can use?