1
$\begingroup$

I have process data in time series data(0min, 1min, ... 999min). I don't know what does the variables mean. They are just written in X1, X2, ... X52. Each row means the data at the time. At certain point, process becomes abnormal. Then the data after the point are all abnormal data. If normal data class value is 0 and 1 for abnormal data, the label would be like below. 0 0 0 . . . 1 1 1 1 So I want to know when the value is changed to 1. I will use change point detection (with python 'ruptures' library). In this situation, should I standardize my data?

I wonder (1) whether standardization is needed or not, and (2) if the standardization downgrades performance of the model, why is it? (3) If the standardization is not needed, is there any advantage for log transformation of data? As far as I know, log transformation is conducted to make the distribution be similar to standard distribution. (Correcting skewness) Is it true?

I would appreciate to the answers for only the part of questions.

$\endgroup$

1 Answer 1

1
$\begingroup$

I am not very familiar with the library and the problem that you are facing, but I took a look at the scientific publication in the Github documentation about ruptures. On page 31, section 8. Presentation of the Python package, under Constraints it says:

All methods can be used whether the number of change points is known or not. In
particular, ruptures implements change point detection under a cost budget and with a linear
penalty term [17, 111].

And for Input it states:

Change point detection can be performed on any univariate or multivariate signal that
fits into a Numpy array. A few standard non-stationary signal generators are included.

Based on this I would say that you don't necessarily need to standardize your data, if that's what you meant by standardize. That is mainly used to speed up the training process for algorithms like logistic/linear regression, neural nets, etc. and I am quite certain that it does not affect the learning outcome of the model.

My suggestion is to try different evaluation metrics and cost function, to plot the results and compare. You can try also standardization as you planned, but IMO it wouldn't give you a better model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.