0
$\begingroup$

I'm new for data analysis. I got some data from the regional environmental center.

Measurements: Datetime, PointID, SubstanceID, Value (substances concentrations in air), MeteoID ,NextValue

Meteorological data: MeteoID, Datetime, temperature, wind speed, wind diretion,humidity, pressure, precipitation.

Substances(8 substances(CO,Cl2,NO2, etc.), 5 of them have about 1.2 million records): SubstanceID, Name, MaxVal (maximum allowable concentration value).

Points(9 static monitoring stations): PointID, Adress, Longitude, Latitude.

Measurments table contains about 8 million records of substances concentratios in air(local mysql). I linked the measurement data and the meterologicala data (closest in time), and added the next value to measurement record. time between measurements 20 min( in average, for some periods data are missed).

I want to make a predictive data analysis and get short-term forecasts for substances concentrations depending on the previous value and weather data(may be for few hours). I am still thinking which methods, techniques can be chosen. I want to consider different methods. which toolkit is most suitable for this case? Should the data be considered as a multidimensional time series? I'm currently looking into the direction of some kind of neural network and implementation in python (but I still don’t know which package).

$\endgroup$
0
$\begingroup$

Should the data be considered as a multidimensional time series?

This depends on whether the target variable (the one that you want to predict) depends on the others. If not, there is no point in doing it. A fast way of checking if the variables are linearly dependent and, therefore, multidimensional forecasting is meaningful is by checking the linear correlation of the variables. Then select only the variables that have high correlation (>0.5) with the target variable to include them in your prediction model.

I am still thinking which methods, techniques can be chosen.

The model that I recommend for time series forecasting is a Recurrent Neural Network. This is because of its inherent ability to store previous timesteps in its memory and to incorporate them into future predictions. This is very important, because it is among the few approaches that exploit the temporal dependencies between samples.

I'm currently looking into the direction of some kind of neural network and implementation in python (but I still don’t know which package).

The most convenient way of implementing a recurrent neural network in Python is by utilizing the Keras framework. Please go carefully through this tutorial, as it will definitely be a very good first step to attack your problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.