5

It seems that the SeqSelfAttention layer is expecting all the time-steps. i.e. return_sequences=True Same is shown in the home page example. Link import tensorflow as tf, numpy as np from tensorflow import keras from tensorflow.keras.layers import Dense, Dropout,Bidirectional,Masking,LSTM from keras_self_attention import SeqSelfAttention X_train = np....


5

From the documentation you referred: "The length of the span is the range of a 64-bit integer times the length of the date or unit." 64 bit integer has values from -2^63 to 2^63-1, which is the same as from -9.2e18 to 9.2e18. So, the time span column shows you which dates would you cover if use only the corresponding units. Note, i.e. that time ...


4

It is an interesting question. I would not completely agree with you though when you say that most time-series models dont use attention. However there is not as much documentation available on the web as there is for other applications. LSTNet was one of the first papers that proposed using an LSTM + attention mechanism for multivariate forecasting time ...


3

Please have a look at your weights after training. I assume your Neurons die due to relu activation as they output Zero for Input < 0. Unfortunately, the ReLU activation function is not perfect. It suffers from a problem known as the dying ReLUs: during training, some neurons effectively “die,” meaning they stop outputting anything other than 0. In some ...


3

Because the data is time series while only Dense layers are used in the model, the problem is caused by model initialization. A model with a 'bad' initialization will constantly predict zero, as you will see by running the script below. import pandas as pd from sklearn.model_selection import train_test_split from sklearn.preprocessing import MinMaxScaler ...


3

The real state of the art here is the Matrix Profile suite, developed by Eamonn Keogh and his team in University of California at Riverside (UCR). Here are some links to get you started: Matrix Profile Foundation homepage The UCR Matrix Profile Page MPA: a novel cross-language API for time series analysis paper (2020) with links to Python, R, and Go ...


3

I would recommend looking into FBProphet. It's a good starting point for automating the creation of forecasts. It's very easy to use, and often produces better results than classical forecasting methods (ARIMA, Holt-Winters, etc.) right out of the box. The default settings offer an additive or multiplicative model, comprised of trend and seasonality. This ...


3

Honestly it seems you are quite far from what would need a supervised vision approach. I suggest you to try a simple non-ML approach first : extract text with a standard library then just label what would count as a 'bullet' then check if there is more than one in a row. This might just work and if it doesn't it will help you understand why. Going the whole ...


3

Generally, I'd pick a very simple, transparent/explainable model and use the results in a semi-automated way. That is, do not just derive a prediction but rather insights. You could, for example, use a (or multiple) decision tree(s) which you pre or post prune. The result could be a tree with, let's say, just 1-3 features to find simple rules like "if a ...


3

The metric to the time series forecastability is "the spectral entropy". I learned it from some talk of Rob Hyndman, so here is the description of his implementation for R tsfeatures package, entropy The spectral entropy is the Shannon entropy −∫π−πf^(λ)logf^(λ)dλ, where f^(λ) is an estimate of the spectral density of the data. This measures the “...


2

One option could be applying the Fourier transform which transforms time to the frequency domain. It is often easy to find recurring patterns when signals are represented in the frequency domain.


2

I don't see why not- it's the loss you wish to minimize. I'm using the following as my loss function and it works well when sMAPE is my metric for prediction accuracy. import tensorflow.keras.backend as K def smape_loss(y_true, y_pred): epsilon = 0.1 summ = K.maximum(K.abs(y_true) + K.abs(y_pred) + epsilon, 0.5 + epsilon) smape = K.abs(y_pred - ...


2

Thanks for this question I think it is a nice use case to play with time series forecasting in all (or many) of its types. As you suggest, there are several possible approaches, and all of them are valid hypothesis a priori to check and validate with your goal. Answering the question: yes, it is possible to build a single model to predict the sales amount ...


2

Is it possible that your model is just mimicking the output of the previous timestep? The predicted population is trailing the true population by 1 time-step. The model is just predicting a value close to the previous population it sees, as it feels this is the best prediction for the next price. For e.g. https://towardsdatascience.com/how-not-to-predict-...


2

Essentially you want to pick a function that will give you the "size" of a matrix. The most obvious way I can think of is by choosing a matrix norm, which is a map $\lVert \cdot \rVert \colon \mathbb{R}^{k, k} \to [0, \infty)$ (or you could generalise to a complex $k \times k$ matrix if you wished). Your suggestion seems similar to computing $$S = \...


2

If you are using information from the future to impute missing data would be data leakage as you would not have this extra information when the model is in production and trying to predict future values. To prevent data leakage, make sure to only use values that are available at the date/time you want to predict. If you were to impute the missing data based ...


2

Transformers can be used for time series forecasting. See the following articles: Adversarial Sparse Transformer for Time Series Forecasting, by Sifan Wu et al. Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case, by Neo Wu, Bradley Green, Xue Ben, & Shawn O'Banion The Time Series Transformer, by Theodoras Ntakouris ...


2

Since the data is recorded every month (i.e. a data point for each month in the year) and we see a yearly seasonality trend we compare the data for the same month against previous year. This is done by taking the difference against the data from last year (.diff()), which is equal to going back 12 observations since each observation is data for a specific ...


2

Yes it is wrong to set shuffle=True. By shuffling the data you allow your model to learn properties of the data distribution that might appear only in the test time periods. For example, if you have a trend in the data, shuffling will 'help' you handle it. In a real-time scenario, you'll never have access to those properties of the distribution.


2

Considering a team like Chelsea has played FA Cup, Champions League, Premier League and other competitions. We need to keep in mind that, other teams would also participate in the same competitions. Sports data from all teams in the competitions would help to identify Chelsea's best win against their toughest competitors that they have faced in FA Cup, ...


2

1 (Y)ear = 3.154e+16 nanoseconds. In scientific notation, "E" refers to a power of 10. | So (9.2E+18) is written as "9.2 × 10^18" in scientific notation. The decimal value of (9.2E+18) would equal 9200000000000000000. In reference to (Y)ears, what does the value (9.2E+18) mean? 9.2 x 10^18 Milliseconds = 291,536,394 Years. 9.2 x 10^18 ...


1

Yes, the aim of auto.arima is for fitting ARIMA models automatically. You do not need to decompose your time series before hand. See how the algorithm works here https://otexts.com/fpp3/arima-r.html. You may still want to look at arguments available in the auto.arima function, and you may want to change default maximum values for p, q, and d, etc.


1

Its the number of features that has to remain consistent not the number of timestamps. Outputs will be batches of one row predictions, so in your case its 1500 and than 1000. Model does not care, features should remain same. And beware of dataleakage with time series.


1

Dynamic Time Warping might be what you're looking for: it measures similarity between two time series based on the optimal alignment between the two sequences. For example point $i$ in sequence 1 might be better aligned with point $i+3$ in sequence 2 based on the evolution of the sequences (as opposed to Euclidean distance which would always compare $i$ in ...


1

It looks to me like what you propose makes sense, but there has been some research done around these questions of time representation already. I'd suggest you check the state of the art in this domain, if only not to reinvent the wheel or miss important cases. I'm not very knowledgeable about it but I can at least point you to TimeML and the related ...


1

TL;DR While the data comes from the NYTimes and seems legit, the presentation is intentionally misleading and the subsequent assertions are baseless. I say "intentionally" because an unbiased and reputable analysis would not propagate such major allegations from the data they have presented. The data does not prove nor disprove voter fraud, so the ...


1

Let's say you have very frequent data across a period of time T and you want to sample N points. Instead of sampling points directly you could sample the time that separate them. You then just need to add 12 hours to all gaps to enforce your constraints. To do so you could sample N points uniformly in [0, T - (N-1) x 12h] and then compute the difference ...


1

What you are looking for is training a classifier with data augmentation. In the context of image classification, this may refer to changing the pose of the object by skewing or rotating the image. In the context of text classification, this can be imagined as classifying different versions of the same sentence with an alternating word sequence (some ...


1

I know that this answer is late but Try to use CNN on top of BLSTM it works very well on your use case.


Only top voted, non community-wiki answers of a minimum length are eligible