Questions tagged [prediction]

prediction is the use of data, statistical algorithms and machine learning techniques to identify the likelihood of future outcomes based on historical data. The goal is to go beyond knowing what has happened to providing a best assessment of what will happen in the future.

183 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8 votes
0 answers
182 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
  • 181
5 votes
1 answer
269 views

Why does CV yield lower score?

My training accuracy was better than my test accuracy, hence I thought my model was over-fitted and tried Cross-validation. The model further degraded. Is that my input data need to be sanitised ...
  • 431
3 votes
1 answer
108 views

How to incorporate the uncertainty of the model coefficients in the prediction interval of a multiple linear regression

I'm dealing with modeling small experimental data sets. As most experimental work does not generate thousands of samples, but rather a handful, I need to be inventive about how to deal with this small ...
3 votes
1 answer
3k views

Making predictions / Loading model in TensorFlow 2.0

I use TensorFlow/Keras on a daily basis to make predictions for a project. Everything works fine but I was getting regular warnings about the transition to TensorFlow 2.0 and I thought this week I ...
  • 1,187
3 votes
0 answers
52 views

Timeseries prediction error measurement. How to deal with diffrent time scales?

I have some time series and a prediction model. Now I would like to measure how good/bad the prediction is for different products. The problem is that for each product the time points (frequency of ...
3 votes
0 answers
173 views

Next events prediction based on previous events

I have data set of sequences of user executed commands sorted in the order of its occurrence. The data looks like this. ...
  • 31
2 votes
1 answer
1k views

Plotting the confidence interval for a plot in python

I have a curve and I want to create the confidence interval for the curve. Here, I provide a simple example: ...
  • 146
2 votes
2 answers
67 views

How do you effectively predict the top 20% most likely customers to churn from a dataset?

I am looking to work out that if I have a dataset with 100,000 existing customers who didn't churn and 20,000 previous customers that churned in the past and the business objective is to target the 20%...
  • 21
2 votes
0 answers
66 views

Time series forecasting for vibration prediction on Industrial machine production?

I'm working on a machine learning project related to an industrial machine. The goal of the project is to build a model that would be able to predict the vibration of the machine while it's in ...
2 votes
0 answers
20 views

History that lead to the word "predict" being used for the application of a model on data

Background The framework scikit-learn uses "predict" for the application of model on (new) input data and I have seen many people use that term. In the scientific papers that I have read (...
  • 732
2 votes
1 answer
222 views

Prediction issue with xgboost custom loss

I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried ...
  • 21
2 votes
0 answers
122 views

Results are too good.. what is wrong? How to predict correctly?

I am about to evaluate a neural network and want to check whether the predictions make sense. The variables: ...
  • 490
2 votes
1 answer
75 views

Predicting when component will fail having its parameters data

I have a component and I need to predict when it will wear out and will need replacement. I monitor, let's say 5 parameters of this component, each one is monitored for every run cycle. So, the ...
2 votes
1 answer
449 views

How to add previous predictions for new predictions in LSTM?

I am trying to train a model on a big data sequence like this [0.2 0.1 0.1 ..... 0.4 0.8] . I create X vectors with length 60 for inputs and Y scaler numbers as ...
  • 1,182
2 votes
1 answer
1k views

How to extract the sample split (values) of decision tree leaves ( terminal nodes) applying h2o library

Sorry for a long story, but it is a long story. :) I am using the h2o library for Python to build a decision tree and to extract the decision rules out of it. I am using some data for training where ...
  • 21
2 votes
2 answers
139 views

How do I use rnn to forecast to n periods with limited data?

So this is my 1st time trying to run a small time-series dataset through an RNN, but after a lot of searching, I haven't been able to find, 1. How I can use this to forecast to n periods ? (like in ...
2 votes
0 answers
107 views

Prediction questions related to the dataset

I have been self-learning data science from different sources. I have a dataset which was sent to me by my friend from one of her college courses. The work is to be done in R preferably. Description: ...
2 votes
0 answers
918 views

Test data predictions yield random results when making predictions from a saved model

I am classifying aerial imagery that is tiled into 256x256 tiles using Keras and TensorFlow. The model splits the training data (i.e. the 256x256 image tiles making up the study area) into 70% ...
  • 327
2 votes
2 answers
437 views

Event prediction sequence

I need to create an app to predict events. I'll explain my requirements: I have a set of intermediate events (eg.: 1, 2, 3...9) and a set of final events (eg.: A for OK and B for KO). These events ...
2 votes
0 answers
604 views

NARX Network Multi-Step Forecasting Question

I've been attempting to do multi-step ahead prediction with the NARX (Non-Linear with Exogenous Inputs) Neural Network. As I understand it, this network can be defined mathematically as follow: $y(t+...
  • 21
1 vote
0 answers
7 views

Strange behaviour around zero for predicted distributions from a deep learning regression model

Can somebody help make sense of these very odd distributions that I obtained from my trained deep learning regression model? The model was trained with either MAE or MSE loss, which is what the ...
1 vote
0 answers
28 views

"Most forecasting algorithms assumes that each point is independent of one another." If so, how forecasting is being possible?

In stationary, if we want to make forecasting, we have to make our data stationary(On classical methods), I get that, but If every data point is independent of each other, how can we make the ...
1 vote
0 answers
25 views

Queries on architecture of production pipeline for a batch predictions

I have a Problem statement to predict Expected delivery time (in days) and below is the information we have from client. Prediction to be batch inference. Each batch inference to have millions of ...
  • 11
1 vote
1 answer
174 views

Linear regression returning negative values for house price prediction

I am trying to do a prediction of real estate (prices are in millions). The mean price for the dataset is 4 million. I do not have any negative values in my dataset,...
1 vote
0 answers
26 views

massively imbalanced data

I am dealing with time series data with +200K (every minute for 6 months)record of gas turbine I am trying to early detect the fault (0 or 1-fault). The issues with the data are: 1.the fault occurred ...
  • 11
1 vote
1 answer
70 views

Logistic Regression for prediction

I would like to ask about the theoretical approach of using Logistic Regression for customer data and more specifically Churn Prediction (in BigQuery and Python). I have my customer data for an online ...
  • 111
1 vote
1 answer
23 views

How to fill missing values in a discrete column in sales predictions for a drug supply chain company

I have been working on a dataset that has data from a famous drug supply chain company. The first few records of the dataset look like the following; Another data accompanies this (primary) dataset. ...
1 vote
0 answers
33 views

Price Predition for Irregular spaced historic data of non independent Prices

I am a little unsure how to proceed. I am not an expert but on a decent intermediate level when it comes to regular Timeseries. Now i am faced with a problem that first seemed related, but is an ...
1 vote
0 answers
23 views

Confusion matrix of 3*2

I would like to include confusion matrix in my research report. I have a binary classification problem. The positive class is further divided into two types for example: Real Positive and Obstruction ...
  • 21
1 vote
0 answers
26 views

Multivariate regression - not enough data?

I have a table with data about 10 agriculture parcels. Each parcel has data in time regard the number of nutrients each parcel has received in each day and in the end I have the total number of ...
1 vote
0 answers
16 views

Prediction method when the time series is not sequential?

I have multivariate time series data consisting of monthly sales of contraceptives at various delivery sites in a certain country, between January 2016 and June 2019. The data looks as follows: The ...
1 vote
0 answers
12 views

Evaluation of reliability of suppliers by on time delivery

I am trying to create a weight that can help me properly evaluate the reliability of a company’s on time delivery ratios. I’m using historic delivery data. For example, comparing company A that ...
1 vote
1 answer
26 views

Rule based prediction for known data

Lets say we have trained our model on 900 records (training data) . During prediction on test data of 100 records, assume model produces 95% accuracy. The question here is, can a mechanism be built, ...
1 vote
0 answers
24 views

Error while using pre-trained model

I'm working on NLP task using RoBERTa model. As training last very long I saved my model, but now for some reason, part of my code doesn't work with this pre-trained model (getting an error), and ...
1 vote
1 answer
424 views

Predicting high frequency sparse time series data in python

I have a dataset of a couple of EV charging stations (10 min frequency) over 1 year. This data consists of lots of 0's, since there is no continuous flow of cars coming to charge but rather ...
1 vote
0 answers
49 views

Difficulty understanding the difference between Poisson, Quasi-Poisson, and Negative Binomial models

I will try to keep this short. As an assignment for my GLM course, we were given a dataset on the # of homicide victims a person knows, as well as the race of the person. The main idea is to answer ...
1 vote
0 answers
13 views

Should I give regularly-spaced or irregular-timestamped data to a price predicting neural network?

I am building an application to predict the price of an item. Data is collected at regular 5-minute intervals while the application is running. Unfortunately, there is downtime, so there is not a full ...
  • 111
1 vote
0 answers
33 views

How to do prediction on survival data, using Random Forest

I should make prediction on survival data, using the random Forest method. My question is: should I follow the same approach as in logistic regression? taking into account only the status variable or ...
1 vote
0 answers
363 views

LSTM giving almost constant output

I have used an LSTM with 4 layers deep each layer having 10 LSTM units to predict the AAPL stock 500 steps away by looking 50 steps back and it was predicting well (only a lag was there). However when ...
  • 85
1 vote
0 answers
16 views

Confidence interval for class membership probabilities

I would like to calculate confidence intervals for predicted probabilities of a class membership obtained with randomForest. I know I could use predict.all in randomForest(), which gives me the ...
1 vote
0 answers
76 views

*(CATEGORIES[int(prediction[0][0])])* giving me different result for single image prediction from saved model

From a saved model, I am trying to predict a single image. I followed this code - https://www.youtube.com/watch?v=A4K6D_gx2Iw I am getting different result for two different command- ...
1 vote
1 answer
44 views

target variable prediction among possible answers

I have a dataset on which I would like to apply a Machine Learning algorithm for multi-class classification. An example of my target variable (in string format, will be later ...
  • 129
1 vote
0 answers
29 views

Why does my model fail to predict on the whole dataset?

So I have about 3000 images with 6 classes and this is what I did: 1 - split into training set and test set prior to anything with 20% test size 2 - performed data augmentation on the under ...
1 vote
0 answers
23 views

Stabilize Neural network prediction for class probability

I could not carry my question from stackoverflow I ve been trying to fit a neural network for binary setting using library(keras) and I am interested in class ...
  • 111
1 vote
0 answers
108 views

Why NARX neural network and Hammerstein-Wiener model perform worse than simple sigmoid network nonlinearity estimator on any predictions?

I am currently working on dynamic modeling and exploring different techniques and algorithms to deploy a dynamic time-series black-box model. My data looks like the following: I have 7 inputs and 1 ...
1 vote
0 answers
155 views

Why are Neural Network predictions "correct", but offset from true value? Not using any past lagged values

I recently asked a similar question, but didn't get a response that really addressed/fixed the issue. Additionally, I've done some more work since then. I'm sorry for the long question below, I just ...
1 vote
0 answers
91 views

When should I reverse normalizations to evaluate loss?

If I am training a neural network and have normalized the data before-hand, should I reverse the normalization to calculate the loss? This tutorial provides an example of this method. What if I'm ...
  • 131
1 vote
1 answer
468 views

YOLO Dense Prediction

I have two questions about dense prediction in YOLOv4 paper What does it mean by the (hard negative, online hard) example mining method is not applicable to one-stage object detector, because this ...
  • 283
1 vote
1 answer
3k views

How to make predictions of multiple input samples at once in tf 2 with keras

I am quite confused on the output of model.predict when after training I validate my model on around 6000 samples I use the following pseudo code: ...
1 vote
0 answers
31 views

Derivative of multi-output Gaussian Process

I am working on a project where I estimate transition and measurements models for a kalman filter using Gaussian Processes. In order to linearize the models I require the Jacobian of the estimated ...
  • 21