3
$\begingroup$

I have csv data file and I design LSTM model to predict values. Then I want to save that prediction value in same csv file. Can I do that? I tried using one code then in my csv file only had prediction values and delete other columns. Can anyone give me a suggestion for that.

import pandas as pd
import numpy as np
data = pd.read_csv('data1.csv')
data = pd.DataFrame(data,columns=['x','x1','x2','y'])
data.columns = ['x', 'x1', 'x2','y']
pd.options.display.float_format = '{:,.0f}'.format
data = data.dropna ()
d = ['y']
y=data['y'].astype(int)
cols=['x', 'x1', 'x2']
x=data[cols].astype(int)
scaler_x = preprocessing.MinMaxScaler(feature_range =(-1, 1))
x = np.array(x).reshape ((len(x),3 ))
x = scaler_x.fit_transform(x)
scaler_y = preprocessing.MinMaxScaler(feature_range =(-1, 1))
y = np.array(y).reshape ((len(y), 1))
y = scaler_y.fit_transform(y)
print("row",len(y))
n = data.shape[0]
p = data.shape[1]
fill_missing(data.values)
train_start = 0
train_end = int(np.floor(0.65*n))
test_start = train_end+1 
test_end = n
x_train = x[np.arange(train_start, train_end), :]
x_test = x[np.arange(test_start, test_end), :]
y_train = y[np.arange(train_start, train_end), :]
y_test = y[np.arange(test_start, test_end), :]
x_train=x_train.reshape(x_train.shape +(1,))
x_test=x_test.reshape(x_test.shape + (1,))
seed = 20
np.random.seed(seed)
fit1 = Sequential ()
fit1.add(LSTM(
output_dim = 10,
activation='relu',
input_shape =(3,1)))
fit1.add(Dense(output_dim =1))
fit1.add(Activation(linear))
batchsize = 10
fit1.compile(loss="mean_squared_error",optimizer="adam")
fit1.fit(x_train , y_train , batch_size = batchsize, nb_epoch =10,   shuffle=True)
print(fit1.summary ())
pred1=fit1.predict(x_test)
pred1=fit1.predict(x_test)
real_test = scaler_y.inverse_transform(np.array(y_test).reshape ((len(y_test), 1))).astype(int)
pred1 = pd.DataFrame(pred1, columns=['pred1']).to_csv('data1.csv')
$\endgroup$
4
$\begingroup$

if you want this column in the same dataframe just do

data['pred'] = pred1
data.to_csv('data1.csv')

The first line automatically adds a column called 'pred' to the dataframe with values coming from pred1.

Hope it helps. Good luck!

$\endgroup$
7
  • $\begingroup$ I tried your code and it gave me an error Length of values does not match length of index. May be because of less value prediction. If I want to save my prediction values comparatively to the actual value.Then how can I code it according to the actual value save prediction value in samme csv file? $\endgroup$ – awa Feb 5 '19 at 15:03
  • $\begingroup$ So the fact is that probably your "test" is a fraction of your data not all of it. You first need to take those rows that you chose for testing and keep it in a dataframe. Then add this column to that dataframe. $\endgroup$ – Kasra Manshaei Feb 5 '19 at 15:39
  • $\begingroup$ I trie it and it gave same error that I mentioned earlier. $\endgroup$ – awa Feb 6 '19 at 4:02
  • 1
    $\begingroup$ Please update your question with full code … let's do it together $\endgroup$ – Kasra Manshaei Feb 6 '19 at 10:46
  • 1
    $\begingroup$ Welcome but please be more clear. At line 5 you use variable "x" before defining it. If this is the original code then you certainly get error as there is no x. If "x" variable is somewhere else in your code and you use it from cache, please include that part. I mean a peace of code which works stand alone. This code will give an error on line 5. $\endgroup$ – Kasra Manshaei Feb 6 '19 at 11:12
1
$\begingroup$

There is no direct method for it but you can do it by the following simple manipulation. Instead of directly appending to the csv file you can open it in python and then append it. Here is the code for the same:

data = pd.read_csv("data1.csv")
data['pred1'] = pred1
df.to_csv('data1.csv')
$\endgroup$
2
  • $\begingroup$ I tried your code and it gave me an error Length of values does not match length of index. May be because of less value prediction. If I want to save my prediction values comparatively to the actual value.Then how can I code it according to the actual value save prediction value in samme csv file? $\endgroup$ – awa Feb 5 '19 at 15:04
  • $\begingroup$ It is difficult to solve without seeing the data. But you should try it yourself. Check the shape of your dataset in which you want to add the column, and also check the shape of the column your adding. Try to find out where is the error coming from by printing both the data. $\endgroup$ – bkshi Feb 6 '19 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.