1
$\begingroup$

I have a dataset that contains consumptions. These consumptions are measured every month. But some months are not measured. So the measured month after the unmeasured month is actually worth the sum of the two months (or more).

My dataset;

            difference
date    
2019-01-01  50.0
2019-02-01  60.0
2019-03-01  NaN
2019-04-01  140.0
2019-05-01  90.0

So we can understand that the 4th month's value is actually a sum of the 3rd and 4th months. It is necessary to organize this data with this logic. Because 140 is not a correct value for the 4th month and the 3rd month's consumption is not zero.

            difference
date    
2019-01-01  50.0
2019-02-01  60.0
2019-03-01  70.0
2019-04-01  70.0
2019-05-01  90.0

This (mean) can be one of an approach to avoid this problem on the dataset. After that, I can use this data set to predict next month's consumption.

I want to know if this approach has a name. What solutions can I implement on this type of time-series datasets? How can I search this problem?

$\endgroup$

1 Answer 1

1
$\begingroup$

The approach you're trying to describe is being able to fill the gaps in your data.

Filling N/A in the data

Since you're working in Python, I'm guessing your data is stored as a Dataframe. Pandas has a specific function for this: DataFrame.fillna().

This lets you fill any NaN values with multiple methods.

There are some similar examples in this answer.

Filling N/A and changing the following item

From my knowledge, Dataframes don't yet have any functionality to do this.

The best option I can think of is to iterate through the series. You could either convert to a list with .tolist() then use a for loop, or use Series.iteritems()

In your loop, you'll need a condition to check if NaN and if so, take the average of the current and next item if the current item is NaN. You may also need to have a condition for the edge case if the final value in the list is NaN

$\endgroup$
3
  • $\begingroup$ Hi @jindustries but my problem is not about filling the gaps. Because i want to edit next filled value too. Please look my dataset's changed values. $\endgroup$ Commented Nov 17, 2021 at 12:51
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Nov 17, 2021 at 14:11
  • $\begingroup$ @ŞevvalKahraman updated my response to clarify as I misinterpreted what you mean't $\endgroup$ Commented Nov 18, 2021 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.