# Automation of finding a starting point of measurement in a large dataset

I am looking for a way to automatically find a starting point of rising in my signal in Python. The data are collected with the frequency 10k (0.0001 s each) so the differences between each point are very small, lost in the noise. I found this point (black dot) manually using data analysis software before but I have multiple files and the manual process is not gonna work well. I was trying to think of something to do with derivative (red dots) or rolling variance (green dots) but it's a dead end for me now. Here's how manual point was chosen:

I pick a point that looks to me that is the closest one to rising signal but is still in the middle of noise before rising. Chosing it manually is just my rough estimation but I don't mind being one or two points wrong from the "correct" starting rising point. I will use it to offset my signal so that rising starts more or less at X = 0.

And now I wanted to find it using python. The full signal looks like this:

The derivative:

The rolling variance:

So they're all close to the interest point (black dot) but I don't know what to do with them next. If I change the limits it all looks like this:

Any ideas how to solve my problem? The simple code sample is below (plotting excluded)

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy.optimize import curve_fit
import scipy.signal as sig

signal = pd.read_csv('dataset.txt', delimiter=' ' )
signal.columns = ['time','current']

#calculating derivative, finding max and min indices of derivative
signal['derivative'] = pd.DataFrame(signal_derivative[:,1])
index_derivative_max = signal['derivative'].index[signal['derivative'] == signal['derivative'].max()]
index_derivative_min = signal['derivative'].index[signal['derivative'] == signal['derivative'].min()]

#calculating rolling variance, range 50 points, finding indices of peaks
signal['rolling_var'] = signal['current'].rolling(window=50,center=False).std()
index_rolling_max = signal['rolling_var'].index[signal['rolling_var'] == signal['rolling_var'].max()]
index_rolling_2nd_max = signal['rolling_var'].index[signal['rolling_var'] == signal['rolling_var'][:100000].max()]

• The main question is how do you determine the first rising point. Speaking based on the experience, they can just be roughly estimated (due to noise) and more or less using what you are doing now. The evidence is the fact that even in the first figure the marked point is not exactly the beginning of raise (next immediate point is lower in magnitude) ... I think u need to expalin ur question more than focusing on python code. How do you determine the beginning of raise? Sep 28 '20 at 9:02
• @KasraManshaei Good point! I didn't specify it (I'll edit post) but the beggining of raise when I choose manually is very dependent from my choosing. I pick a point that is the closest to rising signal but is still somewhere in the middle of signal noise before rising. One point above or below doesn't bother me that much. I need it to offset the whole signal and set the point as 0 to then perform curve fit so the difference between 0 and 0.0001 points won't be a problem. Sep 28 '20 at 9:10