1
$\begingroup$

I'm trying to use isolation forest algorithm for outliers detection.
Data has 2 columns: id and REV. Below code gives me ungrouped result.
Could you pls advise how to get result grouped by first column (id)?

data= pd.read_excel (my_path)
outliers_fraction=0.1
scaler = StandardScaler()
np_scaled = scaler.fit_transform(data)
data = pd.DataFrame(np_scaled)
model =IsolationForest(contamination=outliers_fraction)
model.fit(data)
data['anomaly'] = pd.Series(model.predict(data))
print(data)

I have 2 columns: id and REV. I added a picture of what I expect to see as the final result:

enter image description here


Tried to use function, but have half of data NaN. Any thoughts please?

def groupreg(g):
        outliers_fraction = 0.1
        # train isolation forest
        model = IsolationForest(contamination=outliers_fraction)
        model.fit(g)
        g['anomalyX'] = pd.Series(model.predict(g))
        return g

    df1=data2.groupby('id').apply(groupreg)
    print(df1)
$\endgroup$
  • $\begingroup$ Please provide sample data and sample out that you are expecting and what it is coming out to be. $\endgroup$ – Kiritee Gak Aug 15 '19 at 4:01
0
$\begingroup$

Here is solution if somebody will need it someday

def groupreg(df):
    a=df[['EUR']].values
    # train isolation forest
    model = IsolationForest()
    model.fit(a)
    return pd.Series(model.predict(a))

result=df.groupby('id').apply(groupreg)
#print(result)

#print(isolationforest(data2))
df['outlier1'] = result.tolist()
print(df)
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.