I posted here already but it is marked to close, so thought of posting it here (as this might be the right forum)
Am working on a simple logistic regression with 1000 records and 28 features.
My business users suggest that they want to first see what the AI can do by itself based on our data as it is. Meaning, they don't want me to do feature engineering, trying out multiple algorithms etc.
They want me to avoid all that because they feel it takes time to do feature engineering and they wish to showcase something quicker and earlier. For the 1st cut, they wish to go live with baseline model with no feature engineering (even if it is 50% accuracy).
They are okay with low recall like 30% or 40% (at least for now) for one of the classes because currently there is nothing done to solve this problem. No one is tackling this problem or even thought to solve this problem. So, this is new to them... So, even if it is low recall for negative class, they feel it is something good for them to start (because positive class has high recall). Meaning, they identify those positive cases accurately and go follow up with them. Since this model is reliable (for them) in terms of positive cases, they wish to go live with this.(and focus on those positive cases) Of course, recall for negative cases is a serious concern for them. But at least they have a solution for one of the classes and they are happy. But ultimately, they would like to have solution for negative classes as well. So they suggest me do feature engineering, model experimentation etc after going live. By live, I mean just a simple static dashboard (and not high end MLops etc).
Later, they want to know with all these experiments of model and new features, is recall for negative class is improved or not?
Is this a right way to go further? As a novice data scientist, I don't feel right about this. If it had been at least 80% (my random choice), I would have been bit okay. I don't have any evidence to proof that 80% is the right choice rather than just saying higher no of actuals are predicted correctly.
So my questions are
a) what should I do and what are the pitfalls/points that I should make sure to keep them aware?
b) Is there anything important that I should highlight them?
c) Should this project still be dropped if business is okay to be with 50% acc? Can we continue to use this model as long as business is fine with it?
d) Any real time experience from your model deployment decisions?
Can share your views on this? Would really be helpful for me to learn and also keep them aware?