I know it depends on the problem and various other factors like data availability, the complexity of the use case, the workload of developer ..etc, but can someone suggest effort estimation of building a minimum viable product.

I have taken the following parameters into consideration while preparing effort estimation(close to 3months) for building a price prediction model based on two problem definitions(no sample data supplied by client), which would be integrated with web interface.

Data loading & pre-processing - 1 week Development of model - 2.5 month Deployment - 2 weeks.

not sure whether it's overestimated or underestimated.

Note: Please do not close with the comment "too broad" as I do not have much/can not provide more information. Requesting expert advice.

  • 1
    $\begingroup$ In my experience, you are vastly under-estimating the time needed to find and process the data, and vastly over-estimating the time to build the model. Prob under-estimating deploy too, but this varies quite a bit depending on the application. $\endgroup$
    – Andy M
    May 15 '19 at 13:10

There is something called the 80/20 rule in data science. It comes from surveys that have shown that data scientist usually spend 80% on gather and cleaning data and only 20% on actually using it to build models and the rest.

No one can tell you how long each step is going to take since it depends on your case. But I can tell you that 1 week / 2.5 months differs a lot from 80/20 and I think you should consider re-calibrating those two, but only you know the facts.

You can read more in this survey: CrowdFlower Data Science Report 2016 - check the section called "How a Data Scientist Spends Their Day"

  • Data Loading & Pre-Processing : depending on your data-size, this is a highly variable measure. we can agree that this will be the second longest task.

Edit : This phase ,for me, was about collecting the relevant columns and structuring a csv file to be read and loaded in a python environment.

  • Model Development : again, this will highly depend on your data ( and i'm referring to complexity and structure of your data rather than the size) , if you have a lot of features at hand, and your problem is complex ( demands a lot of effort in feature engineering and feature selection ) , this will be the longest task at hand and will probably take up most of your time
  • I would say deployment would take something in weeks like you mentionned depending on the complexity ( yes again ) of your production environnement.

Please take my statement lightly as i don't have much experience in this field. I've had the opportunity to deploy some models as webservices, and it took me less than your projections. However, our problems are quite different.. so my numbers could be out of match.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.