I am trying to find an ML solution for the following problem.
Objective: Given a list of products of Stores B, C, D … estimate the Order Value (1 year timeframe, let’s say) that each product would bring to Store A if Store A added them to their current offer of products.
I should mention that all stores are e-commerce stores.
Available Data:
• For Store A products we know basically all the information we could want, like Ordered Value/ Quantity/Price for arbitrary timeframes, Page Views, Ratings, age of products …
• For Stores B, C, D … products however, we only know the information available from web scraping: Name of product, Brand, Price, Discount, Rating, Number of Reviews, Listing position in site. We additionally know how to parse the Category tree of Store A for these products.
The approaches tried so far:
• For each product of Stores B, C, D … in the list find top k most similar products from Store A in the same Category (using a compound similarity metric made up of fuzzy-wuzzy scores for Name of product and Brand and Mahalanobis distance for the numerical features). Estimate the Ordered Value for each product of Stores B, C, D … as the average of the Ordered Values of the similar products of Store A. I should mention that the found similar products look reasonable and that we also have a similarity threshold under which the estimated Order Value for a product of Stores B, C, D … defaults to the overall average Order Value of that Category
• Train a regressor on the products of Store A with the restricted features (those available also for products of Stores B, C, D … ) and Order Value as target. Extrapolate to the products of Stores B, C, D … to obtain the estimates. The first solution is very computationally intensive and it doesn’t yield the most plausible estimates sometimes. For the second solution, the regressor underfits (as expected with such few features and a skewed distribution of the target variable) the data of Store A so again the results are not the most satisfactory. I mention also that the volume of data is very large and the lists of products is very volatile, so refinements like training more models or ensembles are not feasible.
My questions:
• Do you know any resources with approaches for this type of problem? I searched the web but failed to find them.
• Based on your experience, do you think it is possible in this context to achieve a performance of say 20-30% MAPE?
Thank you for your help!
