Features marked as boolean values (around 50,000) , need to score a set of fixed output (around 25,000).

I have wikipedia topics relevant to a particular url as features. And the output is the advertisement keywords which will perform well on those url. If I have the training data, i.e. advertisement keywords performance on urls. Given a url (I can extract the features i.e. relevant wikipedia topics), I need to predict which keyword might perform well on that url.

What is the scientific name for such kind of problems in which there is a big boolean input set and I need to score a big fixed output set. Can you name few algorithms for such kind of problems.

  • $\begingroup$ I cannot make sense of what is boolean here. $\endgroup$ – Paul Childs Mar 29 '18 at 3:39
  • $\begingroup$ Boolean here represent whether the particular feature is present or not. In case of this, whether a wikipedia topic is either relevant or not relevant to the url. So if a page is talking about present form of christiano ronaldo, then wikipedia topics such as chirstano_ronaldo and football are relevant to the url, while the wiki topic lionel messi is not relevant to the url. $\endgroup$ – Shubham Jain Mar 29 '18 at 6:29
  • $\begingroup$ Sounds like a standard ML problem, but with very sparse features, so maybe you're looking for an algorithm that can handle very sparse datasets? $\endgroup$ – komodovaran_ Apr 25 '19 at 11:54

Ok. I think I'm getting a clearer picture. Firstly any name isn't going to be different just because the data is large. Of course in the big data community anyone is free to coin a term and time will show if it gets adopted (same goes with what we might consider a scientific name for that matter too).

Another thing is input output. These terms can be confusing as what is input by a comp scientist into a training model is both the input and output of what a statistician would define.

I'm not aware of a name for the problems but names for models addressing such a problem include logistic regression and probit regression. Both deal with a(n) (set of) independent variable(s) - the feature name in your case - and a resultant dependent variable - the binary relevancy. The regression looks at fitting a linear function to a hidden variable; whose threshold gives the binary result. Fitting an "input" training set then allows the scoring based on this linear function, whether you want a binary score or a real valued one (from the hidden variable) that will be more indicative of the probability of your prediction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.