2
$\begingroup$

Is it possible with Orange (only using its widgets, without writing Python code) to implement the following typical machine learning processes?

  1. Train a training set,
  2. Validating a validation set (e.g. using k-fold cross validation), and
  3. Testing the model with a test set.

Using the 'Test & Score" widget, a single dataset is used. Does this mean that only validation is performed, and there is no actual test? Can in this case classification results (accuracy, AUC, etc.) considered reliable?

$\endgroup$
  • 1
    $\begingroup$ Disclaimer. I've never use Orange myself. Still, a quick look at online documentation suggests (and that would be no surprise) that is has all the tools you're looking for. docs.orange.biolab.si/3/visual-programming/widgets/evaluation/… $\endgroup$ – tagoma Sep 23 '17 at 21:07
  • $\begingroup$ I keep getting "test and train datasets have different target variables" error message. This is despite using exact same formats and names (with different values of course). What did i do wrong? $\endgroup$ – Norman Rashid May 30 '18 at 13:19
2
$\begingroup$

The answer is yes. If Test & Score is given only one data set, then all it can do is show results of cross-validation.

To test the models on a separate data set, use separate File widgets to load training and test data. Connect File widget with training data to Test & Score, and the connect File widget with Test data to Test & Score. The connect whatever learner to test (in the workflow on the figure I've used logistic regression and random forest). Make sure "Test on train data" is selected in the Test & Score.

Orange workflo with Test & Score widget

$\endgroup$
  • $\begingroup$ How can i then check accuracy of best model in test set and then save it for use external to Orange $\endgroup$ – Khurram Majeed Jan 25 '19 at 16:31
0
$\begingroup$

In the case of image classification,enter image description here training and testing on different data sets could be achieved with the attached configuration? Is it correct?

$\endgroup$
0
$\begingroup$

You could try use the widget "Save model" to save the model that you've created and "load Model" to validate on new set of data.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.