Classification: How to manage data sets where one data row depends on another data row

Question

I am trying to classify heading, image and image caption of a webpage. I am preparing data by scraping selected URLs(around 1000) using XPath of DOM elements I need. Each data row in the CSV file contains tag name, x-coordinate, y-coordinate, text-size etc with three target labels heading, image, image caption but a text simply will not become image caption until unless it is under an image. Is there any way to create this dependency among rows.

Answer 1

I assume that you are solving a supervised classification problem, that is, you train your model on a labeled sample. I can think of two approaches to this problem.

I. Classify tags, use neighbor tags for the features

For each tag you can calculate features like:

• (x, y) distances to the closest image
• (x, y) distances to the closest image above this tag
• number of images near this tag
• ...

Such features could be fed into any classifier like SVM or decision tree.

II Classify pairs of tags

For each tag $i$, you can consider all other tags $j$ and predict probability that $i$ is a caption of image $j$. This prediction may be based on concatenation of:

• features of $i$
• features of $j$
• joint features of $i$ and $j$, like vertical and horizontal distance from $i$ to $j$.

After these probabilities are predicted, you can classify $i$ as a caption indeed, if any of these probabilities exceeds some threshold.