4
$\begingroup$

If I understand it correctly, then the labels of ImageNet are based on WordNet:

ImageNet is an image dataset organized according to the WordNet hierarchy. Each meaningful concept in WordNet, possibly described by multiple words or word phrases, is called a "synonym set" or "synset". There are more than 100,000 synsets in WordNet, majority of them are nouns (80,000+). In ImageNet, we aim to provide on average 1000 images to illustrate each synset. Images of each concept are quality-controlled and human-annotated. In its completion, we hope ImageNet will offer tens of millions of cleanly sorted images for most of the concepts in the WordNet hierarchy.

Source: http://image-net.org/about-overview

The text above states already that WordNet defines a hierarchy. You can see that by using the Python nltk package:

from nltk.corpus import wordnet
nltk.download()
def get_hyponyms(synset):
    hyponyms = set()
    for hyponym in synset.hyponyms():
        hyponyms |= set(get_hyponyms(hyponym))
    return hyponyms | set(synset.hyponyms())
dog = wordnet.synset('dog.n.01')
print(get_hyponyms(dog))
harrier = wordnet.synset('harrier.n.02')
print(get_hyponyms(harrier))

(thanks to Stephan Ds answer)

gives

set([Synset('harrier.n.02'), Synset('water_spaniel.n.01'), Synset('standard_poodle.n.01'), Synset('dandie_dinmont.n.01'), Synset('courser.n.03'), Synset('wirehair.n.01'), Synset('toy_manchester.n.01'), Synset('puppy.n.01'), Synset('briard.n.01'), Synset('beagle.n.01'), Synset('siberian_husky.n.01'), Synset('manchester_terrier.n.01'), Synset('bloodhound.n.01'), Synset('gordon_setter.n.01'), Synset('leonberg.n.01'), Synset('king_charles_spaniel.n.01'), Synset('yorkshire_terrier.n.01'), Synset('sealyham_terrier.n.01'), Synset('american_water_spaniel.n.01'), Synset('skye_terrier.n.01'), Synset('clumber.n.01'), Synset('pembroke.n.01'), Synset('wire-haired_fox_terrier.n.01'), Synset('shih-tzu.n.01'), Synset('newfoundland.n.01'), Synset('retriever.n.01'), Synset('cocker_spaniel.n.01'), Synset('springer_spaniel.n.01'), Synset('american_foxhound.n.01'), Synset('large_poodle.n.01'), Synset('lapdog.n.01'), Synset('bull_mastiff.n.01'), Synset('affenpinscher.n.01'), Synset('irish_water_spaniel.n.01'), Synset('terrier.n.01'), Synset('keeshond.n.01'), Synset('vizsla.n.01'), Synset('dalmatian.n.02'), Synset('bird_dog.n.01'), Synset('irish_terrier.n.01'), Synset('miniature_poodle.n.01'), Synset('dachshund.n.01'), Synset('australian_terrier.n.01'), Synset('blenheim_spaniel.n.01'), Synset('weimaraner.n.01'), Synset('soft-coated_wheaten_terrier.n.01'), Synset('doberman.n.01'), Synset('kelpie.n.02'), Synset('water_dog.n.02'), Synset('feist.n.01'), Synset('attack_dog.n.01'), Synset('french_bulldog.n.01'), Synset('papillon.n.01'), Synset('bedlington_terrier.n.01'), Synset('foxhound.n.01'), Synset('labrador_retriever.n.01'), Synset('great_dane.n.01'), Synset('kerry_blue_terrier.n.01'), Synset('miniature_schnauzer.n.01'), Synset('pariah_dog.n.01'), Synset('border_terrier.n.01'), Synset('staghound.n.01'), Synset('norwegian_elkhound.n.01'), Synset('redbone.n.01'), Synset('pooch.n.01'), Synset('old_english_sheepdog.n.01'), Synset('police_dog.n.01'), Synset('welsh_terrier.n.01'), Synset('spitz.n.01'), Synset('boxer.n.04'), Synset('tibetan_terrier.n.01'), Synset('shetland_sheepdog.n.01'), Synset('boarhound.n.01'), Synset('border_collie.n.01'), Synset('wolfhound.n.01'), Synset('lhasa.n.02'), Synset('scotch_terrier.n.01'), Synset('coondog.n.01'), Synset('giant_schnauzer.n.01'), Synset('japanese_spaniel.n.01'), Synset('german_short-haired_pointer.n.01'), Synset('entlebucher.n.01'), Synset('griffon.n.03'), Synset('griffon.n.02'), Synset('welsh_springer_spaniel.n.01'), Synset('clydesdale_terrier.n.01'), Synset('hound.n.01'), Synset('brittany_spaniel.n.01'), Synset('corgi.n.01'), Synset('pekinese.n.01'), Synset('mastiff.n.01'), Synset('flat-coated_retriever.n.01'), Synset('sennenhunde.n.01'), Synset('schipperke.n.01'), Synset('english_toy_spaniel.n.01'), Synset('ibizan_hound.n.01'), Synset('airedale.n.01'), Synset('cardigan.n.02'), Synset('miniature_pinscher.n.01'), Synset('bluetick.n.01'), Synset('west_highland_white_terrier.n.01'), Synset('seizure-alert_dog.n.01'), Synset('pomeranian.n.01'), Synset('english_foxhound.n.01'), Synset('bernese_mountain_dog.n.01'), Synset('norfolk_terrier.n.01'), Synset('greater_swiss_mountain_dog.n.01'), Synset('collie.n.01'), Synset('chow.n.03'), Synset('pug.n.01'), Synset('scottish_deerhound.n.01'), Synset('groenendael.n.01'), Synset('golden_retriever.n.01'), Synset('schnauzer.n.01'), Synset('irish_setter.n.01'), Synset('german_shepherd.n.01'), Synset('walker_hound.n.01'), Synset('english_setter.n.01'), Synset('english_springer.n.01'), Synset('sporting_dog.n.01'), Synset('afghan_hound.n.01'), Synset('cairn.n.02'), Synset('rhodesian_ridgeback.n.01'), Synset('chesapeake_bay_retriever.n.01'), Synset('irish_wolfhound.n.01'), Synset('fox_terrier.n.01'), Synset('sled_dog.n.01'), Synset('toy_dog.n.01'), Synset('staffordshire_bullterrier.n.01'), Synset('bullterrier.n.01'), Synset('seeing_eye_dog.n.01'), Synset('samoyed.n.03'), Synset('bouvier_des_flandres.n.01'), Synset('otterhound.n.01'), Synset('kuvasz.n.01'), Synset('cur.n.01'), Synset('guide_dog.n.01'), Synset('malinois.n.01'), Synset('malamute.n.01'), Synset('poodle.n.01'), Synset('curly-coated_retriever.n.01'), Synset('toy_spaniel.n.01'), Synset('basset.n.01'), Synset('toy_terrier.n.01'), Synset('tibetan_mastiff.n.01'), Synset('basenji.n.01'), Synset('field_spaniel.n.01'), Synset('mexican_hairless.n.01'), Synset('setter.n.02'), Synset('great_pyrenees.n.01'), Synset('american_staffordshire_terrier.n.01'), Synset('rottweiler.n.01'), Synset('standard_schnauzer.n.01'), Synset('black-and-tan_coonhound.n.01'), Synset('borzoi.n.01'), Synset('bulldog.n.01'), Synset('sausage_dog.n.01'), Synset('sussex_spaniel.n.01'), Synset('spaniel.n.01'), Synset('working_dog.n.01'), Synset('belgian_sheepdog.n.01'), Synset('watchdog.n.02'), Synset('silky_terrier.n.01'), Synset('eskimo_dog.n.01'), Synset('brabancon_griffon.n.01'), Synset('whippet.n.01'), Synset('plott_hound.n.01'), Synset('liver-spotted_dalmatian.n.01'), Synset('coonhound.n.01'), Synset('saluki.n.01'), Synset('toy_poodle.n.01'), Synset('hearing_dog.n.01'), Synset('chihuahua.n.03'), Synset('lakeland_terrier.n.01'), Synset('shepherd_dog.n.01'), Synset('rat_terrier.n.01'), Synset('italian_greyhound.n.01'), Synset('komondor.n.01'), Synset('saint_bernard.n.01'), Synset('norwich_terrier.n.01'), Synset('pointer.n.04'), Synset('appenzeller.n.01'), Synset('maltese_dog.n.01'), Synset('hunting_dog.n.01'), Synset('smooth-haired_fox_terrier.n.01'), Synset('housedog.n.01'), Synset('boston_bull.n.01'), Synset('pinscher.n.01'), Synset('greyhound.n.01')])

set([])

Now my question:

Are all images of ImageNet in the leaves, e.g. 'harrier.n.02' or are some images only labeled as 'dog.n.01'?

(Sub-question: ImageNet was labeled by ordinary people (not specialists in the topic, e.g. for dogs not biologists with specialization in dogs or something similar) via Amazon Mechanical Turk, if I remember it correctly. How did the people there know all these different kinds of dogs? I wouldn't know what a borzoi is... so how did they check if the label was correct and specific enough?)

Are the 21841 synsets (source) leaves or also inner nodes?

Are all images in ImageNet in the leaves? How many leaves are there?

$\endgroup$
3
$\begingroup$
  1. No, images are present both in the leaves and in the innernodes. A more general category like 'dog.n.01' would be a hypernym to a more specific category like 'harrier.n.02' (which would be one of the hyponyms of dog) but both the synsets would have representative images.

  2. According to this article

    Mechanical Turk brought its own slew of hurdles, with much of the work fielded by two of Li’s Ph.D students, Jia Deng and Olga Russakovsky . For example, how many Turkers needed to look at each image? Maybe two people could determine that a cat was a cat, but an image of a miniature husky might require 10 rounds of validation. What if some Turkers tried to game or cheat the system? Li’s team ended up creating a batch of statistical models for Turker’s behaviors to help ensure the dataset only included correct images.

    According to Karpathy, Turkers query on search engines for images corresponding to a label like 'harrier' and then give a binary 'YES/NO' answer for inclusion of the image in the set. This is a noisy process and sacrifices some correctness to scale at that level.

    If multiple objects are present, it could be classified with the label for a less dominating object in the image since Turkers are unaware of other classes in the dataset and hence wouldn't realize it if a more accurate label exists in wordnet for the given image.

  3. Synsets can be inner nodes or leaves. For example, as of today, the 'dog' synset contains 1603 images whereas its child synset 'harrier' contains 217 images.

  4. All images are not in the leaves. They can be under a leaf or an inner node. The specific number of leaves can be estimated using their api for full hyponym (synset of a whole subtree) on all the high level categories

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.