Release April 15, 2008 ![]() ![]() |
MATLAB Toolbox for the LabelMe Image
Database
|
Citation If you use this dataset or the functions on this toolbox, we would appreciate if you cite: B. C. Russell, A. Torralba, K. P. Murphy, W. T. Freeman, LabelMe: a database and web-based tool for image annotation. International Journal of Computer Vision, pages 157-173, Volume 77, Numbers 1-3, May, 2008. (PDF) Contribute to the dataset If you find this dataset useful, you can help us to make it larger by visiting the annotation tool and labeling several objects. Even if your contribution seems small compared to the size of the dataset, everything counts! We also welcome submissions of copyright free images. Your annotations and images will be made available for download inmediately. Here there are some examples of annotated images:
Content
Download the LabelMe toolbox Download the toolbox and add it to the Matlab path. The toolbox allows you using the dataset online without needing to download
it first. Just execute the next lines to visualize the content of one
of the folders of the collection: 3. List of functions in the toolbox Introduction LabelMe tools for reading and ploting the annotations of individual files LMread - reads one image and the corresponding annotation XML file LMplot - image and polygons visualization LabelMe Database tools LMdatabase - loads all the annotations into a big database struct LMdbshowscenes - shows thumbnails for the all the images in the database LMdbshowobjects - shows crops of the objects in the database. LMobjectnames - returns a list with the name of all the objects in the database LMobjectindex - retuns the indices of an object class within the annotation struct LMcountobject - counts the number of instances of an object class in every image Search tools LMquery - performs a query on the database using any field LMobjectpolygon - search for objects using shape matching qLabelme - search using the online search tool (it does not use the local index) Image manipulation LMimread - reads one image from the database LMimscale - scales the image and the corresponding annotation LMimcrop - crops the image and the corresponding annotation LMimpad - pads an image with PADVAL and modifies the annotation Annotation consistency and synonym lists LMreplaceobjectname - replaces object names LMaddtags - Replaces LabelMe object descriptions with the names in the list tags.txt. You can extend this list to include more synonyms. Use this function to reduce the variability on object labels used to describe the same object class. However, the original labelme descriptions contain information that is more specific and you might want to generate other tag files to account for a specific level of description. Objects, polygons and segmentation LMobjectpolygon - returns all the polygons for an object class within an image LMobjectboundingbox - returns bounding boxes LMobjectmask - returns the segmentation mask for all object instances of one class within an image LMobjectsinsideimage - removes polygons outside the image boundary LMobjectcrop - crops one selected object LM2segments - transforms all the labelme labels into segmentation masks (it takes into account occlusions). LMsortlayers - returns the same annotation file, but the objects are sorted by depth. Creation of training and test images and databases LMcookimage - reformat an image and annotation to fit certain requirements. LMcookdatabase - create tuned databases (you can control the difficulty, the object size, ...) Communication with online annotation tool LMphotoalbum - creates a web page with thumbnails and connected to LabelMe online. LMthumbnailsbar - creates a bar of thumbnails connected to LabelMe online urldir - like 'dir' but takes as argument a web address LMgetfolderlist - returns the list of folders from the online database Install and update images and annotations LMinstall - installs the database LMupdate - authomatic update of the annotations for specific files LMprogressiveinstall - will update your local copy of the images with only the new images and also download all of the annotations Translation from/to other formats PAS2LM - Translates PASCAL format to LabelMe LM2OpenCV - will output a query in format usable for OpenCV Haar Evaluation for object detection LMrecallPrecision - Precision-recall curve XML tools (translates the XML files into MATLAB struct arrays) writeXML - translates a struct array into a XML file loadXML - read XML file drawXML - shows the image and plots the polygons xml2struct - translates a XML string into a matlab struct struct2xml - translates a matlab struct into a XML string Object detection code A simple object detector using boosting - object detection with boosting trained with this dataset |
First, you need to download the LabelMe database or you need to provide the url of the images and annotations online. If you are going to use the dataset frequently, it is better that you get a local copy. Define the root folder for the images 4.1 - Basic functions: indexing, queries The most common functions are LMdatabase, LMimread, and LMquery.
![]() % You can query the database looking for specific objects and visualize each instance in isolation: LMdbshowobjects(LMquery(database, 'object.name', 'bottle'), HOMEIMAGES); ![]()
As there are not specific instructions about how labels should be introduced when using the online annotation tool, this results in different text descriptions used for the same object category. For instace, a person can be described as a "person", "pedestrian", "person walking", "kid", etc. The function LMaddtags replaces LabelMe object descriptions with the names in the list tags.txt. You can extend this list to include more synonyms. Use this function to reduce the variability on object labels used to describe the same object class. However, the original labelme descriptions contain information that is more specific and you might want to generate other tag files to account for a specific level of description. 4.3 - Searching objects by viewpoint Some objects have information related to viewpoint in the annotation file. In the current setup, viewpoints are discretized in 12 different orientations. For instance, for cars we have: In order to get all the frontal cars you can use the query: D = LMquery(Dlabelme, 'object.name', 'car+az270deg''); We have a tool that allows labeling viewpoints for new objects. If you want to add viewpoint information for new objects send us an email and we will send you a link to the tool. By using the online tool, the data that you will provide can be used in the future by other researchers. 4.3 Searching fully annotated images
4.4 - Depth ordering Frequently, an image will contain many partially overlapping polygons. This situation arises when users complete an occluded boundary or when labeling large regions containing small occluding objects. In these situations we need to know which polygon is on top in order to assign the image pixels to the correct object label. The function LMsortlayers will sort the polygons accorting to their relative depth ordering. The function uses some heuristics (see paper) and it will not be correct all the time. 4.5 - Object part hierarchies Despite that object parts are not explicitly labeled as such by the annotation tool, it is posible to authomatically discover object-part hierarchies. When two polygons have a high degree of overlap across many images, this provides evidence for an object-part hierarchy. The toolbox provides a set of functions to suggest parts for an object category. The function partsof will add a new field to each object. The online query tool shows object parts whenever they are labeled. Here there are some examples:
In order to add parts to the object 'car', run the next line: D = partsof(Dlabelme, 'car', 'wheel,door,window,tire,mirror,license+plate,windshield,headlight,light'); 5 - Communication with the online tool You can look in the demo.m file, at the end there is a description that
explains how can you talk, using Matlab, with the online annotation tool
to label only the images that you want and how to get back just those
labels and update you annotation file. The effect is like if the annotation
tool was just working for you, but still all the annotations that you
will enter will be shared and stored online. ![]() |