Benchmarking and Performance Evaluation

The ECVnet Benchmarking Activity has been set up to encourage the development and use of techniques for measuring the performance of computer vision systems and components.

ECVnet resources:

• Survey of image databases for evaluation (in progress)
• Bibliography of evaluation-related papers
• A special issue of Machine Vision and Applications (contents)

Forthcoming and past workshops:

• Workshop alongside CVPR99 in June 1999 in Colorado (call to be announced)
• Call for papers for workshop on vision systems in January 1999
• Workshop alongside CVPR in June 1998 in California (click here for review)
• Workshop in Saarbrucken in March 1998 (programme)
• Workshop in Braunschweig in September 1997 (papers)
• International workshop alongside ECCV in April 1996 (papers)
• Initial ECVnet Paris workshop in July 1995 (minutes)

Performance evaluation-related activities:

• Aerial images: the ARPA RADIUS project
• Cameras: ISO standards in image capture for digital cameras
• Compression (lossy): resource page at University of Waterloo
• Compression (lossless): text/image files of Calgary/Canterbury corpora
• Compression: performance metric references at EPFL
• Contour and corner extraction: Paul Rosin's page at Brunel University
• Digital watermarking: performance evaluation tools
• Line extraction: IAPR TC10 dashed line detection contest
• Line extraction: INSA of Lyon line extraction evaluation
• OCR: CDROM databases at Seattle University (coming soon)
• OCR: NIST databases, tools and reports
• Optic flow: the Barron and Fleet study
• Range image segmentation: University of South Florida evaluation
• Registration of medical images: study on datasets
• Registration: NASA workshop including evaluation apsects (frames only)
• Restoration: evaluation on Hubble Space Telescope images
• Stereo images: ISPRS WG III/3 test of image understanding
• Stereo Images: image sets with ground truth maps at University of Bonn
• Texture: University of Queensland MEASTEX benchmarking

Other web resources:

• DAGM working group on quality characteristics in Germany
• Robust image understanding laboratory at Rutgers University
• Performance characterization page of Visvanathan Ramesh at Washington
• University of Oulu Lumber/Wood grading database
• Videre - an online journal of computer vision research from MIT press
• Online demostrations list at CMU
• Image databases at the Forth ECVnet server
• DARPA Image Understanding datasets website
• Image database at vision1.com

Return to ECVnet home page
For more information and to be added to our mailing list, please contact : Patrick.Courtney@pandora.imag.fr
To be notified of updates to this page, please consider registering with netmind