You are here

C3PI: Computational Photography Project for Pill Identification

Project information

The Computational Photography Project for Pill Identification (C3PI) is creating:

  • The RxIMAGE database of freely available high-quality digital images of prescription pills and associated data
  • The freely-accessible RxIMAGE API (Application Programming Interface) that software developers can use to create apps for text-based search and retrieval from the RxIMAGE database for public use
  • The SPLIMAGE database and SPLIMAGE portal to help the pharmaceutical industry include pill images in the information it provides to the public

By developing and using these resources, C3PI computer scientists conduct computer vision research in text- and image-based search and retrieval. They seek object identification metrics and methods that are invariant with respect to camera angle, lighting, and the color transfer functions often found in digital cameras.

RxIMAGE: DATABASE OF PILL IMAGES AND DATA FOR THE PUBLIC, API FOR SOFTWARE DEVELOPERS TO ACCESS THE DATABASE

  • The RxIMAGE database is the Nation’s only portfolio of curated, freely available, increasingly comprehensive, high-quality digital images of prescription pills and associated data. Examples of pills are capsules and tablets intended for oral use.
  • Photographs of pills for the RxIMAGE database were taken under laboratory lighting conditions, from a camera directly above the front and the back faces of the pill, at high resolution, and using specialized digital macro-photography techniques. Image segmentation algorithms were then applied to create the JPEG images in the database.
  • Software developers can use the freely accessible RxIMAGE API to create apps for text-based search and retrieval from the RxIMAGE database.
  • The general public can use those apps to search for and retrieve images and data from the RxIMAGE database.
  • Usage example: A health care provider uses a mobile device to call up an app that executes an RxIMAGE API request to search the RxIMAGE database for an unknown pill by appearance. That may be the only way to identify the pills an unconscious person or an evacuee has on his or her person during a natural disaster.

PILL IMAGE RECOGNITION CHALLENGE

The NLM Pill Image Recognition Challenge was announced on January 19, 2016, and the winners were announced on August 1, 2016. The Challenge Web site includes information about the Challenge and the winners.

The Challenge invited individuals and teams to submit algorithms and software for pill image recognition. The submissions were to rank images taken from NLM's RxIMAGE database of high-quality prescription pill images by similarity to consumer-quality images of the same pills. NLM plans to use the submissions in creating a future software system for future Application Programming Interfaces (APIs) that can be used in identifying an unknown prescription pill from a photo taken of that pill by a smart phone.

The Challenge was preceded by a Pill Image Recognition pilot. The Pill Image Recognition (PIR) Request For Information (RFI) Pilot and its Instructions for Responding were posted on February 2, 2015.

SPLIMAGE: PILL IMAGE DATABASE AND PORTAL FOR THE PHARMACEUTICAL INDUSTRY

The Computational Photography Project for Pill Identification with its Pill Image Recognition Challenge has reached its conclusion. This service is no longer available from NLM.  A formal announcement is available at the Federal Rgister:  https://www.federalregister.gov/documents/2018/06/07/2018-12273/end-of-the-call-for-participation-for-computational-photography-project-for-pill-identification-c3pi

Questions regarding C3PI and its discontinuation can be sent to nlmlhclhcques@mail.nih.gov

SPL IMAGE GUIDELINES

C3PI developed recommended SPL image guidelines in collaboration with the FDA. The guidelines address image size, color, file format, and file size; layout; scales; and image processing. The overall goal of the guidelines is that a person or a computing system can readily identify a pill’s distinguishing visual characteristics, except for thickness.

DIFFERENCES BETWEEN RxIMAGE IMAGES AND SPLIMAGE IMAGES

The “views” of the fronts and the backs of pills are the same in the RxIMAGE database and the SPLIMAGE database. However, in images in the RxIMAGE database, the background includes an NLM watermark, and other characteristics are different from images in the SPLIMAGE database. Consequently, images in the RxIMAGE database do not meet the guidelines.

RxIMAGE USAGE BY NLM

Images and associated data in the RxIMAGE database are used by NLM’s DailyMed and RxNav.

[1] http://www.fda.gov/forindustry/datastandards/structuredproductlabeling/default.htm

Publications/Tools: 
Caban JJ, Rosebrock A, Yoo TS. Automatic identification of prescription drugs using shape distribution models. In: 19th IEEE International Conference on Image Processing (ICIP); 2012 Sep 30-Oct 3; Lake Buena Vista, Florida. p. 1005-8. DOI: 10.1109/ICIP.2012.6467032