Skip to the content

Machine learning enhanced image recognition & human automation

Fotolia

Fotolia by Adobe is Europe's leading online stock image company. It currently offers over 57 million high-res, royalty-free images and videos available to license.

Fotolia by Adobe is Europe's leading online stock image company. It currently offers over 57 million high-res, royalty-free images and videos available to license.

Problem / Opportunity

Fotolia has a vast, ever expanding picture catalogue. To make these pictures discoverable to users requires very detailed categorisation along many dimensions. In this instance, they needed pictures of people categorised along with the number of people in picture, ethnicity, gender and the age group of each person. Fotolia used to employ a dedicated workforce to enrich their pictures with this highly specialised data. The workforce could not keep up with new image influx nor was it run cost effectively. This is when they approached us.

Innovation Team

We teamed up their search and data owners with our machine learning and image recognition experts to form the innovation task force. We sketched out the integration story, suitable outcome data format and ideal cost per categorised picture so that the solution would pay for itself in a short time.

Tech

We started modelling the decision tree each picture would have to pass to be properly categorised. We ran several tests through a variety of image recognition systems. The results we gathered made it clear that we would also need to utilise machine learning techniques in order to increase the accuracy and improve results over time. Essentially the output from the image recognition systems was often ambiguous. However, there were observable patterns in these ambiguities. Using this insight, we assembled a very large set of training data which had been pre-categorised by Mechanical Turk. These images were then run through image recognition and we were able to build a machine learning model which plotted the patterns in the ambiguous image recognition data and their relationship to the correct categorisations produced by mTurk.

The entire system was exposed via APIs enabling new pictures to flow from the Fotolia website through the machine learning categorisation and be passed back to the website in a matter of seconds.

Impact

Fotolia used the system to categorise 250k pictures in the first month, for just under 6 cents per picture. This means they saved nearly 90% of the cost for picture categorisation. With our image pre-processing and dynamic quality control, we were able to lower the categorisation error rate by 20%.

Methodology Fotolia Created with Sketch. categorisation projects Multiple concurrent Expanded task brief / picture 250k images at <6 cents CONTINUOUS DELIVERY MTurk Task Pipeline Reinforcement training pre-categoriser ML based image ALPHA Hack and Craft Fotolia Picture Search Team Fotolia Data Owners Team members PROJECT ROADMAP 3. Scale 2. Development 1. Ideation
Dev Fotolia Created with Sketch. COMMON JSON RESULT FORMAT CATEGORISER TASKS GROUP CATEGORISER TASKS INDIVIDUAL PEOPLE CATEGORISER MACHINE LEARNING TASK 3 TASK 2 TASK 1 Fotolia

All case studies

Close
See all case studies (14)

Outro

Science and technology are the principal drivers of human progress. The creation of technology is hindered by many problems including cost, access to expertise, counter productive attitudes to risk, and lack of iterative multi-disciplinary collaboration. We believe that the failure of technology to properly empower organisations is due to a misunderstanding of the nature of the software creation process, and a mismatch between that process and the organisational structures that often surround it.