George Grey, CEO of SnapMyLife tells us about his mobile web application and use of Amazon Mechanical Turk to moderate user uploaded photos.
Please describe your company:SnapMyLife is a leading mobile web community for instantly sharing and discovering photos from locations around the world. With just a camera phone and mobile web access, you can share photos, make friends and see location-tagged pictures on maps, add favorites, comments and chat. The company has been in business for about 18 months, and we are growing very rapidly.
How did you first hear of Amazon Mechanical Turk?
To accommodate our growth, all of our services rely on Amazon Web Services – we’re hosting on EC2 and using S3 for all of our storage. As a result, we were introduced to the capabilities of Mechanical Turk and have found it to be an extremely effective tool to moderate pictures in a scalable way.
How did you first use Amazon Mechanical Turk and how has this use evolved?
SnapMyLife’s audience consists of a diverse, global user base, and we go to great lengths to ensure that pictures are appropriate and interesting to users of all ages. Our first batch of HITs involved asking Mechanical Turk workers to moderate photographs submitted by our users. The main goal of using Mechanical Turk for picture moderation is to ensure that inappropriate pictures don’t appear to our audience or the public.
After our successful initial experience with Mechanical Turk for picture moderation, we built a second application that uses Mechanical Turk for picture tagging. When a user uploads a picture on SnapMyLife, we use Mechanical Turk to automatically add tags to that photo. This enables us to find the most interesting and relevant content to users who search for particular topics.
What was your process before using Mechanical Turk?
Before using Mechanical Turk, we manually moderated pictures ourselves, but found it to be a time-consuming activity. As SnapMyLife rapidly grew, an increasing proportion of our time was spent on picture moderation. In addition, keeping coverage 24 hours a day, seven days a week became a growing challenge. With Mechanical Turk, we now outsource or crowdsource the picture moderation aspect of our operations, and as a result, we have simultaneously reduced the load on our internal employees and achieved 24×7 coverage.
What do you estimate your cost savings to be?
Overall, we estimate our cost savings to be about 50% of other outsourcing methods. We are currently at a run-rate of 16 man hours a day in terms of the load that we are putting on to the Mechanical Turk service, and of course, that’s 24×7. The around-the-clock aspect is of particular benefit to us since our service is global and we have users all over the world.
What type of turnaround time are you experiencing?
We are very impressed with Amazon Mechanical Turk’s response times. As photos are uploaded in real time to SnapMyLife, they are immediately transferred to Mechanical Turk. Within three minutes, we are able to moderate a new picture, add the auto tags, and expose it publicly on our site. In essence, users immediately see what other users post. That’s helped us a lot in terms of meeting the needs of our consumers. As mentioned earlier, we estimate that we’re using about 16 hours of Mechanical Turk work a day. If we were to hire a direct workforce to achieve 16 hours a day of uniform coverage through the day, we would clearly need to employ a shift system to achieve 24 hours a day/seven days a week coverage. Therefore the benefit of using Mechanical Turk is that we get access to workers all around the world, available 24×7. There are clear benefits to only paying for the slice of work that we have every time a user uploads a picture. Being able to do so flexibly and scalably is very attractive to us.
What are your costs and how do you ensure quality?
Our average cost per HIT is 1.5 cents, the lowest rate that Amazon makes available. We’re able to achieve a low cost since our questions are relatively simple and very fast to answer. One challenge is maintaining consistency for ambiguous photos – if we ask whether a picture is offensive, different people will answer that question in a different way. In order to ensure greater consistency, we ask the question multiple times and use averages in order to arrive at a reasonable answer.
What do you do to oversee and administer Mechanical Turk?
For overseeing MTurk and the results each week, I estimate that the internal team spends about a tenth of the time that Mechanical Turk spends. In other words, given 16 hours/day of MTurk time, we’re probably spending one to two hours a day ourselves. Most of that time is not spent on actually managing the MTurk process, but dealing with the edge cases and moderation that’s required in cases where we get conflicting results. Our MTurk process is fully automated since we use the MTurk APIs rather than the web service.
How has Mechanical Turk changed your value proposition?
One of the benefits of going to such great lengths to accurately moderate pictures is that it enables SnapMyLife to appeal to a wide range of both consumers and advertisers. In contrast, if we permitted questionable content that was much more racy, it would be difficult to attract premium advertisers to our site.
How have you grown and what are your future plans with Mechanical Turk?
By leveraging Mechanical Turk, we are seeing substantial growth in our service each month. We’ve gone from nothing at inception at the end of last year to a monthly rate of over 1.5 million unique visitors, over 15 million page views, and half a million registered users. We believe that one of the benefits of fully integrating MTurk is that as our business grows, Amazon Mechanical Turk will very effectively allow us to scale the increasing load of picture moderation and tagging. Based on our early positive experiences, we are exploring using MTurk for other similar tasks in the future.
Overall, our use and relationship with both Amazon Mechanical Turk and Amazon Web Services have been a very positive experience. It’s a big benefit to a company like ours, which is dependent on delivering a web service worldwide. Using the Amazon Web Services to host our service gives us scale that would be very difficult for us to achieve on our own, and frees up our engineering resources to innovate and develop our service rather than maintain the service and infrastructure needed to run it.
What advice do you have for those using Mechanical Turk?
Our advice for those who are considering using Mechanical Turk is that it’s an extremely cost effective service for tasks that require relatively quick responses from, and are capable of being done by, a wide range of Internet users. We’ve learned to keep the HIT very simple because that gives us the highest quality and consistency of results. We’ve been very pleased with our use of Mechanical Turk, but we recognize that it may not be suitable for all businesses.
Amazon Case Study: SnapMyLife
Date: 28-08-2013
