Amazon’s AWS cloud compting service on Wednesday morning kicked off its machine learning summit via virtual transmission.
The morning’s keynote talk was lead by Swami Sivasubramanian, AWS’s vice presdient of AI and machine learning; Yoelle Maarek, vice president of research for Amazon Alexa at AWS; Bratin Saha, vice president of machine learning at AWS; and with a special guest appearance by Ashok Srivastava, Chief Data Officer at Intuit.
Sivasubramanian lead off with a talk about machine learning being “one of the most transformative” technologies in a generation. He cited a stat that more than 100 papers in machine learning are published each day. “Machine learning is going mainstream,” said Sivasubramanian. More than 100,000 customers use AWS for machine learning, said Sivasubramanian, citing examples such as pharma giant Roche and The New York Times.
Sivasubramanian Offered examples of “working backward from the customer” in the development of ML. The first example was how a system can “learn with less data.” Accessing and annotating data is “too tedious” as ML becomes mainstream, said Sivasubramanian. He cited the example of the NFL wanting to manage its library of video assets from football games. An 80-year-old pizza maker wanted to ensure every pizza has the same amount of cheese to maintain quality. The company used AWS for an imaging system for pizza inspection. The solution was what’s called “few-shot learning,” where machine learning is supplied with only a limited number of examples.
Few-shot learning is used for custom data labeling in the Amazon Rekognition product, he said. The NFL, for example, assigns custom labels to things such as players and jerseys in video. Amazon’s service for defect inspection, Amazon Lookout for Vision, also uses the service.
Sivasubramanian cited the desire to replicate a real factory setting. So, the team that developed Lookout for Vision built a replica of a factory to try the few-shot approach in the real world.
Sivasubramanian’s next example was understanding “irregular text” with machine learning, where, for example, text is blurred. Accuracy goes way down, he noted. That’s important for real-world instances such as doctors’ handwritten notes.
The traditional language model approach of guessing with the first few letters runs into problems when there is little context. So, the AWS team invented something called “SCATTER,” Selective Context Attentional SceneText Recognizer. It sends an image through additional processing that has a decoder to choose contextual or visual information alone. The SCATTER tech lead to a 3.7% improvement in text recognition, a big improvement, he said. SCATTER is now used in AWS’s automatic text extraction service.
Sivasubramanian then brought up Maarek, to talk about “giving Alexa a sense of humor.” Maarek referenced Alan Turin’s 1950 paper on “Computing Machinery and Intelligence,” in which the mathematician argued against presumptions about computers.
“Think of debuggers,” said Maarek, which is an example of how a computer “thinks about its own thought,” something people thought computers wouldn’t do but Turing said it could. “Already Turing was looking at having a sense of humor being a really hard challenge.”
“We want to look backward, whether customers are funny, and how should the machine respond to it,” explained Maarek. That lead to the challenge of “detecting humor when customers are the one being funny.” To train the system, Amazon looked at humorous customer comments on Amazon. “We actually discovered tons of funny questions,” she said, such as “can you hack into the Matrix” via the Nintendo Switch video game machine, asked one customer. Maarek proceeded to explain the joke…
Another example was customer sarcasm, she said. Will a luxury drink cooler “make me fly.” Said Maarek, “Sarcasm: funny.” Another type of humor is “the superiority theory of humor,” such as asking whether Amazon Show will cook breakfast. Someone asked about the Hutzler Banana Slicer, “will it bend the other way.” ANother example” If a unicorn farts in the woods and no one is around, does it make a sound” (pertaining to Unicorn Meats.)
Maarek said the team built a deep learning model, employing notions about humor such as subjectivity, and using embeddings. “We took into account domain bias, to make sure we didn’t over-fit our model.” As a result the team was able to present a paper with high degrees of humor accuracy at last year’s SIGER conference.
Then, the team moved on to how to detect with speech in Alexa. Would customers appreciate, she asked, Alexa understanding the humor? Or did customers want to feel superior? Maarek cited humorous user utterances toward Alexa, such as “Alexa, can you burp?” “You will see a ton of toilet humor,” she said, “it’s part of a very important area of humor, relief humor.” “Alexa, what is your blood type.” Some examples, she noted, are not so much funny as playful. Such is an example of both personification and superiority by humans. “We defined playfulness,” she said. “The customer doesn’t expect Alexa to take this request literally” and Alexa should not add anything to the shopping list of the user.
Maarek had to go back to the papers about Aristotle, Kant, Schopenhauer, etc., to understand all the forms of humor. Surveying all the forms of humor helped understand the matter of what users will enjoy from Alexa. Will they enjoy if Alexa understand their humor?
The team started with “personification,” where people relate to Alexa as a personality, as a conjecture. They recruited a hundred of students in a blind question-asking exercise, talking to an entity they didn’t know was Alexa (it was labeled as “Shirley,” a play on the movie Airplane.) The students’ questions were examined by a custom version of Google’s BERT transformer neural net. It employed sentiment analysis and such. “We got a pretty good model,” she said, “to detect these funny personification utterances on the fly.” The team went to a speed-dating site to scope out questions people ask when trying to be fun. That lead to a survey of personification questions that people ask “Do you think as good as a woman?” Etc. The result was that all the human questioners enjoyed when Alexa responded. “We really want to have fun not at Alexa but with Alexa,” was the conclusion of the research.
Sivasubramanian then came back on stage and talked about an industrial example: Amazon’s automated fulfillment centers. The company installed 800 sensors on equipment in a fulfillment center in Germany. “We learned a lot,” said Sivasubramanian, including how to reduce faults, and how to better understand the optimal range from a sensor to a gateway. Amazon is going to install “tens of thousands” of its Monitron sensors in the coming months across its fulfillment facilities, said Sivasubramanian.
Sivasubramanian moved on to talk about “horizontal” use cases, where customers don’t have much in the way of ML skills. That includes “embedding” what’s called “autoML,” where customers don’t need to know about model design or tuning. The tech is then used for things such as customer service, document recognition, etc. Sivasubramanian called out domain-specific models for healthcare, such as medical note transcription.
Sivasubramanian called out Amazon SageMaker, the company’s development environment, which is the way the company brings machine learning frameworks such as PyTorch and TensorFlow to data scientists.
Then Sivasubramanian moved on to deploying machine learning at scale, and he invited up Saha. Saha made the point that customers have increased their model deployment from “just a few” in the early days to “thousands” per customer. SageMaker, he noted, now supports hundreds of billions of predictions per month. “From a dozen models to millions of models and hundreds of billions of predictions in just a couple years,” was how Saha summed up the progress.
Saha cited Lyft as a customer. They used SageMaker to reduce model training time for “Level 5” ADAS (self-driving.)
iFood, a leading food delivery company in LatAm, used SageMaker to reduce the travel distance of delivery staff.
Saha cited examples of “using SageMaker in our daily lives.” That includes when you order from Amazon, he said. Amazon needed to integrate and manage tools for fulfillment. That includes “monitoring millions of global shipments annually.” That lead an internal Amazon team to build a computer vision system to scan items at fulfillment centers. That project wasn’t able to handle new requests in production. So, they had to develop ML models offline. That process took three to six months. With SageMaker, said Saha, the team was able to reduce model deployment to two weeks and were able to reduce “prediction latency” by 50%, he said.
“We are building SageMaker along three vectors,” said Saha. “Infrastructure, tools, and ML industrialization.” Saha talked about things built on top of SageMaker, including AWS Inferentia, which is used by customers such as Snap, Autodesk, and Condé Nast for lower cost and higher performance of inference.
Saha talked about the Habana Gaudi-based chips that will be coming to EC2 instances this year. He also called out AWS’s own home-grown “Trainium” chip for ML training, also coming later this year.
Saha moved to talking about the problem of deploying multiple endpoints, one for each ML model. The solution was SageMaker’s “multi-model endpoints,” which allows one to “host hundreds of thousands of models on a single endpoint.” That leads to optimization of prediction accuracy and throughput, he said.
The next technology was SageMaker Clarify, a tool to gain insight into why ML models produce certain predictions. Saha cited the use by SageMaker of “Shapley values,” running experiments on the model or data set to see how predictions improve. Amazon was able to make those Shapley tests run ten times faster than existing open-source implementations, he said.
A third vector, he said, was industrialization of machine learning. “We asked ourselves, How did software go from a niche to an industry?” It involves tools from software such as IDE and CI/CD. SageMaker is the first IDE for ML, he noted. Another analogous tool is the CI/CD tool. Those are rarely available in ML, he noted.
So, Amazon built SageMaker Pipelines, the first purpose-built CI/CD service for machine learning, he said. The tool lets one roll back and troubleshoot ML code at each time step. Customers such as 3M are using Pipelines to let them scale. “With just a few clicks, you can create an entirely automated workflow that reduces months of coding to just a few hours,” said Saha.
Another industrialization avenue is training everyone on ML. Every engineer who joins Amazon has learn ML, he noted. There is a new MOOC, on Coursera, for deep learning. The course, Practical Data Science, is for those ready to implement ML in the enterprise. (Coursera’s founder Andrew Ng is a speaker later today at the Summit.)
Sivasubramanian came back on to introduce Intuit’s chief data officer, Srivastava. Srivastava said the company’s “mission” is “powering prosperity around the world.” He cited stats of tax returns filed via TurboTax (48 million) and “Mint users empowered to make smart money decisions” (over 25 million.)
Srivastava said a lot of that was a result of “AI and machine learning at scale.” Intuit has been working with Amazon AWS since 2018. Intuit has put 250 “AI assets” into production, and is running 2,059 “AI tasks” in production. Intuit has been able to file 600 AI patents in the U.S. in recent years, he said.
Srivastava talked about what he called the AI “hierarchy of needs.” The actual AI model is at the top of the pyramid, the stuff Intuit wanted its developers to focus on, and stuff that is “nonproductive” is the Ml infrastructure below it, and the data infrastructure below that. Those were the things were Intuit relied on Amazon. Srivastava said the company has seen a 60% increase in mobile app deployments. “The benefits to your AI teams are immense,” he said. Intuit has increased by 50% its number of deployed ML models.