Shortly after the launch of ChatGPT, many pundits asked if the company had been slow to get its generative AI tools launched and had left an opening for its competitors on Google Cloud and Microsoft Azure. Garman believes this was more perception than reality, though. He said AWS had offered successful machine learning services, including SageMaker, long before 'generative AI' became a buzzword. He also stated that this company was much more measured about generative AI than perhaps some of its competitors.

"We'd been looking at generative AI before it became a widely accepted thing, but I will say that when ChatGPT came out, there was kind of a discovery of a new area, of ways that this technology could be applied.". And I think everybody was so psyched and energized by it, right? …I think so many people—our competition—kind of just sprinted to put chatbots on top of everything and claim that they're ahead of the curve of generative AI," he said.

I believe that what a lot of people, actually our competition, did was simply rush to be the first in line to slap chatbots on top of everything and show that they were ahead in generative AI.

What Garman said was that the AWS team wanted instead to take a step back and understand how their customers, whether startups or enterprises, could best embed this technology into their applications and then leverage their own differentiated data sets to do so. "They're going to want a platform that they can actually have the flexibility to go build on top of and really think of it as a building platform as opposed to an application that they're going to adapt. And so we took the time to go build that platform," he said.

For AWS, that platform is Bedrock, where it offers access to a wide variety of open and proprietary models. That, by itself, was a bit of an controversy at the time — just doing that, and then letting users chain together many different models," he said. "But for us, we thought probably that's where the world goes, and now it's kind of a foregone conclusion that that's where the world goes," he said. He said he believes everyone would want customized models and bring their own data to them.

Bedrock, Garman said, is "growing like a weed right now."

One thing he still wants to tackle with generative AI, however, is price. "A lot of that is doubling down on our custom silicon and some other model changes in order to make the inference that you're going to be building into your applications [something] much more affordable.".

Amazon Web Services' next generation of its custom Trainium chips, which the company unveiled at its re:Invent conference late in 2023, will launch toward the end of the year, Garman said. "I am really excited that we can really turn that cost curve and start to deliver real value to customers," he said.

One area where AWS hasn't necessarily even tried to compete with some of the other technology giants is in building its own large language models. When I asked Garman about that, he noted that those are still something the company is "very focused on." He thinks it's important for AWS to have first-party models, all while continuing to lean into third-party models as well. But he also wants to make sure that AWS' own models can add unique value and differentiate, whether through the use of its own data or "through other areas where we see opportunity."

Among those areas of opportunity is cost, but also agents, which everybody in the industry seems to be bullish about right now. "Having the models reliably, at a very high level of correctness, go out and actually call other APIs and go do things, that's an area where I think there's some innovation that can be done there," Garman said. Agents, he says will unlock much more utility from generative AI by automating processes on behalf of their users.
 
Q: AI-Powered Chatbot
At the same re:Invent conference, AWS also rolled out Q, its new generative AI-powered assistant. Today, there are basically two flavors of this: Q Developer and Q Business.

Q Developer integrates with many of the most popular development environments, among other things offering code completion and tooling to modernize legacy Java apps.

"We really think about Q Developer as a broader sense of really helping across the developer life cycle," Garman said. "I think a lot of the early developer tools have been super focused on coding, and we think more about how do we help across everything that's painful and is laborious for developers to do?"

At Amazon, Garman said, the teams used Q Developer to update 30,000 Java apps, saving $260 million and 4,500 developer years in the process.

Q Business uses similar technologies under the hood, but its focus is on aggregating internal company data from a wide variety of sources and make that searchable through a ChatGPT-like question-and-answer service. The company is "seeing some real traction there," Garman said.

Shutting down services
While not much has changed since Garman took the helm, something did just happen rather recently on the AWS front: The company said it would phase out some of its services. That's not the kind of move AWS has made very often in the past, but this summer, it said it will close down services such as its web-based Cloud9 IDE, the CodeCommit GitHub competitor, CloudSearch, and others.

It's little bit of a clean-up kind of a thing, where we looked at a bunch of these services, where either, frankly, we launched a better service that people should move to, or we launched one that we just didn't get right, he explained. And, by the way, there's some of these that we just don't get right and their traction was pretty light. We looked at it and said, 'You know what? The partner ecosystem actually has a better solution out there, and we're just going to lean into that.' You can't invest in everything. You can't build everything. We don't like to do that. We take it seriously if companies are going to bet their business on us supporting things for the long term. And so we're very careful about that."

AWS and the open source ecosystem

One relationship that has long been difficult for AWS-or at least has been perceived to be difficult-is with the open source ecosystem. That's changing, and just a few weeks ago, AWS brought its OpenSearch code to the Linux Foundation and the newly formed OpenSearch Foundation.

We love open source. We lean into open source. I think we try to take advantage of the open source community and be a huge contributor back to the open source community.

"I think our view is pretty straightforward," Garman said when I asked him how he thinks of the relationship between AWS and open source going forward. "We love open source. We lean into open source.". I would say that we are really taking advantage of the open source community, and we're really trying to be a huge contributor back to the open source community. I believe that is the whole purpose of open source: benefit from the community, so that is the thing that we take seriously.

He also mentioned that AWS has made some very significant investments in open source and it has open sourced many of its projects.

"Most of the friction has been from companies who originally started open source projects and then decided to kind of un-open source them, which I guess is their right to do. But you know, that's not really the spirit of open source. And so whenever we see people do that, take Elastic as the example of that, and OpenSearch AWS's ElasticSearch fork has been quite popular. … If there's a Linux [Foundation] project or an Apache project or any other place we can lean into, we want to lean into it; we contribute to them. I think we've evolved and learned as an organization how to be a good steward in that community and hopefully that's been noticed by others.