How Amazon Automates Work in Its Corporate Offices: A Conversation With Elaine Kwon
The question isn’t what’s going to get automated. It’s what’s going to get automated last.
As I wrote Always Day One, my book about tech giant culture, I learned of a massive automation program inside Amazon’s corporate offices called Hands Off The Wheel. The initiative took tasks once performed by white-collar employees within Amazon’s retail organization — core work like purchasing, price negotiation, merchandising, and product promotion — and turned them over to the company’s machine learning systems. Elaine Kwon, co-founder and partner at Kwontified, was a vendor manager at Amazon when the company turned her tasks over to the machines. She joins Big Technology Podcast to discuss the experience and what we can learn from it.
Alex Kantrowitz: You joined Amazon in 2014 as a vendor manager. What does a vendor manager do?
Elaine Kwon: In theory, the vendor manager is supposed to be the buyer. And there was a little bit of buying involved, especially within the fashion realm, which is where I was working.
Buying means you’re on the phone with brands like Gucci and Versace saying ‘We need this many handbags, at this price, at these fulfillment centers?’
At the time, a lot of Amazon’s buying spirit was very much about the “Everything Store,” which is: buy one and see if it sticks. Buy one of everything, see if it sticks.
At Amazon, you’re dealing with fulfillment centers that can fit an entire Sunday’s worth of NFL games. So you were just out there buying like crazy and see what the internet demands?
Exactly. The thought process was that we’re Amazon, we didn’t really care about being profitable. We just wanted to be the best e-commerce site in the world. The customer will know what they want, so how about we just have everything and they’ll tell us what we should sell.
A lot of things started changing in 2016 where you start realizing, okay, there’s a lot of inventory sitting at these warehouses because we decided to buy one of everything. A lot of it isn’t going anywhere and what do we start doing about that?
While you’re doing the job, how are you placing the orders? What does this job look like from a practical standpoint?
Every single brand is different. You spend some time working with your portfolio of brands, understanding what their goals are and then ultimately how Amazon can fit into that. It’s a little bit of push and pull. Now, for a lot of brands, it’ll often be a push from my side, meaning — Hey, I really want you to grow this much. In order to do that, these are all the things that we have to do or want to do.
What would those things be?
Price negotiation, preparing for peak seasons in advance and trying to be strategic about it. Sometimes it’s a matter of inventory, logistics, excellence. Understanding that — Hey, we’re opening up a new fulfillment center in this region of the country. We want to make sure that your products are there. Let’s make certain moves, supply chain-wise. So it involves every aspect of what it means to thrive within the e-commerce space today.
But I think where it got difficult is Amazon realized we had to keep more of the money that we’re making, and a lot of things started to change. Meaning even that role, like the vendor manager’s role or what that team’s directives were, one of their new goals was we have to become more profitable. Our category of products has to become more profitable. Our transactions, every single one, are now being measured with a new profitability standpoint.
Then, probably around the time that you started, folks on the machine learning side of Amazon say, “What if we automate these tasks so we can do this as efficiently as possible with the best margins?” How does this stuff start to become operational, and what are the pros and cons?
I remember first hearing the phrase “Hands Off the Wheel,” I think it was late 2015, early 2016. And it was interesting because it was discussed very laissez-faire, very nonchalant, as if…
Don’t worry about these machines.
Exactly. It’s like, this is just something that’s happening. It’s going to change, but it’s no big deal. But I feel like if anyone is taking a critical look at the situation, you can see the writing on the wall, which is, stuff’s changing. The priorities that we had are changing. What does this mean for the work that we used to do in a particular way?
The very first thing that they automated were product promotions. The creation management and execution of things like discounts, pricing changes, coupons, that sort of thing. It felt pretty innocuous. I think most people were like, “Okay, this is just a new tech change. No big deal.” But at least for me, I look at it and I’m like, “All right, this may not seem like a big deal to some, but for me, I saw that as they’re trying to make some of what people are doing every day no longer part of their job. So what’s next? This is just the tip of the iceberg. Right?
One caveat I think we should make is that when we talk about automation at Amazon, we’re not talking about automating something for everyone, for everybody. We’re talking about automation so that Amazon no longer has the responsibility of having to do a certain thing.
So the brands on the other end still have to enter the inputs, but Amazon would basically take the human activity on its end out of it?
Exactly. They no longer have to staff the person, the people, the team, the overhead, in order to accommodate what was happening manually before. What ended up happening, and is still the case to this day, is that they pushed a lot of that work onto brands, manufacturers, and sellers with very little—actually no forewarning, very little training, very little communication. They just woke up one day, logged into their Amazon portal and realized, oh, what’s this?
So somebody from Gucci, let’s say, who’s been speaking with you, all of a sudden is negotiating with a portal because machine learning is doing this on the backend?
Yeah, I would say so. And certain things came more gradually than others, but the long story short is essentially that that’s exactly right. Even to this day, we have a great partnership with Amazon, my company that is. And we talk to them all the time, but they’re very open and honest about their limitations in a way that I think we felt a bit uncomfortable to do back then. So now they’re very transparent. They’re like, “I have no control over that. I have no influence over this. I have no way of helping you with this.”
Sometimes it baffles companies on the other side because they’re like, “Wait a second, you’re actually in charge of this that we’re discussing. What do you mean you can’t change it? What do you mean you can’t make an exception for a mistake that’s actually wrong?” And that’s the conversation I’m having very often.
Tasks beyond price negotiation were automated as well. Back in the day you would send a purchase order for a certain amount of inventory, and now it just showed up in people’s inboxes?
They always showed up in people’s inboxes, but there were usually many people involved before. Usually, it would involve me as someone from the vendor manager team. It would involve the in-stock management team whose entire job was to create, manage, watch, and organize all the purchase orders that are happening at any second. So there used to actually be a set of people that you could talk to. If you said, “Hey, this new thing is happening, a new trend that we’re watching in the market. We want to make sure that we buy enough of X, Y, and Z product for this upcoming season.” You could actually do that, but not anymore. Not once Hands Off the Wheel really got implemented and not once it hit inventory and ordering algorithms.
So everything suddenly became — No, we can’t touch it anymore. Our algorithm is either going to order it or it’s not. We can only hope to try to influence it and hope that what we like, the poking and prodding will influence the animal to behave in a certain way, but we actually have no way of guaranteeing whether certain changes will happen.
Was there a moment of realization where it was like — Oh machine learning is in charge now and it’s our hands off of the wheel?
There were mixed reactions. I think some folks were righteously frustrated, annoyed, angry that thoughtfully laid out plans were suddenly out of their hands. Some of them, including myself, started realizing, okay, this is the end of the line, essentially. It’s not that this role is going to disappear forever and it hasn’t. There are still vendor management teams in existence, but this role is going to change a lot. Instead of having full autonomy over what we’re going to sell or how we’re going to sell it, down to just how they’re going to be merchandised on a detail page, all of that is going away. So what does that mean for me?
I know that for myself, one of the reasons why I took the job was because of that autonomy, was because of the freedom and the ability to really build these partnerships and say, “How are we going to help each other achieve amazing growth together?” When you start taking away all of the tools that the team has in order to achieve any of those goals, now it starts feeling more and more like you’re still being asked to hit astronomical goals, but you have very, very little ability to change anything to achieve them, if that makes sense.
For me, it’s not something that I think spending my time on would be worth. So that’s when I started realizing, all right, we need to find a new way to help brands grow, that’s actually where I started thinking about the future of e-commerce. What does this mean? Because this is not just Amazon. Every single e-commerce marketplace is trying to do both things. They’re trying to grow and trying to scale how they can show the diversity of all their products online. That’s not a small task.
So this is not just an Amazon problem we’re talking about. Amazon may have carried it out in a certain way, but Zalando or Alibaba, Mercado, Libra, you’re talking about the fact that all these companies are fighting to figure out how do we show this product off in the best way possible to convince you to buy it from us while still scaling how we do all this work on the backend to cost as little as possible.
Zalando, the German e-commerce company you mentioned, hired Ralf Herbrich, who headed up this whole program inside Amazon. I’m sure his minions are doing the same thing in all the companies…
Oh yeah. It’s very much happening.
Speaking with you, I don’t hear a lot of ill will toward Amazon, even though, when people speak about automation they’re fearful of what happens to the worker. You seem to have a pretty sunny attitude?
I’m a fairly practical person. Even back then, when I saw what was happening, I didn’t get mad about it. I wasn’t angry about it myself. It was just more like, all right, this makes sense. I understand why this is happening. Now, is this going to hurt some people, some businesses? Absolutely. But for myself, I looked at it as an opportunity because these are what e-commerce marketplaces and companies are doing in order to survive themselves. But that doesn’t mean that the brands and manufacturers and sellers are all alone by themselves without any aid.
That’s where my thoughts went, which is okay, this is going to be a growing chasm in this industry. How do we bridge that gap? What do we need to do in order to solve this? Because this problem may be small today, but it is going to grow and sure enough, five, six years later here we are. And like I said, even this week, I’ve had three conversations with different CEOs about issues related to stymie, or stemming from this exact thing.
So if I’m a vendor manager inside Amazon, what did it look like from my end? How does my life change when tasks get automated?
There were steps that we no longer did anymore. We would go through, line by line, and actually talk about products and what they were going to offer to the customer and decide, do we want to buy it? Yes or no. And again, not super familiar with how the tools may have changed, but my understanding of what we saw happen back then was that the vendor, the brand now creates their new product listings. Whatever they want to sell on Amazon, they create them on their own, they do all that work by themselves, they submit it. If once it’s processed and created by Amazon, which is a process that the vendor manager doesn’t touch anymore, then the algorithm will decide based on the product information submitted, whether or not they want to buy.
From my understanding, Amazon didn’t go out and do mass firings of vendor managers. A lot of people ended up moving into different roles inside the company. What happened there?
Correct. It’s very much sort of choose your own adventure. At least from my experience, they were not going to fire anybody. It was the people that worked in these roles, I’d say for the most part, very talented, very smart, very driven, great. And specifically, at Amazon there’s this idea that if you’re hired into Amazon you’re capable enough to serve in other roles, other teams, other functions, even if you may not have a ton of experience with a particular skill set yet.
There was a cool opportunity for many that was, you know what, I’m going to go interview some other teams internally and see if there’s a different role that might appeal to me. So that’s what I would say many did. A small handful stayed behind within the team itself. Folks like me, I ended up getting poached by a startup at the time and took that route. All the while, still thinking and obsessed with this problem, which is why we’re here today and what I’ve been working on ever since.
It wasn’t a wide public announcement, but I think everyone, again, realized okay, this may be the end of an era. So take it, keep your eyes open, and start thinking about what your next path really looks like.
There’s a lot of criticism of Amazon, a lot of it well-deserved, but there’s some business brilliance there. What could the downside of Hands Off The Wheel be?
I agree. I think that there’s a lot of really interesting and very applicable principles that even I’ve taken into my own business practices, but there’s also a lot of warnings too. I would be the first person to tell you that guess what, there’s still a vendor management team in existence. But the problem in my opinion is that there are so few of them that are now still expected to manage huge goals, huge portfolios of brands.
But because they’ve been given these Hands Off the Wheel automated tools I think the expectation was, well, you should have no problem managing 100 times what people used to before, because you won’t be doing all of these things that people used to have to. But the problem is that automation or any tool that is automated, a piece of work is only as good as the complexity of the rules designed within it.
The fact of the matter is Amazon has so many products. The diversity, the range of products really cannot even be fathomed. So every rule we’ve seen goes into part of whatever tool was being automated and makes mistakes. It makes a lot of mistakes. Even right now, we’re finding that thousands and thousands of products are being mistakenly flagged as pesticides because they implemented a rule to look for anything that could be a pesticide and suppress it, keeping it off the site.
And now you have pearl accessories, headbands, hand creams, even T-shirts…. They’re all being flagged as pesticides. So I give this example to illustrate this idea that there is a lot that can be automated, but when you’re dealing with the level of scale that Amazon’s currently operating at, even the smallest rule can have devastating effects if not managed incredibly well.
So what does this portend for the rest of the economy? Can people who aren’t working for the tech giants anticipate this in the workplace at some point?
Yeah, that’s a great question. And I think it’s one that I have very frequently because as you can imagine, our company is all about this. We are here to help all the brands, manufacturers, and sellers of the world thrive within these spaces. It can be done, but it is incredibly difficult for a single brand or manufacturer to do alone, exactly because of what you just described, which is that very few companies have the software, the data, the expertise, as well as the know-how to navigate what is actually amassed for each of their accounts.
Imagine a brand that sells on Amazon, Walmart, Zalando, Tmall, and Rakuten, it’s just a fraction of the incredible e-commerce marketplace that they could be selling. Let’s say just five, each of those five is its own maze, in that each of these companies was built independently from one another. All of their algorithms on the backend are very unique and specific to that marketplace and that particular piece of software. So from the consumer standpoint, it feels like it’s pretty similar. You can go from Zalando to Amazon and you’re like, “All right, I search for something, I add to cart, I’m done.” But that’s where the similarities pretty much stop.
We talk to and work with again, multinational, publicly traded companies. I would say it’s kind of interesting because they tend to be the first to realize we don’t have what it takes to do this. So they’re usually the first to reach out and work with companies like ours. I would say medium-sized businesses depending on the leadership will reach out immediately or they’ll usually try it on their own. And when things start getting difficult, stagnating, or failing for whatever reason, that’s when they realize we need help.
Then last but not least, we’re seeing a proliferation of awesome small businesses, startups, VC-backed businesses that realize that they know enough to realize they don’t know enough to do this internally and so that’s usually when they reach out. But frankly, the best defense is a killer offense, right?
And I feel like it’s a combination of data analysis and expertise that comes from having an actual team of people that have done this. Not just once, not just twice, but hundreds of times, over many, many years combined with software. And that’s what it takes today.
Where do you think automation goes? Because right now it seems to be in some small number of companies, but people are predicting it’s going to go much broader.
I think the question that we actually ask ourselves isn’t what’s going to get automated, it’s what’s going to get automated last? That’s actually the conversations we have internally. Because especially when you’re in the position we’re in, which is that we work with incredible companies, you realize everyone’s different, but also a lot of what’s happening is not unique.
I think that’s really where the question is. It’s like, what’s the last thing that’s going to be automated. And in my opinion, it would likely be the most difficult thing, which is critical thinking, adapting to new pieces of information and data. I would say Covid has very much thrown all of the forecasting that was run by machine learning into the drain. And that’s been one of the really fun, difficult, but very interesting challenges that we’ve been helping brands with, which is, how do you figure out what’s going to happen when no one knows what’s going to happen.
I would say it just goes to show you, again, that I feel like it’s a long while before we see the level of automation that I think has been depicted in movies and television, but it’s coming and it’s happening step-by-step.
How should people look out for the potential stuff that might be automated and what skills should they be investing in to make sure they can thrive in a situation like this?
I was actually speaking to a graduating class at a university recently. Young people are wondering exactly that. They see all the changes that are happening and they’re wondering, shoot, do I have what it takes to truly make it today? What do I need to know that perhaps I wasn’t taught in school? And frankly, there are two things. One is critical thinking, which I briefly mentioned before, but that is actually the number one trait that I know. For example, we hire and we work in the business of automating a lot of things for others, but that is one factor that frankly, most machines still have not come even close to touching. Being able to actually look at a situation with all of the nuances and complexities at hand and communicate real-time what’s happening and what should happen.
That’s another level that again, I think, is going to be in demand for quite a while yet. The second also may sound counterintuitive, but it’s actually the ability to build trust. And the reason why I say that is because as long as companies are still being run by people, the number one detracting factor that I see coming between two companies or different partnerships is really whether or not the companies understand one another and are able to trust one another. So I say, if you have the ability to connect and truly build trust in a way that goes deeper than just superficial connection, you can find amazing opportunities to survive beyond the age of automation.
I love how you said, ‘as long as companies are still being run by people.’ Maybe one day that will be different.
I’ll sit on the side and take notes for the machine.
Long-term, are you bullish on the future of work and people’s ability to find meaningful employment despite the machines, or are you betting on the machines kicking our butts?
That is a great question. I certainly hope they don’t kick our butts, but I do think that future work is going to require that everyone, every job, every function that we all have is going to incorporate a certain level of technology that we never have had before. And I think that even the way that we are currently adapted to work today is just one step in that direction. But even let’s say, what we would call traditionally non-technological jobs like being UPS delivery drivers, even that today incorporates a level of using technology. We’re only going to see that increase to the point where again, the things that humans are left doing or the things that only humans can do.