LOADING ...

How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS

476K+ views   |   9K+ likes   |   230 dislikes   |  
13:47   |   Jul 18, 2019

Thumbs

How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS
How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS thumb How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS thumb How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS thumb

Transcription

  • Cloud computing is taking over.
  • Demand continues to rise from both companies and consumers that rely
  • on remote storage and computing power accessible from anywhere.
  • Tech giants Google, Microsoft, IBM and others are vying to be the
  • go-to providers.
  • But one company has remained the leader, Amazon.
  • AWS has a commanding lead in the cloud right now.
  • In fact, if you add up number two, three, four and five, they add up
  • to what AWS does.
  • Amazon Web Services is behind a lot of the technology we use.
  • From calling a Lyft to checking your video doorbell, to streaming
  • your favorite shows.
  • When people are watching a Prime movie, or they're watching a Netflix
  • movie, or a Hulu movie, or others like that, they're watching it and
  • streaming off of Amazon Web Services.
  • The Super Bowl streams off us and also Major League Baseball and now
  • NASCAR and Formula One racing as well.
  • If you use Intuit to do your taxes, that runs on AWS.
  • AWS has been one of Amazon's most profitable business endeavors.
  • Last year AWS generated more than $25 billion in sales.
  • Plus, they're still growing like a weed.
  • Get this, they're up 47%.
  • In the first quarter of this year, revenue climbed to $7.7
  • billion, up from $5.44
  • billion a year earlier.
  • We have over 2.2
  • million customers using AWS today.
  • They're usually big companies like Goldman Sachs or Capital One.
  • There's over 4,000 government agencies that run on us today.
  • Companies left and right are abandoning their own data centers for
  • Amazon's or other cloud providers.
  • But moving all of that data online can be a challenge.
  • The transfer fees for moving data over the network online can be quite
  • high. And also, it can take a while if you have petabytes and
  • petabytes and petabytes or yottabytes of data.
  • So Amazon built physical products to make transferring large amounts
  • of data easier.
  • A portable data transfer device capable of operating in a war zone,
  • called Snowball.
  • And even a giant truck called Snowmobile to help companies migrate
  • their data to the cloud.
  • What about if I have exabytes of data?
  • We have a lot of customers who have exabytes of data.
  • And the first thing that came to mind was, we're going to need a
  • bigger box.
  • So why would a company need to move to a cloud service provider like
  • AWS?
  • Most of our customers save between 22 and 54 percent versus running
  • all in, building their own data center, building their own networks,
  • powering it, having people to operate it.
  • One of the biggest reasons that people look to the cloud is not
  • necessarily cost, but around flexibility.
  • Developers can get access to massive amounts of compute and storage
  • and networking resource.
  • AWS says it has the largest global infrastructure footprint of any
  • cloud provider, meaning it has data centers placed in regions around
  • the globe where there is concentrated demand.
  • It has the capacity to allow companies to tap into more server space
  • depending on their needs.
  • When they have a big retail day they can use a million servers, when
  • their normal load is, say, 40 or 50 or 60 servers.
  • And so the ability to do that is astronomically expensive to do on
  • Prem. And that's why you see the startups growing so fast on AWS
  • because they get the access to a Fortune 500 infrastructure for
  • pennies on the dollar.
  • Netflix, for example, has always used Amazon as its cloud provider.
  • But for a company that wants to migrate its data to the cloud,
  • typically a massive data transfer needs to take place.
  • Some companies have hundreds of terabytes, petabytes and even exabytes
  • of data.
  • For some perspective and how big that is, your average MP3 song is
  • about three megabytes.
  • A gigabyte is about a 1,000 megabytes, or around 300 songs.
  • A terabyte is about a 1,000 gigabytes, or 300,000 songs.
  • A petabytes is 1,000 terabytes, or 300 million songs, and an exabyte
  • is around a 1,000 petabytes, or 300 billion songs.
  • A single MP3 file might take a few seconds to transfer over the
  • internet. 300 million or billion, however, might take a while.
  • It's often called in IT the python eating a pig problem.
  • So if you imagine a python, you can visualize it eating a pig.
  • You get this big lump that you have to move through the python.
  • So you have a little network and you've got a big lump to move and so
  • for some of our customers it would've taken them years and years to
  • upload their data over their network.
  • Amazon has tried to solve this problem of cost and time by creating
  • really tough hardware, called Snowballs, which people who operate
  • data centers can connect their infrastructure to.
  • Make copies of the data and then send those snowballs to AWS data
  • centers so that the data can be moved more quickly.
  • The smallest storage Snowball we have is about 50 terabytes.
  • That's 5,000 DVDs and the largest snowballs we have is between 11,000
  • and 14,000 DVDs depending on how you compress it.
  • We work with our Lab 126 folks on the industrial design, our Kindle
  • folks on the e-ink label.
  • So imagine if you're shipping hundreds of these, you could easily put
  • the wrong label, put them in the wrong box.
  • That doesn't happen, it's all automated.
  • It knows where it's going and it labels itself.
  • And my boss, Charlie Bell, worked on the Space Shuttle and we
  • actually used some things off the Space Shuttle where they have to
  • handle the shock of launch and landing.
  • Vass said designing the Snowball to withstand the rigors of transit
  • was not an easy task, since it had to be highly durable as well as
  • less than 50 pounds.
  • We actually went to our shipping partners and we also went to the
  • fulfillment center and talked to them.
  • So from that we learned a really hard problem to solve is that it had
  • to be under 50 pounds.
  • We also wanted people to be able to check it as regular luggage.
  • And that's actually a hard design constraint, to make something as
  • durable as that and as dense compute and storage, in under 50 pounds.
  • The Snowball even passed an explosives test and meets the military's
  • requirements for being airdropped.
  • To meet the specifications, we have to drop the Snowball from 28
  • feet, 80 times, on all four corners and all six sides.
  • And then because we build it so robustly, we are able to also pass
  • the DoD 901 Barge Explosive Test, where you have 83 pounds of plastic
  • explosive going off 20 feet from the device multiple times.
  • Which is a tremendous percussion wave that would turn your insides to
  • jello. If you were standing there it would kill you.
  • And temperature wise, it's designed for the most extreme environments.
  • They can operate at really high temperatures, like 140 degrees
  • ambient temperature and really cold temperatures, like -20.
  • And it can have unconditioned power from a generator and it'll
  • continue to operate.
  • For customers calmly transferring data from the safety of their
  • office, this could all seem like overkill.
  • But in certain instances, it's proven critical.
  • When the volcano was going off in Hawaii, the USGS used the
  • Snowballs. They had local servers and the lava was coming up on their
  • building. And so they didn't want to lose all that extremely valuable
  • data they've collected.
  • They also knew the Snowballs could operate in high temperature
  • environments. And so they shipped the Snowballs there, downloaded the
  • data and shipped the Snowballs out.
  • And so they were able to capture all that data without losing it.
  • Oil rigs is another area we see a lot of them.
  • Military they're very, very popular.
  • So they're used in forward deployed units.
  • They're used on Navy ships.
  • They're used in aircraft.
  • They're used in Special Ops locations all over the world.
  • For cybersecurity, where they're collecting network data and reacting
  • to it locally.
  • There's even a six micron dust filter option you can snap on the
  • front. So if you're operating them in a desert, they can filter the
  • sand out and not have the sand go into the device.
  • Even Hollywood has started taking advantage of Amazon's Snow family.
  • For studio shoots, they're shooting in 8K and 12K cameras now.
  • And so that's a lot of data.
  • And so they put it on the Snowballs, and you can see the screens on
  • the front they're used for quality control.
  • When they're done with the shoot they actually ship the Snowball back
  • and it uploads the data into the cloud and then they post-processing
  • it.
  • There's another, I would call the upper-sell version, which is the
  • Snowball Edge, and that's a 100 terabyte solution.
  • Now, the interesting thing about Snowball Edge is you can actually
  • put compute on there and actually run workloads.
  • AWS was the first public cloud provider to make hardware like this for
  • data transfer, but competitors have since developed similar products.
  • Microsoft is the number two player in the public cloud market, behind
  • AWS. It has Data Box products that have room for 1 petabyte of data,
  • making it larger than what Google and IBM offer today.
  • The Google Cloud, which is behind AWS and Microsoft Azure, has
  • Transfer Appliance products.
  • Which are storage servers that you can install inside a rack in your
  • data center.
  • But it's not as popular as the one that AWS is offering
  • today.
  • But AWS is the only company that felt like it needed to go even
  • bigger. Snowmobile has the equivalent of 1,250 Snowballs in it.
  • And so it's what we call a 100 petabytes truck.
  • To put in context how much data Snowmobile can take.
  • Let's say the typical notebook is 500 gigabytes.
  • A 100 petabytes would be 200 million notebooks that get ingested
  • into this Mack truck.
  • Digital Globe had this challenge where they had a huge amount of
  • satellite imagery.
  • They're one of the largest producers in the world of satellite
  • imagery. And so it would have taken them about 10 years to upload it
  • over the network.
  • And they were actually the first customer that called us and said,
  • hey, can't you just send a truck?
  • And so we built one.
  • We'll drive the truck up to your data center.
  • We have power and network fiber that will connect to your data
  • center. Fill 'er up, and then the truck will come back, put the
  • trailer back on the truck and we'll move it back to AWS.
  • If you think about the idea of moving an exabyte of data, if you
  • basically assign a 10 gigabit per second line to it, which is pretty
  • reasonable, it would take you about 26 years.
  • Using 10 Snowmobiles, it would take you a little less than six
  • months.
  • I remember a few years ago, AWS announced the Snowmobile by driving it
  • onto a stage at an AWS event and people just went nuts.
  • They were like, what?
  • How could a cloud be a truck?
  • And it was cool.
  • It was innovative.
  • But it's not like we've heard about the Snowmobile being a huge
  • business hit.
  • It's not like the Snowmobile is what's making up most of AWS'
  • revenue. Far from it.
  • AWS would not disclose how many customers have used a Snowmobile to
  • make a transfer, or how many trucks it has in service.
  • We weren't able to see the inside of the truck because the technology
  • is safely guarded, but it's essentially a data center on wheels.
  • Snowmobile uses what Amazon calls Zero-G racks, which suspend the
  • system from both the top and the bottom of the truck to handle the
  • impacts while in motion.
  • And it has its own power and cooling.
  • Once the transfer is complete, the truck enters transport mode.
  • An armed guard and an escort accompany the truck as it returns to
  • Amazon's data centers for the upload.
  • Its location is monitored over cellular and satellite communications
  • throughout the entire journey.
  • Amazon intentionally left the truck devoid of branding to keep it
  • discreet. When it connects to the AWS ingestion center.
  • The data is decrypted and hashed.
  • Validated, and then once the data is all unloaded and you validated
  • it's unloaded, then the Snowball set up for another run for another
  • customer.
  • Just as Amazon has disrupted retail with its e-commerce business, the
  • Snow products are an example of how the company has become a force to
  • reckon with in the cloud computing industry as well.
  • They're continually first with features.
  • They're first with different ways of doing things, like networking.
  • They have the most compute variation.
  • They have the widest range of machine learning offerings too.
  • Two years ago, we deployed 1,440 new products and services.
  • We deployed 1,954, that was in 2018.
  • I'm sure we will beat that in 2019.
  • So innovation has been a key point.
  • Security has been a key point.
  • Our systems are the only cloud that's certified to run the
  • intelligence agencies and DoD type clouds, top secret compartmented
  • level of certification.
  • And we continue to have just a wide variety with the most number of
  • databases available to our customers and the most variety of
  • databases. The most robust storage platforms, the most number of
  • options and compute.
  • In addition to that, we've just been doing it longer than anybody
  • else. So we have these years and years of operational excellence and
  • experience behind it.

Download subtitle

Download video & audio

Right click on one of the buttons above and choose "Save Link As..." to download video

Description

Demand for cloud computing from providers like Amazon Web Services continues to rise from both companies and consumers that rely on remote storage and computing power accessible from anywhere. While other tech giants Google, Microsoft, and IBM are vying to be the go-to providers, Amazon remains the undisputed leader in cloud computing.

Amazon Web Services is behind much of the technology we use every day, from streaming your favorite shows on Netflix to calling a car from Lyft. AWS has been one of Amazon’s most profitable business endeavors as companies abandon their own data centers for Amazon Web Services. Amazon said it has more 4,000 government contracts as well.

But moving data from local servers to the AWS cloud servers can be a challenge. Amazondeveloped physical and rugged products called the Snowball and the Snowmobile to help companies transfer data to the cloud. CNBC got a rare inside look at how Amazon Web Services transfers a large amount of data to the cloud.

» Subscribe to CNBC: https://cnb.cx/SubscribeCNBC
» Subscribe to CNBC TV: https://cnb.cx/SubscribeCNBCtelevision
» Subscribe to CNBC Classic: https://cnb.cx/SubscribeCNBCclassic

About CNBC: From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.

Connect with CNBC News Online
Get the latest news: https://www.cnbc.com/
Follow CNBC on LinkedIn: https://cnb.cx/LinkedInCNBC
Follow CNBC News on Facebook: https://cnb.cx/LikeCNBC
Follow CNBC News on Twitter: https://cnb.cx/FollowCNBC
Follow CNBC News on Instagram: https://cnb.cx/InstagramCNBC

How Amazon Uses Explosive-Resistant Devices To Transfer Data To AWS