Case Study: How Airbnb used AWS to expand its Services!!!
Introduction:-
Airbnb is a community marketplace that allows property owners and travelers to connect with each other for the purpose of renting unique vacation spaces around the world. The Airbnb community users’ activities are conducted on the company’s Website and through its iPhone and Android applications. The San Francisco-based Airbnb began operation in 2008 and currently has hundreds of employees across the globe supporting property rentals in nearly 25,000 cities in 192 countries.
Transition period:-
A year after Airbnb launched, the company decided to migrate nearly all of its cloud computing functions to Amazon Web Services (AWS) because of service administration challenges experienced with its original provider. Nathan Blecharczyk, Co-founder & CTO of Airbnb says, “Initially, the appeal of AWS was the ease of managing and customizing the stack. It was great to be able to ramp up more servers without having to contact anyone and without having minimum usage commitments. As our company continued to grow, so did our reliance on the AWS cloud and now, we’ve adopted almost all of the features AWS provides. AWS is the easy answer for any Internet business that wants to scale to the next level.”
Why Airbnb chose AWS:-
Airbnb has grown significantly over the last 3 years. To support demand, the company uses 200 Amazon Elastic Compute Cloud (Amazon EC2) instances for its application, memcache, and search servers. Within Amazon EC2, Airbnb is using Elastic Load Balancing, which automatically distributes incoming traffic between multiple Amazon EC2 instances. To easily process and analyze 50 Gigabytes of data daily, Airbnb uses Amazon Elastic MapReduce (Amazon EMR). Airbnb is also using Amazon Simple Storage Service (Amazon S3) to house backups and static files, including 10 terabytes of user pictures. To monitor all of its server resources, Airbnb uses Amazon CloudWatch, which allows the company to easily supervise all of its Amazon EC2 assets through the AWS Management Console, Command Line Tools, or a Web services API.
In addition, Airbnb moved its main MySQL database to Amazon Relational Database Service (Amazon RDS). Airbnb chose Amazon RDS because it simplifies much of the time-consuming administrative tasks typically associated with databases. Amazon RDS allows difficult procedures, such as replication and scaling, to be completed with a basic API call or through the AWS Management Console. Airbnb currently uses Multi-Availability Zone (Multi-AZ) deployment to further automate its database replication and augment data durability.
Airbnb was able to complete its entire database migration to Amazon RDS with only 15 minutes of downtime. This quick transition was very important to the fast-growing Airbnb because it did not want its community of users to be shut out of its marketplace for an extended period of time. Tobi Knaup, an engineer at Airbnb says, “Because of AWS, there has always been an easy answer (in terms of time required and cost) to scale our site.”
Tech stack used:
ReactJs for creating the web application using AWS AMPLIFY
Profile Management using AWS COGNITO
Static website hosting on S3 bucket
CLOUDFRONT for CDN
Code repository in CODECOMMIT
Backend API’s using Lambda functions(in Python) which will be triggered via API Gateway
AWS ElastiCache for efficient Search functionality
DynamoDB database for storing data in key-value pairs
Static files like images are kept in an S3 bucket
CloudWatch Alarms are being used for monitoring purpose
AWS SES service to send emails to customers
AWS Pinpoint and Athena for analytics purpose
Airbnb Uses Amazon EFS to Scale CI/CD Pipeline for Expanding Online Marketplace:-
Online travel marketplace Airbnb supports hundreds of critical services on its platform, making it essential to maintain a reliable source control infrastructure. The company uses GitHub Enterprise for both source control and management of its continuous integration/continuous delivery (CI/CD) processes. Airbnb has more than 1,000 engineers, who execute more than 100,000 continuous integration jobs on an average working day. GitHub Enterprise provides the engineers with a single source of truth for all code repositories.
However, source control infrastructure had become an operational headache due to the system’s scaling issues. In the previous system, each mirror instance pulled changes from GitHub Enterprise. This became challenging to maintain because the mirrors could get out of sync with each other. The system did not scale with Airbnb’s increasing Git traffic and hindered the team from focusing on higher-level problem solving and implementing new features.
Airbnb sought a solution it could use to re-architect the source code infrastructure with a simpler storage layer. The system needed to update in seconds and read traffic needed to scale.
AWS Services used:-
Amazon EFS :-
Airbnb realized it could utilize Amazon Elastic File System (Amazon EFS), a simple, scalable system for Linux-based workloads for use with AWS Cloud services and on-premises resources. “Through our research, we discovered Amazon EFS, which we could use to share a single file system mounted on all Git mirror instances,” explains Snyder. “When one Git mirror changed, every other Git mirror was guaranteed to have the same update.” In this way, Airbnb used Amazon EFS to back the real-time image of its GitHub repository data. Mirrors were continuously in sync with the production repository at the scale required for the CI automation pipeline.
Queuing Service Helps Sync Repositories:-
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available. Get started with SQS in minutes using the AWS console, Command Line Interface or SDK of your choice, and three simple commands.
SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.
Benefits:-
- Uses single file system to sync GitHub repositories
- Keeps Git mirrors in sync by using a shared file system that allows scaling of CI/CD processes
- Ensures no repository changes are lost during syncing using event-driven queuing approach
- Allows engineers to focus on building system features instead of worrying about scaling
Thank you (●'◡'●)
