Instagram Design

 


Instagram design

Top Features

1.     User registration/login

2.     Feeds on home page from fav people

3.     View Comments/Add comment

4.     Create post

5.     Reels

6.     Keyword search

 

Data store – User Profiles

  • 1.     Millions of users
  • 2.     Users linked with each – Graph DB

Data storge – User content - Relational

  • 1.     User ID
  • 2.     Content ID
  • 3.     Content URL
  • 4.     Uploaded date time

 

Create Post – Rest API –

  • ·       Content Upload – Reviewed – Virus Check - Categorized – Tagged – Stored.
  • ·       Storage – in S3 IA and eventually moved to other S3 buckets
  • ·       Content cached in Cloudfront spread globally
  • ·       API is going to be hit million times a day – API gateway
  • ·       Availability – taken care by multi-AZ, multi- region arch
  • ·       Scalability – taken care by the container platform and elasticity

·        

Queue based Feed publishing

  • ·       Whenever there is new post created – event is triggered – “Post created”.
  • ·       Feed is implemented as message queue. A message is triggered to the feed queue.
  • ·       Program listens to “Post created” event.
  • ·       Pick the post details and push it to the feed of all linked users.
    • o   Get API for picking the post API
    • o   Get API for getting friends of current users
  • ·       Listener program picks up the message and refreshes the feed of friends of the current user. The message header will have the list of friends whose feed should be refreshed.
  • ·       Millions of users are going to be posting to the queue and listener programs will have to deliver the feed to respective users. Should be highly available.
  • ·       Fault tolerant, non duplicate confirmed delivery
  • ·       The feed will be refreshing when the user is viewing the content which will not be a good user experience.

AI based feed generation

  1. 1.     Based on user interest
  2. 2.     Prioritizing feed ranking from different friends of user – based on user interactions with the in the past
    1. 3.     Post since last visit
    2. 4.     Feed table
    3. a.     ContentID
    4. b.     ContentType
    5. c.     ContentAuthor
    6. d.     Status – Pending Push
    7. e.     Last published time
    8. f.      Targeted user
    9. g.     Targeted user last login
  3. 5.     AI engine based on Pt 1-3 will spot the content which is relevant to the user and add it to the feed table.
  4. 6.     When the user logs in the feed will be pushed to the user.

 

Handling of streaming data

  • AWS Firehose can be used for ingesting, transforming and pushing streaming data to clients.
  • Http endpoints can consume streaming data.
  • Traffic from hose to endpoints will be encrypted.

 

 

 

 

Comments

Post a Comment

Popular posts from this blog

AWS Organizations, IAM

Image
Active Directory AD Connector is a directory gateway with which you can redirect directory requests to your on-premises Microsoft Active Directory without caching any information in the cloud. AD Connector comes in two sizes, small and large. You can spread application loads across multiple AD Connectors to scale to your performance needs.  AD Connector is your best choice when you want to use your existing on-premises directory with AWS services. AD Connectors and your on-premises AD domains have a 1-to-1 relationship.  Your end users and IT administrators can use their existing corporate credentials to log on to AWS applications such as WorkSpaces, Amazon WorkDocs, or Amazon WorkMail. Simple AD is a standalone managed directory.  Small - Supports up to 500 users,  Large - Supports up to 5,000 users.  Simple AD does not support features such as multi-factor authentication (MFA), trust relationships with other domains.  AWS Directory Service lets you run Microsoft Active Directory (AD)
Read more

Key Concepts

A  resource group  is a collection of AWS resources that are all in the same AWS Region, and that match the criteria specified in the group's query.  Tag-based queries include lists of resources and tags.  In an AWS CloudFormation stack-based query, you choose an AWS CloudFormation stack in your account in the current region, and then choose resource types within the stack that you want to be in the group.   You can use  resource groups  to organize your AWS resources. AWS Resource Groups is the service that lets you manage and automate tasks on large numbers of resources at one time.  With Resource Groups, you can create a custom console that organizes and consolidates information based on criteria specified in tags, or the resources in an AWS CloudFormation stack.  Transfer Family - Supports connecting to S3 or EFS through FTPS/SFTP/FTP protocols.  The hostname for the endpoint can also configured with R53.  The transfer family can be associated with Elastic IP. Security group pe
Read more

Linear Algebra Concepts

Vector Spaces Rank of matrix is the number of non zero rows in REF form. Alternatively rank of matrix is = total number of columns - total number of dependent columns. In the REF form, If a column of a matrix can be expressed in terms of other columns then it is a dependent vector. The vectors on the left most side with leading 1 at the col number position ( 1 at 1st row 1st col, 1 at 2nd row 2nd col, likewise) then such columns are linearly independent columns. For linerarly independence a1v1+a2v2+a3v3=0 where v1,v2,v3 are vectors and a1,a2,a3 are scalars. Span of a vector space is all the possible vectors which can be formed using the basis vectors. Basis vectors are linearly independent vectors which can span the vector space. Column space is all possible set of vectors formed by the column vectors. Row space is all possible set of vectors formed by the row vectors. Dimension of a row space or column space is equal to its rank. Null space or kernel of a matrix is any vector x which
Read more