Amazon SQS Industry Use Case
What is Amazon SQS:- Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
What is Message Queue:-Message Queue(MQ) is a queue of messages sent between applications. It includes a sequence of work objects that are waiting to be processed. A message is the data transported between the sender and the receiver application; it’s essentially a byte array with some headers at the top.
Some Tools for Message Queue
- Apache Kafka
- Apache ActiveMQ
- AWS SQS
Amazon SQS provides queues for high-throughput, system-to-system messaging. You can use queues to decouple heavyweight processes and to buffer and batch work. Amazon SQS stores messages until microservices and serverless applications process them.
Some Benefits of Amazon Simple Queue Service:-
- It eliminates administrative overhead. AWS manages all in progress operations and underlying infrastructure required to produce an extremely accessible and scalable message queuing service.
- We can Reliably Deliver Messages. Use AWS SQS to transmit any volume of data, at any level of output, while not losing messages or requiring alternative services to be accessible.
- It keeps Sensitive Information Secure. You can use Amazon SQS to exchange sensitive data between applications using server-side secret writing (SSE) to inscribe every message body. The AWS SQS compass point integration with AWS Key Management Service (KMS) permits you to centrally manage the keys that defend SQS messages together with keys that defend your alternative AWS resources.’
- It Scale Elastically and Cost-Effectively. AWS SQS leverages the AWS cloud to dynamically scale supported demand. Amazon SQS scales elastically together with your application.
Industrial Use cases Of Amazon SQS:-
There are so many companies (More precisely 698) who are using Amazon SQS very effectively . Some of these very popular companies are Pinterest , Lyft , Stack ,medium , Accenture , Coursera , Cred , RedBus , NASA , BMW , OYSTER etc.
Case Study of OYSTER:-
Challenge :-
Since its 2009 launch, Oyster has published more than one million high-quality digital images. When this massive volume of images became too cumbersome to handle in-house, the company decided to offload the content to a central repository on Amazon Simple Storage Service (Amazon S3). “We migrated to Amazon S3 in 2010,” says Eytan Seidman, Co-Founder and Vice President of Product. “We chose moving to the cloud and Amazon S3 because storing images in our data center would have been too costly. Amazon S3 was a more economical solution.”
Oyster reprocesses its entire collection of photographic images a few times each year to update the copyright year and, if necessary, to change the watermarks. Using their previous solution, reprocessing the entire collection of photographs required about 800 hours to complete. In addition, Oyster often recreated existing images in new formats and sizes for mobile and tablet devices. Resizing existing images and adding new ones was slowing down the rate at which the company was able to process the collection. “Our processes were slowing down,” says Seidman. “When the iPad with Retina display came out, for example, it took us more than a week to create new sizes specifically for that resolution.” Oyster considered purchasing additional hardware, but found the cost of new hardware and routine maintenance was too high, especially when the machines would sit idle most of the time.
Moreover, there were numerous software bugs in the multiprocessing solution that the company used, but since the solution didn’t scale, Oyster didn’t bother to fix them.
Why they choose AWS:-
There were many reasons to Choose AWS. While the company is still running one local server, the bulk of the processing work now takes place on the AWS Cloud. Oyster is using a customized Amazon Linux AMI within Amazon EC2. Within this new environment, the company connects to Amazon S3 and Amazon Simple Queue Service (Amazon SQS) using boto, a Python interface to AWS. The images themselves are processed with the ImageMagick software available in the AMI package.
Oyster uses Amazon EC2 instances and Amazon SQS in an integrated workflow to generate the sizes they need for each photo. The team processes a few thousand photos each night, using Amazon EC2 Spot Instances. When Oyster processes the entire collection, it can use up to 100 Amazon EC2 instances. The team uses Amazon SQS to communicate the photos that need to be processed and the status of the jobs.
what were the benefits of doing this:-
There were so many benefits of doing this .Oyster’s old system needed approximately 400 hours to process one million photos. By using AWS, the company can process the same number of photos in about 20 hours — a 95 percent improvement. “It took less time to rewrite the code and do a full processing job with AWS than it took to do a single run with the old method,” says Seidman. “It used to take close to a week to produce photos specifically for the iPad. With AWS, we can create the photos in just a few hours. The documentation is straightforward and the dashboards are incredibly helpful.”
Oyster has also been able to reduce in-house hardware expenses by repurposing two of its old servers, which were sitting idle more than 80 percent of the time. “We estimate that we saved roughly $10,000 in capital expenditures by moving to AWS, and reduced our operating expenses by an additional $10,000,” Seidman says. He believes that AWS is a perfect match for any company performing similar batch processing. “AWS lets us move faster without worrying about machine expenditures or maintenance, which frees us to focus on other things.”
(Source Of Knowledge :- Amazon Official Website)
That’s all with this article. I hope you found this post informative .Feel Free to provide Suggestions in comment section or LinkedIn .
