International Acceleration - Scaling Trending Topics with AWS Services

Join the team responsible for trending topics of a fictional social network of short messages and simulate interaction with other company teams to design communication between the various impacted subsystems.

Learn how to architect and develop a scalable solution to integrate functionality, based on distributed messaging and micro-services. Use Spring Boot and AWS services to develop the solution by covering automated testing and using clean architecture concepts and finalize by adding resilience to the solution through observability and incident management.

This acceleration program prepares professionals for exciting opportunities outside Brazil with 100% English technical sessions. It offers free workshops for technical advancement, as well as networking and connections with international recruitment partners for major global companies.

What you will find in this acceleration:

Session 01: Designing a solution for identifying topics of the moment

The first challenge is to interact with the posting team on a fictional social network and to design the communication protocol between the subsystems. We will choose AWS SNS, a tool that supports the scale required to handle the large volume of posts per minute, and ensure that components remain uncoupled. We will then proceed to the architectural design of the solution, which will be based on distributed messaging and microservices. We will compare the defined architecture with simpler alternatives, highlighting where they would present problems if adopted.

Session 02: Developing the solution using Spring Boot and AWS services

Next, our focus will be on developing what was designed. The intake and subject identification of batches of posts will be paper of a lambda fired by SQS. Then we will do an ECS service responsible for aggregating the subject counts of the previous step into shards, using a subject hash prefix as the key, ensuring the scalability of the next step. Each aggregation operation will generate a message per shard, published in another SQS queue. These messages will be consumed by a second lambda, responsible for updating the internal ranking of each shard. Finally, a third lambda will publish the top 10 global subjects every 5 minutes, allowing the other subsystems to consume the result of our work.

Session 03: Adding Observability with AWS NovoWatch and AWS Systems Manager

In the last section, we will address resilience aspects of our current subject subsystem. We will set up metrics, alarms, and dashboards using AWS NovoWatch, evaluating how and when we want to be notified of adverse solution situations. We will also define an on-call rotation and incident management plan using AWS Systems Manager.