This morning, there were reports of a broader AWS service outage reported here and here. Redshift status was also in the red, so depending on your method of continuous sync onto Redshift, you may still be working on the recovery process (I am writing this during our morning, PDT).
From the timeline, Redshift had issues for approximately 6 hours and if you have an automated process to load data, you may have gotten a lot of error messages from your monitoring system. You would have then needed to pause your process and wait for AWS recovery before resuming. This could have put further burden on your Support team, as this started at midnight on a weekend.
By leveraging a SaaS service like FlyData, you can avoid these types of issues and not have to deal with tedious tasks to recover your sync.
# How FlyData handled the outage
The outage affected Redshift clusters of quite a few FlyData customers. FlyData detected the outage and buffered the data during the outage, and resumed the data upload once AWS systems came back up. These were all done automatically without tedious and error-prone manual recovery work. The following steps are how we handle your MySQL change data on our SaaS backend.
- Buffer after fetching from MySQL data source to multiple endpoints
- Retry until the next component is available and receive the data successfully
# FlyData helps make your life easier
What does it mean for you? You won’t need to wake up in the middle of the night for an AWS outage using FlyData! We automatically recover your data upload process after an outage without issues. Our service is created to ease the burden of using Redshift and provide an end-to-end data integration process. In our next article, we will further explain our fault tolerance, so stay tuned!