DELETE Without Losing Insight: How FlyData Sync Append Only Mode Can Help You If you’ve ever designed or managed an application database, you at some point must have thought about the question of whether to really delete data from your database, or to just mark the data as “deleted”. Both have their pros and cons.
# DELETE Without Losing Insight
The benefit of marking the data as deleted is that you still retain the data for future analysis. For example, let’s say you are running a mobile gaming app and wanted to analyze user activity for users who have deleted their account. If we really deleted the user accounts completely, there would be no way for us to analyze what led to their quitting the game. On the other hand, if we simply marked the data as deleted, we can query and answer interesting questions like “what has led to users deleting their account?”
# Growth of Data
The problem is that nowadays we can reach a lot more users faster. In addition, we generate a lot more data per user, as well. As a result, application data usually grows quite fast. So if we simply mark user data as “deleted”, our databases could quickly become too large to handle our everyday queries. Is the solution to give up on historical data analysis and permanently delete the records after all? Or is there another way?
# Keep Your Historical Data and Reduce Burden on Your Application Database
One solution to this problem is to replicate your database to a data warehouse, like Amazon Redshift, and then permanently delete your records on your application database. That way, your application database won’t be burdened by extra data, while you retain a copy of your historical data in Amazon Redshift.
# FlyData Sync Append-only Mode
At FlyData, we understand the need to keep your database quick and light while still having the power to run queries on historical data. For this, we are offering the FlyData Sync Append-only mode for those of you who have faced a similar issue. FlyData Sync Append-only mode will replicate your MySQL database to Amazon Redshift but will ignore any DELETE queries. This allows you to have a copy of your historical data always being replicated over to your Amazon Redshift, while your application database can eliminate such data that is unnecessary on the application-side.