Data Replication between Data Centers
by jkramer from LinuxQuestions.org on (#5279D)
Hi all,
I'm looking for some suggestions and feedback, from a design and security point of view. I have a small personal project in early stages of prod, budget is a bit of a concern which lead me to the below.
I have a VPS (Ubuntu 18.04) sitting in DigitalOcean which hosts my site, which is a SAAS Tool built on Legacy PHP. Database is MySQL. I have many processes running in the background which updates data in the local database, along with creating new db entries (every few seconds). The more jobs I run, the slower the front end becomes if you try and access that same data in the database, so I'm looking to take the processes and jobs off of the server which serves the front end.
I decided to go with another provider, because the pricing was excellent (1/3) and the reviews have been great. The only issue is, the main server is in the US and the second server is in Germany. I set up remote access in between the two and began running my jobs (25-30 python scrupts) where the data is collected from the second server in Germany and updates the remote database in the US. There are two issues. Performance is slow (most likely due to the 25-30 remote connections to the database) and lack of security.
The easiest solution I presume, would be to create another VPS in DigitalOcean and to set up data replication (active/active), so the second server carries most of the resource utilization and passes the data via sync. But, the pricing is getting a little too high with multiple servers in DO, because eventually I'll need a mirror replica and new server for redundancy (bringing me up to 3 servers at that point).
I've hardened the firewall and IP Tables for the second server, but the traffic between the two is still vulnerable to man in the middle attacks and sniffing. My idea was to create a mirror copy of the database in the US, and installing it on the second server in Germany, and then set up active/active replication. For security, I can set up an ssh tunnel in between the servers.
I'm just curious if anyone has any other ideas, suggestions which could improve the design from a performance and security standpoint.
Thanks
I'm looking for some suggestions and feedback, from a design and security point of view. I have a small personal project in early stages of prod, budget is a bit of a concern which lead me to the below.
I have a VPS (Ubuntu 18.04) sitting in DigitalOcean which hosts my site, which is a SAAS Tool built on Legacy PHP. Database is MySQL. I have many processes running in the background which updates data in the local database, along with creating new db entries (every few seconds). The more jobs I run, the slower the front end becomes if you try and access that same data in the database, so I'm looking to take the processes and jobs off of the server which serves the front end.
I decided to go with another provider, because the pricing was excellent (1/3) and the reviews have been great. The only issue is, the main server is in the US and the second server is in Germany. I set up remote access in between the two and began running my jobs (25-30 python scrupts) where the data is collected from the second server in Germany and updates the remote database in the US. There are two issues. Performance is slow (most likely due to the 25-30 remote connections to the database) and lack of security.
The easiest solution I presume, would be to create another VPS in DigitalOcean and to set up data replication (active/active), so the second server carries most of the resource utilization and passes the data via sync. But, the pricing is getting a little too high with multiple servers in DO, because eventually I'll need a mirror replica and new server for redundancy (bringing me up to 3 servers at that point).
I've hardened the firewall and IP Tables for the second server, but the traffic between the two is still vulnerable to man in the middle attacks and sniffing. My idea was to create a mirror copy of the database in the US, and installing it on the second server in Germany, and then set up active/active replication. For security, I can set up an ssh tunnel in between the servers.
I'm just curious if anyone has any other ideas, suggestions which could improve the design from a performance and security standpoint.
Thanks