Posted by j.branson on June 22, 2013 at 3:31pm
Hi! Has anyone successfully setup a multi-instance EC2 VPC on AWS? I'm presently running a bunch of EC2 Classic instances easily. I'm trying to make the move to VPC for lower latency and more security on the DB instances, but getting the gateways and internal routing tables setup is kicking my ass. I can't get a connection without routing back out to the internet for both Code and DB instances...this defeats the point.
FYI I'm running the DBs on an EC2 instance with PHP code backup dumping to S3 as opposed to RDB because of a multi-site Drupal setup that's spanning across several different databases.
I appreciate any suggestions and can arrange things financially if you can actually help me to get it working!
Peace
Comments
Got it going
I've tinkered around and looked through the books worth of documentation from Amazon. Got a Dev VPC going. After two years with DB on EC2, I did take this opportunity to migrate to RDS because it feels more reliable. I'm hosting multiple Drupal DB's with user tables spanning across them. I've tweaked my code to back up individual Drupal DB's so I can go backwards for each site and I'm not cornered into Amazon's backup everything all at once approach (which I'm also using.) Now trying to sort of Cloud Front distributions. Can't tell yet if it's worth it... I'm eager to know your thoughts if you have experience with this.
Different VPC Setups
Amazon provides four different VPC scenarios. The one I have deployed for a Drupal hosting envrionment is "Scenario 2: VPC with Public and Private Subnets": http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
This allows you to "hide" some instances in the Private portion of your cloud (DB, NFS, memcached, etc), but definitely has a few gotchas which are not very obvious when first setting it up. Here are some of my experiences which I found troublesome or just not immediately obvious:
In Scenario 2, you will be provided an EC2 instance as a NAT / router in order to provide outgoing Internet access to the Instances in the Private portion of the cloud. Without this, you will not get ANY Internet access on the Private Instances! The NAT Instance itself is a magical black-box and I have never once logged into it. I did, however, reduce its size (and my costs) by reducing the type from m1.small to t1.micro. Been running like that for a couple of weeks and it's doing just fine - NAT doesn't require a lot of CPU cycles or RAM. If you were to terminate this Instance, all Internet connectivity in the Private portion of the VPC will be lost!
The Public Instances do not seem to use this NAT / router. They need to have Elastic IPs associated with them in order to have outgoing Internet access (and obviously incoming as well).
Your VPC Instances will have their own set of Security Groups. If you login to the Management Console, you will see different Security Groups in the "EC2" and "VPC" sections of the portal! If you like the SGs you had setup for your basic EC2 setup, you will need to recreate them somehow within the VPC (Once a VPC Security Group is setup it can be used across any Instance in that VPC).
Your VPC will initially be allowed to consome something like 5 Elastic IP Addresses. I ran out of these and had to contact Amazon to make a case for why I needed more. I simply explained that I am hosting different sites on different Instances and they gave me 5 more (possibly for a small additional fee). This seemed strange to me at the time I ran out, but Amazon responded quickly and I was able to allocate more within a day or two. Just something to keep in mind - If you can deploy a load balancer to use one IP across 8 web servers (for example), do it. :)
No Special Routing Required
By the way, I was confused by AWS' instructions on configuring routes and such for my VPC. Turns out that the magic black-box NAT provided in Scenario 2 does it all. You just need to decide what IP chunks to use, and attach Elastic IPs to the public Instances, and all is well.
At first it seemed from the instructions that I would need to modify my routing tables or such like that. Not at all. It happens "automagically", as they say.
Another slightly confusing aspect is that the Elastic IPs assigned to public Instances do now show up using "ifconfig". If you are specifying which interface to bind to, for example in your Apache config, you need to specify the "internal" (10.x.x.x) address and not the public Internet-facing one, since the OS doesn't seem to know it exists. Just bind to the 10.x.x.x address and let it rip!
Thanks
Thanks, your comments are helpful. I definitely found too that it's not as confusing as they make it seem once you start clicking launch. and going with it. I've got my EC2 instances behind a load balance and RDS taking calls from within the VPC. I'm only working in DEV mode now so I don't have failover enabled, but will when I set up new production system. One thing I'm not quite sure about is how to setup read replicas for Drupal to know they are there. I'm bet there is documentation on this somewhere that I can find when I get to it.