Best practice for chef, ec2, knife and load time reduction and ami in case of failure
I am in the process of hosting all my servers under chef control for scm using the chef server. It works great ... I love it. I think I am starting to get Zen, but I need to understand better.
Before that, I built ami on ec2, and I always rebuilt ami, but at least they loaded quickly.
With a chef, I'm going to always have to start with a barebones instance and use a knife to create your ami.
It takes about 20 minutes to run the ami, in which the chef's setup is about 5-6 minutes.
The problem is I am running 4 servers, two redis servers, master and slave, and two nginx servers when load balanced.
On a remote machine, I have python based scripts that always check if redis master or slave is running and for load balancing if nginx servers are healthy.
Now that the chef, if the server goes down, I have to wait an additional 15+ minutes when there was time in the prearranged time frame to download a new instance.
I have the following questions:
1) Is there a python api to interact with the knife? In python, I used boto to download the ami. Now, I will have to use popen to execute the knife. My code depends on knowing when the ami is running and will continue further processing.
2) Is it better to use ami where chef client is preinstalled? This shaves off for about 6 minutes.
3) finally, what else am I missing? Any advice would be appreciated.
source to share
1) https://github.com/coderanger/pychef This is a python client for chef api. Not a knife per se, so it won't give you access to plugins you can use, but it should allow you to interact with the chef's server
2) I don't do this because I like to use the ubuntu ami stock, but it looks like it would be a perfectly reasonable approach. There may already be an ami like this, but I don't know what to do. Edit: I forgot, there is a chef installer: http://www.opscode.com/chef/install/ which is installed via deb (IIRC)
3) I don’t think that, to be honest, you don’t have anything deep. But I definitely believe that the chef should be the tool you use to make your life easier, faster, and more consistent. Therefore, I would use it in the way that is most effective for you. If pre-build ami works the best, then that seems like the best solution. Sorry, I don't understand anymore.
source to share