How to auto invoke agent when provisioned image first boots (SoftLayer)
I want newly provisioned images in SoftLayer to do some localisation tasks when they first boot. I'd like to use uDeploy for this. What is the best approach?
I see there are resource templates and support for some cloud systems but not SL.
Is there a public REST (or JMS?) api I can use to have the nascent node get the uDeploy Server to kick things off? I've manually done the remote agent install to a system so could this be automated remotely as well?
Is there a better solution to uDeploy for this kind of initial config step? (and I know this is a uDeploy community so the answer may be no 8-)
Thanks,
Jerh.
1: Define a resource template which shows the basic structure of the environment you'll be provisioning. You have to consider how many VMs will be provisioned, which VMs will serve in which roles, etc. In my example below, I've defined a template with a folder for "DB" machines and a folder for "Web" machines. Once I create a new application environment based on this template, any machines added to that environment's DB or Web folders will inherit any configuration defined in the template under those folders.
Since you'll be spinning up new VMs every time you create an environment like this, you can also tell UCD to expect that some agents will show up whenever you make an environment based on the template. To do this, edit the appropriate Agent Prototype node and enter some name patterns. You can use ${p:application.name} and ${p:environment.name} as variables in the agent name, but whatever you enter here needs to match the name which the UCD agent will be configured with on the VM that gets spun up. Here, we'll say I will expect one web server to be provisioned with this environment, and the name of that agent will be "webserver-[environment name]" (so if I create an environment called "QA5", the agent name would be "webserver-QA5").
2: Define a blueprint for the application-specific information in the template. Below, I'm just showing that I have defined a blueprint using the resource template I created above, and that the DB agent should get my JPetStore-DB component and any Web agents would get JPetStore-APP and JPetStore-WEB.
3: Create the new UCD application environment. For now, I'm showing how this would be done manually - I'll get to automating it in a bit. To make a new environment using the blueprint, go to the application and create an environment. Pick the appropriate blueprint and then select a "Base Resource" - this is the folder in the resource tree where it will create the new deployment targets for that environment.
Once you do this, you can click into the new environment. You will see that it has created the basic structure defined by the blueprint, and because you have set "Agent Name Patterns" in the Agent Prototype entries in the resource template, it will create agents for the environment as well. The components are added to these agents automatically as well, but they won't be deployed until we actually try to run a process on the new environment.
Note that these agents aren't actually associated with a running UCD agent on any system. They're simply placeholders for agents which may come online in the future, based on the agent name. At this point, if you were to create a VM in SoftLayer with a UCD agent on it, you'd set the UCD agent's name to "webserver-QA5" (you can do this by editing the agent's conf/agent/installed.properties file and changing the "locked/agent.name" property). When that agent starts up, it will automatically become associated with the environment in UCD.
So, that basically covers how you would do the configuration of this in UCD itself. But it's also possible to automate all of this using a process in UCD. On the top-level "Processes" tab, you can define a workflow which will create the application environment in UCD, call out to a script you provide which actually creates the SoftLayer instances, wait for the agents to appear, and then run an application deployment to those agents. Here's what it might look like:
Configuration of "Create Environment" - note that I'm using the "Append Unique Suffix" so it generates a new environment name automatically every time:
The "Run Script to Create SoftLayer Instances" step would call whatever shell commands you need - I'm assuming you have a script which can already deal with the actual provisioning work. One thing this would need to do is pass the environment name to the instances so that the agent name is set appropriately. You could access the name of the new environment using: ${p:Create Environment/environment.name}
In "Wait for Resources", you'll have it pause until the new agents show as online. You know what the path to these resources will be based on the options in the "Create Environment" step and the configuration of the blueprint. In my case, it would end up being like this:
/Provisioned Environments/${p:Create Environment/environment.name}/Web/webserver-${p:Create Environment/environment.name}
Finally, you can include the step to run the application deployment on the new environment. That one is pretty basic, but note that I'm using the keyword "latest" for each of the components so that the latest version of each component is deployed to the environment. In the application process you call here, you could include a component which does the first-time OS configuration you're looking to set up:
And because there's a character limit, I'll post this separately:
If you're really only trying to get some OS configuration to happen after the machine boots for the first time, then UCD is a bit heavyweight. It'd be much simpler to use a tool like Puppet or Chef to make one-time, first-boot OS changes. This article [http://server.dzone.com/articles/low-level-infrastructure] is pretty close to what I think you'd want to do. Note that it uses a Puppet Master (a central server which hosts Puppet machine configuration files and modules to be installed) - you wouldn't need to do that even, you can just use the standalone Puppet client to run some modules. The main advantage of using Puppet or Chef here is that they have already done some of the low-level heavy lifting for OS configuration.
However, if you're looking to deploy applications to these systems after provisioning them, introducing UCD makes a lot of sense. You'd do that using the built-in Resource Template feature in UCD.
Also, note that the "Create Environment" plugin step I'm using here is only available in the latest version of this plugin, or in any UCD server version after 6.0.1.2. You can get that here: http://plugins.urbancode.com/IBMUrbanCodeDeploy/plugin/IBMUrbanCodeDeployEnvironments/66.0
Ah, I forgot to mention: You can also use the "Install Agent with SSH" step in the process to deploy the agents on the new VMs once provisioned in SoftLayer. Just add that step to the process to see what configuration it requires - it should be pretty self-explanatory.
At that point, the only thing you'd have to do is make sure you knew which hostnames/IPs the new VMs are at. To get that information, you'd need to make sure either the hostname or IP is returned from the step in which you call out to SoftLayer as an output property.
This involves setting the "post-processing script" on the step which calls out to SoftLayer. Assuming that there is output like this from that step:
WebIP: 10.11.12.13
DatabaseIP: 14.13.12.11
You could use a postprocessing script like this:
if (properties.get("exitCode") != 0) {
properties.put(new java.lang.String("Status"), new java.lang.String("Failure"));
}
else {
properties.put("Status", "Success");
}
scanner.register("WebIP", function(lineNumber, line) { String webIP = line.substring("WebIP: ".length()); properties.put("webIP", webIP); });
scanner.register("DatabaseIP", function(lineNumber, line) { String databaseIP = line.substring("DatabaseIP: ".length()); properties.put("databaseIP", databaseIP); });
You could then use ${p:Run Script to Create SoftLayer Instances/webIP} to get the "webIP" property from the step using that postprocessing script, and that would tell you which IP to install the web agent to.
Cheers Matt - a comprehensive answer - I'll get to grips with it in coming days. Chef/Puppet would fix the bill for immediate need but as you say UCD allows us to expand and work with the nodes post provision. Plenty food for thought.
Wouldn't we also need to add the step for changing the installed agent's name ( by editing the agent's conf/agent/installed.properties file and changing the "locked/agent.name" property) after "Install Agent with SSH" and before 'Run application Process" so it matches with what's configured in resources?
If you're using Install Agent with SSH, you would typically just give it the correct name to start with rather than having to find a way to change it after installation. You only have to worry about updating the installed.properties file on the agent if you had the agent baked into the image itself and needed to update it when spinning up a VM from that image.