In this post we will look at when and why Big SQL triggers a restart of the Ambari server. We will peek under the hood to explain this image and why you should not worry about it:
Big SQL is a custom service. It is not a built-in or native Ambari service. When we are working with Big SQL through the Ambari web UI, Ambari views Big SQL as any other custom service. This distinction is important because it means Big SQL does not have full control over the user interface. For example, here is the Big SQL service menu:
Ambari provides some operations to all services, which you can see in the ‘Service Actions’ menu above: Start, Stop, Restart All, Restart Big SQL Workers, Run Service Check, and Turn On Maintenance Mode. The bottom three operations are known as custom commands. Each service has an XML file that lists the components which define the service and also allows us to define custom commands for the service.
You may wonder what any of this has to do with Big SQL initiating an Ambari Server restart. When the server is restarted, Ambari will read the XML file that defines each service and as a result any change to the custom commands defined in that file will be reflected in the ‘Service Actions’ menu of the UI.
Now we come to the EnableAutoHcatSync custom command. This command turns on a feature that enables automatic synchronization of metadata between Big SQL and HDFS. When you run the command you will see two operations are performed, as indicated by the image near the top of this post.
For ease of use and manageability, Big SQL provides the user with the ability to turn this automatic synchronization on and off. The way Big SQL implements this is via a custom command that modifies the XML file that defines the service. The modification to the XML file removes the ‘enable’ command and adds a ‘disable’ command.
An Ambari Server restart is required to refresh the Service Actions menu. The ‘Restarting Ambari Server’ operation is orange because Ambari loses the handle on the operation when it restarts. If you were to click the ‘Restarting Ambari Server’ link all the way through to the stderr and stdout logs you would see they are empty.
This is not ideal and we hope to revisit it in the future, but for now you should know this behavior is expected and does not indicate a problem with your Big SQL service.