Service Configuration Streamlining and Sequencing

Hi,

In order to further improve the simplicity and robustness of the control path for provisioning infrastructure and delivery services, we are currently considering ways to streamline management and operations.

Currently, when applying changes in traffic-control that require the synchronization between the traffic-router and traffic-servers, the user should be conscious to do so in a certain order. Otherwise, "black holes" may be created. Furthermore, in some of the scenarios the user have to wait and verify that the configuration reached all traffic server before he may apply it to the traffic-router.

The main use-cases we would like to address at this point are:

• Assign servers to a Delivery-Service.
  For this operation, the configuration must first be applied to the added traffic servers, propagate, and only then applied to the traffic-router.
• Remove servers assignment to a Delivery-Service.
  For this operation, the configuration must first be applied to the traffic-router, and only then to the traffic-servers.
• Add a new delivery service.
  This is practically a private case of servers assignment to a delivery-service.
• Delete a delivery service.
  This is practically a private case of servers assignment removal from a delivery-service.
• Update settings that must be applied together on the traffic servers and the router.

We would like to simplify the procedure, allowing the deployment of new configuration in a single operation, instead of doing it step by step.

One solution can be based on the insight that deploying such configuration changes may be done by initially updating the traffic server with added functionality (e.g remap-rule), then updating the router, and lastly, removing old functionality from the traffic servers. Such a solution can be orchestrated by traffic-ops, probably without complicating other components.

Other solutions may provide more flexibility, but would probably involve adding complexity to other components such as traffic-router.
An example to such a solution, already under discussion in a mail tread (10x Eric may be based on the concept of a delivery-service configuration "generation" which would be an ordinal identifier for the a delivery service configuration. A "generation" changes whenever the remap rule changes or the consistent hash mapping of content to server changes (e.g. due to additional server assignment).
In such a solution, each traffic-server may hold a single generation for each delivery service configuration, while traffic-router may hold a history of generations and know which server holds which configuration generation.

This approach introduces a considerable flexibility. It allows configurations to be set one after the other with no need to wait between them.
On the other hand, complicated algorithms for solving the issue may impose more risk to the network when applied, comparing to a simple "traffic-ops" orchestrated solution.

Both approaches fit well with TC-129: Create TO API endpoint to queue all servers for a delivery service

I'm not sure what is preferable from an operator point of view. I'm also not familiar with TC 3.0 "Config State Machine" solution to validate he different approaches against.

Nir