In this section, we show how to extend NCGuard’s programmatic inter- face (see Section 8.3) to support and pilot an entire network reconfigura- tion process.
In NCGuard programmatic interface, a reconfiguration process is en- coded as a directed acyclic graph G = (V, E), or reconfiguration graph. The reconfiguration graph is either computed automatically, for instance,
manually fixed by the network operator. Each node Ci in V represents a high-level network representation, whereCinit andCf inrepresent the ini- tial and the final representation, respectively. Each edge e = (Ci, Ci+1) in E represents two consecutive high-level representations of the reconfigu- ration process. Observe that consecutive representations can differ com- pletely or differ by only one line on one router. Each edge in the graph is further associated with a list configuration statements and a predicate. When applied to the network, the list of configuration statements trans- formCi intoCi+1. A predicate is a condition that must be satisfied before executing the reconfiguration operation. Currently, we have identified four predicates which can be combined to form more complex conditions: #1 WaitForProtocolConvergence This predicate assesses that a protocol
instance has converged. Assessing the convergence of a protocol can be done by monitoring the state of the routers either by using “show commands” or by directly collecting the routing updates. Among oth- ers, assessing the convergence is important to avoid the problems re- lated to transient states. This predicate can also be used to monitor the stability of a protocol instance. Notice that the implementation of the predicate must be particularized depending on the protocol. #2 WaitForProtocolCompleteness This predicate assesses that a protocol
instance contains required routing information. The required routing information should be provided beforehand by the network opera- tor. For instance, this predicate can verify that a IGP has propagated a route for all the internal prefixes. This predicate is important to avoid loss of visibility during the reconfiguration. With respect to the WaitForProtocolConvergence predicate, this predicate checks the presence of routing information, not its stability.
#3 WaitForTimerExpiration This simple predicate assesses that a given amount of time has elapsed before executing the associated transi- tion. It is useful as in most reconfiguration scenarios each intermedi- ate step takes a bounded time to finish (e.g. a few minutes).
#4 WaitForManualInput This predicates waits for a manual input from the network operator. It is useful when the operator wants to keep full control of some critical reconfiguration steps (e.g., the launching time).
An actual migration corresponds to a pathCinit. . . Ck. . . Cf inin the mi- gration graph. If multiple paths exist betweenCinit andCf in, one path is chosen before the migration. Several criteria can be used to discriminate among several paths. For instance, an operator might prefer the path with the minimal amount of intermediate states or a path in which the recon- figuration steps are kept as local as possible.
As illustration, we consider a reconfiguration scenario performed in the Internet2 network (see Fig.1.2) where an iBGP full-mesh is replaced with a three-level route-reflection hierarchy. As reconfiguration strategy, we use
Cinit Cf in CA CL CZ . . . network-wide add UP(SEAT,SALT) add UP(SEAT,LOSA) WaitForManualInput predicate actions CK add UP(NEWY,CHIC) add UP(NEWY,ATLA) WaitForManualInput
remove all SEAT OVER WaitForBGPConvergence
CV
remove all NEWY OVER WaitForBGPConvergence
. . .
configuration
migration of the bottom-layer
Figure 8.2 Snapshot of a migration graph.
the best current practices [156, 198, 68] which consist in migrating routers in a bottom-up manner. Each router is migrated in two steps. First, the final UP sessions are established along with the initial OVER sessions. Once the duplicated routes have been propagated on the UP sessions, the initial OVER sessions are removed. Regarding the final route-reflection topology, it also follows the best current practices by closely following the physical topology. In particular, routers KANS and HOUS form the top-layer, routers SALT, LOSA, CHIC and ATLA, the intermediate one, while routers SEAT, NEWY and WASH form the lowest one. For redundancy purposes, each router is connected to the two route-reflectors located immediately above it.
A snapshot of the corresponding reconfiguration graph is depicted in Fig. 8.2. Also, Listing 8.10 shows how NCGuard can represent the entire migration scenario in less than 30 lines (lines 69–98). In this example, NC- Guard generates first all the intermediate configurations and associates each of them with an intermediate node in the reconfiguration graph (lines 72–77). The nodes are then linked together and each transition is associ- ated with a predicate (lines 79–92). A timer based predicate of five seconds is used between each transition. In addition to pilot the reconfiguration, NCGuard is also able to gather statistics during the migration, e.g., by log- ging the output of “show commands”. In this particular example, a “show ip bgp all” is performed on all routers after each migration step. The ac- tual migration process is finally launched (line 95). During this process, NCGuard uses the Command Line Interface (CLI) to interact with the net- work equipments. Since configuration changes often concern only specific parts of the configuration, only the changes are pushed on the devices. On average, the complete Internet2 migration process took less than 10 min-
to migrate an entire network within a single maintenance window (which typically lasts for a few hours only [199]).
Finally, a network operator will probably want to test its migration pro- cess beforehand, even when NCGuard is used. One way to do that is to perform the migration in a virtual environment. Thanks to NCGuard’s pro- grammatic interface, generating and provisioning virtual environments is really easy. At the time of writing, NCGuard supports the generation of dynamips [158] labs, which allow to virtualize Cisco routers. To further help network operators, NCGuard provides the ability to inject tailored (IGP or BGP) routes or routes feed collected from the existing network or from Routeviews [190]. The ability to generate a virtual lab out of a high- level representation is also useful for researchers. Indeed, it enabled us to validate the intended behavior of all the gadgets described in this thesis.
1 ################# Migration ######################
2 # We create the Lab
3 ucl_lab = Lab("UCLab") 4
5 # Add servers to the lab (multiple server support)
6 ucl_lab.add_server(Server(name="nostromo")) 7
8 # Generate the initial network-wide configurations
9 translate([abilene], lab=ucl_lab) 10
11 ################ Migration graph definition ########################
12 network_states = [] 13
14 # Generate the network-wide configuration of each intermediate state
15 for rid in compute_migration_order(final_ibgp, randomize=True):
16 print "Generating configuration for router %s" % (rid)
17 migrate_router(abilene.routers[rid], final_ibgp, lab=ucl_lab) 18 translate([abilene], lab=ucl_lab)
19 network_states.append(MigrationState(name="ITER#"+str(i)) 20
21 # Link the state together and associate the transitions with a predicate
22 migration_process = MigrationProcess(name="ABILENE_FM2RR", network=abilene) 23
24 for (head_state, tail_state) in zip(network_states, network_states[1:]):
25 migration_process.add_transition(
26 MigrationTransition(head_state, tail_state, TimerPredicate(5), 27 options = {’show_commands’ :
28 {
29 ’internal_routers’ : {
30 ’commands’: [’show ip bgp all’], 31 ’routers’ : abilene.routers.values()
32 }
33 }
34 }))
35
36 # Start the migration process !
37 migration_process.start()
Listing 8.10 Thanks to NCGuard’s programmatic interface, it is possible to describe the complete full-mesh to route-reflection scenario of the Internet2 network. On average, less than 10 minutes are required to complete the migration scenario.