Although the path service is “centralized” (in the sense that each source sends queries to a specific place), the global path service can be distributed to improve computa- tional scalability. In particular, since each access domain only needs paths from itself to other access domains (and possibly paths from other domains to itself), the com- putation of paths from one access domain can be handled by a local path service as
shown in Figure 3.3. Moreover the local path service needs to compute paths from
the local access domain to all other domains, not all-to-all. This greatly reduces the computational complexity.
Deploying a path service at each access domain improves the computational scal- ability of the global path service, but at the same time increases the bandwidth overhead. In particular, path services at each access domain must periodically collect relay advertisements from each transit domain. There are possible ways of reducing the bandwidth overhead of the advertisements. Our approach is to reduce the over- head by controlling the frequency of advertisement collection by the path service. An important observation is that the path service can use a pull (i.e., on-demand) model to collect relay advertisements. By using an on-demand collection mechanism, the path service can control the frequency at which the relay advertisements are collected from the transit domains. For example, it is expected that the large (e.g., backbone) transit domains, where a large number of flows are aggregated, have relatively stable performance measurements compared to smaller transit providers. The path service can also select the frequency of relay advertisement collection from different transit domains according to their “popularity”. For example, if a set of transit domains are heavily used (i.e., popular) by the end-systems to relay packets compared to the other transit domains, then the path service can collect relay advertisements from those domains more frequently than from the others. A comprehensive solution to collecting performance information with minimal bandwidth overhead is described
by Wu in [84]. In this thesis, we mostly focus on the computational overhead of computing paths rather than the bandwidth overhead.
Transit Provider Transit Provider Transit Provider PS Access Provider PS Access Provider PS Access Provider Transit Provider PS Access Provider PS Access Provider PS Access Provider PS Access Provider PS List of Paths Path Query Sender Path Service Relay Advertisements
Figure 3.3: Inter-network with transit providers and access providers with local path services.
The conventional wisdom on source routing mandates that accurate topology maps of the global Internet be used to compute end-to-end paths, and accurate topology maps require awareness to changes in the routing state that happen at very short time- scales (in milliseconds). Based on this conventional wisdom, source routing is very hard to scale because of (i) the high frequency of route updates, and (ii) path services repeating path computations with the frequent arrival of new routing information rather than being able to use pre-computed paths.
Our hypothesis is that awareness of changes at very short time-scales is not nec- essary because end systems can observe the state of the relevant parts of the network by performing end-to-end measurements. The measurements can either be made in- band during a data transfer or by simply sending probe packets. Instead of notifying the path service of short-term changes in the network, transit providers can provide performance information that contains information such as a statistical summary of measurements over a longer (e.g., 10 seconds to minutes) period.
plexities and having to return paths to end systems, upon path requests, in a timely manner. The path service must respond to queries (i.e., compute and return paths) with latency comparable to the time it takes it takes for users in the current Internet to resolve a DNS query, prior to sending their first data packets. In particular, the time to obtain paths from the path service should not exceed an Internet round-trip time of up to a few tens of milliseconds for the majority (say, 95%) of the queries.
Periodically collecting large amounts of routing information containing dynamic performance information is also a challenge. However, we only consider domain- level routing information that is aggregated to hide the inner details of the domains. Therefore, the amount of routing information is on the order of the number of transit domains in the global Internet and not the number of routers. However, each transit domain can produce routing information (i.e., relay advertisement) that is quadratic in terms of the number of channels that the domain has (in theory, a domain can advertise multiple relays for each of its ingress, egress channel pairs). The path service must keep up with the advertisements that are periodically obtained from the transit domains. More importantly, the service must provide the most recent available information about the performance characteristics of any returned paths, to help the application—the path characteristics are derived from the information for the individual relays making up the path.
Given the above model, the challenge is to design a path service that can provide, on demand, multiple paths that satisfy a set of application-specific constraints from a set of source domains (one or many) to any given destination access domain in a timely manner. It must perform this function in a graph at least the scale of
today’s Internet AS graph with 105 – 106 nodes. On-demand computation of paths
that satisfy multiple constraints require the path service to consider thousands to millions of possible paths between access providers in a limited amount of time. The computation should use the most recently collected performance information. The
second challenge is to scale to large networks with arbitrarily large number of paths between access domains.
3.4
Assumptions
In this section, we list our assumptions about the routing architecture that the path
service requires. In Section 3.4.1, we describe a possible money flow between the
stakeholders—that is, the path services, the domains, and the users. Then, we ex- plain mechanisms by which transit domains can enforce their transit policies in Sec- tion 3.4.2.
3.4.1
Economic Considerations
The flow of compensation in our system is as follows. Access domains make money by charging users to connect to the network. Transit domains make money by charging for transit service. Path services make money by charging users—either directly, or indirectly by charging their access providers—for providing paths, and access to the transit services that implement those paths. Thus, users pay access providers and path services; path services pay transit providers for the use of their relay services. Transit providers should receive compensation (directly or indirectly) from any users or access domains whose traffic they forward. One thing that makes this feasible is that the path service acts as a clearinghouse for payments from users to transit providers. Such payments might occur on various time scales. In the limit, a path service might charge for each individual path query it fulfills. More plausible, however, would be a subscription type service, in which users (or their access domains) contract with a path service for a longer time (e.g., over a month period) interval.
One of the features of this system is that money follows traffic flow, while traffic flow is only weakly constrained. This is in contrast to the current Internet ecosystem, in which traffic is (mostly) constrained to flow between customer AS’s and their
providers. In other words, in the current Internet ecosystem, traffic flow follows money flow. Because the money flow changes very slowly, competition based on service quality is effectively inhibited.
It is quite possible that brokers (which act as intermediaries among users, ac- cess domains, and path services) or other kinds of “middlemen” would arise in this ecosystem. We do not consider them here, although they are quite compatible with our model.
3.4.2
Policy Enforcement
It remains to be shown that transit providers can enforce their policies—that is, control access to their resources. If all path services receive all relay advertisements, and a path service constructs many possible paths (up to some maximum path length), in principle any access domain can send packets via any of a large set of transit providers. However, a transit domain should only carry traffic that it knows it has been (or will be) compensated for. The problem is that once a source knows a path, it can continue to use it forever (including after it stops paying the path service), unless some access control mechanism is provided. The form of such a mechanism will depend on the way forwarding is implemented. Here we discuss two possibilities that correspond to an SDN-based forwarding approach and a source-routed approach to forwarding.
• Stateful, Software Defined Networking (SDN) based enforcement: In this method,
when a path service returns a path to a user, it informs an SDN controller for each transit domain in the path. Each controller then installs state in its do- main, which causes packets arriving on the specified ingress channel that match a specified pattern—say, source and destination IP address (or prefix) and/or flow ID—to be forwarded through the network to the specified egress channel. The transaction between the user and the path service gives the user access
to the returned paths for some period of time, after which the switch state is removed from the transit domains, revoking that user’s access. Note that no co-ordination is required between the SDN controllers for the different domains. The advantage of this approach is that it can be made backward-compatible, using existing protocol headers. It has the usual disadvantages of stateful for- warding in networks, and also increases the pre-transmission latency. We discuss
SDN in Section 4.1.
• Stateless, in-band enforcement: In this approach, packets carry an explicit rep-
resentation of the path in the packet, along with a proof of policy-compliance for each domain in the path. Such a proof could take the form of a cryptographically- derived token, based on a secret shared between the path service and the tran- sit domain, and containing an expiration time. Such a scheme for verifying
policy-compliance of source-routed packets is described in Platypus [66], for
example. The ingress switch in each transit domain verifies compliance, then simply forwards the packet to the indicated egress channel. This approach has the advantage of greatly simplifying the state requirements of transit domains. It has the disadvantage of adding to the computational load on the data plane, and significantly increasing the overhead in the packet. Both of these costs can be amortized over multiple uses, for example by verifying packets at random intervals.