A worker is a combined fast-joiner and path sorter as shown in Figure6.3. The main
task of the worker is to compute results for a particular query. The workers compute the results of a query in parallel. In order to distribute the work of computing query results, each worker is assigned a single partition of slow-changing path records, and the workers compute the results using only the partition that is assigned to them. After the master forwards the query to the workers, each worker computes and returns a list of path records. The master then merges the individual results from each worker to produce the final results, as discussed before.
The task of the fast-joiner component is to compute the fast-changing attributes of the path records in the assigned partition of path records using the fast-changing relay (routing) information. The fast-changing information consists of a list of relay records, which contains both slow- and fast-changing attributes of relays. Similar to path records, relay records are kept in the main memory for fast access by the workers. Once the fast-changing attributes of paths are computed, they are stored as part of the path record to use for future queries. However, the fast-changing attributes of a path record need to be re-computed when the path record becomes stale. As mentioned before, a path record becomes stale when the fast-changing attributes of one or more relays used in the path change. In order to check for staleness, both path records and relay records contain a member to record the time of the most recent write to the fast-changing attribute. We refer to path records whose fast-changing
attributes are computed as fast-changing path records in Figure6.3. The actual worker
implementation uses a single copy of each path record; the two partitions—slow-
changing and fast-changing—are used in the Figure 6.3 for presentation purposes.
In our experiments, we use latency and available bandwidth as the two fast- changing attributes. Both of the attributes are represented in the form of histograms with five bins that represent probability distributions. The value of each bin is an
integer between zero and 100, and the sum of the bin values in a histogram is 100. As mentioned before, an external (simulation) component named update-generator periodically sends relay (i.e., routing) information updates to the path service. These updates contain fast-changing attributes for a list of relays, with randomly generated histograms of probability distributions. The fast-joiner component computes convolu- tions on the latency histograms and minimum operations on the available bandwidth histograms to produce path histograms.
The path sorter component performs a linear search on path records to obtain the
top kmax paths, where kmax is the largest k value that the system allows for queries.
We set kmax to five in our experiments as this provides senders with a reasonable
number of alternatives. As the path sorter goes through the list of path records, it maintains a list of the current top five path records. Once the search is complete, the list of the top five path records is returned to the master. In order to obtain the top five paths, the path sorter computes the likelihood of each path record satisfying the constraints on latency and available bandwidth. The constraint on latency is an upper-bound, while the constraint on available bandwidth is a lower-bound value. In the implementation, the fast-join and path sorter components operate together in a sequence—they are implemented as part of the worker thread. After the fast-joiner computes the attributes of a path, the path sorter computes its likelihood of satisfying the constraints.
Each worker maintains an index to locate the path records for a specific source and destination channel in its path record partition. Upon receiving a query, a worker first looks up in its index to locate paths whose source and destination matches the source and the destination requested in the path query. Our hypothesis is that when each partition of path records contains roughly the same number of paths with similar lengths for a given source-destination pair, each worker has roughly the same
computational load.4 We test our hypothesis in Section 6.4.4. The pre-computed paths by the slow-joiner are partitioned in such a way as to ensure that each worker obtains a similar load, as follows:
1. Pre-computed paths are grouped by source and destination domains.
2. Within each group, paths are sorted by their length (i.e., number of relays). 3. Grouped and sorted paths are assigned to workers in a round-robin fashion.
The majority of the time that a worker spends computing query results is during the fast-join operation. During this operation, the worker iterates over the paths in its partition and computes the path attributes through a sequence of histogram operations (i.e., convolution and minimum). In order to improve the performance of this step, we perform an additional optimization step to reduce the repeated com- putations for paths that have common portions. This optimization step is explained next.