4 CHAPTER Spatial analysis of shallow geothermal energy at a regional scale
4.7 Shallow geothermal energy economic potential in the Region of Murcia
The order notation captures the idea of time and space complexities in precise mathematical terms. Let us start with the following important definition:
Let f and g be two positive real-valued functions on the set N of natural numbers. We call g(n) to be of the order of f(n) if there exist a positive real constant c and a natural number n0 such that g(n) <= cf(n) for all n >= n0. In this case we write
g(n) = O(f(n)) and also say that g(n) is big-Oh of f(n).
The following figure illustrates the big-Oh notation. In this example, we take c=2.
Figure: Explaining the big-Oh notation Examples
• Take f(n) = n and g(n) = 2n + 3. For n >= 3 we have 2n + 3 <= 3n. Thus taking the constants c = 3 and n0 = 3 shows that 2n + 3 = O(n). Conversely, for any n >= 1 we have n <= 2n + 3, i.e., n = O(2n + 3) too.
• Since 100n <= n2 for all n >= 100, we have 100n = O(n2). Now I will show that n2 is not of the order of 100n. Assume otherwise, i.e., n2 = O(100n), i.e., there exist constants c and n0 such that n2 <= 100cn for all n >= n0. This implies that n <= 100c for all n >= n0. This is clearly absurd, since c is a finite positive real number.
• The last two examples can be easily generalized. Let f(n) be a polynomial in n of degree d. It can be easily shown that f(n) = O(nd). In other words, the highest degree term in a polynomial determines its order. That is intuitively clear, since as n becomes sufficiently large, the largest degree term dominates over other terms.
Now let g(n) be another polynomial in n of degree e. Assume that d <= e. Then it can be shown that f(n) = O(g(n)). If d = e, then g(n) = O(f(n)) too.
However, if d < e, then g(n) is not O(f(n)).
Any function that is O(nd) for some positive integral constant d is said to be of polynomial order. A function which is O(n) is said to be of linear order. We can analogously define functions of quadratic order (O(n2) functions), cubic order (O(n3) functions), and so on.
• A distinguished case of polynomial order O(nd) corresponds to the value d = 0. A function f(n) of this order is an O(1) function. For all sufficiently big n, f(n) is by definition bounded from above by a constant value and so is said to have constant order.
• I will now show n = O(2n). We prove by induction on n that n <= 2n for all n >= 1. This is certainly true for n = 1. So assume that n >= 2 and that n 1 <= 2n-1. We also have 1 <= 2n-1. Adding the two inequalities gives n <= 2n. The converse of the last order relation is not true, i.e., 2n is not of the order of n. We prove this by contradiction. Assume that 2n = O(n), i.e., 2n <= cn for all n >= n0. Simple calculus shows that the function 2x / x on a real variable x tends to infinity as x tends to infinity. In particular, c cannot be a bounded constant in this case.
A function which is O(an) for some real constant a > 1 is said to be of exponential order. It can be shown that for any a > 1 and d >= 1 we have nd = O(an), but an is not of the order of nd. In other words, any polynomial function grows more slowly than a (truly) exponential function.
• A similar comparison holds between logarithmic and polynomial functions. For any positive integers d and e, the function (log n)d is O(ne), but ne is not O((log n)d). Functions of polynomial, exponential and logarithmic orders are most
widely used for analyzing algorithms.
We now explain how the order notation is employed to characterize the time and space complexities of a program. We count the number of basic operations performed by an algorithm and express that count as having the order of a simple function. For example, if an algorithm performs 2n2 - 3n + 1 operations on an input of size n, we say that the algorithm runs in O(n2) time, i.e., in quadratic time, or that it is a quadratic time algorithm. Any algorithm that runs in polynomial time is said to be a polynomial-time algorithm. An algorithm that does not run in polynomial time, but in exponential time, is called an exponential-time algorithm. An exponential function (like 2n) grows so rapidly (compared to polynomial functions) with the input n that exponential-time algorithms are usually much slower compared to polynomial-time algorithms, even when the input is not too big. By an efficient solution of a problem, one typically means devising an algorithm for that problem, that runs in some polynomial time O(nd) with d as small as possible.
Examples
We now analyze the complexities of some popular algorithms discussed earlier in the notes.
• Computation of factorials
In this case we express the running-time as a function of the integer n whose factorial is to be computed. Let us first look at the following iterative algorithm:
int factorialIter ( int n ) {
int prod, i;
if (n <= 1) return 1;
prod = 1;
for (i=2; i<=n; ++i) prod *= i;
return prod;
}
The function first compares n with 1. If n is indeed less than or equal to 1, the constant value 1 is returned. Thus for n = 0,1 the algorithm does only one basic operation (comparison). Here we neglect the cost of returning a value. If n > 2, then prod is first initialized to 1. Then the loop starts. The loop contains an initialization of i, exactly n-1 increments of i and exactly n comparisons of i with n. Inside the function body the variable prod is multiplied by i. The loop is executed n-1 times. This accounts for a total of n-1 multiplications. Thus the total number of basic operations done by this iterative function is
1 + 1 + 1 + (n-1) + n + (n-1) = 3n + 1.
Since 3n + 1 is O(n), it follows that the above algorithm runs in linear time.
Next consider the following recursive function for computing factorials:
int factorialRec ( int n ) {
if (n <= 1) return 1;
return n * factorialRec(n-1);
}
Let T(n) denote the running time of this recursive algorithm for the input n. If n
= 0,1, then T(n) = 1, since computation in these cases involves only a single comparison. If n >= 2, then in addition to this comparison, factorialRec is called on input n-1 and then the return value is multiplied by n. To sum up, we have:
T(0) = 1, T(1) = 1,
T(n) = 1 + T(n-1) + 1 = T(n-1) + 2 for n >= 2.
This is not a closed-form expression for T(n). A formula for T(n) can be derived by repeatedly using the last relation until the argument becomes too small (0 or 1) so that the constant value 1 can be substitued for it.
T(n) = T(n-1) + 2
= (T(n-2) + 2) + 2 = T(n-2) + 4 = T(n-3) + 6
...
= T(1) + 2(n-1) = 1 + 2(n-1) = 2n - 1.
Therefore, T(0) = 1,
T(n) = 2n - 1 for n >= 1.
It follows that the recursive function also runs in linear time. Note that both the iterative and recursive versions run in O(n) time. But the actual running times are respectively 3n + 1 and 2n - 1. It may appear to the reader that the recursive function is faster (since 2 is smaller than 3). But in the analysis, we have neglected the cost of function calls and returns. The iterative version makes no recursive calls, whereas the recursive version makes n-1 recursive calls. It depends on the compiler and the run-time system whether n-1 recursive calls is slower or faster than the overhead associated with the loop in the iterative version.
Still, we should feel happy to end the story by rephrasing the fact that both the two versions are equally efficient -- as efficient as an O(n) function.
• Computation of Fibonacci numbers
With Fibonacci numbers, the iterative and recursive versions exhibit marked difference in running times. We start with the iterative version.
int fibIter ( int n ) {
int i, p1, p2, F;
if (n <= 1) return n;
i = 1; F = 1; p1 = 0;
while (i < n) { ++i;
p2 = p1;
p1 = F;
F = p1 + p2;
}
return F;
}
The function initially makes a comparison and if n = 0,1 the value n is returned.
For n >= 2, it proceeds further down. First, three variables (i,F,p1) are
initialized. The subsequent while loop is executed exactly n-1 times. The body of the loop involves four basic operations (one increment, two copies and one
addition). Moreover, the loop continuation condition is checked n times. So the number of basic operations performed by this iterative algorithm is
1 + 3 + 4(n-1) + n = 5n.
In particular, fibIter runs in linear time.
Let us now investigate the recursive version:
int fibRec ( int n ) {
if (n <= 1) return n;
return fibRec(n-1) + fibRec(n-2);
}
Let T(n) denote the running time of this recursive function on input n. Simple investigation of the function shows that:
T(0) = 1, T(1) = 1,
T(n) = T(n-1) + T(n-2) + 2 for n >= 2.
Now it is somewhat complicated to find a closed-form formula for T(n). We instead give an upper bound and a lower bound on T(n). To that effect let us first introduce a new function S(n) as:
S(n) = T(n) + 2 for all n.
We then have:
S(0) = 3, S(1) = 3,
S(n) = S(n-1) + S(n-2) for n >= 2.
Denote by F(n) the n-th Fibonacci number and use induction on n. S(0) <=
F(4) and S(1) <= F(5). Moreover, S(n) = S(n-1) + S(n-2) <= F(n+3) + F(n+2) = F(n+4). A lower bound on S(n) can be derived by induction on n as:
S(0) >= F(3) and S(1) >= F(4). Moreover, S(n) = S(n-1) + S(n-2) >=
F(n+2) + F(n+1) = F(n+3). It follows that:
F(n+3) - 2 <= T(n) <= F(n+4) - 2 for all n >= 0.
The next question is to find a closed form formula for the Fibonacci numbers. We will not do it here, but present the well-known result:
F(n) = [1/sqrt(5)]
[
((1+sqrt(5))/2)n - ((1-sqrt(5))/2)n]
.The number r = (1+sqrt(5))/2 = 1.61803... is called the golden ratio. Also (1-sqrt(5))/2 = -0.61803... is the negative of the reciprocal of the golden ratio and has absolute value less than 1. The powers [(1-sqrt(5))/2]n become very small for large values of n and so
F(n) is approximately equal to [1/sqrt(5)]rn. For all sufficiently large n, we then have
[1/sqrt(5)]rn+3 - 2 <= T(n) <= [1/sqrt(5)]rn+4 - 2
The first inequality shows that T(n) cannot have polynomial order, whereas the second inequality shows that T(n) is of exponential order.
To sum up, recursion helped us convert a polynomial-time (in fact, linear) algorithm to a truly exponential algorithm. This teaches you two lessons. First, use recursion judiciously. Second, different algorithms (or implementations) for the same problem may have widely different complexities. Performance analysis of programs is really important then!
• Linear search
We are given an array A of n integers and another integer x. The task is to locate the existence of x in A. Here n is taken to be the input size. We assume that A is not sorted, i.e., we will do linear search in the array. Here is the code:
int linSearch ( int A[] , int n , int x ) {
int i;
for (i=0; i<n; ++i) if (A[i] == x) return 1;
return 0;
}
The time complexity of the above function depends on whether x is present in A and if so at which location. Clearly, the worst case (longest running time) occurs when x is not present in the array and the last statement (return 0;) is executed.
In this case the loop requires one initialization of i, n increments of i and n+1 comparisons of i with n. Inside the loop body there is a single comparison which fails in all of the n iterations of the loop in the worst-case scenario. Thus the total time needed by this function is:
1 + n + (n+1) + n = 3n + 2.
This is O(n), i.e., the linear search is a linear time algorithm.
• Binary search
In order to curtail the running time of linear search, one uses the binary search algorithm. This requires the array A to be sorted a priori. We do not compute the running time for sorting now, but look at the running time of binary search in a sorted array.
int binSearch ( int A[] , int n , int x ) {
int L, R, M;
L = 0; R = n-1;
while (L < R) { M = (L + R) / 2;
if (x > A[M]) L = M+1; else R = M;
}
return (A[L] == x);
}
For simplicity assume that the array size n is a power of 2, i.e., n = 2k for some integer k >= 0. Initially, the boundaries L and R are adjusted to the leftmost and rightmost indices of the entire array. After each iteration of the while loop the central index M of the current search window is computed. Depending on the result of comparison of x with A[M], the boundaries (L,R) is changed either to (L,M) or to (M+1,R). In either case, the size of the search window (i.e., the subarray delimited by L and R) is reduced to half. Thus after k iterations of the while loop the search window reduces to a subarray of size 1, and L and R
become equal. After the loop terminates, a comparison is made between x and an array element. So the number of basic operations done by this algorithm equals:
2 + (k+1) + k x (2 + 1 + 1) + 1
(Init) (Loop condn) (No of iter) (ops in loop body) (last comparison)
= 5k + 4.
But k = log2n, so the running time of binary search is O(log n), i.e.,
logarithmic. This is far better than the linear running time of the linear search algorithm.
• Bubble sort
It is interesting to look at the running times of different sorting algorithms. Let us start with a non-recursive sorting algorithm. Here is the code that bubble sorts an array of size n.
void bubbleSort ( int A[] , int n ) {
for (i=n-2; i>=0; --i) {
for (j=0; j<=i; ++j) { if (A[j] > A[j+1]) { t = A[j];
A[j] = A[j+1];
A[j+1] = t;
} } } }
This is an example of a nested for loop. The outer loop runs over i for the values n-2,n-3,...,0 and for a value of i the inner loop is executed i+1 times. This means that the inner loop is executed a total number of
(n-1) + (n-2) + ... + 2 + 1 = n(n-1)/2
times. Each iteration of the inner loop involves a comparison and conditionally a set of three assignment operations. Thus the inner loop performs at most
4 x n(n-1)/2 = 2n(n-1)
basic operations. This quantity is O(n2). We should also add the costs associated with the maintenance of the loops. The outer loop requires O(n) time, whereas for each i the inner loop requires O(i) time. The n-1 iterations of the outer loop then leads to a total of O((n-1) + (n-2) + ... + 1), i.e., O(n2), basic operations for maintaining all of the inner loops. To sum up, we conclude that the bubble sort algorithm runs in O(n2) time.
• Matrix multiplication
Here is the straightforward code for multiplying two n x n matrices. We take n as the input size parameter.
/* Multiply two n x n matrices A and B and store the product in C */
void matMul ( int C[SIZE][SIZE] , int A[SIZE][SIZE] , int B[SIZE][SIZE] , int n )
{
int i, j, k;
for (i=0; i<n; ++i) { for (j=0; j<n; ++j) { C[i][j] = 0;
for (k=0; k<n; ++k) C[i][j] += A[i][k] * B[k][j];
} } }
This is another example of nested loops with an additional level of nesting (compared to bubble sort). The outermost and the intermediate loops run
independently over the values of i and j in the range 0,1,...,n-1. For each of
these n2 possible values of i,j, the element C[i][j] is first initialized and then the innermost loop on k is executed exactly n times. Each iteration in the
innermost loop involves one multiplication and one addition. Therefore, for each i,j the innermost loop takes O(n) running time. This is also the cost associated with maintaining the loop on k. Thus each execution of the body of the
intermediate loop takes a total of O(n) time and this body is executed n2 times leading to a total running time of O(n3). It is easy to argue that the cost for maintaining the loop on i is O(n) and that for maintaining all of the n executions of the intermediate loop is O(n2).
So two n x n matrices can be multiplied in O(n3) time. Can we make any better than that? The answer is: yes. There are algorithms that multiply two n x n matrices in time O(nw) time, where w < 3. One example is Straßen's algorithm that takes time O(nlog2(7)), i.e., O(n2.807...). The best known matrix multiplication algorithm is due to Coppersmith and Winograd. Their algorithm has a running time of O(n2.376). It is clear that for setting the value of all C[i][j]'s one must perform at least n2 basic operations. It is still an open question whether O(n2) running time suffices for matrix multiplication.
• Stack ADT operations
Look at the two implementations of the stack ADT detailed earlier. It is easy to argue that each function (except print) performs only a constant number of operations irrespective of the current size of the stack and so has a running time of O(1). This is the reason why we planned to write seperate routines for the stack and queue ADTs instead of using the routines for the ordered list ADT. Insertion or deletion in the ordered list ADT may require O(n) time, where n is the current size of the list.
• Partitioning in quick sort
This example illustrates the space complexity of a program (or function). We concentrate only on the partitioning stage of the quick sort algorithm. The
following function takes the first element of the array as the pivot and returns the last index of the smaller half of the array. The pivot is stored at this index.
int partition1 ( int A[] , int n ) {
int *L, *R, lIdx, rIdx, i, pivot;
L = (int *)malloc((n-1) * sizeof(int));
R = (int *)malloc((n-1) * sizeof(int));
pivot = A[0];
lIdx = rIdx = 0;
for (i=1; i<n; ++i) {
if (A[i] <= pivot) L[lIdx++] = A[i];
else R[rIdx++] = A[i];
}
for (i=0; i<lIdx; ++i) A[i] = L[i];
A[lIdx] = pivot;
for (i=0; i<rIdx; ++i) A[lIdx + 1 + i] = R[i];
free(L); free(R);
return lIdx;
}
Here we collect elements of A[] smaller than or equal to the pivot in the array L and those that are larger than the pivot in the array R. We allocate memory for these additional arrays. Since the sizes of L and R are not known a priori, we have to prepare for the maximum possible size (n-1) for both. In addition, we use a constant number (six) of variables. The total additional space requirement for this function is therefore
2(n-1) + 6 = 2n + 4, which is O(n).
Let us plan to reduce this space requirement. A possible first approach is to store L and R in a single array LR of size n-1. Though each of L and R may be
individually as big as having a size of n-1, the total size of these two arrays must be n-1. We store elements of L from the beginning and those of R from the end of LR. The following code snippet incorporates this strategy:
int partition2 ( int A[] , int n ) {
int *LR, lIdx, rIdx, i, pivot;
LR = (int *)malloc((n-1) * sizeof(int));
pivot = A[0];
lIdx = 0; rIdx = n-1;
for (i=1; i<n; ++i) {
if (A[i] <= pivot) LR[lIdx++] = A[i];
else LR[rIdx--] = A[i];
}
for (i=0; i<lIdx; ++i) A[i] = LR[i];
A[lIdx] = pivot;
for (i=rIdx+1; i<n; ++i) A[i] = LR[i];
free(LR);
return lIdx;
}
The total amount of extra memory used by this function is (n-1) + 5 = n + 4,
which, though about half of the space requirement for partition1, is still O(n). We want to reduce the space complexity further. Using one or more additional arrays will always incur O(n) space overhead. So we would avoid using any such
extra array, but partition A in A itself. This is called in-place partitioning. The function partition3 below implements in-place partitioning. It works as follows.
It maintains the loop invariant that at all time the array A is maintained as a concatenation LUR of three regions. The leftmost region L contains elements smaller than or equal the pivot. The rightmost region R contains elements bigger than the pivot. The intermediate region U consists of yet unprocessed elements.
Initially, U is the entire array A (or A without the first element which is taken to be the pivot), and finally U should be empty. The region U is delimited by two indices lIdx and rIdx indicating respectively the first and last indices of U. During each iteration, the element at lIdx is compared with the pivot, and depending on the comparison result this element is made part of L or R.
The function partition3 uses only four extra variables and so its space complexity is O(1). That is a solid improvement over the earlier versions.
It is easy to check that the time complexity of each of these three partition routines is O(n).