Simulations were conducted in order to analyze the performance of the three models including [11] in terms of interference or collision probability Piunder different utiliza-
tion values, which is illustrated in Figure6.4. It is widely accepted that the complete MDP has a superior performance than the POMDP and IMM. The major concern is the
behaviour of these models for varying Λ and µ, which are the underlying basic con- straints in all three models. Figure6.4illustrates the behaviour of the IMM and MDP in terms of Pi and Pw for varying λ and µ values. As λ increases, Pw decreases prior
to an initial increase due to the user of interest having sensing model parameters such as δtnot altered, hence leading to an initial increase in the waste. The MDP follows a
similar behaviour but with a lower bound. On the other hand, Pi tends to increase for
increasing values of λ, where the difference between the models is negligible compared to the Pw.
Occupancy Rate λ
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Interference and Wastage
10-2 10-1 100 Pw, IMM, µ=0.2 Pw, IMM, µ=0.4 Pi, IMM, µ=0.4 Pi, MDP, µ=0.2 Pi, MDP, µ=0.4 Pw, MDP, µ=0.4 Pw, MDP, µ=0.2 Pi, IMM, µ=0.2
Utilization 10-1 Interference Probability P i 10-2 10-1 100 POMDP [1], λ,µ=0.2 POMDP [1], λ=0.2,µ=0.1 MDP, λ,µ=0.2 MDP, λ=0.2,µ=0.1 IMM [2], λ,µ=0.2 IMM [2], λ=0.2,µ=0.1
FIGURE6.5: Interference Probability of the three System Models
Figure6.5illustrates the utilization behaviour of the three models: IMM, POMDP, and MDP for varying interference probabilities. It shows that the gradual increase in utilization results in an increase in interference. The MDP model provides the least interference, while the IMM and POMDP are more significant.
One should note that the three models that are compared, tends to have different sys- tem and Markov model formulation. Hence, one cannot completely acknowledge the performance of each model as completely valid. This provides a novel concept of the MDP formulation and determines the degree of performance with the IMM and exist- ing POMDP of similar relevance.
6.6
Summary of Contribution & Conclusion
The main contribution of this chapter is the channel occupancy model behavior be- tween competing users as a four-state MDP process, taking into account of the Re- ceiver Operating Characteristics (ROCs) which comprise of PDand probability of false
alarm PF. The other contributions are the analysis of the trade-off between interference
and wastage probabilities of the channel, and the comparison of the degree of perfor- mance in utilization and interference against other models such as IMM and POMDP.
This chapter describes a decision making strategy through use of a Markov Decision Process for an environment comprising of the ongoing transmissions in the channel. The decision-making scheme is performed and analysed in terms of interference and wastage probabilities for various occupancy rates. Furthermore, comparisons are made between the IMM, POMDP and MDP in terms of utilization.
Conclusion and Future Work
7.1
Conclusion
In this thesis, we explore the three key areas of optimal decision making of Cognitive Radio in Cognitive Radio Networks as research objectives, and present some significant contributions. The thesis clearly identifies some of the objectives that are addressed in cognitive radio networks, and have provided some novel research material to address the relevant research objectives. The research objectives and the relevant contributions of this thesis are addressed individually in chapters 3 − 6. The three research objectives are based on some areas of cognitive radio networks, where some gaps were found in current literature. They are in threefold: research objective 1 sensing and learning in robust aerial-to-terrestrial communications, research objective 2 resiliency of authorized cognitive radio in cognitive radio networks against jamming attacks from unautho- rized cognitive radios, and research objective 3 optimal decision making of cognitive radio to improve spectral efficiency.
The first contribution in research objective 1 was an extensive model featuring aerial-to- terrestrial communications which considered fading channels from a theoretical per- spective, followed by a data set obtained from simulation where theory was limited (especially in the Rician case). The second contribution was the modelling of Pri- mary user transmissions as a simple Markov model, followed with Markov models for the Secondary user transmissions; which incorporated a Reinforcement Learning algorithm to determine the performance of the Secondary user transmissions in terms of interference and wastage of the channel.
The contribution in research objective 2 was the proposal of a means of successful trans- mission for the authorized cognitive radio under jamming attacks from unauthorized cognitive radios, which is formulated as a zero-sum game to model the resilience of the authorized cognitive radio. The framework was then extended by incorporating Fictitious Play learning for both users, to reach a Nash equilibrium. Additionally, we model the action set of the authorized cognitive radio as a multi-armed bandit model that uses Q-learning to learn the behaviour of the jamming attacks and decide upon a suitable strategy against the jamming strategy.
The contribution in research objective 3 was modelling the interaction between primary user and secondary user transmissions as a complete four-state Markov Decision Pro- cess. The utilization performance of the secondary user was compared with other exist- ing models such as Incomplete Markov model and Partial Observable Markov Decision Process. It is further extended by the performance analysis on the secondary user ’s uti- lization against wastage and interference of the channel of interest and compared with the Incomplete Markov model.
Throughout this timespan of the conducted research for this dissertation, we heavily relied upon verifying the analytical results and formulas against computer simulations using Monte-Carlo technique. We addressed practical numerical examples to reflect the usefulness of the stated methodologies. The vast majority of the work presented in this thesis was published in-part or as a whole in peer-reviewed journals, conference proceedings, book chapters, or otherwise currently undergoing a peer review process.