5.5 Illustrative Example - Portfolio Proving
Now let us consider an example that is more relevant to Automated Reasoning. The framework makes it easy to design an automated theorem proving system using a portfolio approach. We can do this as follows:
Broadcastable Artefacts: three broadcastable artefacts are defined - axioms, theorem and proof. They each are string lists of length two. The first string indicates the artefact type and can be “a”, “t” or “p”. The second is either a set of axioms, a theorem or the details of the proof of a theorem. Axioms and theorems are presented in TPTP [171] format.
Processes: three types of process are defined, namely AxiomBG, TheoremBG and Prover.
In each round, the AxiomBG process, proposes an axioms broadcast, containing the ax-ioms to be used for proving. It gives this proposal an importance rating of 1. These axioms are in TPTP format and pre-defined by the user. AxiomBG detaches when an axioms artefact is broadcast (it identifies that its proposal has been successful). Theo-remBG reacts to the broadcast of an axioms artefact by proposing a theorem artefact containing the theorem to be proved. Again this is rated as 1 and the theorem is in TPTP format. TheoremBG detaches when a theorem artefact is broadcast. Each Prover embod-ies a different automated theorem prover. These could be different prover applications or the same prover application configured or parameterised differently. Each theorem prover must be able to understand the TPTP format for problem specification. These processes react to the broadcast of an axioms artefact by storing those axioms for future use in performing proofs. They react to theorem artefacts by appealing to their prover to prove the theorem using their previously seen axioms, subject to a user-defined timeout limit. If the prover is successful, the Prover process proposes a proof artefact with an importance rating of 1. Otherwise it proposes nothing.
Initial State: a number of Prover processes are attached. One for each prover/prover-configuration the user wishes to apply. In addition, there is one AxiomBG process, configured with the user’s required axioms, and one TheoremBG, which has the theorem the user wishes to prove. Processing begins with a null broadcast.
We have not yet implemented such a system. However, an example of how processing might progress in the hypothetical system is shown in table 5.3. AxiomBG proposes axioms until they are broadcast, whereupon it detaches. This broadcast triggers the proposal of the theorem by TheoremBG. When this is broadcast, the Prover processes attempt to prove the theorem. In the example, only Prover2 finds a proof in the allotted time and this is broadcast in round 4.
The reasoning process ends once a proof has been proposed and broadcast. Alternatively, if no prover can find a proof, then there will no proposals following the theorem broadcast and, again, processing, and reasoning, will end.
Processes
AxiomBG TheoremBG Prover1 Prover2 . . .
Rnd B/c Prop. Imp. Prop. Imp. Prop. Imp. Prop. Imp. . . .
1 null [a,axioms] 1 - - - . . .
2 [a,axioms] detach [t,theorem] 1 - - - - . . .
3 [a,theorem] detach - - [p,proof ] 1 . . .
4 [p,proof ] - - - - . . .
Table 5.3: Processing in the portfolio prover system.
The configuration above allows users to prove theorems using a number of different provers simultaneously. This example demonstrates how this architecture lends itself quite well to a distributed version of this system. Each process could be placed on a different processor. When a theorem artefact is broadcast, the provers on these separate processors will simultaneously attempt the proof using their local resources. Under our framework, all processes are given time to consider their response before a new round begins. Hence, we have included a timeout here to stop processes from attempting, unsuccessfully, to prove theorems for long periods, potentially locking the system and preventing the broadcast of a successful proof. In §13.4.1, we consider asynchronous processing models which would allow processing to continue without waiting for all processes to respond.
5.5. Illustrative Example - Portfolio Proving 99
5.5.1 Extensions to the Portfolio Prover
This example is perhaps the most simple design of a portfolio prover system that we could consider, there are many potential upgrades to this design. For example, the ratings of impor-tance do not play a significant part in guiding overall processing. In early rounds, only one proposal is made, which means it is automatically chosen for broadcast. If there are a number of successful proofs, the above configuration would broadcast one chosen randomly from them all. A simple enhancement to the rating approach would be to make Prover processes assign importance values based upon the details of the proof, such as proof length or some other com-plexity measure. This would guarantee broadcast of the simplest of several proposed proofs, which may be more desirable for the user. In the configurations we describe in chapter 7 and thereafter, we will see more complicated process interactions where importance rating schemes are used.
Another enhancement would be translation. Some users may prefer to provide background axioms in another format, for example the Otter-format syntax [129]. We could introduce a new Translator process, with the ability to convert Otter into TPTP. In addition, we would add additional flags to the axioms and theorem artefacts by making them lists of length three rather than two. The additional string would read either “otter” or “tptp” to indicate the format. AxiomBG and TheoremBG would be pre-configured by the user with an indicator of their format and create proposals appropriately. The new Translator process would respond to the Otter format flag of axioms and theorem artefacts by proposing translated versions with the tptp format flag. TheoremBG would only react to artefacts with the tptp format. We would now be able to introduce provers which use different input formats. For example, we could now use provers that only understand Otter format. We would encapsulate them in Prover processes that are configured to react to the appropriate format flag.
5.6 Conclusions
In this chapter, we described our framework for Combined Reasoning based upon the Global Workspace Architecture. The simple examples in this chapter demonstrate how the framework operates and how users can configure the background for different runs. We also discussed how simple design decisions affect the efficiency of design. Furthermore, our portfolio prover system shows how it is possible to design effective reasoning systems using the framework and how these have to potential to easily translate into parallel implementations.
The framework itself is very simple. We have only deviated from the underlying GWA theory by allowing the termination and spawning of processes. Such behaviour is not considered by the basic theory, which only considers a static set of attached specialist processes. Maintaining this low complexity has several benefits for clarity of design and extensibility. For example, processes encapsulate specific component skills. Adding particular system capabilities is aligned with introducing one or more new specialist processes. Such modularity makes the system design clear. They also allow for straightforward interpretation of the overall system as a parallel implementation. In addition, there is only one point of communication between processes. It is very easy to see the information transfer between processes and monitor how overall processing is progressing. This common communication point simplifies enhancement of a system. In our examples, we saw how we could introduce new capabilities by hooking into this point of communication. For example, we introduced Otter translation into the portfolio prover by re-specifying the broadcastable artefacts. We amended how they are proposed and how processes react to them. In essence, we broke the chain of broadcasts to introduce a new processing loop. The simplicity, clarity and extensibility of the framework are appealing from a software engineering viewpoint. In later chapters, we will see how, despite its simplicity, the framework allows us to create effective combined reasoning systems.
In the next chapter, we describe the GC toolkit. This is a software toolkit which allows users to develop concrete systems according to the framework and allows them to parameterise and run systems for specific tasks.