PR ESUPUESTO Concepto

Broadly speaking,SyGuScan be thought of as a_{meta-synthesis}framework: it essentially allows a concise description ofanysynthesis problem whose solution space can be described using a

context-free grammar and function symbols in some combination of background SMT theories, and whose properties can be described using a universally quantified formula. We now compare and contrastSyGuSwith some other frameworks that have been proposed in recent literature, which have similar goals.

5.4.1

sketch

and Rosette

sketch[SLRBE05, STB+_{06, SAT}+_{07, SLJB08, Sol09] and Rosette [TB13, TB14] are both}

meta-synthesis frameworks that were designed to beembeddedwithin a language: thesketch

language isC-like, whereas Rosette is embedded within the functional language, Racket. As withSyGuS, the space of programs is described using a context-free grammar in Rosette, and usinggenerators— which use a combination of regular and context-free constructs to describe

the search space — insketch.

UnlikeSyGuS, these frameworks allow the specification for the program to be synthesized to be written as a program. sketchuses a subset of theCprogramming language to describe the behavior of the program to be synthesized. ThisCprogram could possibly be sub-optimal or unoptimized, with the sketch for the program describing theshapeof an optimized version.

SyGuS sketch Rosette FlashMeta

Specification language SMTLIB-like C-like Racket Inductive spec.

Program space CFG Generators CFG CFG

Full formal specifications Yes Yes Yes No

Inductive specifications Yes Yes Yes Yes

Solvers extensively use ranking? No No No Yes

Language and Platform agnostic? Yes No No Relatively

Intended audience _ResearchersSynthesis Programmers Programmers,_Students Domain_experts

Existence of multiple solvers Yes No No No

Table 5.1: Comparison of various meta-synthesis frameworks

combination of assertions, pre-conditions and post-conditions on the program. Needless to say, these specification techniques can be much more expressive than the first order specifications thatSyGuS allows. As a consequence, these techniques sometimes require inputs from the programmer — in the form ofpragmasin the case ofsketch— or restrict the language to a safe, and decidable subset — as is the case with Rosette.

The differences betweensketchand Rosette on the one hand andSyGuSon the other stem from the design choices made with the intended audience in mind.sketchand Rosette are both intended to enable programmers synthesize usable code, whereasSyGuSintends to cleanly abstract the core synthesis problem in a language and platform agnostic manner, to encourage adoption and spur research in program synthesis techniques. Indeed, the relatively low entry barrier has led to a multitude of solvers competing in the 2015SyGuScompetition. Lastly, we note that regardless of the exact logic used to specify properties of the program to be synthesized,SyGuS,sketchand Rosette all support full and formal specifications,_i.e., it is possible for specifications to unambiguously and formally describe the behavior of the program to be synthesized foranyinput.

5.4.2

FlashMeta

FlashMeta [PG15] is another meta-synthesis framework which is geared towards synthesis frominductivespecifications [PG15]. An inductive specification is a quantifier free first-order

predicate, where each atom constrains the behavior of the desired program on aspecificconcrete

input. Various other techniques for program synthesis using inductive specifications [Gul11, SG12, LG14, BGHZ15, KG15] can be expressed using the FlashMeta framework [PG15].

LikeSyGuS, FlashMeta uses a context-free grammar to describe the space of candidate programs. However, unlikeSyGuS, FlashMeta does not assume the existence of background SMTLIB theories, and thus does not restrict the space of programs to consist only of function symbols from some background theory. FlashMeta allowsanyfunction that can be expressed

as a pureC#function to be used in the context-free grammar that describes the search space for candidate programs. For programs that operate on infinite domains, such as the domain of strings and integers, inductive specifications can be viewed as an under-approximation of a complete specification. It is possible that two behaviorally different programs both satisfy a given inductive specification. FlashMeta uses domain specificrankingschemes to determine

which program is most likely to be the program desired by the user from among a set of programs which all satisfy the inductive specification [PG15, SG15]. Ranking is especially important when inductive specifications are used, as there always exists a trivial solution which is a large case split over all the concrete inputs referred to in the inductive specification. Such a solution is undesirable, because it does not generalize well to unseen inputs.

A novel feature of FlashMeta, that is not present in any of the other meta-synthesis frame- works discussed in this dissertation, is the use ofwitness functions[PG15]. A witness function is

specified by a programmer, who, in this case is assumed to be an expert, with a deep knowledge of the kinds of programs that are likely to be useful for an end user. Consider an inductive specificationϕ, for a functionfwhich is to be synthesized. Further, suppose that the synthesizer is exploring the possibility that the top-level operator forfisF. The_shapeof the program is thus hypothesized to beF(a₁,a₂, . . .a_n), where the argumentsa_inow need to be synthesized. A witness functionωj(ϕ)deducesa specificationϕj on thejth argument toF. This essentially

allows FlashMeta todecomposethe synthesis problems into multiple sub-goals, which in turn

leads to scalable synthesis algorithms.

We conclude the comparison with other meta-synthesis frameworks by noting that Table 5.1 compares and contrasts the various meta-synthesis frameworks along different dimensions and summarizes the comparison that we have just presented.

6

Enumerative Strategies for

SyGuS

Solvers

This chapter describes how enumerative strategies can be used to solve instances of theSyGuS problem. The first strategy we describe is a straightforward extension of the algorithm used to infer expressions intransit, presented in Section 4.3. We then discuss recent advances made in the area ofSyGuSsolvers, and present an algorithm for a class ofSyGuSinstances variously termed single invocation[RDK+15], separable [ACR15], or single-point definable [MNS16]

in recent literature. The algorithm is enumerative in spirit, but uses a divide-and-conquer approach by synthesizing multiple expressions, each of which is correct for a subset of inputs, and then attempts tounify[ACR15] these expressions using conditionals.

6.1

esolver: An Enumerative

SyGuS

Solver

Having defined theSyGuSproblem, as well as the language to describe instances of theSyGuS problem, we built a solver for such instances based on enumerating candidate expressions, which we dubesolver. The core algorithms used inesolverare similar to the algorithms for inferring expressions intransit, described in Algorithms 4.1 and 4.2. We use the notion of a signature to prune the space of expressions to be searched. The key differences from the algorithms presented in Algorithms 4.1 and 4.2 are that:

• esolverdoes not assume that all well-typed expressions are a part of the candidate space, and instead enumerates expressions using the grammar provided as part of the problem instance.

• The notion of a signature, which we use to prune the search space, now needs to take into account thenon-terminalin the grammar from which an expression was derived, to avoid

• esolverhandles several extensions to the SyGuS solver — such as thelet construct in constraints and grammars [RU14], which we have not described here.

We do not present the details about the implementation of esolver, as it is a rather straightforward extension of the algorithms presented in Section 4.3.esolverwon the 2014 SyGuScompetition with four other solvers participating. The implementation ofesolver— along with two other implementations, one based on symbolic search [GJTV11, JGST10] and the other based on a stochastic search [SSA13] — has been made available as a base- line for other participants to compare against, and possibly build upon, and is continually maintained [JRU13].

The 2015 SyGuS competition had several new solvers competing, the most notable of general-purpose solver being the CVC4 solver [RDK+_{15]. The CVC4 solver was the overall}

winner of the 2015SyGuScompetition, withesolvercoming in second place overall. However, despite CVC4 being the overall winner, there were a set of benchmarks which could not be solved by the CVC4 solver, but whichesolvercould solve, as well as the other way around. In addition, a solver based on a unification approach was also proposed by Radhakrishna et. al. [ACR15], which did not participate in the 2015SyGuScompetition, but has an impressive performance nonetheless. The next section provides a brief overview of these new algorithms to solve theSyGuSproblem, and discusses the capabilities and limitations ofesolver(and enumerative strategies in general) with respect to the newer algorithms.