COOPERACIÓN ESPAÑOLA CON SEIS DIREPROS – CETMAR

Now that we know how to parse function literals the next step is to parse the calling of a function: call expressions. Here is their structure:

<expression>(<comma separated expressions>)

What? Yup, that’s it, but granted, a few examples are needed. Here is the normal call expression we all know:

add(2, 3)

Now think about this: theadd is an identifier. And identifiers are expressions. The arguments 2and3are expressions too - integer literals. But they don’t have to be, the arguments are just a list of expressions:

That’s valid, too. The first argument is the infix expression 2 + 2 and the second one is 3 * 3 * 3. So far, so good. Now, let’s look at the function that’s being called here. In this case the function is bound to the identifier add. The identifier add returns this function when it’s evaluated. That means, we could go straight to the source, skip the identifier and replace add with a function literal:

fn(x, y) { x + y; }(2, 3)

Yes, that’s valid. We can also use function literals as arguments:

callsFunction(2, 3, fn(x, y) { x + y; });

Let’s look at the structure again:

<expression>(<comma separated expressions>)

Call expressions consist of an expression that results in a function when evaluated and a list of expressions that are the arguments to this function call. As an AST node they look like this:

// ast/ast.go

type CallExpression struct {

Token token.Token // The '(' token

Function Expression // Identifier or FunctionLiteral

Arguments []Expression }

func (ce *CallExpression) expressionNode() {}

func (ce *CallExpression) TokenLiteral() string { return ce.Token.Literal }

func (ce *CallExpression) String() string {

var out bytes.Buffer args := []string{}

for _, a := range ce.Arguments { args = append(args, a.String()) } out.WriteString(ce.Function.String()) out.WriteString("(") out.WriteString(strings.Join(args, ", ")) out.WriteString(")") return out.String() }

The test case for call expressions is just like the rest of our test suite and makes assertions about the*ast.CallExpressionstructure:

// parser/parser_test.go

func TestCallExpressionParsing(t *testing.T) { input := "add(1, 2 * 3, 4 + 5);" l := lexer.New(input) p := New(l) program := p.ParseProgram() checkParserErrors(t, p) if len(program.Statements) != 1 {

}

stmt, ok := program.Statements[0].(*ast.ExpressionStatement)

if !ok {

t.Fatalf("stmt is not ast.ExpressionStatement. got=%T", program.Statements[0])

}

exp, ok := stmt.Expression.(*ast.CallExpression)

if !ok {

t.Fatalf("stmt.Expression is not ast.CallExpression. got=%T", stmt.Expression)

}

if !testIdentifier(t, exp.Function, "add") {

return

}

if len(exp.Arguments) != 3 {

t.Fatalf("wrong length of arguments. got=%d", len(exp.Arguments)) }

testLiteralExpression(t, exp.Arguments[0], 1) testInfixExpression(t, exp.Arguments[1], 2, "*", 3) testInfixExpression(t, exp.Arguments[2], 4, "+", 5) }

As with function literals and parameter parsing it’s also a good idea to add a separate test for the argument parsing. Just to make sure that every corner case works and is covered by a test. I added a TestCallExpressionParameterParsing test function that does exactly this. You can see it in the code for this chapter.

So far, so familiar. But now comes the twist. If we run the tests we get this error message: $ go test ./parser

--- FAIL: TestCallExpressionParsing (0.00s) parser_test.go:853: parser has 4 errors

parser_test.go:855: parser error: "expected next token to be ), got , instead" parser_test.go:855: parser error: "no prefix parse function for , found" parser_test.go:855: parser error: "no prefix parse function for , found" parser_test.go:855: parser error: "no prefix parse function for ) found" FAIL

FAIL monkey/parser 0.007s

Huh, that doesn’t make a lot of sense. Why is there no error message telling us to register a prefixParseFn for call expressions? Because there are no new token types in call expressions. So what do we do instead of registering aprefixParseFn? Take at look at this:

add(2, 3);

The add is an identifier that’s parsed by a prefixParseFn. And after the identifier comes a token.LPAREN, right between the identifier and the list of arguments, just in the middle, in infix position… Yes, we need to register an infixParseFn for token.LPAREN. This way we parse the expression that is the function (either an identifier, or a function literal), then check for an infixParseFn associated with token.LPAREN and call it with the already parsed expression as argument. And in this infixParseFnwe can then parse the argument list. Perfect!

func New(l *lexer.Lexer) *Parser {

// [...]

p.registerInfix(token.LPAREN, p.parseCallExpression)

// [...]

}

func (p *Parser) parseCallExpression(function ast.Expression) ast.Expression { exp := &ast.CallExpression{Token: p.curToken, Function: function}

exp.Arguments = p.parseCallArguments()

return exp }

func (p *Parser) parseCallArguments() []ast.Expression { args := []ast.Expression{} if p.peekTokenIs(token.RPAREN) { p.nextToken() return args } p.nextToken()

args = append(args, p.parseExpression(LOWEST))

for p.peekTokenIs(token.COMMA) { p.nextToken()

p.nextToken()

args = append(args, p.parseExpression(LOWEST)) } if !p.expectPeek(token.RPAREN) { return nil } return args }

parseCallExpressionreceives the already parsedfunction as argument and uses it to construct an *ast.CallExpression node. To parse the argument list we call parseCallArguments, which looks strikingly similar to parseFunctionParameters, except that it’s more generic and returns a slice of ast.Expressionand not *ast.Identifier.

There is nothing here we haven’t seen before. All we did was register a new infixParseFn. The tests still fail though:

$ go test ./parser

--- FAIL: TestCallExpressionParsing (0.00s) parser_test.go:853: parser has 4 errors

parser_test.go:855: parser error: "expected next token to be ), got , instead" parser_test.go:855: parser error: "no prefix parse function for , found" parser_test.go:855: parser error: "no prefix parse function for , found" parser_test.go:855: parser error: "no prefix parse function for ) found" FAIL

FAIL monkey/parser 0.007s

The reason that it still doesn’t work is that the ( in add(1, 2) acts like an infix operator now, but we haven’t assigned a precedence to it. It doesn’t have the right “stickiness” yet, so parseExpressiondoesn’t return what we want. But call expressions have the highest precedence of all, so it’s important that we fix our precedences table:

// parser/parser.go

var precedences = map[token.TokenType]int{

// [...]

token.LPAREN: CALL, }

To make sure that call expressions really have the highest precedence we can just extend our TestOperatorPrecedenceParsing test function:

// parser/parser_test.go

func TestOperatorPrecedenceParsing(t *testing.T) { tests := []struct { input string expected string }{ // [...] { "a + add(b * c) + d", "((a + add((b * c))) + d)", }, { "add(a, b, 1, 2 * 3, 4 + 5, add(6, 7 * 8))", "add(a, b, 1, (2 * 3), (4 + 5), add(6, (7 * 8)))", }, { "add(a + b + c * d / f + g)", "add((((a + b) + ((c * d) / f)) + g))", }, } // [...] }

If we now run the tests again, we can see that all of them pass: $ go test ./parser

ok monkey/parser 0.008s

Yes, all of them: the unit test, the test for argument parsing and the precedence tests - wow! They all pass! And if that wasn’t enough, here’s some more good news: we are done. Yes, the parser is finished. Granted, we’ll come back to it later, at the end of the book, to extend it once more. But for now: that’s it! The AST is fully defined and the parser works - it’s time to move on to the topic of evaluation.

Before we do that though, let’s remove the TODOs we left in the code and extend our REPL to integrate the parser.