1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
|
# go tree-sitter
[](https://github.com/mitjafelicijan/go-tree-sitter/actions/workflows/test.yml?query=branch%3Amaster)
[](https://godoc.org/github.com/mitjafelicijan/go-tree-sitter)
Golang bindings for [tree-sitter](https://github.com/tree-sitter/tree-sitter)
## Usage
Create a parser with a grammar:
```go
import (
"context"
"fmt"
sitter "github.com/mitjafelicijan/go-tree-sitter"
"github.com/mitjafelicijan/go-tree-sitter/javascript"
)
parser := sitter.NewParser()
parser.SetLanguage(javascript.GetLanguage())
```
Parse some code:
```go
sourceCode := []byte("let a = 1")
tree, _ := parser.ParseCtx(context.Background(), nil, sourceCode)
```
Inspect the syntax tree:
```go
n := tree.RootNode()
fmt.Println(n) // (program (lexical_declaration (variable_declarator (identifier) (number))))
child := n.NamedChild(0)
fmt.Println(child.Type()) // lexical_declaration
fmt.Println(child.StartByte()) // 0
fmt.Println(child.EndByte()) // 9
```
### Custom grammars
This repository provides grammars for many common languages out of the box.
But if you need support for any other language you can keep it inside your own project or publish it as a separate repository to share with the community.
See explanation on how to create a grammar for go-tree-sitter [here](https://github.com/mitjafelicijan/go-tree-sitter/issues/57).
Known external grammars:
- [Salesforce grammars](https://github.com/aheber/tree-sitter-sfapex) - including Apex, SOQL, and SOSL languages.
- [Ruby](https://github.com/shagabutdinov/go-tree-sitter-ruby) - Deprecated, grammar is provided by main repo instead
- [Go Template](https://github.com/mrjosh/helm-ls/tree/master/internal/tree-sitter/gotemplate) - Used for helm
### Editing
If your source code changes, you can update the syntax tree. This will take less time than the first parse.
```go
// change 1 -> true
newText := []byte("let a = true")
tree.Edit(sitter.EditInput{
StartIndex: 8,
OldEndIndex: 9,
NewEndIndex: 12,
StartPoint: sitter.Point{
Row: 0,
Column: 8,
},
OldEndPoint: sitter.Point{
Row: 0,
Column: 9,
},
NewEndPoint: sitter.Point{
Row: 0,
Column: 12,
},
})
// check that it changed tree
assert.True(n.HasChanges())
assert.True(n.Child(0).HasChanges())
assert.False(n.Child(0).Child(0).HasChanges()) // left side of the tree didn't change
assert.True(n.Child(0).Child(1).HasChanges())
// generate new tree
newTree := parser.Parse(tree, newText)
```
### Predicates
You can filter AST by using [predicate](https://tree-sitter.github.io/tree-sitter/using-parsers#predicates) S-expressions.
Similar to [Rust](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_rust) or [WebAssembly](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web) bindings we support filtering on a few common predicates:
- `eq?`, `not-eq?`
- `match?`, `not-match?`
Usage [example](./_examples/predicates/main.go):
```go
func main() {
// Javascript code
sourceCode := []byte(`
const camelCaseConst = 1;
const SCREAMING_SNAKE_CASE_CONST = 2;
const lower_snake_case_const = 3;`)
// Query with predicates
screamingSnakeCasePattern := `(
(identifier) @constant
(#match? @constant "^[A-Z][A-Z_]+")
)`
// Parse source code
lang := javascript.GetLanguage()
n, _ := sitter.ParseCtx(context.Background(), sourceCode, lang)
// Execute the query
q, _ := sitter.NewQuery([]byte(screamingSnakeCasePattern), lang)
qc := sitter.NewQueryCursor()
qc.Exec(q, n)
// Iterate over query results
for {
m, ok := qc.NextMatch()
if !ok {
break
}
// Apply predicates filtering
m = qc.FilterPredicates(m, sourceCode)
for _, c := range m.Captures {
fmt.Println(c.Node.Content(sourceCode))
}
}
}
// Output of this program:
// SCREAMING_SNAKE_CASE_CONST
```
## Development
### Updating a grammar
Check if any updates for vendored files are available:
```
go run _automation/main.go check-updates
```
Update vendor files:
- open `_automation/grammars.json`
- modify `reference` (for tagged grammars) or `revision` (for grammars from a branch)
- run `go run _automation/main.go update <grammar-name>`
It is also possible to update all grammars in one go using
```
go run _automation/main.go update-all
```
|
