1berry/ruby
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Lrama v0.6.11

Browse Source
This commit is contained in:
ydah 2024-12-22 23:12:35 +09:00 committed by Yuichiro Kaneko
parent 34e6bb48af
commit d8c152eead
Notes: git 2024-12-23 06:16:50 +00:00 
Merged: https://github.com/ruby/ruby/pull/12429
16 changed files with 654 additions and 564 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

95
tool/lrama/NEWS.md
View File

@ -1,12 +1,47 @@
# NEWS for Lrama
## Lrama 0.6.11 (2024-12-23)
### Add support for %type declarations using %nterm in Nonterminal Symbols
Allow to use `%nterm` in Nonterminal Symbols for `%type` declarations.
```yacc
%nterm <type> nonterminal…
```
This directive is also supported for compatibility with Bison, and only non-terminal symbols are allowed. In other words, definitions like the following will result in an error:
```yacc
%{
// Prologue
%}
%token EOI 0 "EOI"
%nterm EOI
%%
program: /* empty */
;
```
It show an error message like the following:
```command
❯ exe/lrama nterm.y
nterm.y:6:7: symbol EOI redeclared as a nonterminal
%nterm EOI
^^^
```
## Lrama 0.6.10 (2024-09-11)
### Aliased Named References for actions of RHS in parameterizing rules
Allow to use aliased named references for actions of RHS in parameterizing rules.
```
```yacc
%rule sum(X, Y): X[summand] '+' Y[addend] { $$ = $summand + $addend }
;
```
@ -18,7 +53,7 @@ https://github.com/ruby/lrama/pull/410
Allow to use named references for actions of RHS in parameterizing rules caller side.
```
```yacc
opt_nl: '\n'?[nl] <str> { $$ = $nl; }
;
```
@ -29,7 +64,7 @@ https://github.com/ruby/lrama/pull/414
Allow to define parameterizing rules in the middle of the grammar.
```
```yacc
%rule defined_option(X): /* empty */
| X
;
@ -52,8 +87,8 @@ https://github.com/ruby/lrama/pull/420
Support to report unused terminal symbols.
Run `exe/lrama --report=terms` to show unused terminal symbols.
```
❯ exe/lrama --report=terms sample/calc.y
```console
$ exe/lrama --report=terms sample/calc.y
11 Unused Terms
0 YYerror
1 YYUNDEF
@ -74,8 +109,8 @@ https://github.com/ruby/lrama/pull/439
Support to report unused rules.
Run `exe/lrama --report=rules` to show unused rules.
```
❯ exe/lrama --report=rules sample/calc.y
```console
$ exe/lrama --report=rules sample/calc.y
3 Unused Rules
0 unused_option
1 unused_list
@ -96,8 +131,8 @@ https://github.com/ruby/lrama/pull/446
Support to warning redefined parameterizing rules.
Run `exe/lrama -W` or `exe/lrama --warnings` to show redefined parameterizing rules.
```
❯ exe/lrama -W sample/calc.y
```console
$ exe/lrama -W sample/calc.y
parameterizing rule redefined: redefined_method(X)
parameterizing rule redefined: redefined_method(X)
```
@ -117,7 +152,7 @@ https://github.com/ruby/lrama/pull/457
Allow to specify tag on callee side of parameterizing rules.
```
```yacc
%union {
int i;
}
@ -130,7 +165,7 @@ Allow to specify tag on callee side of parameterizing rules.
Allow to use named references for actions of RHS in parameterizing rules.
```
```yacc
%rule option(number): /* empty */
| number { $$ = $number; }
;
@ -142,7 +177,7 @@ Allow to use named references for actions of RHS in parameterizing rules.
Allow to nested parameterizing rules with tag.
```
```yacc
%union {
int i;
}
@ -179,8 +214,8 @@ User can use `'symbol'?`, `'symbol'+` and `'symbol'*` in RHS of user defined par
Support trace actions for debugging.
Run `exe/lrama --trace=actions` to show grammar rules with actions.
```
❯ exe/lrama --trace=actions sample/calc.y
```console
$ exe/lrama --trace=actions sample/calc.y
Grammar rules with actions:
$accept -> list, YYEOF {}
list -> ε {}
@ -199,7 +234,7 @@ expr -> '(', expr, ')' { $$ = $2; }
Support inlining for rules.
The `%inline` directive causes all references to symbols to be replaced with its definition.
```
```yacc
%rule %inline op: PLUS { + }
| TIMES { * }
;
@ -213,7 +248,7 @@ expr : number { $$ = $1; }
as same as
```
```yacc
expr : number { $$ = $1; }
| expr '+' expr { $$ = $1 + $3; }
| expr '*' expr { $$ = $1 * $3; }
@ -226,7 +261,7 @@ expr : number { $$ = $1; }
User can specify the type of mid rule action by tag (`<bar>`) instead of specifying it with in an action.
```
```yacc
primary: k_case expr_value terms?
{
$<val>$ = p->case_labels;
@ -241,7 +276,7 @@ primary: k_case expr_value terms?
can be written as
```
```yacc
primary: k_case expr_value terms?
{
$$ = p->case_labels;
@ -266,7 +301,7 @@ Bison supports this feature from 3.1.
Support `preceded`, `terminated` and `delimited` rules.
```
```text
program: preceded(opening, X)
// Expanded to
@ -302,7 +337,7 @@ In general, these resources are freed by actions or after parsing.
However if syntax error happens in parsing, these codes may not be executed.
Codes associated to `%destructor` are executed when semantic value is popped from the stack by an error.
```
```yacc
%token <val1> NUM
%type <val2> expr2
%type <val3> expr
@ -350,7 +385,7 @@ Lrama provides these five callbacks. Registered functions are called when each e
User also needs to access semantic value of their stack in grammar action. `$:n` provides the way to access to it. `$:n` is translated to the minus index from the top of the stack.
For example
```
```yacc
primary: k_if expr_value then compstmt if_tail k_end
{
/*% ripper: if!($:2, $:4, $:5) %*/
@ -375,7 +410,7 @@ https://github.com/ruby/lrama/pull/344
Allow to pass an instantiated rule to other parameterizing rules.
```
```yacc
%rule constant(X) : X
;
@ -392,7 +427,7 @@ program : option(constant(number)) // Nested rule
Allow to use nested parameterizing rules when define parameterizing rules.
```
```yacc
%rule option(x) : /* empty */
| X
;
@ -419,7 +454,7 @@ https://github.com/ruby/lrama/pull/337
Allow to define parameterizing rule by `%rule` directive.
```
```yacc
%rule pair(X, Y): X Y { $$ = $1 + $2; }
;
@ -442,7 +477,7 @@ https://github.com/ruby/lrama/pull/285
Allow to specify type of rules by specifying tag, `<i>` in below example.
Tag is post-modification style.
```
```yacc
%union {
int i;
}
@ -469,7 +504,7 @@ https://github.com/ruby/lrama/pull/197
Support `separated_list` and `separated_nonempty_list` parameterizing rules.
```
```text
program: separated_list(',', number)
// Expanded to
@ -500,7 +535,7 @@ https://github.com/ruby/lrama/pull/204
Parameterizing rules are template of rules.
It's very common pattern to write "list" grammar rule like:
```
```yacc
opt_args: /* none */
| args
;
@ -532,7 +567,7 @@ https://github.com/ruby/lrama/pull/62
### Runtime configuration for error recovery
Meke error recovery function configurable on runtime by two new macros.
Make error recovery function configurable on runtime by two new macros.
* `YYMAXREPAIR`: Expected to return max length of repair operations. `%parse-param` is passed to this function.
* `YYERROR_RECOVERY_ENABLED`: Expected to return bool value to determine error recovery is enabled or not. `%parse-param` is passed to this function.
@ -555,7 +590,7 @@ https://github.com/ruby/lrama/pull/44
Instead of positional references like `$1` or `$$`,
named references allow to access to symbol by name.
```
```yacc
primary: k_class cpath superclass bodystmt k_end
{
$primary = new_class($cpath, $bodystmt, $superclass);
@ -564,7 +599,7 @@ primary: k_class cpath superclass bodystmt k_end
Alias name can be declared.
```
```yacc
expr[result]: expr[ex-left] '+' expr[ex.right]
{
$result = $[ex-left] + $[ex.right];

2
tool/lrama/lib/lrama/bitmap.rb
View File

@ -13,7 +13,7 @@ module Lrama
end
def self.to_array(int)
a = []
a = [] #: Array[Integer]
i = 0
while int > 0 do

6
tool/lrama/lib/lrama/context.rb
View File

@ -405,7 +405,7 @@ module Lrama
@check = []
# Key is froms_and_tos, value is index position
pushed = {}
userd_res = {}
used_res = {}
lowzero = 0
high = 0
@ -430,7 +430,7 @@ module Lrama
end
end
if ok && userd_res[res]
if ok && used_res[res]
ok = false
end
@ -458,7 +458,7 @@ module Lrama
@base[state_id] = res
pushed[froms_and_tos] = res
userd_res[res] = true
used_res[res] = true
end
@yylast = high

28
tool/lrama/lib/lrama/counterexamples.rb
View File

@ -32,8 +32,10 @@ module Lrama
conflict_state.conflicts.flat_map do |conflict|
case conflict.type
when :shift_reduce
# @type var conflict: State::ShiftReduceConflict
shift_reduce_example(conflict_state, conflict)
when :reduce_reduce
# @type var conflict: State::ReduceReduceConflict
reduce_reduce_examples(conflict_state, conflict)
end
end.compact
@ -48,7 +50,7 @@ module Lrama
@reverse_transitions = {}
@states.states.each do |src_state|
trans = {}
trans = {} #: Hash[Grammar::Symbol, State]
src_state.transitions.each do |shift, next_state|
trans[shift.next_sym] = next_state
@ -66,6 +68,7 @@ module Lrama
@transitions[[src_state_item, sym]] = dest_state_item
# @type var key: [StateItem, Grammar::Symbol]
key = [dest_state_item, sym]
@reverse_transitions[key] ||= Set.new
@reverse_transitions[key] << src_state_item
@ -82,7 +85,7 @@ module Lrama
@states.states.each do |state|
# LHS => Set(Item)
h = {}
h = {} #: Hash[Grammar::Symbol, Set[States::Item]]
state.closure.each do |item|
sym = item.lhs
@ -97,6 +100,7 @@ module Lrama
sym = item.next_sym
state_item = StateItem.new(state, item)
# @type var key: [State, Grammar::Symbol]
key = [state, sym]
@productions[state_item] = h[sym]
@ -109,6 +113,7 @@ module Lrama
def shift_reduce_example(conflict_state, conflict)
conflict_symbol = conflict.symbols.first
# @type var shift_conflict_item: ::Lrama::States::Item
shift_conflict_item = conflict_state.items.find { |item| item.next_sym == conflict_symbol }
path2 = shortest_path(conflict_state, conflict.reduce.item, conflict_symbol)
path1 = find_shift_conflict_shortest_path(path2, conflict_state, shift_conflict_item)
@ -153,12 +158,14 @@ module Lrama
prev_state_item = prev_path&.to
if target_state_item == state_item || target_state_item.item.start_item?
result.concat(reversed_reduce_path[_j..-1].map(&:to))
result.concat(
reversed_reduce_path[_j..-1] #: Array[StartPath|TransitionPath|ProductionPath]
.map(&:to))
break
end
if target_state_item.item.beginning_of_rule?
queue = []
queue = [] #: Array[Array[StateItem]]
queue << [target_state_item]
# Find reverse production
@ -174,15 +181,17 @@ module Lrama
end
if si.item.beginning_of_rule?
# @type var key: [State, Grammar::Symbol]
key = [si.state, si.item.lhs]
@reverse_productions[key].each do |item|
state_item = StateItem.new(si.state, item)
queue << (sis + [state_item])
end
else
# @type var key: [StateItem, Grammar::Symbol]
key = [si, si.item.previous_sym]
@reverse_transitions[key].each do |prev_target_state_item|
next if prev_target_state_item.state != prev_state_item.state
next if prev_target_state_item.state != prev_state_item&.state
sis.shift
result.concat(sis)
result << prev_target_state_item
@ -195,9 +204,10 @@ module Lrama
end
else
# Find reverse transition
# @type var key: [StateItem, Grammar::Symbol]
key = [target_state_item, target_state_item.item.previous_sym]
@reverse_transitions[key].each do |prev_target_state_item|
next if prev_target_state_item.state != prev_state_item.state
next if prev_target_state_item.state != prev_state_item&.state
result << prev_target_state_item
target_state_item = prev_target_state_item
i = j
@ -224,9 +234,9 @@ module Lrama
def shortest_path(conflict_state, conflict_reduce_item, conflict_term)
# queue: is an array of [Triple, [Path]]
queue = []
visited = {}
start_state = @states.states.first
queue = [] #: Array[[Triple, Array[StartPath|TransitionPath|ProductionPath]]]
visited = {} #: Hash[Triple, true]
start_state = @states.states.first #: Lrama::State
raise "BUG: Start state should be just one kernel." if start_state.kernels.count != 1
start = Triple.new(start_state, start_state.kernels.first, Set.new([@states.eof_symbol]))

11
tool/lrama/lib/lrama/counterexamples/derivation.rb
View File

@ -18,7 +18,7 @@ module Lrama
alias :inspect :to_s
def render_strings_for_report
result = []
result = [] #: Array[String]
_render_for_report(self, 0, result, 0)
result.map(&:rstrip)
end
@ -44,18 +44,19 @@ module Lrama
str << "#{item.next_sym.display_name}"
length = _render_for_report(derivation.left, len, strings, index + 1)
# I want String#ljust!
str << " " * (length - str.length)
str << " " * (length - str.length) if length > str.length
else
str << " • #{item.symbols_after_dot.map(&:display_name).join(" ")} "
return str.length
end
if derivation.right&.left
length = _render_for_report(derivation.right.left, str.length, strings, index + 1)
str << "#{item.symbols_after_dot[1..-1].map(&:display_name).join(" ")} "
left = derivation.right&.left #: Derivation
length = _render_for_report(left, str.length, strings, index + 1)
str << "#{item.symbols_after_dot[1..-1].map(&:display_name).join(" ")} " # steep:ignore
str << " " * (length - str.length) if length > str.length
elsif item.next_next_sym
str << "#{item.symbols_after_dot[1..-1].map(&:display_name).join(" ")} "
str << "#{item.symbols_after_dot[1..-1].map(&:display_name).join(" ")} " # steep:ignore
end
return str.length

11
tool/lrama/lib/lrama/counterexamples/example.rb
View File

@ -38,9 +38,10 @@ module Lrama
private
def _derivations(paths)
derivation = nil
derivation = nil #: Derivation
current = :production
lookahead_sym = paths.last.to.item.end_of_rule? ? @conflict_symbol : nil
last_path = paths.last #: Path
lookahead_sym = last_path.to.item.end_of_rule? ? @conflict_symbol : nil
paths.reverse_each do |path|
item = path.to.item
@ -57,12 +58,14 @@ module Lrama
when ProductionPath
derivation = Derivation.new(item, derivation)
current = :production
else
raise "Unexpected. #{path}"
end
if lookahead_sym && item.next_next_sym && item.next_next_sym.first_set.include?(lookahead_sym)
state_item = @counterexamples.transitions[[path.to, item.next_sym]]
derivation2 = find_derivation_for_symbol(state_item, lookahead_sym)
derivation.right = derivation2
derivation.right = derivation2 # steep:ignore
lookahead_sym = nil
end
@ -89,7 +92,7 @@ module Lrama
end
def find_derivation_for_symbol(state_item, sym)
queue = []
queue = [] #: Array[Array[StateItem]]
queue << [state_item]
while (sis = queue.shift)

4
tool/lrama/lib/lrama/counterexamples/path.rb
View File

@ -20,6 +20,10 @@ module Lrama
"#<Path(#{type})>"
end
alias :inspect :to_s
def type
raise NotImplementedError
end
end
end
end

4
tool/lrama/lib/lrama/grammar.rb
View File

@ -30,7 +30,7 @@ module Lrama
:after_shift, :before_reduce, :after_reduce, :after_shift_error_token, :after_pop_stack,
:symbols_resolver, :types, :rules, :rule_builders, :sym_to_rules, :no_stdlib, :locations
def_delegators "@symbols_resolver", :symbols, :nterms, :terms, :add_nterm, :add_term,
def_delegators "@symbols_resolver", :symbols, :nterms, :terms, :add_nterm, :add_term, :find_term_by_s_value,
:find_symbol_by_number!, :find_symbol_by_id!, :token_to_symbol,
:find_symbol_by_s_value!, :fill_symbol_number, :fill_nterm_type,
:fill_printer, :fill_destructor, :fill_error_token, :sort_by_number!
@ -382,7 +382,7 @@ module Lrama
end
def validate_rule_lhs_is_nterm!
errors = []
errors = [] #: Array[String]
rules.each do |rule|
next if rule.lhs.nterm?

2
tool/lrama/lib/lrama/grammar/parameterizing_rule/rhs.rb
View File

@ -16,7 +16,7 @@ module Lrama
return unless user_code
resolved = Lexer::Token::UserCode.new(s_value: user_code.s_value, location: user_code.location)
var_to_arg = {}
var_to_arg = {} #: Hash[String, String]
symbols.each do |sym|
resolved_sym = bindings.resolve_symbol(sym)
if resolved_sym != sym

2
tool/lrama/lib/lrama/grammar/rule_builder.rb
View File

@ -67,7 +67,7 @@ module Lrama
end
def resolve_inline_rules
resolved_builders = []
resolved_builders = [] #: Array[RuleBuilder]
rhs.each_with_index do |token, i|
if (inline_rule = @parameterizing_rule_resolver.find_inline(token))
inline_rule.rhs_list.each do |inline_rhs|

4
tool/lrama/lib/lrama/grammar/symbols/resolver.rb
View File

@ -57,6 +57,10 @@ module Lrama
nterm
end
def find_term_by_s_value(s_value)
terms.find { |s| s.id.s_value == s_value }
end
def find_symbol_by_s_value(s_value)
symbols.find { |s| s.id.s_value == s_value }
end

1
tool/lrama/lib/lrama/lexer.rb
View File

@ -16,6 +16,7 @@ module Lrama
%union
%token
%type
%nterm
%left
%right
%nonassoc

2
tool/lrama/lib/lrama/lexer/token/user_code.rb
View File

@ -16,7 +16,7 @@ module Lrama
def _references
scanner = StringScanner.new(s_value)
references = []
references = [] #: Array[Grammar::Reference]
until scanner.eos? do
case

1039
tool/lrama/lib/lrama/parser.rb
View File

File diff suppressed because it is too large Load Diff

5
tool/lrama/lib/lrama/state/reduce.rb
View File

@ -26,9 +26,8 @@ module Lrama
end
def selected_look_ahead
if @look_ahead
# @type ivar @look_ahead: Array<Grammar::Symbol>
@look_ahead - @not_selected_symbols
if look_ahead
look_ahead - @not_selected_symbols
else
[]
end

2
tool/lrama/lib/lrama/version.rb
View File

@ -1,5 +1,5 @@
# frozen_string_literal: true
module Lrama
VERSION = "0.6.10".freeze
VERSION = "0.6.11".freeze
end