1berry/ruby
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[ruby/yarp] ignore state on embexpr_end

Browse Source 
Ripper state can carry over from the previous node type making coparison less useful

https://github.com/ruby/yarp/commit/74509728d4
This commit is contained in:
HParker 2023-08-25 11:12:09 -07:00 committed by git
parent 4825600da5
commit f84690c6e2
1 changed files with 8 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
lib/yarp/lex_compat.rb
View File

@ -208,18 +208,9 @@ module YARP
end end
end end
# It is extremely non obvious which state the parser is in when comments get # Tokens where state should be ignored
# dispatched. Because of this we don't both comparing state when comparing # used for :on_comment, :on_heredoc_end, :on_embexpr_end
# against other comment tokens. class IgnoreStateToken < Token
class CommentToken < Token
def ==(other)
self[0...-1] == other[0...-1]
end
end
# Heredoc end tokens are emitted in an odd order, so we don't compare the
# state on them.
class HeredocEndToken < Token
def ==(other) def ==(other)
self[0...-1] == other[0...-1] self[0...-1] == other[0...-1]
end end
@ -624,12 +615,12 @@ module YARP
when :on___end__ when :on___end__
EndContentToken.new([[lineno, column], event, value, lex_state]) EndContentToken.new([[lineno, column], event, value, lex_state])
when :on_comment when :on_comment
CommentToken.new([[lineno, column], event, value, lex_state]) IgnoreStateToken.new([[lineno, column], event, value, lex_state])
when :on_heredoc_end when :on_heredoc_end
# Heredoc end tokens can be emitted in an odd order, so we don't # Heredoc end tokens can be emitted in an odd order, so we don't
# want to bother comparing the state on them. # want to bother comparing the state on them.
HeredocEndToken.new([[lineno, column], event, value, lex_state]) IgnoreStateToken.new([[lineno, column], event, value, lex_state])
when :on_embexpr_end, :on_ident when :on_ident
if lex_state == Ripper::EXPR_END if lex_state == Ripper::EXPR_END
# If we have an identifier that follows a method name like: # If we have an identifier that follows a method name like:
# #
@ -649,6 +640,8 @@ module YARP
else else
Token.new([[lineno, column], event, value, lex_state]) Token.new([[lineno, column], event, value, lex_state])
end end
when :on_embexpr_end
IgnoreStateToken.new([[lineno, column], event, value, lex_state])
when :on_ignored_nl when :on_ignored_nl
# Ignored newlines can occasionally have a LABEL state attached to # Ignored newlines can occasionally have a LABEL state attached to
# them which doesn't actually impact anything. We don't mirror that # them which doesn't actually impact anything. We don't mirror that