Two fixes were necessary:
- ensure we are handling newlines correctly
- accept two consecutive string tokens without a separator
https://github.com/ruby/yarp/commit/4e707937cb
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
* BOM should not impact looking for the encoding string
* We should re-encode tokens when the encoding changes
* BOM should change the column of comments only
https://github.com/ruby/yarp/commit/119fc2d7b2
* Ensure the node gets initialized in case of failure with no assertions
* Include lambda and top-level nodes for scopes
* Small indentation fixes
https://github.com/ruby/yarp/commit/be29e07391
This commit creates a scope node, and exposes a method to get
ScopeNodes from other existing nodes. It still has TODOs around
creating ScopeNodes for other types of nodes, which will be
done in future commits.
https://github.com/ruby/yarp/commit/3b9ac5764d
* `le_len` to `eol_length`
* Braces on the same line as switch case
* `peek_addr` -> `peek_at`
* `peek_at` -> `peek_offset`
* `match_line_ending_addr` -> `match_eol_at`
* `match_line_ending_at` -> `match_eol_offset`
https://github.com/ruby/yarp/commit/d7ffa9e64e
Introduce three new inline helper functions:
- `match_line_ending`
- `match_line_ending_at`
- `match_line_ending_addr`
These functions are similar in signature to the `peek*` functions, but
return the length of the line ending being inspected (or 0 if no line
ending was found).
These functions are then used to simplify how we're detecting line
endings throughout "src/yarp.c".
Also:
- test coverage backfilled for `__END__` comments with CRLF line endings.
- error message for invalid `%` tokens updated to not include
the potential line endings.
- some small refactorings for readability along the way
https://github.com/ruby/yarp/commit/a00067386d
In many places in the code we use the idiom:
x < parser->end && *x == 'y'
which is essentially an extension of an existing pattern:
- `peek()` looks at `parser->current.end`
- `peek_at()` looks at `(parser->current.end + offset)`
This commit introduces a new inline function, `peek_addr`, which
accepts a pointer and encapsulates the address value check and
conditional dereferencing. The result is more readable code, and more
ubiquitous safety checks on pointer values, allowing us to rewrite the
above as:
peek_addr(parser, x) == 'y'
Also:
- change the type of `peek_at()`'s offset argument from `size_t` to
`ptrdiff_t` so that it can accept negative offsets.
- use `current_token_starts_line` in one place where the equivalent
code is inline.
- use `peek` or `peek_at` to replace inline code in a few places
These changes simplify the code and make it easier to visually spot
patterns, particularly around line-endings (which will be a subject of
a future pull request).
https://github.com/ruby/yarp/commit/4c608d53ea
YARP commits were synced out of order. We are doing this reset
commit to ensure that all files are currently correct and we can
proceed with monitoring syncs so this doesn't happen again.
Previously, parsing a snippet like this:
%r\nfoo\n
would result in tracking the second newline twice, resulting in a
failed runtime assertion.
Fixing that issue reveals another bug, which is that the _first_
newline was not being tracked at all. So we introduce a call to
yp_newline_list right when we construct the REGEXP_BEGIN token.
https://github.com/ruby/yarp/commit/0d5d759091
It makes it more difficult to find all locations where a variable
is written to if they're just read nodes. To keep things consistent
we should make them write nodes.
https://github.com/ruby/yarp/commit/840e094045
