wasi-libc 22 and later support realpath(3)
(https://github.com/WebAssembly/wasi-libc/pull/463) but the underlying
host syscall may return ENOTSUP. This is typically the case when using
incomplete WASI polyfills on web browsers. For such cases, we should
fallback to the emulated realpath.
macOS 15.0 24A5331b seems to have a weird issue that
`CFStringCreateWithBytesNoCopy` does not return `NSTaggedPointerString`
instance for the first call. This issue is fixed in macOS 15.1 but we
need to workaround it for macOS 15.0.
This does not change any actual behaviour, but provides a choke point for blocking IO operations.
* Update `IO::Buffer` to use `rb_io_blocking_region`.
* Update `File` to use `rb_io_blocking_region`.
* Update `IO` to use `rb_io_blocking_region`.
Do not release GVL around get{pwuid,pwnam,grgid,grnam} calls,
as doing so is not thread-safe. Another C extension could have
a concurrent call, and derefencing the returned pointer from
these calls could result in a segfault.
Have rb_home_dir_of call rb_getpwdirnam_for_login if available,
so it can use getpwnam_r and release GVL in a thread-safe manner.
This is related to GVL releasing work in [Bug #20587].
As `__has_attribute` macro is always defined in internal/compilers.h,
gcc warns `-Wunguarded-availability-new` as unknown option. Check if
the warning option is usable instead.
* Release GVL for fdopendir calls
* Release GVL for readdir calls
* Release GVL for chdir call in dir_chdir0
* Release GVL for fchdir call in dir_fchdir
* Release GVL for chroot calls
* Release GVL for lstat calls
* Release GVL for stat calls
* Release GVL for fstatat calls
* Release GVL for getpwnam call in rb_home_dir_of
(technically in file.c, but called from dir.c)
This does not release GVL for readdir/stat/lstat on Windows,
as that causes issues because the emulation functions that
are called in win32.c require the GVL.
This also removes some explicit casts either to or from void *,
which are allowed implicitly. The remaining casts to or from
void * are part of function pointer casts, which are not
allowed implicitly and will generate a warning.
File.expand_path leaks the dir if the encodings are not compatible.
For example:
Encoding.default_external = Encoding::UTF_16BE
10.times do
100_000.times do
File.expand_path("./a")
rescue
end
puts `ps -o rss= -p #{$$}`
end
Before:
12288
15488
18656
21872
25056
28240
31392
34688
37856
41056
After:
9680
9728
9728
9792
9792
9792
9792
9792
9792
This reverts commit 18e55fc1e1ec20e8f3166e3059e76c885fc9f8f2.
fix [Bug #19704]
https://bugs.ruby-lang.org/issues/19704
This breaks compatibility for extension libraries. Such changes
need a discussion.
* Add rb_io_path and rb_io_open_descriptor.
* Use rb_io_open_descriptor to create PTY objects
* Rename FMODE_PREP -> FMODE_EXTERNAL and expose it
FMODE_PREP I believe refers to the concept of a "pre-prepared" file, but
FMODE_EXTERNAL is clearer about what the file descriptor represents and
aligns with language in the IO::Buffer module.
* Ensure that rb_io_open_descriptor closes the FD if it fails
If FMODE_EXTERNAL is not set, then it's guaranteed that Ruby will be
responsible for closing your file, eventually, if you pass it to
rb_io_open_descriptor, even if it raises an exception.
* Rename IS_EXTERNAL_FD -> RUBY_IO_EXTERNAL_P
* Expose `rb_io_closed_p`.
* Add `rb_io_mode` to get IO mode.
---------
Co-authored-by: KJ Tsanaktsidis <ktsanaktsidis@zendesk.com>
This reverts commit 35136e1e9c232ad7a03407b992b2e86b6df43f63.
test-spec has been failing since this revision.
.github/workflows/compilers.yml:82
https://github.com/ruby/ruby/actions/runs/4276884159/jobs/7445299562
```
env:
# Minimal flags to pass the check.
default_cc: 'gcc-11 -fcf-protection -Wa,--generate-missing-build-notes=yes'
optflags: '-O2'
LDFLAGS: '-Wl,-z,now'
# FIXME: Drop skipping options
# https://bugs.ruby-lang.org/issues/18061
# https://sourceware.org/annobin/annobin.html/Test-pie.html
TEST_ANNOCHECK_OPTS: "--skip-pie --skip-gaps"
```
Failure:
```
1)
An exception occurred during: Kernel#require (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:317
Kernel#require (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.ext.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:5:in `<top (required)>'
2)
An exception occurred during: Kernel#require ($LOADED_FEATURES) stores an absolute path
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:330
Kernel#require ($LOADED_FEATURES) stores an absolute path ERROR
LeakError: Closed file descriptor: 8
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:5:in `<top (required)>'
3)
An exception occurred during: Kernel#require ($LOADED_FEATURES) does not load a non-canonical path for a file already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:535
Kernel#require ($LOADED_FEATURES) does not load a non-canonical path for a file already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:5:in `<top (required)>'
4)
An exception occurred during: Kernel#require ($LOADED_FEATURES) does not load a ../ relative path for a file already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:551
Kernel#require ($LOADED_FEATURES) does not load a ../ relative path for a file already loaded ERROR
LeakError: Leaked file descriptor: 9 : #<File:../code/load_fixture.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:5:in `<top (required)>'
5)
An exception occurred during: Kernel#require ($LOADED_FEATURES) complex, enumerator, rational, thread, ruby2_keywords are already required
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:563
Kernel#require ($LOADED_FEATURES) complex, enumerator, rational, thread, ruby2_keywords are already required ERROR
LeakError: Closed file descriptor: 8
Closed file descriptor: 9
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:5:in `<top (required)>'
6)
An exception occurred during: Kernel.require (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:317
Kernel.require (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.ext.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:23:in `<top (required)>'
7)
An exception occurred during: Kernel.require ($LOADED_FEATURES) stores an absolute path
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:330
Kernel.require ($LOADED_FEATURES) stores an absolute path ERROR
LeakError: Closed file descriptor: 8
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:23:in `<top (required)>'
8)
An exception occurred during: Kernel.require ($LOADED_FEATURES) does not load a non-canonical path for a file already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:535
Kernel.require ($LOADED_FEATURES) does not load a non-canonical path for a file already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:23:in `<top (required)>'
9)
An exception occurred during: Kernel.require ($LOADED_FEATURES) does not load a ../ relative path for a file already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:551
Kernel.require ($LOADED_FEATURES) does not load a ../ relative path for a file already loaded ERROR
LeakError: Leaked file descriptor: 9 : #<File:../code/load_fixture.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:23:in `<top (required)>'
10)
An exception occurred during: Kernel.require ($LOADED_FEATURES) complex, enumerator, rational, thread, ruby2_keywords are already required
/__w/ruby/ruby/src/spec/ruby/core/kernel/shared/require.rb:563
Kernel.require ($LOADED_FEATURES) complex, enumerator, rational, thread, ruby2_keywords are already required ERROR
LeakError: Closed file descriptor: 8
Closed file descriptor: 9
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_spec.rb:23:in `<top (required)>'
11)
An exception occurred during: Kernel#require_relative with a relative path (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:197
Kernel#require_relative with a relative path (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.ext.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:4:in `<top (required)>'
12)
An exception occurred during: Kernel#require_relative with a relative path ($LOADED_FEATURES) stores an absolute path
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:205
Kernel#require_relative with a relative path ($LOADED_FEATURES) stores an absolute path ERROR
LeakError: Closed file descriptor: 8
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:4:in `<top (required)>'
13)
An exception occurred during: Kernel#require_relative with an absolute path (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:399
Kernel#require_relative with an absolute path (file extensions) does not load a C-extension file if a complex-extensioned .rb file is already loaded ERROR
LeakError: Leaked file descriptor: 8 : #<File:/__w/ruby/ruby/src/spec/ruby/fixtures/code/load_fixture.ext.rb>
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:277:in `<top (required)>'
14)
An exception occurred during: Kernel#require_relative with an absolute path ($LOAD_FEATURES) stores an absolute path
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:407
Kernel#require_relative with an absolute path ($LOAD_FEATURES) stores an absolute path ERROR
LeakError: Closed file descriptor: 8
/__w/ruby/ruby/src/spec/ruby/core/kernel/require_relative_spec.rb:277:in `<top (required)>'
```
When loading Ruby source files, we can save the result of
successful opens as open(2)/openat(2) are a fairly expensive
syscalls. This also avoids a time-of-check-to-time-of-use
(TOCTTOU) problem.
This reduces open(2) syscalls during `require'; but should be
most apparent when users have a small $LOAD_PATH. Users with
large $LOAD_PATH will benefit less since there'll be more
open(2) failures due to ENOENT.
With `strace -c -e openat ruby -e exit' under Linux, this
results in a ~14% reduction of openat(2) syscalls
(glibc uses openat(2) to implement open(2)).
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
0.00 0.000000 0 296 110 openat
0.00 0.000000 0 254 110 openat
Additionally, the introduction of `struct ruby_file_load_state'
may make future optimizations more apparent.
This change cannot benefit binary (.so) loading since the
dlopen(3) API requires a filename and I'm not aware of an
alternative that takes a pre-existing FD. In typical
situations, Ruby source files outnumber the mount of .so
files.
