1046 Commits

Author SHA1 Message Date
Koichi Sasada
be1bbd5b7d M:N thread scheduler for Ractors
This patch introduce M:N thread scheduler for Ractor system.

In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.

From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.

Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.

There are additional settings by environment variables:

`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).

`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).

This patch will be reverted soon if non-easy issues are found.

[Bug #19842]
2023-10-12 14:47:01 +09:00
Nobuyoshi Nakada
457971f4e2
[DOC] State the precision of Process.times as platform-defined
Remove the bad example that can lead to misunderstanding as if this
precision is defined in Ruby.
2023-10-03 14:42:47 +09:00
Burdette Lamar
f9490110e1
[DOC] Another example for spawn (#8575) 2023-10-01 19:37:40 -04:00
BurdetteLamar
84312e688f [DOC] Fix minor inconsistencies 2023-09-27 14:42:50 -04:00
Peter Zhu
154bd04ee2 [DOC] Fix link in process.c 2023-09-26 11:28:36 -04:00
BurdetteLamar
a40ab12768 [DOC] Refactor doc in process.c 2023-09-26 11:26:19 -04:00
Jean Boussier
efe2822708 Process.warmup: invoke malloc_trim if available
Similar to releasing free GC pages, releasing free malloc pages
reduce the amount of page faults post fork.
2023-09-15 17:45:21 +02:00
Nobuyoshi Nakada
998ae7c3f3
[Bug #19868] Deprecate Process::Status#& and Process::Status#>> 2023-09-14 15:26:25 +09:00
Nobuyoshi Nakada
b6de0a6c69
[Bug #19868] Suggest other Process::Status method for & and >>
`Process::Status#&` and `Process::Status#>>` are provided only for
the backward compatibility with older Ruby than 1.8 where `$?` was
a `Fixnum`, and the knowledge about internals of system dependent
macros is necessary to use them.  Modern programs and libraries
should not need these methods.
2023-09-14 14:57:57 +09:00
Nobuyoshi Nakada
efe5e6e8d0
Negative value to Process::Status method for compatibility 2023-09-14 14:38:14 +09:00
Burdette Lamar
5edabd1cd5
[DOC] RDoc for Process::Status (#8416) 2023-09-12 16:31:31 -04:00
Nobuyoshi Nakada
ac65fc833d
[DOC] Fix up Process::Status#>> 2023-09-07 10:05:10 +09:00
Burdette Lamar
54274b8c65
[DOC] Rdoc for Process::Status (#8386) 2023-09-06 18:26:11 -04:00
Burdette Lamar
a17a1cd535
[DOC] Fix for RDoc for Process.kill (#8370) 2023-09-05 12:53:11 -04:00
Nobuyoshi Nakada
5b157997e2
[DOC] Fix up markup 2023-09-05 21:09:41 +09:00
Nobuyoshi Nakada
7c8932365f
[DOC] Fix signal list
- Signal names can be symbols, as stated above.
- Supported signals and those values are platform dependent.
- Key sequences to send signal are configurable.
- Fix description of signal 0.

Co-authored-by: Peter Zhu <peter@peterzhu.ca>
2023-09-05 20:36:45 +09:00
Burdette Lamar
c9af911489
[DOC] RDoc for process.c (#8366) 2023-09-04 21:16:48 -04:00
Burdette Lamar
40ab77eb3d
[DOC] Process doc (#8363) 2023-09-03 15:36:51 -04:00
BurdetteLamar
3602e253e7 [DOC] RDoc for process.c 2023-09-02 19:46:35 -04:00
Burdette Lamar
ef4dab6983
[DOC] RDoc for process.c (#8358) 2023-09-02 11:43:10 -04:00
Burdette Lamar
cfcb4a4e26
[DOC] RDoc for process.c (#8353) 2023-09-02 09:04:04 -04:00
Burdette Lamar
84fa8ae84e
[DOC] RDoc for #spawn (#8342) 2023-08-31 14:02:09 -04:00
Burdette Lamar
eb3d94f4ba
[DOC] RDoc for Kernel#system (#8309) 2023-08-31 09:06:11 -04:00
Samuel Williams
e46e48d690
Expose rb_process_status_wait and hide rb_process_status_waitv. (#8316) 2023-08-29 22:24:55 +12:00
Samuel Williams
901b6d9c50
Validate the typed data before dereferencing the internal struct. (#8315) 2023-08-29 20:04:14 +12:00
Burdette Lamar
1cc700907d
[DOC] More on method exec (#8302) 2023-08-27 13:39:18 -04:00
Peter Zhu
b7237e3bbd Free all empty heap pages in Process.warmup
This commit adds `free_empty_pages` which frees all empty heap pages and
moves the number of pages freed to the allocatable pages counter. This
is used in Process.warmup to improve performance because page
invalidation from copy-on-write is slower than allocating a new page.
2023-08-27 09:39:29 -04:00
Burdette Lamar
f9aea75747
[DOC] RDoc for Process (#8282) 2023-08-24 17:41:43 -04:00
Burdette Lamar
75c9c2c431
[DOC] Process doc (#8279) 2023-08-24 13:46:14 -04:00
Bo Anderson
84a12d6578 Fix compile error on older systems without clock_get* 2023-08-24 19:09:06 +09:00
Burdette Lamar
7e0d2c6143
[DOC] RDoc for Process (#8253) 2023-08-23 23:21:44 -04:00
Burdette Lamar
589c01c411
[DOC] RDoc for Process (#8179) 2023-08-07 09:51:03 -04:00
Burdette Lamar
533dcb8e52
RDoc for module Process (#8141) 2023-07-31 21:44:56 -04:00
Nobuyoshi Nakada
8b390a456c
[DOC] Update description of Process.daemon arguments
These arguments can only be `true` or `false` now.  Also add markups.
2023-08-01 01:47:19 +09:00
Jean Boussier
9b405a18be Process.warmup: precompute strings coderange
This both save time for when it will be eventually needed,
and avoid mutating heap pages after a potential fork.

Instrumenting some large Rails app, I've witnessed up to
58% of String instances having their coderange still unknown.
2023-07-26 11:41:23 +02:00
Jean Boussier
fa30b99c34 Implement Process.warmup
[Feature #18885]

For now, the optimizations performed are:

  - Run a major GC
  - Compact the heap
  - Promote all surviving objects to oldgen

Other optimizations may follow.
2023-07-17 11:20:15 +02:00
Nobuyoshi Nakada
c8d0470bb0
Use File::NULL instead of hard coded null device names 2023-07-10 19:21:47 +09:00
Samuel Williams
a218ed5692
Hide the usage of rb_io_t where possible. (#7880)
This retries the compatible parts of the previously reverted PR so we can continue to update related code without breaking backwards compatibility.
2023-06-01 14:23:30 +09:00
NARUSE, Yui
85dcc4866d Revert "Hide most of the implementation of struct rb_io. (#6511)"
This reverts commit 18e55fc1e1ec20e8f3166e3059e76c885fc9f8f2.

fix [Bug #19704]
https://bugs.ruby-lang.org/issues/19704
This breaks compatibility for extension libraries. Such changes
need a discussion.
2023-06-01 08:43:22 +09:00
Samuel Williams
18e55fc1e1
Hide most of the implementation of struct rb_io. (#6511)
* Add rb_io_path and rb_io_open_descriptor.

* Use rb_io_open_descriptor to create PTY objects

* Rename FMODE_PREP -> FMODE_EXTERNAL and expose it

FMODE_PREP I believe refers to the concept of a "pre-prepared" file, but
FMODE_EXTERNAL is clearer about what the file descriptor represents and
aligns with language in the IO::Buffer module.

* Ensure that rb_io_open_descriptor closes the FD if it fails

If FMODE_EXTERNAL is not set, then it's guaranteed that Ruby will be
responsible for closing your file, eventually, if you pass it to
rb_io_open_descriptor, even if it raises an exception.

* Rename IS_EXTERNAL_FD -> RUBY_IO_EXTERNAL_P

* Expose `rb_io_closed_p`.

* Add `rb_io_mode` to get IO mode.

---------

Co-authored-by: KJ Tsanaktsidis <ktsanaktsidis@zendesk.com>
2023-05-30 10:02:40 +09:00
Nobuyoshi Nakada
7f7a8fa555
Put rb_fork back into process.c
Now, calling `rb_fork` directly breaks the PID cache and the timer
thread, so must use `rb_fork_ruby` or similar instead.
2023-05-21 23:00:27 +09:00
Nobuyoshi Nakada
3612b1bed8 [Feature #19590] Show the invalid clock argument
Include the failed clock argument in the error message from
`Process.clock_gettime` and `Process.clock_getres`.
2023-04-13 14:07:27 +09:00
Daniel DeLorme
4b100f75d2 [DOC] Clarify behavior of abort() with no argument
Since Ruby 2.7 it prints the message and backtrace of $!

Fixes [Bug #19578]
2023-04-05 07:37:07 -07:00
Aaron Patterson
8525603c72
Revert "Fix transient heap mode"
This reverts commit 87253d047ce35e7836b6f97edbb4f819879a3b25.

Revert "Implement `Process.warmup`"

This reverts commit ba6ccd871442f55080bffd53e33678c0726787d2.
2023-04-04 12:59:14 -07:00
Jean Boussier
ba6ccd8714 Implement Process.warmup
[Feature #18885]

For now, the optimizations performed are:

  - Run a major GC
  - Compact the heap
  - Promote all surviving objects to oldgen

Other optimizations may follow.
2023-04-04 19:49:08 +02:00
Nobuyoshi Nakada
1d529f382c
PID cache is never cleared where fork is not used 2023-04-04 11:29:57 +09:00
Nobuyoshi Nakada
3e7d1cbceb Cache pid itself instead of converted VALUE 2023-03-24 14:22:02 +09:00
Jean Boussier
3563e1383f thread_pthread.c: Use a fork_gen to protect against fork instead of getpid()
[Feature #19443]

Until recently most libc would cache `getpid()` so this was a cheap check to make.

However as of glibc version 2.25 the PID cache is removed and calls to getpid() always
invoke the actual system call which significantly degrades the performance of existing applications.

The reason glibc removed the cache is that some libraries were bypassing fork(2)
by issuing system calls themselves, causing stale cache issues.

That isn't a concern for Ruby as bypassing MRI's primitive for forking would
render the VM unusable, so we can safely cache the PID.
2023-03-23 09:08:19 +00:00
Peter Zhu
3210130993 Suppress a compiler warning on GCC in process.c
rb_pid_t is 32 bits on some platforms, which will cause a warning on GCC
due to POSFIXABLE always returning true.

    include/ruby/internal/arithmetic/fixnum.h:43:31: warning: comparison is always true due to limited range of data type [-Wtype-limits]
2023-03-22 17:25:18 -04:00
Nobuyoshi Nakada
758e4db551
Mark cached pid if necessary 2023-03-20 18:08:05 +09:00