110 Commits

Author SHA1 Message Date
Peter Zhu
4f88acc833 Fix compiler warnings in objspace_dump.c when assertions are turned on
Example:

```
In file included from ../../../include/ruby/defines.h:72,
                 from ../../../include/ruby/ruby.h:23,
                 from ../../../gc.h:3,
                 from ../../../ext/objspace/objspace_dump.c:15:
../../../ext/objspace/objspace_dump.c: In function ‘dump_append_ld’:
../../../ext/objspace/objspace_dump.c:95:26: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
   95 |     RUBY_ASSERT(required <= width);
      |                          ^~
```
2021-04-26 19:26:50 -04:00
Jean Boussier
3a888398a6 objspace_dump.c: tag singleton classes and reference the superclass 2021-02-04 09:53:31 -08:00
Jean Boussier
6ca3d1af33 objspace_dump.c: Handle allocation path and line missing 2021-01-20 10:48:13 -08:00
Aaron Patterson
18b3f0f54c Make ext/objspace ASAN friendly
ext/objspace iterates over the heap, but some slots in the heap are
poisoned, so we need to take care of that when running with ASAN
2020-09-28 08:20:23 -07:00
Jean Boussier
fbba6bd4e3 Parse ObjectSpace.dump_all / dump arguments in Ruby to avoid allocation noise
[Feature #17045] ObjectSpace.dump_all should allocate as little as possible in the GC heap

Up until this commit ObjectSpace.dump_all allocates two Hash because of `rb_scan_args`.

It also can allocate a `File` because of `rb_io_get_write_io`.

These allocations are problematic because `dump_all` dumps the Ruby
heap, so it should try modify as little as possible what it is
observing.
2020-09-15 09:18:13 -07:00
Kazuhiro NISHIYAMA
406559a268
Add missing break
pointed out by Coverity Scan
2020-09-11 11:02:24 +09:00
Jean Boussier
5001cc4716 Optimize ObjectSpace.dump_all
The two main optimization are:
  - buffer writes for improved performance
  - avoid formatting functions when possible

```

|                   |compare-ruby|built-ruby|
|:------------------|-----------:|---------:|
|dump_all_string    |       1.038|   195.925|
|                   |           -|   188.77x|
|dump_all_file      |      33.453|   139.645|
|                   |           -|     4.17x|
|dump_all_dev_null  |      44.030|   278.552|
|                   |           -|     6.33x|
```
2020-09-09 11:11:36 -07:00
Jean Boussier
b49a870414 Add a :since option to dump_all
This is useful to see what a block of code allocated, e.g.

```
GC.start
GC.disable
ObjectSpace.trace_object_allocations do
  # run some code
end
gc_gen = GC.count
allocations = ObjectSpace.dump_all(output: :file, since: gc_gen)
GC.enable
GC.start
retentions = ObjectSpace.dump_all(output: :file, since: gc_gen)
```
2020-09-09 08:05:14 -07:00
John Hawthorn
971857c332 Fix method name escaping in ObjectSpace.dump
It's possible to define methods with any name, even if the parser
doesn't support it and it can only be used with ex. send.

This fixes an issue where invalid JSON was output from ObjectSpace.dump
when a method name needed escaping.
2020-08-17 09:47:53 -07:00
Nobuyoshi Nakada
27f7b047e0
Also escape DEL code 2020-08-17 22:36:48 +09:00
Nobuyoshi Nakada
7b4b5e0840
Fixed the radix for control chars 2020-08-17 22:30:26 +09:00
Jean Boussier
6a0cb1d649 Avoid allocating a string when dumping an anonymous module or class 2020-07-23 10:52:30 +09:00
Alan Wu
cbf52087a2 Fix missing imemo cases in objspace_dump by refactoring
imemo_callcache and imemo_callinfo were not handled by the `objspace`
module and were showing up as "unknown" in the dump. Extract the code for
naming imemos and use that in both the GC and the `objspace` module.
2020-07-10 22:42:35 -04:00
Nobuyoshi Nakada
e474c189da
Suppress -Wswitch warnings 2020-04-08 15:13:37 +09:00
卜部昌平
5e22f873ed decouple internal.h headers
Saves comitters' daily life by avoid #include-ing everything from
internal.h to make each file do so instead.  This would significantly
speed up incremental builds.

We take the following inclusion order in this changeset:

1.  "ruby/config.h", where _GNU_SOURCE is defined (must be the very
    first thing among everything).
2.  RUBY_EXTCONF_H if any.
3.  Standard C headers, sorted alphabetically.
4.  Other system headers, maybe guarded by #ifdef
5.  Everything else, sorted alphabetically.

Exceptions are those win32-related headers, which tend not be self-
containing (headers have inclusion order dependencies).
2019-12-26 20:45:12 +09:00
git
e315f3a134 * expand tabs. 2019-07-31 10:22:47 +09:00
Koichi Sasada
72825c35b0 Use 1 byte hint for ar_table [Feature #15602]
On ar_table, Do not keep a full-length hash value (FLHV, 8 bytes)
but keep a 1 byte hint from a FLHV (lowest byte of FLHV).
An ar_table only contains at least 8 entries, so hints consumes
8 bytes at most. We can store hints in RHash::ar_hint.

On 32bit CPU, we use 4 entries ar_table.

The advantages:
* We don't need to keep FLHV so ar_table only consumes
  16 bytes (VALUEs of key and value) * 8 entries = 128 bytes.
* We don't need to scan ar_table, but only need to check hints
  in many cases. Especially we don't need to access ar_table
  if there is no match entries (in many cases).
  It will increase memory cache locality.

The disadvantages:
* This technique can increase `#eql?` time because hints can
  conflicts (in theory, it conflicts once in 256 times).
  It can introduce incompatibility if there is a object x where
  x.eql? returns true even if hash values are different.
  I believe we don't need to care such irregular case.
* We need to re-calculate FLHV if we need to switch from ar_table
  to st_table (e.g. exceeds 8 entries).
  It also can introduce incompatibility, on mutating key objects.
  I believe we don't need to care such irregular case too.

Add new debug counters to measure the performance:
* artable_hint_hit - hint is matched and eql?#=>true
* artable_hint_miss - hint is not matched but eql?#=>false
* artable_hint_notfound - lookup counts
2019-07-31 09:52:03 +09:00
Lourens Naudé
612cad5d20
Let struct dump_config in objspace fit in a single cache line
Let dump_config boolean members roots and full_heap be bit flags
instead

Closes: https://github.com/ruby/ruby/pull/2274
2019-07-08 13:30:21 +09:00
nobu
97fffcadcb ext/objspace/objspace_dump.c: print addresses consistently
The format addresses are printed in are different if you use
`ObjectSpace.dump_all(output: :stdout)` vs.
`ObjectSpace.dump_all(output: :string)` (or `ObjectSpace.dump`) due to
differences in the underlying `vfprintf` implementation.

Use `"%#"PRIxVALUE` to format `VALUE`.

Co-authored-by: Ashe Connor <ashe@kivikakk.ee>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64974 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-09 03:21:57 +00:00
svn
e20ac9442d * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64973 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-09 02:01:56 +00:00
naruse
b9d00698c8 Revert "ext/objspace/objspace_dump.c: print addresses consistently"
This reverts commit r64970.

Visual C++ 12.0 doesn't have PRIxPTR.
Anyway we have our own vfprintf implementation BSD_vfprintf().
If you want to have portable vfprintf, replace it with BSD_vfprintf like
vsnprintf or just use BSD_vfprintf.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64972 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-09 02:01:54 +00:00
svn
9f81048b45 * expand tabs.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64971 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-08 23:43:48 +00:00
tenderlove
c5768e4366 ext/objspace/objspace_dump.c: print addresses consistently
The format addresses are printed in are different if you use
`ObjectSpace.dump_all(output: :stdout)` vs.
`ObjectSpace.dump_all(output: :string)` (or `ObjectSpace.dump`) due to
differences in the underlying `vfprintf` implementation.

Use %"PRIxPTR" instead to be consistent across both.

Co-authored-by: Ashe Connor <ashe@kivikakk.ee>

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64970 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-10-08 23:43:45 +00:00
mame
e9303e1a00 Rename imemo_alloc with imemo_tmpbuf
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-09 07:11:59 +00:00
nobu
c8094ff154 no ID cache in Init functions
Init functions are called only once, cache is useless.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-02-16 08:39:48 +00:00
kazu
51edb30042 Include ruby/{io,encoding}.h before internal.h
because of r61712 and r61713

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61725 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-01-09 12:35:12 +00:00
tenderlove
2c075d3863 Fix imemo_name to dump new imemo types
New IMEMO types were introduced, this just fixes the function that
converts the type to support the new types.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-12-21 00:40:54 +00:00
mame
b6e80c2260 Remove RNODE cast from NODE utility functions
Now, casting NODE to VALUE is not recommended.  This change requires an
explicit cast from VALUE to NODE to use the NODE utility functions such
as `nd_type`.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-11-04 11:37:19 +00:00
nobu
37a9da68a7 objspace_dump.c: remove unnecessary break
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-27 05:27:06 +00:00
tenderlove
ef76e3cfb6 Add IMEMO type to heap dump output.
IMEMO objects have many types.  Without this change, we cannot see what
types of IMEMO objects are being used when dumping the heap.  Adding the
type to the IMEMO object will allow us to gather statistics about IMEMO
objects being used.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-31 17:46:51 +00:00
tenderlove
ed2abc4362 Add full option to ObjectSpace.dump_all
The `full` option includes all slots (even `T_NONE`) in the JSON output.
This is to help with debugging heap fragmentation.

Here is an example usage:

```ruby
File.open('heap.json', 'w') do |f|
  ObjectSpace.dump_all(output: f, full: true)
end
```

The `heap.json` file contains all slots, including empty slots.

[Feature #13001] [ruby-core:78468]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57260 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-03 22:42:10 +00:00
nobu
cf6ca2d8b2 objspace_dump.c: dc is no longer used [ci skip]
* ext/objspace/objspace_dump.c (heap_i): remove no longer used
  variable.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56177 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-17 09:22:26 +00:00
tenderlove
45e8ccde00 Use JSON lines format for full heap dumps.
This commit changes full heap dumps back to using JSON lines format
(http://jsonlines.org) so that we can process very large heaps without
loading the entire heap in to memory at once.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-16 17:44:45 +00:00
rhe
2f551849c4 Use PRIuSIZE format specifier for size_t values
Use PRIuSIZE instead of PRIdSIZE. This fixes the exception message shown
on too large xmalloc2. This commit also fixes other incorrect use of
PRIdSIZE in other functions; though most of them are debug print.

* gc.c (heap_extend_pages, get_envparam_size, ruby_malloc_size_overflow,
  gc_profile_dump_on): Use PRIuSIZE instead of PRIdSIZE as the passed
  value is size_t, not ssize_t.

* iseq.c (get_line_info, rb_iseq_disasm_insn): Ditto.

* sprintf.c (rb_str_format): Ditto.

* thread_win32.c (native_thread_create): Ditto.

* vm.c (get_param): Ditto.

* ext/objspace/objspace_dump.c (dump_append_string_content,
  dump_object): Ditto.

* ext/socket/raddrinfo.c (host_str, port_str): Ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-13 12:33:13 +00:00
nobu
cfb2b14ddc ObjectSpace.dump_all: valid JSON
* ext/objspace/objspace_dump.c: generate valid JSON for dump_all.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55377 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-06-11 05:04:22 +00:00
nobu
d7d64d9147 objspace_dump.c: fix format
* ext/objspace/objspace_dump.c (dump_append_string_content),
  (dump_object): fix the format specifiers.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54948 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-08 09:59:36 +00:00
naruse
f2535e1467 * configure.in: add -Wsuggest-attribute=format and suppress warnings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54945 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-05-08 00:55:29 +00:00
naruse
3738fe3333 * variable.c: use uint32_t instead of long to avoid confusion about
the type of ivtbl->numiv.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54700 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-04-22 09:47:34 +00:00
nobu
c92322b537 objspace_dump.c: special const
* ext/objspace/objspace_dump.c: support special constant objects.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-10-19 06:25:01 +00:00
tenderlove
3b90363f76 * ext/objspace/objspace_dump.c (obj_type): add IMEMO types to the heap
dump information.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51892 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-09-17 22:40:24 +00:00
nobu
e833adaa69 ext/objspace/objspace_dump.c: adjust indent
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50983 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-21 03:21:34 +00:00
sorah
cc77375d2d * ext/objspace/objspace_dump.c(dump_object): Return empty JSON object when
passed object is a special const, instead of SEGV.
  Based patch by Kohei Suzuki (eagletmt). [ruby-core:69692] [Bug #11291]

* test/objspace/test_objspace.rb(test_dump_special_consts): Test for above fix.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-20 20:50:20 +00:00
akr
7cd76ab0c5 * internal.h: Include ruby.h and ruby/encoding.h to be
includable without prior inclusion.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48447 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-15 11:49:06 +00:00
nobu
27c4fb3361 ext: get rid of inadvertent ID creation
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-08-03 01:56:16 +00:00
shyouhei
39fd13bc5c * ext/objspace/objspace_dump.c (dump_append): avoid fflush.
because dump_append_string_value() iterates over each chars,
	  fflush()-ing here effectively issues system calls on every single
	  bytes exist in a ruby process.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46959 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-07-26 08:28:05 +00:00
charliesome
c8c539a198 ext/objspace: Check fptr before trying to dump FILE object fd
Patch by Scott Francis. Closes GH-562.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45308 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-10 21:55:51 +00:00
ko1
d76b382683 * ext/objspace/objspace_dump.c (dump_object): use STR_SHARED_P()
instead of removed STR_NOCAPA_P() macro.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44832 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-02-05 05:41:04 +00:00
ko1
c77a745aab * string.c: remvoe STR_ASSOC related code.
By r44804, string objects can not have STR_ASSOC flag.
* internal.h: ditto.
* ext/objspace/objspace_dump.c (dump_object): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44805 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-02-04 07:12:49 +00:00
tmm1
7170baa878 objspace_dump.c: include object's gc flags in dump
* ext/objspace/objspace_dump.c (dump_object): include fstring flag on
  strings. include gc flags (old, remembered, wb_protected) on all objects.
* ext/objspace/objspace_dump.c (Init_objspace_dump): initialize lazy
  IDs before first use.
* gc.c (rb_obj_gc_flags): new function to retrieve object flags
* internal.h (RB_OBJ_GC_FLAGS_MAX): maximum flags allowed for one obj
* test/objspace/test_objspace.rb (test_dump_flags): test for above
* test/objspace/test_objspace.rb (test_trace_object_allocations):
  resolve name before dump (for rb_class_path_cached)

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-12-10 02:26:09 +00:00
tmm1
adcd0174b9 * ext/objspace/objspace_dump.c (dump_append_string_value): Escape
control characters for strict json parsers.
* ext/objspace/objspace_dump.c (objspace_dump): Document File/IO
  output option.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-11-26 05:27:22 +00:00