* string.c (mapping_buffer): get rid of zero-length array member,
which is not a part of C90.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56915 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c: Implement String#casecmp? and Symbol#casecmp? by using
String#downcase :fold for Unicode case folding. This does not include
options such as :turkic, because these currently cannot be combined
with the :fold option. This implements feature #12786.
* test/ruby/test_string.rb/test_symbol.rb: Tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
a hash value of Object might be Bignum, but it causes many troubles
expecially the Object is used as a key of a hash. so I've gave up
to do so.
* array.c (rb_ary_hash): use above macro.
* bignum.c (rb_big_hash): ditto.
* hash.c (rb_obj_hash, rb_hash_hash): ditto.
* numeric.c (rb_dbl_hash): ditto.
* proc.c (proc_hash): ditto.
* re.c (rb_reg_hash, match_hash): ditto.
* string.c (rb_str_hash_m): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56340 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
size with int, and of course also not guaranteed the value can be
Fixnum.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56320 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (lstrip_offset): add a fast path in the case of single
byte optimizable strings, as well as rstrip_offset.
[ruby-core:77392] [Feature #12788]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56250 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (enc_strlen, rb_enc_strlen_cr): Avoid signed integer
overflow. The result type of a pointer subtraction may have the same
size as long. This fixes String#size returning an negative value on
i686-linux environment:
str = "\x00" * ((1<<31)-2))
str.slice!(-3, 3)
str.force_encoding("UTF-32BE")
str << 1234
p str.size
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56247 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
actually check its availability rather to check GCC's version.
* configure.in (WARN_UNUSED_RESULT): moved to here.
* configure.in (RUBY_FUNC_ATTRIBUTE): change function declaration
to return int rather than void, because it makes no sense for a
warn_unused_result attributed function to return void.
Funny thing however is that it also makes no sense for noreturn
attributed function to return int. So there is a fundamental
conflict between them. While I tested this, I confirmed both
GCC 6 and Clang 3.8 prefers int over void to correctly detect
necessary attributes under this setup. Maybe subject to change
in future.
* internal.h (UNINITIALIZED_VAR): renamed to MAYBE_UNUSED, then
moved to configure.in for the same reason we move
WARN_UNUSED_RESULT.
* configure.in (MAYBE_UNUSED): moved to here.
* internal.h (__has_attribute): deleted, because it has no use now.
* string.c (rb_str_enumerate_lines): refactor macro rename.
* string.c (rb_str_enumerate_bytes): ditto.
* string.c (rb_str_enumerate_chars): ditto.
* string.c (rb_str_enumerate_codepoints): ditto.
* thread.c (do_select): ditto.
* vm_backtrace.c (rb_debug_inspector_open): ditto.
* vsnprintf.c (BSD_vfprintf): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56169 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
The behavior on signed integer overflow is undefined. On platform with
sizeof(long)==4, it's fairly easy that 'len + termlen' overflows, where
len is the string length and termlen is the terminator length.
So, prevent the integer overflow by avoiding adding to a string length,
or casting to size_t before adding where the total size is passed to
{RE,}ALLOC*().
* string.c (STR_HEAP_SIZE, RESIZE_CAPA_TERM, str_new0, rb_str_buf_new,
str_shared_replace, rb_str_init, str_make_independent_expand,
rb_str_resize): Avoid overflow by casting the length to size_t. size_t
should be able to represent LONG_MAX+termlen.
* string.c (rb_str_modify_expand): Check that the new length is in the
range of long before resizing. Also refactor to use RESIZE_CAPA_TERM
macro.
* string.c (str_buf_cat): Fix so that it does not create a negative
length String. Also fix the condition for 'string sizes too big', the
total length can be up to LONG_MAX.
* string.c (rb_str_plus): Check the resulting String length does not
exceed LONG_MAX.
* string.c (rb_str_dump): Fix integer overflow. The dump result will be
longer then the original String.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56157 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_EMBEDDABLE_P): Renamed from STR_EMBEDABLE_P(). And use
it in more places.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56155 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_EMBEDABLE_P): extract the predicate macro to tell
if the given length is capable in an embedded string, and fix
possible integer overflow.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56151 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_change_terminator_length): fix integer overflow
in the case growing the terminator length and the string length
is around LONG_MAX.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56149 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_set_len): The buffer overflow check is wrong. The
space for termlen is allocated outside the capacity returned by
rb_str_capacity(). This fixes r41920 ("string.c: multi-byte
terminator", 2013-07-11). [ruby-core:77257] [Bug #12757]
* test/-ext-/string/test_set_len.rb (test_capacity_equals_to_new_size):
Test for this change. Applying only the test will trigger [BUG].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56148 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* array.c (rb_ary_concat_multi): take multiple arguments. based
on the patch by Satoru Horie. [Feature #12333]
* string.c (rb_str_concat_multi, rb_str_prepend_multi): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56021 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_fs_setter): check and convert $; value at
assignment.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55990 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_split_m): show $; name in error message when it
is a wrong object.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* *.c: rename rb_funcall2 to rb_funcallv, except for extensions
which are/will be/may be gems. [Fix GH-1406]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55773 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Bug #12628]
This patch introduce many changes.
* Introduce concept of "Block Handler (BH)" to represent
passed blocks.
* move rb_control_frame_t::flag to ep[0] (as a special local
variable). This flags represents not only frame type, but also
env flags such as escaped.
* rename `rb_block_t` to `struct rb_block`.
* Make Proc, Binding and RubyVM::Env objects wb-protected.
Check [Bug #12628] for more details.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
UTF-8 to use upper-case four-digit hexadecimal escapes without braces
where possible [Feature #12419].
* test/ruby/test_string.rb (test_dump): Add tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55728 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (STR_BUF_MIN_SIZE): reduce from 128 to 127
[ruby-core:76371] [Feature #12025]
* string.c (rb_str_buf_new): adjust for above reduction
From Jeremy Evans <code@jeremyevans.net>:
This changes the minimum buffer size for string buffers from 128 to
127. The underlying C buffer is always 1 more than the ruby buffer,
so this changes the actual amount of memory used for the minimum
string buffer from 129 to 128. This makes it much easier on the
malloc implementation, as evidenced by the following code (note that
time -l is used here, but Linux systems may need time -v).
$ cat bench_mem.rb
i = ARGV.first.to_i
Array.new(1000000){" " * i}
$ /usr/bin/time -l ruby bench_mem.rb 128
3.10 real 2.19 user 0.46 sys
289080 maximum resident set size
72673 minor page faults
13 block output operations
29 voluntary context switches
$ /usr/bin/time -l ruby bench_mem.rb 127
2.64 real 2.09 user 0.27 sys
162720 maximum resident set size
40966 minor page faults
2 block output operations
4 voluntary context switches
To try to ensure a power-of-2 growth, when a ruby string capacity
needs to be increased, after doubling the capacity, add one. This
ensures the ruby capacity will be odd, which means actual amount
of memory used will be even, which is probably better than the
current case of the ruby capacity being even and the actual amount
of memory used being odd.
A very similar patch was proposed 4 years ago in feature #5875. It
ended up being rejected, because no performance increase was shown.
One reason for that is that ruby does not use STR_BUF_MIN_SIZE
unless rb_str_buf_new is called, and that previously did not have
a ruby API, only a C API, so unless you were using a C extension
that called it, there would be no performance increase.
With the recently proposed feature #12024, String.buffer is added,
which is a ruby API for creating string buffers. Using
String.buffer(100) wastes much less memory with this patch, as the
malloc implementation can more easily deal with the power-of-2
sized memory usage. As measured above, memory usage is 44% less,
and performance is 17% better.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55686 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
termlen and resize heap for the terminator. This is split from
rb_str_fill_terminator (str_fill_term) because filling terminator
and changing terminator length are different things. [Bug #12536]
* internal.h: declaration for rb_str_change_terminator_length.
* string.c (str_fill_term): Simplify only to zero-fill the terminator.
For non-shared strings, it assumes that (capa + termlen) bytes of
heap is allocated. This partially reverts r55557.
* encoding.c (rb_enc_associate_index): rb_str_change_terminator_length
is used, and it should be called whenever the termlen is changed.
* string.c (str_capacity): New static function to return capacity
of a string with the given termlen, because the termlen may
sometimes be different from TERM_LEN(str) especially during
changing termlen or filling terminator with specific termlen.
* string.c (rb_str_capacity): Use str_capacity.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
ChangeLog about the reverted changes are also deleted in this file.
[Bug #12536] [ruby-dev:49699] [ruby-dev:49702]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55559 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
of memory for termlen should always be needed.
In this fix, if possible, decrease capa instead of realloc.
[Bug #12536] [ruby-dev:49699]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55557 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Additional fix for [Bug #12536] [ruby-dev:49699].
* string.c (rb_usascii_str_new, rb_utf8_str_new): Specify termlen
which is apparently 1 for the encodings.
* string.c (str_new0_cstr): New static function to create a String
object from a C string with specifying termlen.
* string.c (rb_usascii_str_new_cstr, rb_utf8_str_new_cstr): Specify
termlen by using new str_new0_cstr().
* string.c (str_new_static): Specify termlen from the given encoding
when creating a new String object is needed.
* string.c (rb_tainted_str_new_with_enc): New function to create a
tainted String object with the given encoding. This means that
the termlen is correctly specified. Curretly static function.
The function name might be renamed to rb_tainted_enc_str_new
or rb_enc_tainted_str_new.
* string.c (rb_external_str_new_with_enc): Use encoding by using the
above rb_tainted_str_new_with_enc().
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55555 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
is used, TERM_LEN(str) should be considered with it because
embedded strings are also processed by TERM_FILL.
Additional fix for [Bug #12536] [ruby-dev:49699].
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55552 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
[Bug #12536] [ruby-dev:49699]
* string.c (TERM_LEN_MAX): Macro for the longest TERM_FILL length,
the same as largest value of rb_enc_mbminlen(enc) among encodings.
* string.c (str_new, rb_str_buf_new, str_shared_replace): Allocate
+TERM_LEN_MAX bytes instead of +1. This change may increase memory
usage.
* string.c (rb_str_new_with_class): Use TERM_LEN of the "obj".
* string.c (rb_str_plus, rb_str_justify): Use str_new0 which is aware
of termlen.
* string.c (str_shared_replace): Copy +termlen bytes instead of +1.
* string.c (rb_str_times): termlen should not be included in capa.
* string.c (RESIZE_CAPA_TERM): When using RSTRING_EMBED_LEN_MAX,
termlen should be counted with it because embedded strings are
also processed by TERM_FILL.
* string.c (rb_str_capacity, str_shared_replace, str_buf_cat): ditto.
* string.c (rb_str_drop_bytes, rb_str_setbyte, str_byte_substr): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55547 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* string.c (rb_str_casemap): do not put code with side effects
inside RSTRING_PTR() macro which evaluates the argument multiple
times.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
