1berry/ruby
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
91,898 Commits 27 Branches 1,171 Tags
Commit Graph

420 Commits

Author SHA1 Message Date
akr
7908180df1 * tool/transcode-tblgen.rb: record offsets array as index of 
byte_array to avoid relocation.

* transcode.c (transcode_restartable0): add byte_array to get offsets
  array.

* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
  uintptr_t.
  (rb_transcoder): add fields: byte_array, word_array and word_size.

* enc/trans/newline.trans: follow rb_transcoder change.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 17:40:32 +00:00
akr
752e053a1d * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array. 
(BYTE_LOOKUP_BASE): follow the type change.
  (BYTE_LOOKUP_INFO): ditto.
  (PType): ditto.
  (rb_transcoding): ditto.

* tool/transcode-tblgen.rb: follow the type change.

* transcode.c: ditto.

* enc/trans/newline.trans: ditto.

* enc/trans/iso2022.trans: ditto.

* enc/trans/utf_16_32.trans: ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 16:22:49 +00:00
akr
2ae7b0328e * transcode_data.h (BYTE_LOOKUP_BASE): abstract accessor for 
BYTE_LOOKUP.
  (BYTE_LOOKUP_INFO): ditto.

* transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and
  BYTE_LOOKUP_INFO.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-09-01 14:33:56 +00:00
akr
e7ee4fabd4 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 15:31:14 +00:00
akr
a017cecf42 fix typos. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 14:24:51 +00:00
akr
c3434e35ac rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:38:12 +00:00
akr
69610b07c3 * transcode.c (transcode.c): set source_encoding and 
destination_encoding as encoding object.
  (ecerr_source_encoding): new method.
  (ecerr_destination_encoding): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 09:08:31 +00:00
akr
72ee10b19c * transcode.c (make_econv_exception): rename instance variable names 
for storing encoding names.
  (ecerr_source_encoding_name): method renamed.
  (ecerr_destination_encoding_name): ditto.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 08:17:48 +00:00
akr
2d71485ec1 * transcode.c (econv_putback): associate encoding to the result. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:59:03 +00:00
akr
d427cf3af8 * transcode.c (econv_insert_output): raise ArgumentError on failure. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 07:43:19 +00:00
akr
c3f55e6d48 * transcode.c (econv_last_error): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:27:52 +00:00
akr
9d2accff2b * transcode.c (econv_primitive_convert): make two arguments, 
destination_byteoffset and destination_bytesize, optional.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:18:29 +00:00
akr
f3c43ae0c2 * transcode.c (make_econv_exception): error message simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-31 05:12:32 +00:00
akr
d9c5f88abc * transcode.c (econv_putback): make max argument optional. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:39:16 +00:00
akr
97a5808a14 * transcode.c (rb_econv_putback): putback from last byte. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:36:38 +00:00
akr
998a4ec5af * transcode.c (econv_putback): renamed from econv_primitive_putback. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:27:35 +00:00
akr
5bbc51ed43 * transcode.c (econv_insert_output): renamed from 
econv_primitive_insert_output.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 19:23:23 +00:00
akr
5e222d6945 * transcode.c (econv_primitive_errinfo): 5th element (partial_input) 
removed from result.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-30 18:45:48 +00:00
akr
c2ba5cf9c5 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-29 16:25:27 +00:00
akr
d4afbe49f3 rdoc update. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:28:52 +00:00
akr
2c80188627 * transcode.c (econv_finish): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18913 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 18:00:02 +00:00
akr
23d10cbb00 * transcode.c (econv_convert): new method. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:46:18 +00:00
akr
ad8cfb1b94 * transcode.c (econv_primitive_convert): accept nil as 
destination_bytesize for unlimited destination size.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18911 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:39:02 +00:00
akr
3811bb53bd * transcode.c (econv_primitive_convert): accept nil as input for empty 
input.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 17:13:49 +00:00
akr
c23ec90e81 * transcode.c (sym_invalid_byte_sequence): new variable. 
(sym_undefined_conversion): ditto.
  (sym_destination_buffer_full): ditto.
  (sym_source_buffer_empty): ditto.
  (sym_finished): ditto.
  (sym_output_followed_by_input): ditto.
  (sym_incomplete_input): ditto.
  (econv_result_to_symbol): use above variables.
  (Init_transcode): initialize above variables.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18908 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-28 16:59:17 +00:00
akr
1366802a1f * transcode.c (transcode_loop): simplified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18876 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 16:14:49 +00:00
akr
db6ec3105e * include/ruby/encoding.h (rb_econv_result_t): enumeration constant: 
econv_incomplete_input. 

* io.c (finish_writeconv): check econv_incomplete_input.

* transcode.c (transcode_restartable0): return econv_incomplete_input
  for unexpected end of source buffer.
  (trans_sweep): check econv_incomplete_input.
  (rb_trans_conv): ditto.
  (rb_econv_convert0): ditto.
  (rb_econv_convert): ditto.
  (transcode_loop): ditto.
  (make_econv_exception): change message for econv_incomplete_input.
  (econv_result_to_symbol): return :incomplete_input for
  econv_incomplete_input.
  (ecerr_incomplete_input): new method.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 16:09:29 +00:00
akr
a68e409eaa * include/ruby/encoding.h (rb_econv_elem_t): move to transcode.c 
(rb_econv_t): defined as an incomplete type.

* transcode.c (rb_econv_elem_t): moved from encoding.h.
  (rb_econv_t): complete type defined.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 15:01:11 +00:00
akr
19ab08653e * transcode.c (rb_econv_open): disable newline conversion for ASCII 
incompatible encodings.
  (str_transcode0): don't need disable newline conversion here.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18870 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 12:55:14 +00:00
akr
18b097b73b * transcode.c (rb_econv_binmode): binmode is effective only once. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18869 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-26 12:45:20 +00:00
akr
873987f2e0 * transcode.c (str_transcode0): disable newline conversion for ASCII 
incompatible encoding.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18853 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 16:27:23 +00:00
akr
1a805010ed * transcode.c (str_transcode0): don't short cut for newline conversion. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18852 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:58:35 +00:00
akr
86154f63b3 * transcode.c (make_econv_exception): show readagain part for invalid 
byte sequence exception.  store the part as an instance variable.
  (ecerr_readagain_bytes): new method to access the readagain part.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18850 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:26:54 +00:00
akr
0486026389 * transcode.c (rb_econv_open): make last_tc NULL if there are only 
additional transcoders.
  (econv_description): extracted from rb_econv_open_exc.
  (rb_econv_open_exc): use econv_description.
  (econv_inspect): use econv_description.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18843 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 15:01:36 +00:00
akr
aeffc67dbb * transcode.c (rb_econv_open_by_transcoder_entries): initialize 
last_error.  num_trans may be zero.
  (rb_econv_convert0): num_trans may be zero.
  (rb_econv_putbackable): ditto.
  (rb_econv_putback): ditto.
  (rb_econv_convert): input_ptr and output_ptr may be NULL.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18835 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 13:04:16 +00:00
nobu
10906176b4 * transcode.c (rb_econv_substr_append): fix for buffer overrun. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 08:35:42 +00:00
nobu
8c6040edca * transcode.c (rb_econv_open_exc): constified. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18828 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-25 01:42:36 +00:00
akr
2c78899641 * configure.in (BUILTIN_TRANSSRCS): defined. 
(BUILTIN_TRANSOBJS): defined.

* enc/Makefile.in (BUILTIN_TRANSES): defined.

* enc/make_encmake.rb (BUILTIN_TRANSES): defined.

* enc/depend: don't generate rules for builtin transcoders.

* common.mk (COMMONOBJS): add BUILTIN_TRANSOBJS.
  (enc.mk): pass BUILTIN_TRANSOBJS.
  (newline.c): new rule.
  (newline.$(OBJEXT)): new ru.e
  (srcs): newline.c added.

* Makefile.in (BUILTIN_TRANSSRCS): defined.
  (BUILTIN_TRANSOBJS): defined.

* transcode.c (Init_transcode): call Init_newline.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18826 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 15:58:43 +00:00
akr
bf5e559ce0 * transcode.c (declare_transcoder): arguments order changed. 
(rb_declare_transcoder): call declare_transcoder just once.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18824 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 15:13:37 +00:00
akr
ae1b02a8da * include/ruby/encoding.h (rb_econv_t): use rb_econv_option_t. 
* transcode.c: follow the rb_econv_t change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 10:49:36 +00:00
akr
071319fde1 * io.c (rb_io_s_pipe): accept optional hash. 
(rb_io_set_encoding): ditto.
  (rb_io_extract_modeenc): use rb_econv_opts to initialize
  ecopts.
  (rb_file_open_generic): ditto.
  (rb_file_open_internal): ditto.
  (io_encoding_set): new argument: opt.
  (argf_set_encoding): copy fptr->encs.opts to argf_ecopts.

* transcode.c (rb_econv_opts): accept Qnil for initialization.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18817 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 09:40:31 +00:00
akr
09c7eba7b1 * include/ruby/encoding.h (rb_str_transcode): make 3rd argument 
rb_econv_option_t*.

* transcode.c (transcode_loop): take rb_econv_option_t* as a argument.
  (str_transcode0): ditto.
  (str_transcode): make rb_econv_option_t and call str_transcode0 with
  it.
  (rb_str_transcode): take rb_econv_option_t*.

* io.c (io_fwrite): follow the rb_str_transcode change.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18814 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 08:39:09 +00:00
akr
f27b74142e * include/ruby/encoding.h (rb_econv_opts): declared. 
* transcode.c (rb_econv_opts): defined.

* io.c (rb_io_extract_modeenc): use rb_econv_opts.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18811 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 07:20:21 +00:00
akr
9b8adfed22 * include/ruby/encoding.h (rb_econv_option_t): defined. 
(rb_econv_open): 3rd arg changed.
  (rb_econv_open_exc): ditto.

* io.c (make_writeconv): use rb_econv_option_t.
  (make_readconv): ditto.
  (rb_econv_open): take rb_econv_option_t for options.
  (rb_econv_open_exc): ditto.
  (transcode_loop): use rb_econv_option_t.
  (econv_init): use rb_econv_option_t.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18810 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 07:12:44 +00:00
akr
b47c46106a * transcode.c (rb_econv_substr_append): associate dst with destination 
encoding when dst is created.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18809 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 06:44:21 +00:00
akr
c6d291b003 * include/ruby/encoding.h (rb_str_transcode): add ecflags argument. 
* transcode.c (econv_opts): extracted from str_transcode.
  (str_transcode_enc_args): extracted from str_transcode.
  (str_transcode0): extracted from str_transcode.
  (str_transcode): use econv_opts, str_transcode_enc_args,
  str_transcode0.
  (rb_str_transcode): call str_transcode0.
  (econv_primitive_insert_output): give the additional argument for
  rb_str_transcode.

* io.c (make_writeconv): use invalid/undef flags.
  (io_fwrite): ditto.
  (rb_scan_open_args): give the additional argument for
  rb_str_transcode.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18808 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 06:25:24 +00:00
akr
51ad3df2ce * transcode.c (str_transcode): check last hash only if 0 < argc. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18807 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 04:28:12 +00:00
akr
bc0efbc9cd * include/ruby/encoding.h (rb_econv_substr_append): renamed from 
rb_econv_string.
  (rb_econv_str_convert): declared.
  (rb_econv_substr_convert): declared.
  (rb_econv_str_append): declared.

* io.c (io_fwrite): use rb_econv_str_convert instead of
  rb_econv_string.

* transcode.c (rb_econv_substr_append): renamed from rb_econv_string.
  (rb_econv_str_append): new function.
  (rb_econv_substr_convert): ditto.
  (rb_econv_str_convert): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18806 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 03:22:43 +00:00
akr
7c2c7ea315 * transcode.c (rb_econv_open): return NULL on error. 
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18804 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 03:03:18 +00:00
akr
8f9ed3c464 * include/ruby/encoding.h (rb_econv_open_exc): declared. 
* transcode.c (rb_eNoConverter): new exception.
  (rb_econv_open_exc): new function.
  (transcode_loop): use rb_econv_open_exc.

* io.c (make_writeconv): use rb_econv_open_exc.
  (make_readconv): ditto.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18803 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
 2008-08-24 02:42:37 +00:00