1berry/ruby
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

* lib/csv.rb (CSV#read_to_char): set encoding and verify data

Browse Source 
which read from io before encode it to @encoding.

* lib/csv.rb (CSV#raw_encoding): add to get @io's encoding.

* lib/csv.rb (CSV#read_io): add to read string and set @io's
  encoding.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25353 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
naruse 2009-10-15 18:19:15 +00:00
parent e5e49b3a27
commit e3c7548363
2 changed files with 29 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
ChangeLog
View File

@ -1,3 +1,13 @@
Fri Oct 16 03:15:52 2009 NARUSE, Yui <naruse@ruby-lang.org>
* lib/csv.rb (CSV#read_to_char): set encoding and verify data
which read from io before encode it to @encoding.
* lib/csv.rb (CSV#raw_encoding): add to get @io's encoding.
* lib/csv.rb (CSV#read_io): add to read string and set @io's
encoding.
Thu Oct 15 18:26:12 2009 Nobuyoshi Nakada <nobu@ruby-lang.org> Thu Oct 15 18:26:12 2009 Nobuyoshi Nakada <nobu@ruby-lang.org>
* parse.y (rb_intern3): check symbol table overflow before generate * parse.y (rb_intern3): check symbol table overflow before generate

28
lib/csv.rb
View File

@ -1550,12 +1550,7 @@ class CSV
# create the IO object we will read from # create the IO object we will read from
@io = if data.is_a? String then StringIO.new(data) else data end @io = if data.is_a? String then StringIO.new(data) else data end
# honor the IO encoding if we can, otherwise default to ASCII-8BIT # honor the IO encoding if we can, otherwise default to ASCII-8BIT
@encoding = if @io.respond_to? :internal_encoding @encoding = raw_encoding || Encoding.default_internal || Encoding.default_external
@io.internal_encoding || @io.external_encoding
elsif @io.is_a? StringIO
@io.string.encoding
end
@encoding ||= Encoding.default_internal || Encoding.default_external
# #
# prepare for building safe regular expressions in the target encoding, # prepare for building safe regular expressions in the target encoding,
# if we can transcode the needed characters # if we can transcode the needed characters
@ -1989,7 +1984,6 @@ class CSV
sample = read_to_char(1024) sample = read_to_char(1024)
sample += read_to_char(1) if sample[-1..-1] == encode_str("\r") and sample += read_to_char(1) if sample[-1..-1] == encode_str("\r") and
not @io.eof? not @io.eof?
# try to find a standard separator # try to find a standard separator
if sample =~ encode_re("\r\n?|\n") if sample =~ encode_re("\r\n?|\n")
@row_sep = $& @row_sep = $&
@ -2272,8 +2266,9 @@ class CSV
# #
def read_to_char(bytes) def read_to_char(bytes)
return "" if @io.eof? return "" if @io.eof?
data = @io.read(bytes) data = read_io(bytes)
begin begin
raise unless data.valid_encoding?
encoded = encode_str(data) encoded = encode_str(data)
raise unless encoded.valid_encoding? raise unless encoded.valid_encoding?
return encoded return encoded
@ -2281,11 +2276,26 @@ class CSV
if @io.eof? or data.size >= bytes + 10 if @io.eof? or data.size >= bytes + 10
return data return data
else else
data += @io.read(1) data += read_io(1)
retry retry
end end
end end
end end
private
def raw_encoding
if @io.respond_to? :internal_encoding
@io.internal_encoding || @io.external_encoding
elsif @io.is_a? StringIO
@io.string.encoding
else
@io.encoding
end
end
def read_io(bytes)
@io.read(bytes).force_encoding(raw_encoding)
end
end end
# Another name for CSV::instance(). # Another name for CSV::instance().