Prevent GC compaction and unintended sweeps for preventing checksum mismatch

[cosmo0920]

With this repro which is based on #112 (comment)

require"digest"require"tempfile"
$LOAD_PATH.unshiftFile.expand_path('lib',__dir__)require'zstd-ruby'puts"Zstd::VERSION=#{Zstd::VERSION}"defcompare_compressed(original:,compressed:)begindecompressed=Zstd.decompress(compressed)rescue=>edecompress_error=eendiforiginal != decompressedifdecompress_errorputs"Decompression error for #{original.bytesize} bytes input (#{decompress_error})"elseputs"Content mismatch for #{original.bytesize} bytes input"endputs" Original: #{original.bytesize} bytes, #{Digest::SHA256.hexdigest(original)[0,10]} checksum"ifdecompressedputs" Zstd.decompress: #{decompressed.bytesize} bytes, #{Digest::SHA256.hexdigest(decompressed)[0,10]} checksum"endbegincli_decompressed=Tempfile.create(binmode: true)do |temp_write|
temp_write.write(compressed)temp_write.closeTempfile.create(binmode: true)do |temp_read|
system"zstd","--decompress","--quiet","--force","-o",temp_read.path,temp_write.path,exception: trueFile.read(temp_read.path,binmode: true)endendputs" zstd cli: #{cli_decompressed.bytesize} bytes, #{Digest::SHA256.hexdigest(cli_decompressed)[0,10]} checksum"rescue=>eputs" zstd cli error: #{e}"endendenddeftest_stream_write(1..256_000).eachdo |length|
original="a" * lengthstream=Zstd::StreamingCompress.newstream << originalres=stream.finishcompare_compressed(original: original,compressed: res)endenddeftest_stream_compress(1..256_000).eachdo |length|
original="a" * lengthstream=Zstd::StreamingCompress.newres=stream.compress(original)res << stream.finishcompare_compressed(original: original,compressed: res)endenddeftest_compress(1..256_000).eachdo |length|
original="a" * lengthres=Zstd.compress(original)compare_compressed(original: original,compressed: res)endenddefdebug_specific_size(length)puts"=== Debugging size #{length} ==="original="a" * length# 1. Generate the "good" stream using the working .compress methodstream_good=Zstd::StreamingCompress.newgood_compressed=stream_good.compress(original)good_compressed << stream_good.finishFile.write("good.zst",good_compressed)puts"Saved working output to good.zst (#{good_compressed.bytesize} bytes)"# 2. Generate the "bad" stream using the failing << methodstream_bad=Zstd::StreamingCompress.newstream_bad << originalbad_compressed=stream_bad.finishFile.write("bad.zst",bad_compressed)puts"Saved failing output to bad.zst (#{bad_compressed.bytesize} bytes)"putsendcaseARGV[0]when"stream_write"puts"=== Zstd::StreamingCompress.new with << ==="test_stream_writewhen"stream_compress"puts"=== Zstd::StreamingCompress.new with .compress ==="test_stream_compresswhen"compress"puts"=== Zstd.compress ==="test_compresswhen"debug"size=ARGV[1].to_iifsize <= 0abort"Please provide a specific size to debug, e.g., 'ruby zstd_repro.rb debug 20086'"enddebug_specific_size(size)elseabort"Unknown test mode: #{ARGV[0].inspect}"end

And using with rspec tests, there is no issues and checksum mismatches reported.

$ bundle exec rspec
<snip>
Finished in 15.39 seconds (files took 0.12929 seconds to load)
60 examples, 0 failures

% ruby zstd_repro.rb compressZstd::VERSION=2.0.0=== Zstd.compress ===
% ruby zstd_repro.rb stream_compressZstd::VERSION=2.0.0=== Zstd::StreamingCompress.new with .compress ===
% ruby zstd_repro.rb stream_writeZstd::VERSION=2.0.0=== Zstd::StreamingCompress.new with << ===

[SpringMT]

Thank you!

[SpringMT]

I just released v2.0.1 !