How to block IO shared by fork in ruby
How can we block an IO that has been shared by multiple ruby processes?
Consider this script:
#!/usr/bin/ruby -w
# vim: ts=2 sw=2 et
if ARGV.length != 2
$stderr.puts "Usage: test-io-fork.rb num_child num_iteration"
exit 1
end
CHILD = ARGV[0].to_i
ITERATION = ARGV[1].to_i
def now
t = Time.now
"#{t.strftime('%H:%M:%S')}.#{t.usec}"
end
MAP = %w(nol satu dua tiga empat lima enam tujuh delapan sembilan)
IO.popen('-', 'w') {|pipe|
unless pipe
# Logger child
File.open('test-io-fork.log', 'w') {|log|
log.puts "#{now} Program start"
$stdin.each {|line|
log.puts "#{now} #{line}"
}
log.puts "#{now} Program end"
}
exit!
end
pipe.sync = true
pipe.puts "Before fork"
CHILD.times {|c|
fork {
pid = Process.pid
srand
ITERATION.times {|i|
n = rand(9)
sleep(n / 100000.0)
pipe.puts "##{c}:#{i} #{MAP[n]} => #{n}, #{n} => #{MAP[n]} ##{c}:#{i}"
}
}
}
}
And try like this:
./test-io-fork.rb 200 50
As expected, the test-io-fork.log files will contain the IO race sign.
What I want to achieve is to make a TCP server for custom GPS protocol that will store the GPS points in the database. Since this server will handle 1000 concurrent clients, I would like to limit the database connection to only one child, instead opening 1000 database connections at a time. This server will run on Linux.
source to share
UPDATE
It might be a bad form to update after accepting the answer, but the original is a little misleading. Whether or not ruby makes a separate call write(2)
for the auto-added newline depends on the buffering state of the IO input object.
$stdout
(when connected to a tty) is usually line buffered, so the effect puts()
- of a given reasonably sized line - with an implicitly added new line is one call write(2)
. However not so, with IO.pipe
and $stderr
as OP discovered.
ORIGINAL ANSWER
Change your main argument pipe.puts()
to be a terminated newline string:
pipe.puts "##{c} ... #{i}\n" # <-- note the newline
Why? You set up pipe.sync
hoping that the pipe write will be atomic and unmoved as they are (presumably) smaller than PIPE_BUF
bytes.But that didn't work because the ruby pipe implementation puts()
makes a separate write (2) call to add a trailing newline and so your entries sometimes alternate where you expected a newline.
Here's a confirmation snippet from the following version of your script:
$ strace -s 2048 -fe trace=write ./so-1326067.rb
....
4574 write(4, "#0:12 tiga => 3, 3 => tiga #0:12", 32) = 32
4574 write(4, "\n", 1)
....
But including your newline solves the problem, making sure your entire record is passed in a single system call:
....
5190 write(4, "#194:41 tujuh => 7, 7 => tujuh #194:41\n", 39 <unfinished ...>
5179 write(4, "#183:38 enam => 6, 6 => enam #183:38\n", 37 <unfinished ...>
....
If for some reason this doesn't work for you, you will have to coordinate the interprocess mutex (for example File.flock()
).
source to share