Tempfile and garbage collection
I have this command in my Rails controller
open(source) { |s| content = s.read }
rss = RSS::Parser.parse(content, false)
and this results in temporary files filling up (limited) disk space.
I've looked into the problem to some degree and it ends up somewhere on the stack:
io = Tempfile.new('open-uri')
but it looks like this Tempfile instance is never closed explicitly. He received
def _close # :nodoc:
which can automatically fire on garbage collection?
Any help understanding what is going on or how to clear the tempfiles would be really helpful.
source to share
looks like it _close
closes the file and then waits for garbage collection to undo (delete) the file. In theory, you could force it to be disabled by calling a method Tempfile
close!
instead, close
or calling close(true)
(which calls close!
internally).
edit: But the problem is in open-uri, which is out of your hands - and that doesn't make promises to clean up after itself: it just assumes that the garbage collector will complete everything Tempfile
in due time.
In this case, you have no choice but to call the garbage collector using ObjectSpace.garbage_collect
( see here ). This should delete all temporary files.
source to share
If you really want to force open-uri
not to use a temporary file, you can mess up the constant OpenURI::Buffer::StringMax
:
> require 'open-uri'
=> true
> OpenURI::Buffer::StringMax
=> 10240
> open("http://www.yahoo.com")
=> #<File:/tmp/open-uri20110111-16395-8vco29-0>
> OpenURI::Buffer::StringMax = 1_000_000_000_000
(irb):10: warning: already initialized constant StringMax
=> 1000000000000
> open("http://www.yahoo.com")
=> #<StringIO:0x6f5b1c>
Which is because of this snippet from open-uri.rb
:
class Buffer
[...]
StringMax = 10240
def <<(str)
[...]
if [...] StringMax < @size
require 'tempfile'
source to share
Definitely not an error, but a faulty I / O processing error. Buffer.io is either StringIO if @size is less than 10240 bytes or Tempfile if over that amount. The provision condition in OpenURI.open_uri () calls the close () function, but since it might be a StringIO object that doesn't have a close! () Method, it simply can't just call close! ().
The fix I think would be one of the following:
Checks the clure clause for the class and calls either StringIO.close or Tempfile.close! if necessary.
- or -
The Buffer class requires a finalizer that handles the validation of the class and calls the correct method.
Of course, none of them fix this unless you are using a block to handle the IO, but I suppose you can do your own check in that case, since open () returns an IO object, not a Buffer object.
lib is a big chunk of messy code, imho, so it can recycle to clean it up. I think I can do this, just for fun. ^. ^
source to share