CreateReadStream doesn't close the file

[petersirka]

I found a big problem in streams. I performed multiple tests on Ubuntu 14.04 LTS and OSX too and I see multiple opened files through lsof -p PID. I use autoClose: true (in default) and I destroy the stream manually with this module https://github.com/stream-utils/destroy.

This problem is not recorded for every request, but only if I performed multiple requests at the same time. My website falls down.

How can I prevent this problem? Is there a bug? How to close opened files? Extending ulimit is not a solution.

Thanks for any advice.

PS: io.js v1.5.1
I reported the same problem on node.js GitHub.

[mscdex]

Can you show the relevant code, or at least a minimal, reproducible example?

[vkurchatkin]

There is a known problem when you pipe a stream to a response and the client aborts request. Try this package: https://github.com/jshttp/on-finished

[petersirka]

@mscdex
here is a video with low quality:
https://www.dropbox.com/s/3avchxknfc2nrxj/nodejs-iojs-stream-problem.mov
Example: https://www.dropbox.com/s/fl3ic54zw3zdglb/iojs-stream-problem.zip

@vkurchatkin the problem still persists, but the simulation is harder. Here is a video:
https://www.dropbox.com/s/p0soq6289yn5o6h/nodejs-iojs-problem-destroy-onfinished.mov
Example: https://www.dropbox.com/s/5wga3la8j10zq7y/iojs-stream-problem-harder.zip

@vkurchatkin I tested this problem more. I destroyed the stream via stream.close() and stream.destroy(), but the file has been still opened.

Thanks!

[jonathanong]

please don't ask people to download zip files to test your code. just show us a gistfile.

the second example is incorrect.

var destroy = require('destroy');
var http = require('http');
var onFinished = require('on-finished');
var fs = require('fs');

http.createServer(function (req, res) {
    res.writeHead(200, { 'Content-Type': 'image/jpeg' });
    var stream = fs.createReadStream('bg.jpg');
    stream.pipe(res);
    onFinished(res, function(err) {
        res.on('finish', function() {
            destroy(stream);
        });
    });
}).listen(1337, '127.0.0.1');

console.log(process.argv[0] + ': ' + process.pid + '\nhttp://127.0.0.1:1337');

you're basically doing something like:

res.on('finish', =>
  res.on('finish', =>
    destroy(stream)
  )
)

which isn't going to work because finish is only going to be destroyed once. you just have to do this:

onFinished(res, function () {
  destroy(stream)
})

if the above doesn't work, please file an issue with on-finished and destroy.

[jonathanong]

see the bottom example: https://github.com/jshttp/on-finished#example

[petersirka]

@jonathanong ohh sorry, my mistake.
I'll test it today and I'll give you a feedback. Thanks!

[petersirka]

I have tested it and with on-finished it works correctly. But I wonder why the stream is not closed after the end of the reading (with autoClose: true)? So I detected: the stream must be manually closed after the pipe is closed.

I think that it's a bug. The test result with onFinished:

• streams opened: 311 504
• still opened: 7 files

Can you explain to me, why does it behaving like that?
Thanks.

[Trott]

This has been dormant for a long time. Does anyone know if it is a confirmed bug? If so, any idea if it's been fixed?

[jasnell]

Without further confirmation it's difficult to say. I'd recommend closing and we can reopen later if necessary.