`fs/promises`'s `readFile` returns truncated content for files in `/proc/net/`

[notcake]

Version

v21.1.0

Platform

Linux ubuntu 6.2.0-35-generic #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Oct 6 10:23:26 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

fs/promises

What steps will reproduce the bug?

Compare the length of the content returned by fs/promises.readFile and fs.readFile when reading any file larger than 4 KiB in /proc/net. fs/promises only returns the first 4 KiB or so.

$ node
Welcome to Node.js v21.1.0.
Type ".help" for more information.
> require("fs/promises").readFile("/proc/net/unix", "utf-8").then(x => console.log(x.length))
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 31,
  [Symbol(trigger_async_id_symbol)]: 25
}
> 4059
require("fs").readFile("/proc/net/unix", "utf-8", (_, x) => console.log(x.length))
undefined
> 65432

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

fs/promises's readFile and fs.readFile return the same untruncated file contents.

What do you see instead?

fs/promises's readFile truncates procfs file contents to around 4 KiB.

Additional information

No response

[bnoordhuis]

Can confirm. The bug here is that the promise-based version allocates a 64k buffer and thinks it's done when it reads less than that. That optimization is valid for regular files but not for files under /proc. It should keep reading until EOF (zero read.)

[debadree25]

(rookie question) Is there a way to way to check whether the file we are reading is one of these special files? other than maybe checking the path?

[bnoordhuis]

Checking the path isn't enough because a procfs can be mounted anywhere; /proc is just the most common.

Checking fs.statfsSync(path).type === 0x9FA0 (PROC_SUPER_MAGIC) works but isn't race-free.

I'm not 100% sure if it's okay to assume that fs.fstatSync(fd).dev === 22 means the file is on procfs.

[bnoordhuis]

Forgot to mention, linux has a fstatfs(fd) syscall but that isn't exposed by libuv. Would IMO be okay to use in a linux-only code path though.

edit: having said that... it's easier all around to simply read until EOF. Zero-sized files are rare enough that they aren't worth optimizing for. You're making at least two system calls either way.

[benjamingr]

@nodejs/fs

[avivkeller]

Hi! It's been a few months since any activity on this issue. I just attempted to reproduce in v22.6.0, and I was unable to do:

require("fs/promises").readFile("/proc/net/unix", "utf-8").then(x => console.log(x.length))
require("fs").readFile("/proc/net/unix", "utf-8", (_, x) => console.log(x.length))

$ node index.js
75897
75897

For that reason, I'm optimistically closing this issue, but feel free to reopen.