investigate flaky parallel/test-http-writable-true-after-close on macOS

[Trott]

• Version: v9.0.0-pre
• Platform: macOS
• Subsystem: test

https://ci.nodejs.org/job/node-test-commit-osx/12717/nodes=osx1010/console

not ok 757 parallel/test-http-writable-true-after-close
  ---
  duration_ms: 0.195
  severity: fail
  stack: |-
    Mismatched <anonymous> function calls. Expected exactly 1, actual 0.
        at Object.exports.mustCall (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/common/index.js:485:10)
        at ClientRequest.common.mustCall (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/parallel/test-http-writable-true-after-close.js:16:29)
        at ClientRequest.<anonymous> (/Users/iojs/build/workspace/node-test-commit-osx/nodes/osx1010/test/common/index.js:517:15)
        at Object.onceWrapper (events.js:316:30)
        at emitOne (events.js:115:13)
        at ClientRequest.emit (events.js:210:7)
        at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:548:21)
        at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
        at Socket.socketOnData (_http_client.js:437:20)

This test was only recently adjusted for CentOS 5. See #15505 and #15520

Ping @mcollina

[mcollina]

I propose we remove that test. We would be a little less covered, but maintaining it is probably not worth it.

[Trott]

@mcollina It looks like there's support for removing CentOS 5 from the test cluster for 8.x.x and newer (which presumably would also mean master). See nodejs/build#898

If that happens, might it make sense to put the test back to the simpler way it was initially and see if that's robust/reliable?

[Trott]

(Another option: Mark the test as flaky everywhere. If it turns a build yellow, we'll still notice it, and I and some other folks do review the tests-marked-as-flaky from time to time.)

[Trott]

I'm able to reproduce the flakiness locally on my macOS laptop, but it takes a lot of runs and high concurrency:

$ tools/test.py -j 96 --repeat 1920 test/parallel/test-http-writable-true-after-close.js 
=== release test-http-writable-true-after-close ===                     
Path: parallel/test-http-writable-true-after-close
Mismatched <anonymous> function calls. Expected exactly 1, actual 0.
    at Object.exports.mustCall (/Users/trott/io.js/test/common/index.js:485:10)
    at ClientRequest.common.mustCall (/Users/trott/io.js/test/parallel/test-http-writable-true-after-close.js:16:29)
    at ClientRequest.<anonymous> (/Users/trott/io.js/test/common/index.js:517:15)
    at Object.onceWrapper (events.js:316:30)
    at emitOne (events.js:115:13)
    at ClientRequest.emit (events.js:210:7)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:548:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
    at Socket.socketOnData (_http_client.js:437:20)
Command: out/Release/node /Users/trott/io.js/test/parallel/test-http-writable-true-after-close.js
[01:32|% 100|+ 1919|-   1]: Done                                        
$ 

Would be curious if Linux laptops and other operating systems show the same issue. This issue might be a simpler race condition or something than what we were seeing with CentOS 5 (which was a reliable failure and not intermittent flakiness like this). If so, moving the test to sequential seems a likely fix.

[Trott]

Update: The idea of reverting to the previous version of the test doesn't appear to be the way to go. That test is much more flaky under high concurrency (in local testing) than the current version.

tools/test.py -j 96 --repeat 9999 test/parallel/test-http-writable-true-after-close.js had 15 failures in the previous version, while the same command had only 5 failures with the current version.

Running the test 9999 times with -j 1 had no failures locally for me.

Therefore, I think moving the test to sequential is the way to go. I'll open a pull request for that now.

[Trott]

#15705