Hot reload and debugger broken on dynamic pages in Next.js 14.2.0-canary.43 with --turbo (broken for all versions of --turbo) - vscode debugger

[avianion]

Link to the code that reproduces this issue

https://github.com/avianion/nextjs-turbo-broken-dynamic-page-hot-reload-and-vscode-debugger

To Reproduce

To Reproduce:

Set up a Next.js project with Turbopack (--turbo).
Create a dynamic page (e.g., pages/posts/[id].js).
Run the development server.
Make changes to the dynamic page.
Observe that the hot reload does not work, and the vscode debugger breaks.

Current vs. Expected behavior

Current vs. Expected behavior:
Current behavior: Hot reload does not work on dynamic pages, and the debugger breaks, overloading the jest-worker. Next.js searches for the dynamic page in the list of pages but cannot find it.
Expected behavior: Hot reload should work seamlessly on dynamic pages, allowing changes to be reflected immediately without reloading the entire page. The debugger should function correctly without breaking or overloading the jest-worker.

Provide environment information

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 10 Pro
Binaries:
  Node: 20.0.0
  npm: 9.6.4
  Yarn: 1.22.19
  pnpm: 7.29.1
Relevant packages:
  next: 14.2.0-canary.43
  react: 18.2.0
  react-dom: 18.2.0

Which area(s) are affected? (Select all that apply)

Turbopack (--turbo)

Which stage(s) are affected? (Select all that apply)

next dev (local)

Additional context

I am running Next.js in a Docker container on Windows 11.
The issue specifically occurs on dynamic pages (e.g., [id]).
I suspect this might be related to how Next.js handles dynamic pages in combination with Turborepo.

_PACK-2881

[wottpal]

Same issue here and assuming it's related to dynamic pages specifically. Everything works fine when omitting --turbo.

[avianion]

I can't believe this is not fixed yet. This is literally a development breaking bug for people who use dynamic page routes.

And I assume that's a lot of people.

I've been diligently following up and trying with each new canary version to no avail for the past few weeks.

I am wondering when all turbopack tests are done will it still work or will there be issues outstanding?

[timneutkens]

Hey, thank you for trying out Turbopack and sharing your experience!

Sorry we haven't been able to get back to your particular issue yet. It landed on the backlog as we weren't able to reproduce the specific issue of hot reloading not working, only that vscode debugger does not work which was not release candidate blocking (bug that could be fixed before the stable release).

I've opened a PR on your reproduction to have some code that might reproduce it based on the steps provided: PR.
The provided repository only holds the reproduction template without changes, maybe you made the changes but didn't push them to the repository?

Could you share more details about what you're seeing exactly? From reading the reproduction steps it sounds like you're trying to report 2 separate issues:

• HMR does not work on dynamic routes
• VS Code debugger does attach to code

However from running the reproduction steps I was only able to confirm one issue:

• VS Code debugger does not attach to code (both on dynamic routes as well as normal routes)

We tried running the reproduction both on macOS and Windows and HMR/Fast Refresh are being applied correctly.

Could you please share additional details so that we can investigate it further? Then we can implement a fix for your particular case.
We're going to have a look at the vscode debugger issue regardless 👍

Thanks in advance!

[timneutkens]

Just did some investigating with @sokra and found that the reason it doesn't apply is that VS Code's debugger has built-in handling for webpack specific sourcemap mapping, i.e. webpack:// and such. In order to support Turbopack out of the box we have to send a PR to the VS Code extension (going to do that now). On the Turbopack side of things we're renaming one of the paths so that it becomes turbopack://: vercel/turborepo#8034

In the meantime, while we work on landing the change in VS Code, you can use a launch.json config that has Turbopack specific sourceMapPathOverrides.

E.g. I was able to attach the debugger using this config:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Launch",
      "type": "node",
      "request": "launch",
      "cwd": "${workspaceFolder}",
      "runtimeExecutable": "npm",
      "runtimeArgs": ["run", "dev"],
      "skipFiles": ["<node_internals>/**"],
      "sourceMapPathOverrides": {
        "webpack:///./~/*": "${workspaceFolder}/node_modules/*",
        "webpack://?:*/*": "${workspaceFolder}/*",
        // Temporary Turbopack path
        "/turbopack/[project]/*": "${workspaceFolder}/*",
        // Eventual Turbopack path
        "turbopack://[project]/*": "${workspaceFolder}/*"
      }
    }
  ]
}

[timneutkens]

As discussed above I've opened a PR on VS Code's JavaScript debugger extension: https://vercel.slack.com/archives/C06BW4K7XPT/p1713960193753119

[avianion]

Hi @timneutkens

This launch config worked fine for me for the VScode debugger. Although your PR might make it better and easier for others.

  "configurations": [
    {
      "name": "Docker Nextjs Debug",
      "type": "node",
      "request": "attach",
      "address": "chat.lvh.me",
      "localRoot": "${workspaceFolder}",
      "remoteRoot": "/app",
      "sourceMaps": true,
      "cwd": "${workspaceFolder}",
      "port": 9230,
      "sourceMapPathOverrides": {
        "/turbopack/[project]/*": "${webRoot}/nextjs/*"
      }
    },

The debugger attaches perfectly fine. However, once attaching to one of the breakpoints, and being WITHIN a dynamic route, this error crops up repeatedly.

Exception has occurred: PageNotFoundError: Cannot find module for page: /chat/mQyUtnWCiXkS1pxm23Spp

This only happens on dynamic page routes, not regular page routes.

The source of the error is within this code

    if (!pagePath && pagesDir) {
        pagePath = await (0, _findpagefile.findPageFile)(pagesDir, normalizedPagePath, extensions, false);
    }
    if (pagePath !== null && pagesDir) {
        const pageUrl = (0, _ensureleadingslash.ensureLeadingSlash)((0, _removepagepathtail.removePagePathTail)((0, _normalizepathsep.normalizePathSep)(pagePath), {
            extensions
        }));
        return {
            filename: (0, _path.join)(pagesDir, pagePath),
            bundlePath: _path.posix.join("pages", (0, _normalizepagepath.normalizePagePath)(pageUrl)),
            page: _path.posix.normalize(pageUrl)
        };
    }
    if (page === "/_error") {
        return {
            filename: require.resolve("next/dist/pages/_error"),
            bundlePath: page,
            page: (0, _normalizepathsep.normalizePathSep)(page)
        };
    } else {
        throw new _utils1.PageNotFoundError(normalizedPagePath);

within next/dist/server/dev/on-demand-entry-handler.js

Here's the stack trace

 at findPagePathData (/app/nextjs/node_modules/next/dist/server/dev/on-demand-entry-handler.js:328:15)
    at async Object.ensurePage (/app/nextjs/node_modules/next/dist/server/dev/hot-reloader-turbopack.js:600:42)
    at async ensure (/app/nextjs/node_modules/next/dist/server/lib/router-utils/setup-dev-bundler.js:162:13)
    at async Object.getItem (/app/nextjs/node_modules/next/dist/server/lib/router-utils/filesystem.js:498:45)
    at async handleRoute (/app/nextjs/node_modules/next/dist/server/lib/router-utils/resolve-routes.js:266:36)
    at async resolveRoutes (/app/nextjs/node_modules/next/dist/server/lib/router-utils/resolve-routes.js:525:28)
    at async handleRequest (/app/nextjs/node_modules/next/dist/server/lib/router-server.js:211:96)
    at async requestHandlerImpl (/app/nextjs/node_modules/next/dist/server/lib/router-server.js:377:13)
    at async Server.requestListener (/app/nextjs/node_modules/next/dist/server/lib/start-server.js:141:13)

As you can see, it seems to be an issue with the turbopack hot reloader colliding with the debugger itself, and not being able to find the dynamic route.

I believe that this issue only occurs when the process is being debugged, however I could be wrong on this and it could be occuring anyway every time the page is hot reloaded.

Here is how the dynamic route is set up

import React from "react";
import { notFound } from "next/navigation";
import { FindAllChats } from "@/features/chat/chat-services/chat-service";
import { FindChatThreadByID } from "@/features/chat/chat-services/chat-thread-service";
import { ChatUI } from "@/features/chat/chat-ui/chat-ui";

export default async function Home({ params }: { params: { id: string } }) {
  const items = await FindAllChats(params.id);
  const threads = await FindChatThreadByID(params.id);

  if (threads.length === 0) {
    notFound();
  }

  const chatThread = threads[0];

  if (params.id !== chatThread.id) {
    return notFound();
  }

  return <ChatUI messages={items} chatThread={chatThread} />;
}

and the path is done like this nextjs/app/chat/[id]/page.tsx

The error basically makes us unable to use the vscode debugger in development mode with turbopack, specifically with dynamic routes.

[timneutkens]

Thanks for the additional details, that clarifies a lot more. It seems you are using Docker but the dockerfile is not provided. Can you share the setup as I still can't be reproduced with the provided steps unfortunately.

Could you also clarify if you're actively on a breakpoint when you make changes to the file and then save or if it's something else?

[avianion]

Thanks for your response @timneutkens . I am not saving the file at all. I am literally moving through the breakpoints as normal with the Vscode debugger.

Sometimes the error will appear before even hitting a breakpoint and sometimes after. The error will continuously occur then in a kind of loop until it triggers an issue with the jest worker, which basically nukes the Nextjs server and requires a restart. So basically, one error causes another and then when visiting any page, the app does not work on a development environment

Server Error

Error: Jest worker encountered 2 child process exceptions, exceeding retry limit
This error happened while generating the page. Any console logs will be displayed in the terminal window.
Call Stack
Next.js
ChildProcessWorker.initialize
file:/app/nextjs/node_modules/next/dist/compiled/jest-worker/index.js (1:11580)
ChildProcessWorker._onExit
file:/app/nextjs/node_modules/next/dist/compiled/jest-worker/index.js (1:12545)
ChildProcess.emit
node:events (518:28)
ChildProcess.emit
node:domain (488:12)

Here's the dockerfile

FROM node:20-alpine AS base

# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat

# Set the working directory
WORKDIR /app/nextjs

# Rebuild the source code and install dependencies in a single stage
FROM base AS builder
WORKDIR /app/nextjs

# Copy package.json and package-lock.json
COPY package.json ./
COPY package-lock.json ./

# Install the dependencies
RUN npm install --force

# Copy the rest of the app files
COPY . .

# Dev image, copy all the files and run next
FROM base AS runner
WORKDIR /app/nextjs

# Copy node_modules from the builder stage
# COPY --from=builder /app/nextjs/node_modules ./node_modules
# Copy the rest of the app files from the builder stage
COPY --from=builder /app/nextjs/ .

# Set NODE_ENV to development
ENV NODE_ENV development

# Create user and group for security reasons
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
# Change ownership of the /app/nextjs directory to the nextjs user
# RUN chown -R nextjs:nodejs /app/nextjs

# USER nextjs

ENV PORT 3000
ENV HOSTNAME localhost


EXPOSE 3000

and the docker-compose.yaml

  nextjs:
    develop:
      watch:
        - action: sync
          path: ./nextjs
          target: /app/nextjs/

    build:
      context: ./nextjs
      dockerfile: ../compose/local/nextjs/Dockerfile
    image: avian_local_nextjs
    container_name: avian_local_nextjs
    env_file:
      - ./.envs/.local/.nextjs
    ports:
      - "3000:3000"
      - "9230:9230"
      - "9229:9229"
    # volumes:
    #   - ./nextjs/.next/:/app/nextjs/.next/:z
    command: npm run dev
    restart: always
    environment:
      - VIRTUAL_HOST=chat.lvh.me
      - VIRTUAL_PORT=3000