## Description
Refactor the [Next.js
CLI](https://nextjs.org/docs/app/api-reference/next-cli) to use
[commander](https://github.com/tj/commander.js) instead of
[arg](https://github.com/vercel/arg).
## Why?
- Auto-generated, properly formatted help command + output. With `arg`,
much of the help commands were manually added via a single
`console.log`, causing deviations over time.
- Ergonomic, ease of adding new subcommands and rules
## Breaking Changes
- Update the experimental `next experimental-compile` and `next
experimental-generate` build commands in favor of `next build
--experimental-build-mode=compile/generate`
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
## Description
We need to manually update the `@capsizecss/metrics` every time it
updates so we get the latest fallback fonts.
## Changes
- https://github.com/vercel/next.js/issues/47115
Closes NEXT-2547
### Why?
While developing https://github.com/vercel/next.js/pull/57656, using
`webpack`(which runs `bundle5.js`) gives me the following error:
```
Error: Cannot find module 'next/dist/build/webpack/plugins/terser-webpack-plugin'
Require stack:
- .../node_modules/next/dist/compiled/webpack/bundle5.js
```
### What?
The path to
[terser-webpack-plugin](https://github.com/vercel/next.js/blob/canary/packages/next/src/build/webpack/plugins/terser-webpack-plugin/src/index.ts)
seems to be modified, but the `taskfile.js` for building `bundle5.js`
hasn't been updated since https://github.com/vercel/next.js/pull/29168.
### How?
- Added `src` after the `terser-webpack-plugin` path in `taskfile.js`
- Run `pnpm ncc-compiled` (not sure if this is the right command to
run), built `bundle5.js` only.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
### What
Merges the package into the internal overlay.
### Detailed Changes
* Move `@next/react-dev-overlay` into next package and rename the import
paths.
* Fix the `getErrorSource` symbol issue, use `Symbol.for()` instead of
`Symbol`
* Extra `getErrorSource` into single lib as it's being used in a lot of
places, this will keep the same bundle size
Closes PACK-2261
Closes NEXT-1977
---------
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
## History
Previously, we added support for `squoosh` because it was a wasm
implementation that "just worked" on all platforms when running `next
dev` for the first time. However, it was slow so we always recommended
manually installing `sharp` for production use cases running `next
build` and `next start`.
Now that [`sharp` supports
webassembly](https://sharp.pixelplumbing.com/install#webassembly), we no
longer need to maintain `squoosh`, so it can be removed. We also don't
need to make the user install sharp manually because it can be installed
under `optionalDependencies`. I left it optional in case there was some
platform that still needed to manually install the wasm variant with
`npm install --cpu=wasm32 sharp` such as codesandbox/stackblitz (I don't
believe sharp has any fallback built in yet).
Since we can guarantee `sharp`, we can also remove `get-orientation` dep
and upgrade `image-size` dep.
I also moved an [existing `sharp`
test](https://github.com/vercel/next.js/pull/56674) into its own fixture
since it was unrelated to image optimization.
## Related Issues
- Fixes https://github.com/vercel/next.js/issues/41417
- Closes https://github.com/vercel/next.js/pull/54670
- Related https://github.com/vercel/next.js/issues/54708
- Related https://github.com/vercel/next.js/issues/44804
- Related https://github.com/vercel/next.js/issues/48820
The PR continues from #58038 aiming to reduce the installation size
further, and **improve performance in doing so**.
Most of `micromatch`'s APIs are essentially wrappers/alias of
`picomatch`, supplemented with additional features facilitated by a
transitive dependency `braces`. However, Next.js doesn't use those
"extra features". Switching from `micromatch` to `picomatch` can avoid
introducing the transitive dependency `braces`.
The PR also improves some globs' performance by hoisting.
- `micromatch.makeRe` is just an alias of `picomatch.makeRe`
-
90dc0cd3e1/index.js (L387C30-L387C30)
- `micromatch.isMatch(str, patterns, options)` equals
`picomatch(patterns, options)(str)`
-
90dc0cd3e1/index.js (L123)
- `micromatch.matcher` is just an alias of `picomatch`
-
90dc0cd3e1/index.js (L104C1-L104C40)
---------
Co-authored-by: Sam Ko <sam@vercel.com>
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Re-add precompile `watchpack`
- Next.js has stopped bundling the `watchpack` since
https://github.com/vercel/next.js/pull/50792. Currently, `watchpack` is
a direct dependency of Next.js. The PR adds the pre-compile back.
Co-authored-by: Steven <steven@ceriously.com>
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Remove `find-cache-dir`
- The usage of `find-cache-dir` was removed **4 years ago**
(https://github.com/vercel/next.js/pull/7013) but the dist and the build
script were never removed.
- Remove `@segment/ajv-human-errors`
- The usage of `@segment/ajv-human-errors` was removed in
https://github.com/vercel/next.js/pull/56383 by me. Though the build
script was also removed in that PR, the dist never got removed.
## What
When users specify `"type": "module"` in Next.js app, especially with
`create-next-app`, `Image` component is not working. An error
`Unsupported Server Component type: {...}` is thrown.
## Why
`next/image` API is mixing with a client component as default export and
a named export as server component. But the entry file of the API is
still CJS file, which will import the module as the object. So you'll
get `{ default, unstable_getImageProps }` when you do `import Image from
'next/image'` instead of `Image` component itself, where the CJS module
load all the exports as an object. This is expected behavior for ESM but
breaks the usage.
It only errors when you're using js extensions, if you're using
typescript, it still works. If you're using turbopack, it works in dev
mode.
This is also because webpack can't analyze the exports from CJS module
of that `next/image` entry file. Usually we can assign the default
export to the module itself, then attach other named exports onto it, so
the default export equals the `module.exports` itself. But for
`next/image` since the default export is an client component, doing that
will error with React as you cannot modify the react client reference.
Turbopack doesn't use the same way to analyze the default export, so it
doesn't have this problem.
## How
We create few ESM version of entry files of nextjs APIs, then pick up
them to let app router for bundling, instead of using the `next/<api
name>.js` CJS files. Those ESM entries still point to the `next/dist/..`
CJS files. In this way webpack and directly gets the exports from the
`next/dist/...` files and be aware of the module exports. No more CJS
module wrapping the ESM module, the default and named exports can
preserve correctly.
Fixes#54777
Closes NEXT-1774
Closes NEXT-1879
Closes NEXT-1923
This ensures we don't spam build logs with dynamic usage errors or
similar unexpectedly as they can be caught by this worker process
listener but shouldn't be logged.
Closes NEXT-1763
This is a 6.6kB (gzipped) win for the Node.js worker size baseline. `jsonwebtoken` uses semver to ensure that the Node.js version is larger than `16.9.0` (which is always true with Next.js) so we can just alias it to be a noop function that returns `true`.
This implements support for properly tracing sourcemaps when presenting
error stacks to the user. It also adds code frames when possible.
Closes WEB-1764
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### What?
Note: This is not a breaking change, just removing some unused code.
### Why?
Since #56896 we don't need this, as Node.js 18+ has `fetch` exposed by default.
### How?
Depends on #56896, #56909
We already didn't load `fetch` if `globalThis` had it (ie. Node.js 18+ environments), and since we are dropping support for Node.js 16, these code paths should have no effect on runtime behavior.
### Story
Since we introduced `ImageResponse` into `next/server` export, there're a few libraries relying on `next/server` that accidentally ended up with bundling og image into the bundle. As og package is quite large that could easily raise the size concern for middleware, edge runtime routes.
### Struggles
We've done optimizations. The tree-shaking strategies are tricky, we tried modularize imports and also optimize cjs require/exports to make sure you're not including og package into bundle if it's not being used. However, it's still not 100% can handle all the bundle optimization cases, such as `import {..} from "next/server.js"` could also ended up with the cjs bundle that failed the tree-shaking.
### Move on
So we decide to move og `ImageResponse` into a separate export `next/og`
Closes NEXT-1660
This is the first in a series of PRs replacing our use of fs-extra with node's own `node:fs`.
Test Plan:
With clean and existing artifacts in each case, ran:
- `pnpm build`
- `pnpm dev`
- `./node_modules/.bin/taskr ncc`
Closes WEB-1717
The PR supersedes the https://github.com/vercel/next.js/pull/53150, which is way too outdated, has way too many conflicts, and also heavily relies on GitHub Copilot (which makes the progress slow and tedious).
The PR uses [`json-schema-to-zod`](https://github.com/StefanTerdell/json-schema-to-zod) (instead of the GitHub Copilot) to generate the zod schema, and manually replaces all generated `z.customRefine` with my hand-written zod schema.
TODO:
- [x] Convert schema
- [x] Reduce `z.any()` usage
- [x] Create human-readable errors from the `ZodError`
- [x] Update test cases to reflect the latest error message
-----
The benefit of using zod over ajv:
- Easier maintenance: zod schema is straightforward to compose.
- Better typescript support: config schema now strictly reflects the `NextConfig` type.
- Smaller installation size: by replacing `ajv` and `@segment/ajv-human-errors` w/ `zod`, I am able to reduce installation size by 114 KiB.
- Better Extension: the zod error message is easy to customize.
-----
In the previous PR https://github.com/vercel/next.js/pull/56083, @feedthejim replaces `zod` w/ `superstruct`. `superstruct` is lightweight and fast, which makes it perfect for creating simple schemas for RSC payload. But, this also means `superstruct` has its limitations compared to `zod`:
- `superstruct`'s syntax is different, and some utilities's usage is counter-intuitive:
- `z.array(z.string()).gt(1)` vs `s.size(s.array(s.string()), 1)`
- `z.numer().gt(1)` vs `s.size(s.number(), 1)`, `s.min(s.number(), 1)`
- `z.boolean().optional().nullable()` vs `s.nullable(s.optional(z.boolean()))`
- `superstruct` has weaker TypeScript support and worse DX compared to `zod` when composing huge schema:
- `zod.ZodType + z.object()` can provide a more detailed type mismatch message on which specific property is the culprit, while `Describe + s.object()` provides almost no information at all.
- `zod`'s schema is more powerful
- `z.function()` supports `z.args()` and `z.returns()`, while `superstruct` only has `s.func()`
- zod also has Promise type `z.promise()` and intersection type `z.and()`
- `superstruct`'s error is harder to parse compared to `zod`'s `ZodError`
So in the PR, I re-introduced `zod` for `next.config.js` validation.
Today when we hydrate an SSR'd RSC response on the client we encounter import chunks which initiate code loading for client components. However we only start fetching these chunks after hydration has begun which is necessarily after the initial chunks for the entrypoint have loaded.
React has upstream changes that need to land which will preinitialize the rendered chunks for all client components used during the SSR pass. This will cause a `<script async="" src... />` tag to be emitted in the head for each chunk we need to load during hydration which allows the browser to start fetching these resources even before the entrypoint has started to execute.
Additionally the implementation for webpack and turbopack is different enough that there will be a new `react-server-dom-turbopack` package in the React repo which should be used when using Turbopack with Next.
This PR also removes a number of patches to React src that proxy loading (`__next_chunk_load__`) and bundler requires (`__next_require__`) through the `globalThis` object. Now the react packages can be fully responsible for implementing chunk loading and all Next needs to do is supply the necessary information such as chunk prefix and crossOrigin attributes necessary for this loading. This information is produced as part of the client-manifest by either a Webpack plugin or Turbopack.
Additionally any modifications to the chunk filename that were previously done at runtime need to be made in the manifest itself now. This means we need to encode the deployment id for skew protection and encode the filename to make it match our static path matching (and resolutions on s3) when using `[` and `]` segment characters.
There are a few followup items to consider in later PRs
1. we currently bundle a node and edge version of react-server-dom-webpack/client. The node version has an implementation for busboy whereas the edge version does not. Next is currently configured to use busboy when handling a fetch action sent as multipart with a node runtime. Ideally we'd only bundle the one platform we are buliding for but some additional refactoring to support better forking is possibly required here
This PR also updates react from 09285d5a7 to d900fadbf.
### React upstream changes
- https://github.com/facebook/react/pull/27439
- https://github.com/facebook/react/pull/26763
- https://github.com/facebook/react/pull/27434
- https://github.com/facebook/react/pull/27433
- https://github.com/facebook/react/pull/27424
- https://github.com/facebook/react/pull/27428
- https://github.com/facebook/react/pull/27427
- https://github.com/facebook/react/pull/27315
- https://github.com/facebook/react/pull/27314
- https://github.com/facebook/react/pull/27400
- https://github.com/facebook/react/pull/27421
- https://github.com/facebook/react/pull/27419
- https://github.com/facebook/react/pull/27418
Noticed a couple of directories had unit tests in `dist` which are being
run and reported during the Turbopack runs. These are generally quite
small so I'm not expecting this to have much effect on the package size
or CI runs, just making sure the data is correct.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
This was causing issues due to needing to run the watcher in a detached process to avoid it clobbering the other build tasks. For now this disables the watcher so we still get the initial types which is generally all that's needed
When running the `dev` task, types aren't emitted so there's a good chance of running into a TypeError like:
```
Type error: Module '"next/dist/lib/metadata/types/metadata-interface.js"' has no exported member 'ResolvingMetadata'.
```
during a `next build` from within the monorepo.
This creates a task to run `types` from within taskr as part of the build step.
## What
This PR changes Next.js to bundle its vendored React libraries so that the App Router pages can use those built-in versions.
## Why
Next.js supports both Pages and App Router and we've gone through a lot of iteration to make sure that Next.js stays flexible wrt to the version of React used: in Pages, we want to use the React provided by the user and in the App Router, to be able to use it, we need to use the canary version of React, which we've built into Next.js for convenience.
The problem stems from the fact that you can't run two different instances of React (by design).
Previously we have a dual worker setup, where we would separate completely each Next.js versions (App and Pages) so that they would not overlap with each other, however this approach was not great performance and memory wise.
We've recently tried using an ESM loader and a single process, but this change would still opt you into the React canary version if you had an app page, which breaks some assumptions.
## How
A list of the changes in this PR:
### New versions of the Next.js runtime
Since we now compile a runtime per type of page (app/route/api/pages), in order to bundle the two versions of React that we vendored, we introduced a new type of bundle suffixed by `-experimental`. This bundle will have the bleeding edge React needed for Server Actions and Next.js will opt you in into that runtime automatically.
For internal contributors, it means that we now run a compiler for 10 subparts of Next.js:
- next_bundle_server
- next_bundle_pages_prod
- next_bundle_pages_turbo
- next_bundle_pages_dev
- next_bundle_app_turbo_experimental
- next_bundle_app_prod
- next_bundle_app_prod_experimental
- next_bundle_app_turbo
- next_bundle_app_dev_experimental
- next_bundle_app_dev
### Simplified require-hook
Since the versions of React are correctly re-routed at build time for app pages, we don't need the require hook anymore
### Turbopack changes
The bundling logic in Turbopack has been addressed to properly follow the new logic
### Changes to the shared contexts system
Some context files need to have a shared instance between the rendering runtime and the user code, like the one that powers the `next/image` component. In general, the aliasing setup takes care of that but we need the require hook for code that is not compiled to reroute to the correct runtime. This only happens for pages node_modules.
A new Turbopack resolving plugin has been added to handle that logic in Turbopack.
### Misc changes
- `runtime-config` (that powers `next/config`) has been converted to an `.external` file, as it should have been
- there are some rules that have been added to the aliases to support the usage of `react-dom/server` in a server-components. We can do that now since the runtime takes care of separating the versions of React.
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
### What?
I made `styled-jsx` configurable. It uses `swc_css` by default, and with
`useLigntningcss: true`, it uses it.
https://github.com/swc-project/plugins/pull/207
### Why?
### How?
Closes NEXT-
Fixes #
Closes WEB-1532
## What?
In Next, rendering a route involves 3 layers:
- the routing layer, which will direct the request to the correct route to render
- the rendering layer, which will take a route and render it appropriately
- the user layer, which contains the user code
In #51831, in order to optimise the boot time of Next.js, I introduced a change that allowed the routing layer to be bundled. In this PR, I'm doing the same for the rendering layer. This is building up on @wyattjoh's work that initially split the routing and the rendering layer into separate entry-points.
The benefits of having this approach is that this allows us to compartmentalise the different part of Next, optimise them individually and making sure that serving a request is as efficient as possible, e.g. rendering a `pages` route should not need code from the `app router` to be used.
There are now 4 different rendering runtimes, depending on the route type:
- app pages: for App Router pages
- app routes: for App Router route handlers
- pages: for legacy pages
- pages api: for legacy API routes
This change should be transparent to the end user, beside faster cold boots.
## Notable changes
Doing this change required a lot of changes for Next.js under the hood in order to make the different layers play well together.
### New conventions for externals/shared modules
The big issue of bundling the rendering runtimes is that the user code needs to be able to reference an instance of a module/value created in Next during the render. This is the case when the user wants to access the router context during SSR via `next/link` for example; when you call `useContext(value)` the value needs to be the exact same reference to one as the one created by `createContext` earlier.
Previously, we were handling this case by making all files from Next that were affected by this `externals`, meaning that we were marking them not to be bundled.
**Why not keep it this way?**
The goal of this PR as stated previously was to make the rendering process as efficient as possible, so I really wanted to avoid extraneous fs reads to unoptimised code.
In order to "fix" it, I introduced two new conventions to the codebase:
- all files that explicitly need to be shared between a rendering runtime and the user code must be suffixed by `.shared-runtime` and exposed via adding a reference in the relevant `externals` file. At compilation time, a reference to a file ending with this will get re-written to the appropriate runtime.
- all files that need to be truly externals need to be suffixed by `.external`. At compilation time, a reference to it will stay as-is. This special case is needed mostly only for the async local storages that need to be shared with all three layers of Next.
As a side effect, we should be bundling more of the Next code in the user bundles, so it should be slightly more efficient.
### App route handlers are compiled on their own layer
App route handlers should be compiled in their own layer, this allows us to separate more cleanly the compilation logic here (we don't need to run the RSC logic for example).
### New rendering bundles
We now generate a prod and a dev bundle for:
- the routing server
- the app/pages SSR rendering process
- the API routes process
The development bundle is needed because:
- there is code in Next that relies on NODE_ENV
- because we opt out of the logic referencing the correct rendering runtime in dev for a `shared-runtime` file. This is because we don't need to and that Turbopack does not support rewriting an external to something that looks like this `require('foo').bar.baz` yet. We will need to fix that when Turbopack build ships.
### New development pipeline
Bundling Next is now required when developing on the repo so I extended the taskfile setup to account for that. The webpack config for Next itself lives in `webpack.config.js` and contains the logic for all the new bundles generated.
### Misc changes
There are some misc reshuffling in the code to better use the tree shaking abilities that we can now use.
fixes NEXT-1573
Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
Reverts vercel/next.js#53578
This PR (#53578) will break client components test, revert it for now.
Can repro by adding `"use client"` to `app/page.tsx` in `test/production/jest/server-only.test.ts`
```
FAIL app/page.test.jsx
● Test suite failed to run·
Cannot find module 'private-next-rsc-mod-ref-proxy' from 'app/page.jsx'·
Require stack:
```
### 🧐 What's in there?
This is another attempt to allow testing server-only code with Jest.
### 🧪 How to test?
There's an integration tests which can be triggered with `pnpm testheadless server-only`
Here is a more comprehensive setup:
<details>
<summary><code>app/lib/index.ts</code></summary>
```ts
import 'server-only'
export function add(num1: number, num2: number) {
return num1 + num2
}
```
</details>
<details>
<summary><code>app/lib/index.test.ts</code></summary>
```ts
import { add } from '.'
it('adds two numbers', () => {
expect(add(1, 3)).toEqual(4)
})
```
</details>
<details>
<summary><code>app/client-component.tsx</code></summary>
```ts
'use client'
import { useState } from 'react'
export default function ClientComponent() {
const [text, setText] = useState('not clicked yet')
return <button onClick={() => setText('clicked!')}>{text}</button>
}
```
</details>
<details>
<summary><code>app/client-component.test.tsx</code></summary>
```ts
import { fireEvent, render, screen } from '@testing-library/react'
import ClientComponent from './client-component'
it('can be clicked', async () => {
render(<ClientComponent />)
const button = screen.getByRole('button')
expect(button).toHaveTextContent('not clicked yet')
await fireEvent.click(button)
expect(button).toHaveTextContent('clicked!')
})
```
</details>
<details>
<summary><code>app/server-component.tsx</code></summary>
```ts
import { add } from '@/lib'
export default function ServerComponent({ a, b }: { a: number; b: number }) {
return (
<code role="comment">
{a} + {b} = {add(a, b)}
</code>
)
}
```
</details>
<details>
<summary><code>app/server-component.test.tsx</code></summary>
```ts
import { render, screen } from '@testing-library/react'
import ServerComponent from './server-component'
it('renders', () => {
render(<ServerComponent a={2} b={3} />)
expect(screen.getByRole('comment')).toHaveTextContent('2 + 3 = 5')
})
```
</details>
<details>
<summary><code>app/page.tsx</code></summary>
```ts
import Link from 'next/link'
import ClientComponent from './client-component'
import ServerComponent from './server-component'
export default function Page() {
return (
<>
<h1>Hello World</h1>
<Link href="/dave">Dave?</Link>
<p>
<ClientComponent />
</p>
<p>
<ServerComponent a={5} b={2} />
</p>
</>
)
}
```
</details>
<details>
<summary><code>app/page.test.tsx</code></summary>
```ts
import { render, screen } from '@testing-library/react'
import Page from './page'
it('greets', () => {
render(<Page />)
expect(screen.getByRole('link')).toHaveTextContent('Dave?')
expect(screen.getByRole('heading')).toHaveTextContent('Hello World')
expect(screen.getByRole('button')).toHaveTextContent('not clicked yet')
expect(screen.getByRole('comment')).toHaveTextContent('5 + 2 = 7')
})
```
</details>
<details>
<summary><code>app/[blog]/page.tsx</code></summary>
```ts
import { Metadata } from 'next'
import Link from 'next/link'
type Props = {
params: { blog: string }
}
export async function generateMetadata({
params: { blog: title },
}: Props): Promise<Metadata> {
return { title, description: `A blog post about ${title}` }
}
export default function Page({ params }: Props) {
return (
<>
<div>
<Link href="/">Back</Link>
</div>
<h1>All about {params.blog}</h1>
</>
)
}
```
</details>
<details>
<summary><code>app/[blog]/page.test.tsx</code></summary>
```ts
import { render, screen } from '@testing-library/react'
import Page from './page'
it('has the appropriate title', () => {
const title = 'Jane'
render(<Page params={{ blog: title }} />)
expect(screen.getByRole('heading')).toHaveTextContent(`All about ${title}`)
expect(screen.getByRole('link')).toHaveTextContent('Back')
})
```
</details>
<details>
<summary><code>app/layout.tsx</code></summary>
```ts
export default function RootLayout({ children }) {
return (
<html lang="en">
<body>{children}</body>
</html>
)
}
```
</details>
<details>
<summary><code>jest.config.js</code></summary>
```ts
const nextJest = require('next/jest')
const createJestConfig = nextJest({ dir: './' })
module.exports = createJestConfig({
testEnvironment: 'jsdom',
setupFilesAfterEnv: ['<rootDir>/test-setup.ts'],
})
```
</details>
<details>
<summary><code>package.json</code></summary>
```ts
{
"name": "rsc-test",
"version": "0.0.0",
"private": true,
"scripts": {
"test": "jest"
},
"devDependencies": {
"@testing-library/jest-dom": "latest"
}
}
```
</details>
<details>
<summary><code>test-setup.ts</code></summary>
```ts
import '@testing-library/jest-dom'
```
</details>
The app should run and all test should pass.
### ❗ Notes to reviewers
#### The problem:
1. next/jest configures jest with a transformer ([jest-transformer](https://github.com/vercel/next.js/blob/canary/packages/next/src/build/swc/jest-transformer.ts)) to compile react code with next -swc
2. the transformers configures next -swc for a given environment: Server or Client, based on jest global environment
3. Based on the environment, next -swc checks for invalid usage of `import('server-only')` `“use client”`, `export const metadata` or `export async function generateMetadata`
4. Because the global test environment is either jsdom or node, the same test suite can not include both client and server components
#### Possible mitigations
*A. Using jest projects*
When configured with [multiple projects](https://jestjs.io/docs/next/configuration/#projects-arraystring--projectconfig), Jest can launch different runners with different environment. This would allow running server-only code in node and client-only code in jsdom.
However, it requires user to completely change their jest configuration. It would also require a different setup when scaffolding new app-directory project with create-next.
*B. Using doc blocks*
Jest allows changing the environment per test file [with docBlock](https://jestjs.io/docs/configuration#testenvironment-string).
However, by the time jest is invoking next -swc on a source file to transform it, this information is gone, and next -swc is still invoked with the (wrong) global environment.
The PR #52393 provides a workaround for files with `import('server-only')`, but does not allow testing pages with metadata.
*C. Always compile for node*
Our jest-transformer could always configure next -swc for server:
- pass Server-specific validations `import('server-only')` `export const metadata` or `export async function generateMetadata`
- does not complain about `"use client"`
This is what this PR is about!
Fixes#47299
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
Followup on the https://github.com/vercel/next.js/pull/52520. The enhancements:
1. Both `fetch` and `http(s)` APIs are intercepted in the testmode.
2. The loopback mode that would allow the test-side to use any fetch-mocking library of their choice.
## Vendoring
Updates all module resolvers (node, webpack, nft for entrypoints, and nft for next-server) to consider whether vendored packages are suitable for a given resolve request and resolves them in an import semantics preserving way.
### Problem
Prior to the proposed change, vendoring has been accomplished but aliasing module requests from one specifier to a different specifier. For instance if we are using the built-in react packages for a build/runtime we might replace `require('react')` with `require('next/dist/compiled/react')`.
However this aliasing introduces a subtle bug. The React package has an export map that considers the condition `react-server` and when you require/import `'react'` the conditions should be considered and the underlying implementation of react may differ from one environment to another. In particular if you are resolving with the `react-server` condition you will be resolving the `react.shared-subset.js` implementation of React. This aliasing however breaks these semantics because it turns a bare specifier resolution of `react` with path `'.'` into a resolution with bare specifier `next` with path `'/dist/compiled/react'`. Module resolvers consider export maps of the package being imported from but in the case of `next` there is no consideration for the condition `react-server` and this resolution ends up pulling in the `index.js` implementation inside the React package by doing a simple path resolution to that package folder.
To work around this bug there is a prevalence of encoding the "right" resolution into the import itself. We for instance directly alias `react` to `next/dist/compiled/react/react.shared-subset.js` in certain cases. Other times we directly specify the runtime variant for instance `react-server-dom-webpack/server.edge` rather than `react-server-dom-wegbpack/server`, bypassing the export map altogether by selecting the runtime specific variant. However some code is meant to run in more than one runtime, for instance anything that is part of the client bundle which executes on the server during SSR and in the browser. There are workaround like using `require` conditionally or `import(...)` dynamically but these all have consequences for bundling and treeshaking and they still require careful consideration of the environment you are running in and which variant needs to load.
The result is that there is a large amount of manual pinning of aliases and additional complexity in the code and an inability to trust the package to specify the right resolution potentially causing conflicts in future versions as packages are updated.
It should be noted that aliasing is not in and of itself problematic when we are trying to implement a sort of lightweight forking based on build or runtime conditions. We have good examples of this for instance with the `next/head` package which within App Router should export a noop function. The problem is when we are trying to vendor an entire package and have the package behave semantically the same as if you had installed it yourself via node_modules
### Solution
The fix is seemingly straight forward. We need to stop aliasing these module specifiers and instead customize the resolution process to resolve from a location that will contain the desired vendored packages. We can then start simplifying our imports to use top level package resources and generally and let import conditions control the process of providing the right variant in the right context.
It should be said that vendoring is conditional. Currently we only vendor react pacakges for App Router runtimes. The implementation needs to be able to conditionally determine where a package resolves based on whether we're in an App Router context vs a Page Router one.
Additionally the implementation needs to support alternate packages such as supporting the experimental channel for React when using features that require this version.
### Implementation
The first step is to put the vendored packages inside a node_modules folder. This is essential to the correct resolving of packages by most tools that implement module resolution. For packages that are meant to be vendored, meaning whole package substitution we move the from `next/(src|dist)/compiled/...` to `next/(src|dist)/vendored/node_modules`. The purpose of this move is to clarify that vendored packages operate with a different implementation. This initial PR moves the react dependencies for App Router and `client-only` and `server-only` packages into this folder. In the future we can decide which other precompiled dependencies are best implemented as vendored packages and move them over.
It should be noted that because of our use of `JestWorker` we can get warnings for duplicate package names so we modify the vendored pacakges for react adding either `-vendored` or `-experimental-vendored` depending on which release channel the package came from. While this will require us to alter the request string for a module specifier it will still be treating the react package as the bare specifier and thus use the export map as required.
#### module resolvers
The next thing we need to do is have all systems that do module resolution implement an custom module resolution step. There are five different resolvers that need to be considered
##### node runtime
Updated the require-hook to resolve from the vendored directory without rewriting the request string to alter which package is identified in the bare specifier. For react packages we only do this vendoring if the `process.env.__NEXT_PRIVATE_PREBUNDLED_REACT` envvar is set indicating the runtime is server App Router builds. If we need a single node runtime to be able to conditionally resolve to both vendored and non vendored versions we will need to combine this with aliasing and encode whether the request is for the vendored version in the request string. Our current architecture does not require this though so we will just rely on the envvar for now
##### webpack runtime
Removed all aliases configured for react packages. Rely on the node-runtime to properly alias external react dependencies. Add a resolver plugin `NextAppResolverPlugin` to preempt perform resolution from the context of the vendored directory when encountering a vendored eligible package.
##### turbopack runtime
updated the aliasing rules for react packages to resolve from the vendored directory when in an App Router context. This implementation is all essentially config b/c the capability of doing the resolve from any position (i.e. the vendored directory) already exists
##### nft entrypoints runtime
track chunks to trace for App Router separate from Pages Router. For the trace for App Router chunks use a custom resolve hook in nft which performs the resolution from the vendored directory when appropriate.
##### nft next-server runtime
The current implementation for next-server traces both node_modules and vendored version of packages so all versions are included. This is necessary because the next server can run in either context (App vs Page router) and may depend on any possible variants. We could in theory make two traces rather than a combined one but this will require additional downstream changes so for now it is the most conservative thing to do and is correct
Once we have the correct resolution semantics for all resolvers we can start to remove instances targeting our precompiled instances for instance making `import ... from "next/dist/compiled/react-server-dom-webpack/client"` and replacing with `import ... from "react-server-dom-webpack/client"`
We can also stop requiring runtime specific variants like `import ... from "react-server-dom-webpack/client.edge"` replacing it with the generic export `"react-server-dom-webpack/client"`
There are still two special case aliases related to react
1. In profiling mode (browser only) we rewrite `react-dom` to `react-dom/profiling` and `scheduler/tracing` to `scheduler/tracing-profiling`. This can be moved to using export maps and conditions once react publishses updates that implement this on the package side.
2. When resolving `react-dom` on the server we rewrite this to `react-dom/server-rendering-stub`. This is to avoid loading the entire react-dom client bundle on the server when most of it goes unused. In the next major react will update this top level export to only contain the parts that are usable in any runtime and this alias can be dropped entirely
There are two non-react packages currently be vendored that I have maintained but think we ought to discuss the validity of vendoring. The `client-only` and `server-only` packages are vendored so you can use them without having to remember to install them into your project. This is convenient but does perhaps become surprising if you don't realize what is happening. We should consider not doing this but we can make that decision in another discussion/PR.
#### Webpack Layers
One of the things our webpack config implements for App Router is layers which allow us to have separate instances of packages for the server components graph and the client (ssr) graph. The way we were managing layer selection was a but arbitrary so in addition to the other webpack changes the way you cause a file to always end up in a specific layer is to end it with `.serverlayer`, `.clientlayer` or `.sharedlayer`. These act as layer portals so something in the server layer can import `foo.clientlayer` and that module will in fact be bundled in the client layer.
#### Packaging Changes
Most package managers are fine with this resolution redirect however yarn berry (yarn 2+ with PnP) will not resolve packages that are not defined in a package.json as a dependency. This was not a problem with the prior strategy because it was never resolving these vendored packages it was always resolving the next package and then just pointing to a file within it that happened to be from react or a related package.
To get around this issue vendored packages are both committed in src and packed as a tgz file. Then in the next package.json we define these vendored packages as `optionalDependency` pointing to these tarballs. For yarn PnP these packed versions will get used and resolved rather than the locally commited src files. For other package managers the optional dependencies may or may not get installed but the resolution will still resolve to the checked in src files. This isn't a particularly satisfying implemenation and if pnpm were to be updated to have consistent behavior installing from tarballs we could actually move the vendoring entirely to dependencies and simplify our resolvers a fair bit. But this will require an upstream chagne in pnpm and would take time to propogate in the community since many use older versions
#### Upstream Changes
As part of this work I landed some other changes upstream that were necessary. One was to make our packing use `npm` to match our publishing step. This also allows us to pack `node_modules` folders which is normally not supported but is possible if you define the folder in the package.json's files property.
See: #52563
Additionally nft did not provide a way to use the internal resolver if you were going to use the resolve hook so that is now exposed
See: https://github.com/vercel/nft/pull/354
#### additional PR trivia
* When we prepare to make an isolated next install for integration tests we exclude node_modules by default so we have a special case to allow `/vendored/node_modules`
* The webpack module rules were refactored to be a little easier to reason about and while they do work as is it would be better for some of them to be wrapped in a `oneOf` rule however there is a bug in our css loader implementation that causes these oneOf rules to get deleted. We should fix this up in a followup to make the rules a little more robuts.
## Edits
* I removed `.sharedlayer` since this concept is leaky (not really related to the client/server boundary split) and it is getting refactored anyway soon into a precompiled runtime.
### 🧐 What's in there?
At the moment, it is not possible to test code with `import 'server-only'` in app directory.
When trying to load such file in jest (even with `testEnvironment: node`), the error will be:
```
● Test suite failed to run··
x NEXT_RSC_ERR_CLIENT_IMPORT: server-only
,-[lib/util.js:1:1]
1 | /** @jest-environment node */·
2 | import 'server-only'
: ^^^^^^^^^^^^^^^^^^^^
3 | export const PI = 3.14;
`----·
at Object.transformSync (node_modules/next/src/build/swc/index.ts:443:25)
at transformSync (node_modules/next/src/build/swc/index.ts:629:19)
at Object.process (node_modules/next/src/build/swc/jest-transformer.ts:117:25)
at ScriptTransformer.transformSource (node_modules/@jest/transform/build/ScriptTransformer.js:619:31)
at ScriptTransformer._transformAndBuildScript (node_modules/@jest/transform/build/ScriptTransformer.js:765:40)
at ScriptTransformer.transform (node_modules/@jest/transform/build/ScriptTransformer.js:822:19)·
```
In a nutshell:
- next/swc is looking for ‘server-only’ [text in the source](https://github.com/vercel/next.js/blob/canary/packages/next-swc/crates/core/src/react_server_components.rs#L576), and throw if not configured for server
- next's jest-transformer will only configure next/swc for server [if the environment is node](https://github.com/vercel/next.js/blob/canary/packages/next/src/build/swc/jest-transformer.ts#L88)
- when testing Next.js apps, your jest testEnvironment is most likely jsdom. But you can configure it [per file with docBlock](https://jestjs.io/docs/configuration#testenvironment-string), which jest-transformer ignores because it only reads the top-level configuration.
This PR fixes this, by
1. reading the docblock to configure next/swc accordingly and bypass its hardcoded guard
2. mocking `server-only` so [it does not throw](https://github.com/vercel/next.js/blob/canary/packages/next/src/compiled/server-only/index.js) when loaded (jest does not read `react-server` export from package.json)
Users would still have to annotate their `server-only` files with `/** @jest-environment node */` in order to test them.
### 🧪 How to test?
There's a full test available: `pnpm testheadless --testPathPattern jest/server-only`
Here is also a minimal reproduction:
<details>
<summary>app/layout.tsx</summary>
```typescript
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (<html lang="en"><body>{children}</body></html>)
}
```
</details>
<details>
<summary>app/page.tsx</summary>
```typescript
import { PI } from '@/lib/utils'
export default function Home() {
return <h1>{PI}</h1>
}
```
</details>
<details>
<summary>lib/utils.ts</summary>
```typescript
import 'server-only'
export const PI = 3.14
```
</details>
<details>
<summary>lib/utils.test.ts</summary>
```typescript
import { PI } from './utils'
it('works', () => expect(PI).toEqual(3.14))
```
</details>
<details>
<summary>jest.config.js</summary>
```typescript
const nextJest = require('next/jest')
module.exports = nextJest({ dir: './' })({ testEnvironment: 'jsdom' })
```
</details>
### ❗ Notes to reviewers
[jest-docblock](https://packagephobia.com/result?p=jest-docblock) with dependencies is only 12.5 kB.
Fixes#47448
This PR:
- adds a minified bundled server for Next with some optimisations applied
- a test server in minimal-server.js
- misc changes:
- makes some polyfills lazy
- adds a cached version of node-html-parser
-
`reactProductionProfiling` was a next config working for pages before mainly for profiling react, this PR also enables it for pages.
### Chanegs
* Add react profiling entry and related alias
* Fix `reactProductionProfiling` is missing in next config type and schema
Closes#51131
## What?
While investigating slow compilation for a page on vercel.com in
development I found that there was close to 10 seconds of time
unaccounted for in `.next/trace`. Ran a profile and found that time was
spent in watchpack `batch`, specifically calling `close` many times.
When I tried to debug this further by running unbundled webpack I
noticed the same issue didn't exists.
### Before
<img width="1329" alt="Before"
src="https://github.com/vercel/next.js/assets/6324199/9ace4628-db04-4de7-993f-65aef9dffc55">
### After
<img width="1278" alt="After"
src="https://github.com/vercel/next.js/assets/6324199/55d5e58b-4a27-4235-8dea-723a7a78c117">
## Raw numbers
<table>
<tr>
<td>Before</td>
<td>After</td>
<td>Delta</td>
<td>Delta (percent)</td>
</tr>
<tr>
<td>13840 ms</td>
<td>3580 ms</td>
<td>-10260 ms</td>
<td>-74.13%</td>
</tr>
</table>
## How?
Investigated further and found that specifically not minifying watchpack
solved the issue.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Bumping `@edge-runtime/*` <picture data-single-emoji=":edge-runtime:" title=":edge-runtime:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/edge-runtime/b940e917443aa49f.png" alt=":edge-runtime:" align="absmiddle"></picture> packages to their latest beta, as we had some huge improvements and big changes on how it works internally.
* Using the latest `undici` which provides a WebSocket implementation
* Less polyfill usage as we load modules in the Node.js realm, and can reuse modules from Node.js as time goes on <picture data-single-emoji=":just-right2:" title=":just-right2:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/just-right2/588cf34d02f4b3bd.png" alt=":just-right2:" align="absmiddle"></picture>
* `instanceof` checks now work within the realm <picture data-single-emoji=":mind_blown:" title=":mind_blown:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/mind_blown/0186b6f181040126.gif" alt=":mind_blown:" align="absmiddle"></picture>
This PR does three things:
- Vendors the package `react-server-dom-webpack@experimental` as
`react-server-dom-webpack-experimental` similar to React and React DOM
- Upgrades all vendored React packages
- Re-lands the `experimentalReact` flag to switch between `@next` and
`@experimental` channels of React for app dir.
Fix NEXT-926.
