This PR adds some improved `modularizeImports` rules to ensure that some
of the popular UI libraries (mostly icons) can be correctly optimized
instead of creating thousands of modules. Here's an example of applying
this to `lucide-react`:
With https://github.com/swc-project/plugins/pull/196 being landed, we
can add different transform rules for specific names in the import. For
example, for `lucide-react` there're now 3 transforms:
- `'Lucide(.*)': ...`
- `'(.*)Icon': ...`
- `'*': ...`
On top of that, another critical change made in this PR is the
introducing of `modularize-import-loader`. With this new loader, we can
delegate the task of re-exporting the value to Webpack, where we can
adjust the type of the export and the target path. This is very
necessary and can't be done in SWC alone because libs like
`lucide-react` is using `.mjs` and don't have `exports` values for the
individual icons in the package.json files.
Because of that, if we simply transform the expression to `import Icon
from 'lucide-react/esm/icons/foo` it will fail because:
1. It's missing a `.mjs` extension
2. It doesn't have `/esm/icons/foo.mjs` in package.json `exports`
For 1), in fact that they moved to `.mjs` only in a recent upgrade so we
can't even hard code it to be `.mjs` in the transform (breaks old
versions).
Because of all the above, I decided to go with Webpack's own resolution
logic with the loader as a _temporary_ solution. It's temporary because
it's still assuming that the file structure of these libs doesn't
change.
This uses an IPC server (if available) for incremental cache methods to help prevent race conditions when reading/writing from cache and also to dedupe requests in cases where multiple cache reads are in flight. This cuts down on data fetching across the different build-time workers.
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
### What?
This reimplements our stream cancellation code for a few more cases:
1. Adds support in all stream-returning APIs
2. Fixes cancellation detection in node 16
3. Implements out-of-band detection, so can cancel in the middle of a
read
It also (finally) adds tests for all the cases I'm aware of.
### Why?
To allow disconnecting from an AI service when a client disconnects. $$$
### How?
1. Reuses a single pipe function in all paths to push data from the
dev's `ReadableStream` into our `ServerResponse`
2. Uses `ServerResponse` to detect disconnect, instead of the
`IncomingMessage` (request)
- The `close` event fire once all incoming body data is read
- The request `abort` event will not fire after the incoming body data
has been fully read
3. Using `on('close')` on the writable destination allows us to detect
close
- Checking for `res.destroyed` in the body of the loop meant we had to
wait for the `await stream.read()` to complete before we could possibly
cancel the stream
- - -
#52157 (and #51594) had an issue with Node 16, because I was using
`res.closed` to detect when the server response was closed by the client
disconnecting. But, `closed` wasn't
[added](https://github.com/nodejs/node/pull/45672) until
[v18.13.0](https://nodejs.org/en/blog/release/v18.13.0#:~:text=%5Bcbd710bbf4%5D%20%2D%20http%3A%20make%20OutgoingMessage%20more%20streamlike%20(Robert%20Nagy)%20%2345672).
This fixes it by using `res.destroyed`.
Reverts #52277
Relands #52157Fixes#52809
---------
Follows #53115
- Replace `chalk` with `picocolors`
- Note that `chalk.hex('#007acc')` has been replaced with `picocolors.blue`
- Replace `glob` with `fast-glob@2.2.7`
- Not only does `fast-glob` is a faster drop-in replacement of `glob` with first-party `Promise`-based API support, but also `fast-glob` is already a dependency of `cpy`:
<img width="812" alt="image" src="https://github.com/vercel/next.js/assets/40715044/8efa24c4-5312-4b1c-bf8d-68255ca30b60">
Together the PR removes about `50 KiB` from the `create-next-app/dist/index.js`:
<img width="570" alt="image" src="https://github.com/vercel/next.js/assets/40715044/db2f3723-14cc-48ce-9cb2-8aa1fb1d5e95">
Co-authored-by: Steven <229881+styfle@users.noreply.github.com>
Fixes#52548
This PR fixes the issue of the `<Image />` "blur" placeholder where the
placeholder image appears "washed out" and feathered around the edges.
The fix does NOT involve any API changes - only the underlying SVG
filter is updated.
The filter works for any image type (JPEG, GIF, WEBP) no matter if it
contains transparency or not.
## How it works
```html
<filter id='b' color-interpolation-filters='sRGB'>
<feMorphology in='SourceAlpha' operator='dilate' radius='15' result='dilate' />
<feGaussianBlur in='dilate' stdDeviation='15' result='mask' />
<feGaussianBlur in='SourceGraphic' stdDeviation='20' result='blur' />
<feComponentTransfer in='blur' result='solid'>
<feFuncA type='table' tableValues='1 1' />
</feComponentTransfer>
<feComposite in2='mask' in='solid' operator='in' result="comp" />
<feMerge>
<feMergeNode in="SourceGraphic" />
<feMergeNode in="comp" />
</feMerge>
</filter>
```
Currently the underlying filter uses `feComponentTransfer ` to get rid of the feathered edges but only for JPEG images. This is because if alpha channel was to be mapped to 1 for images with transparency in it, all of the transparent pixels would turn black (which is obviously undesirable).
The way I fixed it is by creating a mask from the SourceAlpha (effectively the "shape" of the object in an image), slightly blurring it for a smoother look (feMorphology + feGaussianBlur), and then clipping the `feComponentTransfer` result to the shape of the mask (`<feComposite in2='mask' in='solid' operator='in' result="comp" />`).
Then finally `feMerge` is used to stack original image and the clipping result from the previous step just to make sure that any left over artifacts from clipping are hidden.
## Result
Here's a comparison between the current implementation (left column), and the filter above (right column)
<img width="614" alt="Screenshot 2023-07-12 at 10 44 56" src="https://github.com/vercel/next.js/assets/28541613/afdc1f88-eb1a-4a21-a88a-06057a875e1b">
---------
Co-authored-by: Steven <steven@ceriously.com>
### What?
Allow the following:
```ts
new NextRequest(new Request(...))
```
### Why?
Cloning a request by passing it to the constructor of another `Request` is allowed by the spec: https://developer.mozilla.org/en-US/docs/Web/API/Request/Request#parameters
### How?
If the passed argument is an instance of `Request`, we pass it as-is to `super()`
Fixes#52967
Closes NEXT-1468
Previously, this warning message assumed the user knew what `yarn` was and had it installed.
This PR changes the warning message to assume the user knows what `npm` is and has it installed, since `npm` ships with the official `node` installation.
### Why
In #52985 the not found solution introduces `NODE_ENV` to determine if it his the not found boundary and should render the not found, as in the next build mode, we have `/_not-found` as a special route which has a empty parallel route, but in next dev mode so far it his the `parallel-default-route`. This could dependend on the `NODE_ENV` passing to next server but not necessarily.
### What
Fixes#53082Fixes#53083
### How
When server actions `not-found` hits, now we create a new loader tree based on the previous one, including `layout` and other components but not the children parallel routes
For production case, to make the rendering independent from the `NODE_ENV`, we're using original pathname to check if it's `/_not-found` to determine if it's production build 404 page
To support replace the loader tree of action, did a little refactor that passing down the loader tree from top level to `bodyResult`. Then we can change the loader tree itself before rendering, in short, we tweak it from original tree to one for not-found case, so server actions could render it properly
### Why
We calculate the “next url” depending on the router state and the previous router state so that when you navigate to a route, the proxy matches with that header and returns you the intercepted route if matching
### What
- Fixes#52745
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
This breaks out routing handling from `next-server`, `next-dev-server`,
and `base-server` so that these are only handling the "render" work and
eventually these will be deleted completely in favor of the bundling
work being done.
The `router` process and separate `render` processes are still
maintained here although will be investigated further in follow-up to
see if we can reduce the need for these.
We are also changing the `require-cache` IPC to a single call instead of
call per entry to reduce overhead and also de-dupes handling for
starting the server between the standalone server and normal server.
To maintain support for existing turbopack route resolving this
implements the new route resolving in place of the existing
`route-resolver` until the new nextturbo API is fully landed.
After these initial changes we should continue to eliminate non-render
related code from `next-server`, `base-server`, and `next-dev-server`.
This PR tries to address some feedback around prefetching, like in #49607, where they were some warnings because we were over prefetching.
The tweaks in this PR:
- if there are no loading boundary below, we don't prefetch the full page anymore. I made that change a while ago but I think it wasn't the original intent from @sebmarkbage. Fixing that now. So now, we will prefetch the page content until the nearest loading boundary, only if there is any.
- there's now a queue for limiting the number of concurrent prefetches. This is to not ruin the bandwidth for complex pages. Thanks @alvarlagerlof for that one.
- also, if the prefetch is in the queue when navigating, it will get bumped.
- bonus: fixes a bug where we were wrongly stripping headers in dev for static pages
Test plan:
<img width="976" alt="CleanShot 2023-07-20 at 17 53 43@2x" src="https://github.com/vercel/next.js/assets/11064311/2ea34419-c002-4aea-94df-57576e3ecb2e">
In the screenshot you can see that:
- only 5 requests at a time are authorised
- when I clicked on 15, it got prioritised in the queue (did not cancel the rest)
- the prefetch only included the content until the loading boundary
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
### What
This PR changes the flow of not-found rendering process.
### Why
`not-found.js` was rendered in two ways before:
* 1 is SSR rendering the not-found as 404
* 2 is triggering the error on RSC rendering then the error will be
preserved in inline flight data, on the client it will recover the error
and trigger the proper error boundary.
The solution has been through a jounery:
No top-level not found boundary -> introduce metadata API -> then we
create a top level root not found boundary -> then we delete it due to
duplicated rendering of root layout -> now this
So the solution before this PR is still having a root not found boundary
wrapped in the `AppRouter`, it's being used in a lot of places including
HMR. As we discovered it's doing duplicated rendering of root layout,
then we removed it and it started failing with rendering `not-found` but
missing root layout. In this PR we redesign the process.
### How
Now the rendering architecture looks like:
* For normal root not-found and certain level of not-found boundary
they're still covered by `LayoutRouter`
* For other error renderings including not-found
* Fully remove the top level not-found boundary, when it renders with
404 error it goes to render the fallback page
* During rendering the fallback page it will check if it should just
renders a 404 error page or render nothing and let the error from inline
flight data to trigger the error boundary
pseudo code
```
try {
render AppRouter > PageComponent
} catch (err) {
create ErrorComponent by determine err
render AppRouter > ErrorComponent
}
```
In this way if the error is thrown from top-level like the page itself
or even from metadata, we can still catch them and render the proper
error page based on the error type.
The problematic is the HMR: introduces a new development mode meta tag
`<meta name="next-error">` to indicate it's 404 so that we don't do
refresh. This reverts the change brougt in #51637 as it will also has
the duplicated rendering problem for root layout if it's included in the
top level not found boundary.
Also fixes the root layout missing issue:
Fixes#52718Fixes#52739
---------
Co-authored-by: Shu Ding <g@shud.in>
We have the logic to group the client compiler's entry names to make sure we generate one single manifest file for the page. This is complicated and requires a special step to "group" the entry names because a page can depend on a bunch of files from everywhere.
And currently, the normalization of "entryName → groupName" doesn't cover interception routes' conventions (`(.)`, `(..)` and `(...)`). This PR fixes that.
Closes#52862, closes#52681, closes#52958.
### What?
- Visiting a page in the app router without a proper component export doesn't show the dev overlay, but logs errors to the console
- When it does show the error overlay (e.g. during an HMR event), the error message was sharing the module code itself rather than the component path, making it hard to debug
### Why?
`createComponentTree` can throw these errors before the AppRouter tree is mounted, leaving the errors uncaught by the dev overlay.
### How?
This wraps the server root in the `ReactDevOverlay` when in dev mode with a minimal "HMR" for when the server component is edited (to reload the page).
Closes NEXT-308
This updates our `moduleResolution` to `bundler` as this matches our heuristics much more closely so is more accurate. This shouldn't be a breaking change is it should be compatible with our previous default.
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
- Adds `next-router-worker` as the process title for the routing worker.
- Adds `next-render-worker-app` as the process title for the rendering
worker for `app`
- Adds `next-render-worker-pages` as the process title for the rendering
worker for `pages`
Currently it's hard to find these processes in e.g. activity monitor,
this helps explain what they are used for when someone is looking at the
running processes.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
When doing dynamic imports like `import(variable)`, Webpack _tries_ to statically analyze it and creates a regex like context module for it (which includes all possible modules). This `ContextModule` doesn't have a resource path so we need to use the identifier to track it.
Tested with @alexkirsz's repro here https://github.com/vercel/next.js/issues/50243#issuecomment-1628675346 and confirmed that it fixes the problem.
Closes#50243.
This ensures we properly detect `not-found` as `runtime = 'edge'` and treat it as such instead of attempting to render it like normal.
x-ref: https://github.com/vercel/next.js/issues/52648
### What?
Support metadata exports for `not-found.js` conventions
### Why?
We want to define metadata such as title or description basic properties for error pages, including 404 and 500 which referrs to `error.js` and `not-found.js` convention. See more requests in #45620
Did some research around metadata support for not-found and error convention. It's possible to support in `not-found.js` when it's server components as currently metadata is only available but for `error.js` it has to be client components for now so it's hard to support it for now as it's a boundary.
### How?
We determine the convention if we're going to render is page or `not-found` boundary then we traverse the loader tree based on the convention type. One special case is for redirection the temporary metadata is not generated yet, we leave it as default now.
Fixes#52636
### What
This was an issue introduced in #52416 when we check the images and icons property while merging with static file conventions. Where we should check if the properties are present but we only checked if they value is falsy. So when you're using `icons` with array value it breaks.
Now we properly check all the possible case of `metadata.icons` value, so then decide if we are using the static file convention or the icons property of defined metadata.
Also add few more tests cases to cover icons resolving.
References for Client Components need to be aggregated to the page entry level, and emitted as files in the correct directory for the SSR server to read from. For normal routes (e.g. `app/foo/bar/page`), we can go through and collect all entries (layout, loading, error, ...) from the current and parent segments, to aggregate all necessary client references.
However, for routes with special conventions like `app/(group)/@named/foo/bar/page`, it needs to be normalized (remove the named slot and group segments) so it can be grouped together with `app/(group)/@named2/foo/bar/loading`.
When there's a runtime error showing in root layout (server components), it should be able to catch by `global-error`.
For server components, we caught it and gonna render the error fallback components (either not-found or error page), and the response status is `200`, and since we'll display error dev overlay in developmenet mode so we only render `global-error` for production.
So that you can catch more errors with `global-error` and maybe do potential error tracking on client side.
Follow up of #52573
Closes NEXT-1442
minor refactor: move `appUsingSizeAdjust` into `Metadata` component so that we can just tune the flag as prop
We have some special bundle path handling logic for `/index` routes in
`normalizePagePath`, which is missing in the new
`AppBundlePathNormalizer`. This already broke `/index/page.js` in dev in
the past, and now become noticeable in prod as well because of the
manifest change.
b98469c86b/packages/next/src/shared/lib/page-path/normalize-page-path.ts (L5-L14)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Introduce a new way to search for `not-found` component that based on
the request pathname and current loader tree of that route. And we
search the proper not-found in the finall catch closure of app
rendering, so that we don't have to pass down the root layout to
app-router to create the extra error boundary.
This ensures the root layout doesn't have duplicated rendering for
normal requests
Fixes NEXT-1220
Fixes#49115
Transitions the App Pages renderer into the entrypoint bundle.
- Adjusts the static path detection to handle route module's with App Pages
- Fixes bug in font manifest loading on Edge
This PR ensures that both Webpack and the config won't be initiated in render workers. This is great for performance but also avoids potential issues (e.g. Next.js plugin with side effects). Instead, we pass a serialized config from the router worker to the render workers.
Closes#52366.
### 🧐 What's in there?
At the moment, it is not possible to test code with `import 'server-only'` in app directory.
When trying to load such file in jest (even with `testEnvironment: node`), the error will be:
```
● Test suite failed to run··
x NEXT_RSC_ERR_CLIENT_IMPORT: server-only
,-[lib/util.js:1:1]
1 | /** @jest-environment node */·
2 | import 'server-only'
: ^^^^^^^^^^^^^^^^^^^^
3 | export const PI = 3.14;
`----·
at Object.transformSync (node_modules/next/src/build/swc/index.ts:443:25)
at transformSync (node_modules/next/src/build/swc/index.ts:629:19)
at Object.process (node_modules/next/src/build/swc/jest-transformer.ts:117:25)
at ScriptTransformer.transformSource (node_modules/@jest/transform/build/ScriptTransformer.js:619:31)
at ScriptTransformer._transformAndBuildScript (node_modules/@jest/transform/build/ScriptTransformer.js:765:40)
at ScriptTransformer.transform (node_modules/@jest/transform/build/ScriptTransformer.js:822:19)·
```
In a nutshell:
- next/swc is looking for ‘server-only’ [text in the source](https://github.com/vercel/next.js/blob/canary/packages/next-swc/crates/core/src/react_server_components.rs#L576), and throw if not configured for server
- next's jest-transformer will only configure next/swc for server [if the environment is node](https://github.com/vercel/next.js/blob/canary/packages/next/src/build/swc/jest-transformer.ts#L88)
- when testing Next.js apps, your jest testEnvironment is most likely jsdom. But you can configure it [per file with docBlock](https://jestjs.io/docs/configuration#testenvironment-string), which jest-transformer ignores because it only reads the top-level configuration.
This PR fixes this, by
1. reading the docblock to configure next/swc accordingly and bypass its hardcoded guard
2. mocking `server-only` so [it does not throw](https://github.com/vercel/next.js/blob/canary/packages/next/src/compiled/server-only/index.js) when loaded (jest does not read `react-server` export from package.json)
Users would still have to annotate their `server-only` files with `/** @jest-environment node */` in order to test them.
### 🧪 How to test?
There's a full test available: `pnpm testheadless --testPathPattern jest/server-only`
Here is also a minimal reproduction:
<details>
<summary>app/layout.tsx</summary>
```typescript
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (<html lang="en"><body>{children}</body></html>)
}
```
</details>
<details>
<summary>app/page.tsx</summary>
```typescript
import { PI } from '@/lib/utils'
export default function Home() {
return <h1>{PI}</h1>
}
```
</details>
<details>
<summary>lib/utils.ts</summary>
```typescript
import 'server-only'
export const PI = 3.14
```
</details>
<details>
<summary>lib/utils.test.ts</summary>
```typescript
import { PI } from './utils'
it('works', () => expect(PI).toEqual(3.14))
```
</details>
<details>
<summary>jest.config.js</summary>
```typescript
const nextJest = require('next/jest')
module.exports = nextJest({ dir: './' })({ testEnvironment: 'jsdom' })
```
</details>
### ❗ Notes to reviewers
[jest-docblock](https://packagephobia.com/result?p=jest-docblock) with dependencies is only 12.5 kB.
Fixes#47448
Previously `global-error` only caught the error on client side, this PR adds the support for catching the errors thrown during client components SSR or server components RSC rendering.
Closes#46572Closes#50119Closes#50723
The problem was introduced in #52450, that the client reference manifest isn't being tracked and included in the function.
Verified that this fixes the issue.
### What?
This PR fixes build crashing when `output: 'standalone'` and `experimental.appDir` is enabled but there is no app pages.
### How?
It does that by checking whether `.next/server/app` exists before copying the folder to `.next/standalone/...`
Closes#51828Fixes#44442Fixes#44120
This PR changes client manifest generation process. Instead of one big
manifest file that contains client references for the entire app, we're
now generating one manifest file per entry which only covers client
components that can be reached in the module graph.
