### What?
The second step in our chunking refactoring, this removes our use of Module::as_chunk and Module::as_root_chunk. Instead, the only way to generate a chunk is directly from a root ChunkItem.
### Why?
In the end we want to avoid creating chunks from modules directly, but enforce everything going through the ChunkingContext to be chunked. This allows us to replace the existing chunking algorithm with a much more efficient one that avoid duplication between chunks in first place and doesn't require a post-chunking optimization.
### How?
https://github.com/vercel/turbo/pull/6120
Re: https://github.com/vercel/next.js/pull/56467
Closes WEB-1721
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
When we landed #51179 it broke library like `apollo-client` as it's bundling client hooks into RSC bundle, so our RSC linter caught them and reported fatal errors. But those client hook APIs won't get executed in RSC. The original purpose of erroring on invalid hooks for server & client components was to catch bugs easier, but it might be too strict for the 3rd party libraries like `apollo-client` due to few reasons.
We changed the rules only applying on user land source code. For 3rd party packages if they're not being imported correctly into proper server or client components, we're still showing runtime errors instead of fatal build errors.
x-ref: https://github.com/apollographql/apollo-client/issues/10974
Closes NEXT-1673
This is the first in a series of PRs replacing our use of fs-extra with node's own `node:fs`.
Test Plan:
With clean and existing artifacts in each case, ran:
- `pnpm build`
- `pnpm dev`
- `./node_modules/.bin/taskr ncc`
Closes WEB-1717
### What?
This change moves a few methods around.
Module::as_chunk is moved to ChunkItem::as_chunk as temporary intermediate state.
EcmascriptPlaceable::as_chunk_item is moved to ChunkableModule::as_chunk_item. This generalizes the concept of converting a Module into a ChunkItem
### Why?
This is the first step of refactoring the chunking. In the end we want to avoid creating chunks from modules directly, but enforce everything going through the ChunkingContext to be chunked. This allows us to replace the existing chunking algorithm with a much more efficient one that avoid duplication between chunks in first place and doesn't require a post-chunking optimization.
### How?
see https://github.com/vercel/turbo/pull/6104
Closes WEB-1715
Co-authored-by: Justin Ridgewell <112982+jridgewell@users.noreply.github.com>
This replaces the `(global as any)._nextDevHandlers` invocation with references to a specific service instance while also removing the module scoped `devInstances`. This ensures that correct types are used.
This was done while changing the `match` parameter in `ensurePage` to `definition` which didn't cause a typescript error (it should have).
The PR supersedes the https://github.com/vercel/next.js/pull/53150, which is way too outdated, has way too many conflicts, and also heavily relies on GitHub Copilot (which makes the progress slow and tedious).
The PR uses [`json-schema-to-zod`](https://github.com/StefanTerdell/json-schema-to-zod) (instead of the GitHub Copilot) to generate the zod schema, and manually replaces all generated `z.customRefine` with my hand-written zod schema.
TODO:
- [x] Convert schema
- [x] Reduce `z.any()` usage
- [x] Create human-readable errors from the `ZodError`
- [x] Update test cases to reflect the latest error message
-----
The benefit of using zod over ajv:
- Easier maintenance: zod schema is straightforward to compose.
- Better typescript support: config schema now strictly reflects the `NextConfig` type.
- Smaller installation size: by replacing `ajv` and `@segment/ajv-human-errors` w/ `zod`, I am able to reduce installation size by 114 KiB.
- Better Extension: the zod error message is easy to customize.
-----
In the previous PR https://github.com/vercel/next.js/pull/56083, @feedthejim replaces `zod` w/ `superstruct`. `superstruct` is lightweight and fast, which makes it perfect for creating simple schemas for RSC payload. But, this also means `superstruct` has its limitations compared to `zod`:
- `superstruct`'s syntax is different, and some utilities's usage is counter-intuitive:
- `z.array(z.string()).gt(1)` vs `s.size(s.array(s.string()), 1)`
- `z.numer().gt(1)` vs `s.size(s.number(), 1)`, `s.min(s.number(), 1)`
- `z.boolean().optional().nullable()` vs `s.nullable(s.optional(z.boolean()))`
- `superstruct` has weaker TypeScript support and worse DX compared to `zod` when composing huge schema:
- `zod.ZodType + z.object()` can provide a more detailed type mismatch message on which specific property is the culprit, while `Describe + s.object()` provides almost no information at all.
- `zod`'s schema is more powerful
- `z.function()` supports `z.args()` and `z.returns()`, while `superstruct` only has `s.func()`
- zod also has Promise type `z.promise()` and intersection type `z.and()`
- `superstruct`'s error is harder to parse compared to `zod`'s `ZodError`
So in the PR, I re-introduced `zod` for `next.config.js` validation.
This adds types for the component modules so that the modules no longer need to rely on `ComponentMod` being `any`. This allows stricter typing for those elements.
This also improves types around template files to allow their exports to form the `ComponentMod` types.
Updates `.node-version` to allow any version of `18` which will allow development machines to update to newer minor releases without the need to update the repository each time.
### What?
This implements Server Actions inside the new Turbopack next-api bundles.
### How?
Server Actions requires:
1. A `.next/server/server-reference-manifest.json` manifest describing what loader module to import to invoke a server action
2. A "loader" entry point that then imports server actions from our internal chunk items
3. Importing the bundled `react-experimental` module instead of regular `react`
4. A little 🪄 pixie dust
5. A small change in the magic comment generated in modules that export server actions
I had to change the magic `__next_internal_action_entry_do_not_use__` comment generated by the server actions transformer. When I traverse the module graph to find all exported actions _after chunking_ has been performed, we no longer have access to the original file name needed to generate the server action's id hash. Adding the filename to comment allows me to recover this without overcomplicating our output pipeline.
Closes WEB-1279
Depends on https://github.com/vercel/turbo/pull/5705
Co-authored-by: Tim Neutkens <6324199+timneutkens@users.noreply.github.com>
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
There's lots of situations in Next.js where we want to ensure that only one operation is in progress at a time for a given task. An example of this is our response cache. The expectation is that for multiple requests for the same page should only result in a single invocation. This isn't new behavior, but this abstracts the batching interface away so we don't duplicate it.
There should be no shared react packages in our server runtime. rsc should always be separate from ssr.
This update reconfigures the runtiem to eliminate shared react modules. the jsx runtime will now be separate for RSC and SSR. this is necessary because the implementations for the jsx runtime rely on React and they need to see the right versions.
Additionally I fixed an alias so that the shared subset react is used when using react-server condition.
I also fixed a bug in 2 tests related to class/className.
Note: this PR blocks upgrading React canary due to internal changes in how React state is managed in when using the `react-server` condition
### What?
* node.js doesn't allow module requests starting with `.` since node.js
18, so we need to remove out .pnpm special case
* externals should be resolvable from the project dir instead of the
root dir
* fix a problem when the list of externals is empty
### Why?
### How?
Closes WEB-1703
Example usage of `readFileSync().toString()` first creates a `Buffer` instance and later does another C++ call to convert Buffer to UTF-8. This pull request reduces the C++ calls.
### What?
Merged a bunch of dependabot alerts in my own canary branch, mainly
postcss patch updates, and one graphql minor update, to fix moderate
security vulnerabilities in examples. Spot checked most and look good
still. EDIT: also one in scripts/send-trace-to-jaeger
### Why?
Because safety
### How?
Dependabot
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Steven <steven@ceriously.com>
When the function name is omitted webpack generates a name from the loader config which can be really long un-necessarily. Providing a name for the default function alleviates this. cc @shuding who found this
Using `await fs.access` has couple of downsides. It creates unnecessary
async contexts where async scope can be removed. Also, it creates the
possibility of race conditions such as `Time-of-Check to Time-of-Use`.
It would be nice if someone can benchmark this. I'm rooting for a
performance improvement.
Some updates from Node.js land:
- There is an open pull request to add V8 Fast API to `existsSync`
method - https://github.com/nodejs/node/pull/49893
- Non-existing `existsSync` executions became 30% faster -
https://github.com/nodejs/node/pull/49593
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Today when we hydrate an SSR'd RSC response on the client we encounter import chunks which initiate code loading for client components. However we only start fetching these chunks after hydration has begun which is necessarily after the initial chunks for the entrypoint have loaded.
React has upstream changes that need to land which will preinitialize the rendered chunks for all client components used during the SSR pass. This will cause a `<script async="" src... />` tag to be emitted in the head for each chunk we need to load during hydration which allows the browser to start fetching these resources even before the entrypoint has started to execute.
Additionally the implementation for webpack and turbopack is different enough that there will be a new `react-server-dom-turbopack` package in the React repo which should be used when using Turbopack with Next.
This PR also removes a number of patches to React src that proxy loading (`__next_chunk_load__`) and bundler requires (`__next_require__`) through the `globalThis` object. Now the react packages can be fully responsible for implementing chunk loading and all Next needs to do is supply the necessary information such as chunk prefix and crossOrigin attributes necessary for this loading. This information is produced as part of the client-manifest by either a Webpack plugin or Turbopack.
Additionally any modifications to the chunk filename that were previously done at runtime need to be made in the manifest itself now. This means we need to encode the deployment id for skew protection and encode the filename to make it match our static path matching (and resolutions on s3) when using `[` and `]` segment characters.
There are a few followup items to consider in later PRs
1. we currently bundle a node and edge version of react-server-dom-webpack/client. The node version has an implementation for busboy whereas the edge version does not. Next is currently configured to use busboy when handling a fetch action sent as multipart with a node runtime. Ideally we'd only bundle the one platform we are buliding for but some additional refactoring to support better forking is possibly required here
This PR also updates react from 09285d5a7 to d900fadbf.
### React upstream changes
- https://github.com/facebook/react/pull/27439
- https://github.com/facebook/react/pull/26763
- https://github.com/facebook/react/pull/27434
- https://github.com/facebook/react/pull/27433
- https://github.com/facebook/react/pull/27424
- https://github.com/facebook/react/pull/27428
- https://github.com/facebook/react/pull/27427
- https://github.com/facebook/react/pull/27315
- https://github.com/facebook/react/pull/27314
- https://github.com/facebook/react/pull/27400
- https://github.com/facebook/react/pull/27421
- https://github.com/facebook/react/pull/27419
- https://github.com/facebook/react/pull/27418
### What?
Deduplicates our env var injection between the JS and rust side
Closes WEB-937
---------
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
When trying to update my own website to use App Router, I found the current Next.js Docs were missing specifics to get it working (most notably, no mention of `pageExtensions`). Coupled with community feedback, we decided to revisit and rework the entire page.
The page has been walked through and tested on brand new create-next-app projects for both App and Pages Routers to ensure completeness.
Closes [DX-2095](https://linear.app/vercel/issue/DX-2095/improve-nextjs-mdx-docs)
Rendered pages while running locally, in case they are more helpful in seeings the changes and different between routers.
### App Router
![localhost_3332_docs_pages_building-your-application_configuring_mdx (1)](https://github.com/vercel/next.js/assets/446260/2a2b1a66-c794-4831-ab89-a6a61d02eb84)
### Pages Router
![localhost_3332_docs_pages_building-your-application_configuring_mdx](https://github.com/vercel/next.js/assets/446260/cc60a8a2-a931-4033-aaba-e989a719692d)
This refactors a bit of the typescript types around the export and build pipelines. This does not introduce any new functionality.
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
This updates to make build/server tracing parallelized with our webpack
compilations so that it can start right after the server compilation is
finished instead of after all compilations are finished. This
parallelizing requires the `experimental.webpackBuildWorker` config be
enabled and doesn't change behavior without it.
Part of this also cleans up our plugin state restoring/handling to use
serializable structures instead of serializing/deserializing manually.
### What?
Fixes the pages router not receiving a hash when being linked from the
app router.
### Why?
The hash being removed breaks sites that rely on it for client side
features.
### How?
The hash gets omitted from the URL when used as a key for the
preflightCache. Once it's clear that it's an external URL and that it's
not empty we can use the initial href to send the hash as well.
Not completely sure if there's an edge case that might break, I added an
extra check for when the hash is only used to scroll the page.
This might need an additional test case just for
`navigate-reducer.test.tsx`.
Fixes#56212
---------
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Co-authored-by: JJ Kasper <jj@jjsweb.site>