This PR adds proof-of-concept support for the App Router to `next build
--experimental-turbo`.
It introduces a new way to generate Next.js manifests in Turbopack.
Currently, in dev, we pass proxy objects in lieu of manifests, and rely
on the entries to know which chunks they need loaded on the client.
However, this can't work for builds because it requires control over
Next.js rendering, which is not compatible with a Next->Turbo approach.
We would need to modify Next.js to support these "lazy" entries. So for
now, we add well-known assets (`NextDynamicAsset`,
`NextServerComponentAsset`, `NextClientReferenceAsset`, etc.) to the
graph, which will get picked up when walking it during asset processing.
This lets us collect all possible entries before chunking.
This two-step process (collecting all entries, then chunking them) is
also a good first step towards production chunking.
## Turbopack updates
* https://github.com/vercel/turbo/pull/5494 <!-- Tobias Koppers - add
reporting of console messages -->
* https://github.com/vercel/turbo/pull/5448 <!-- Alex Kirszenberg -
Misc. changes to support App Router build -->
### What?
Update SWC crates to `v0.79.13`.
### Why?
- Explicit resource management proposal is now fully implemented, although it's behind a parser flag because it's stage 3
- Some bugs of `swcMinify` are fixed.
### How?
Closes WEB-1272
## Turbopack updates
* https://github.com/vercel/turbo/pull/5475
Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
This fixes Turbopack integration tests.
In https://github.com/vercel/next.js/pull/51928, I made it so all dev chunking contexts output to the same directory. However, in the case of the web entry source, this can lead to conflicts. This PR ensures the chunking context is different for that one case.
The client chunking context must remain the same for a given Next.js compilation. For production builds, we only need to create it once for both the Pages Router and the App Router. As such, it doesn't make sense for it to accept a context type, at least not as an enum. A vector of enums or flags could also work. Considering the only case where the chunking context differes today is the web entry source, we might as well make the web entry source also serve under `_next` (or remove it entirely?).
This is a follow up to PR https://github.com/vercel/next.js/pull/52291
which removed the `next.config.js` which started causing an error during
CI:
```
@next/swc:test-cargo-bench: error: couldn't read packages/next-swc/crates/next-dev/benches/next.config.js: No such file or directory (os error 2)
@next/swc:test-cargo-bench: --> packages/next-swc/crates/next-dev/benches/bundler.rs:74:13
@next/swc:test-cargo-bench: |
@next/swc:test-cargo-bench: 74 | include_bytes!("next.config.js"),
@next/swc:test-cargo-bench: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@next/swc:test-cargo-bench: |
@next/swc:test-cargo-bench: = note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)
@next/swc:test-cargo-bench:
@next/swc:test-cargo-bench: error: could not compile `next-dev` (bench "mod") due to previous error
@next/swc:test-cargo-bench: warning: build failed, waiting for other jobs to finish...
@next/swc:test-cargo-bench: ELIFECYCLE Command failed with exit code 101.
```
https://github.com/vercel/next.js/actions/runs/5476339074/jobs/9973767389#step:27:959
Depends on https://github.com/vercel/turbo/pull/5398 (and downstack) on
the Turbo side.
This PR makes it so the output path of Node.js entry chunks for pages is
determined solely from the pathname. This isn't actually necessary for
pages, but it makes for easier debugging anyway. See the Turbo PR for
more details.
This also changes the page structure so it also works if there is no
`pages` directory. In this case, we still want to use pages' default
_app, _document, and _error pages, even with existing app routes. So an
empty page structure should still have an effect.
## Turbopack updates
* https://github.com/vercel/turbo/pull/5415 <!-- Will Binns-Smith -
Turbopack: Execution tests in node.js -->
* https://github.com/vercel/turbo/pull/5461 <!-- Tobias Koppers - use a
lock to ensure atomic invalidation from file changes -->
* https://github.com/vercel/turbo/pull/5398 <!-- Alex Kirszenberg -
Configure the path of the Node.js entry chunk -->
* https://github.com/vercel/turbo/pull/5469 <!-- Tobias Koppers - allow
hmr tests to correctly detect hmr event -->
### What?
I've found 2 more spots that didn't properly cancel the streaming response when the client disconnects. This fixes `RenderResult.pipe()` (called during dynamic render results) and `sendResponse()` (used during Route Handlers using `nodejs` runtime).
It also (finally) adds tests for all the cases I'm aware of.
### Why?
To allow disconnecting from an AI service when a client disconnects. $$$
### How?
Just checks for `response.closed`, which will be closed when the client's connection disconnects.
The current message isn't very clear about `"use server" function` and
`"use server" file`, and there's no link to docs to explain it further:
```
"use server" functions are not allowed in client components.
You can import them from a "use server" file instead.
```
This PR makes it a bit more verbose:
```
It is not allowed to define inline "use server" annotated Server Actions in Client Components.
You can either mark the entire file by putting "use server" at the top, or pass them down through props from a Server Component.
Read more: https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions#with-client-components
```
### What?
Add all passing tests
### Why?
We enforced that they all run turbopack and these are passing.
If any of these tests would become red, that indicated a problem.
Currently we are validating the `experimental.serverActions` flag when creating the actual entries for Server Actions, this causes two problems. One is that syntax errors caught at compilation time are still shown, even if you don't have this flag enabled. Another problem is we still traverse the client graph to collect these action modules even if the flag isn't enabled.
This PR moves that check to be happening at compilation time, which addresses the two above but also brings the extra benefit of showing the exact span and module trace that errors:
<img width="974" alt="CleanShot 2023-07-03 at 20 26 34@2x" src="https://github.com/vercel/next.js/assets/3676859/1676b1f6-e205-4963-bce4-5b515a698e9c">
When looking at [some sites](https://rsc-llm-on-the-edge.vercel.app/) with a large amount of chunks streamed, I noticed that the inlined Flight data array can be optimized quite a lot. Currently we do:
```js
self.__next_f.push([1,"d5:[\"4\",[\"$\",\"$a\",null,..."])
```
1. The `self.` isn't needed (except for the initial bootstrap tag) as React itself has `<script>$RC("B:f","S:f")</script>` too.
2. After the bootstrap script tag, all items are an array with `[1, flight_data]` and `flight_data` is always a string. We can just push only these strings.
3. We use `JSON.stringify(flight_payload)` to inline the payload where the payload itself is a string with a lot of double quotes (`"`), this results in a huge amount of backslashes (`\`). Here we can instead replace it to use a pair of single quotes on the outside and un-escape the double quotes inside.
Here's a side-by-side comparison of a small page:
<img width="1710" alt="CleanShot 2023-06-30 at 11 41 02@2x" src="https://github.com/vercel/next.js/assets/3676859/398356ec-91d5-435c-892d-16fb996029e8">
For a real production page I saw the HTML payload reduced by 11,031 bytes, a 3% improvement.
Note that all the tests are not considering gzip here, so the actual traffic impact will be smaller.
