### What?
Update SWC crates to `v0.79.13`.
### Why?
- Explicit resource management proposal is now fully implemented, although it's behind a parser flag because it's stage 3
- Some bugs of `swcMinify` are fixed.
### How?
Closes WEB-1272
## Turbopack updates
* https://github.com/vercel/turbo/pull/5475
Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
This fixes Turbopack integration tests.
In https://github.com/vercel/next.js/pull/51928, I made it so all dev chunking contexts output to the same directory. However, in the case of the web entry source, this can lead to conflicts. This PR ensures the chunking context is different for that one case.
The client chunking context must remain the same for a given Next.js compilation. For production builds, we only need to create it once for both the Pages Router and the App Router. As such, it doesn't make sense for it to accept a context type, at least not as an enum. A vector of enums or flags could also work. Considering the only case where the chunking context differes today is the web entry source, we might as well make the web entry source also serve under `_next` (or remove it entirely?).
This is a follow up to PR https://github.com/vercel/next.js/pull/52291
which removed the `next.config.js` which started causing an error during
CI:
```
@next/swc:test-cargo-bench: error: couldn't read packages/next-swc/crates/next-dev/benches/next.config.js: No such file or directory (os error 2)
@next/swc:test-cargo-bench: --> packages/next-swc/crates/next-dev/benches/bundler.rs:74:13
@next/swc:test-cargo-bench: |
@next/swc:test-cargo-bench: 74 | include_bytes!("next.config.js"),
@next/swc:test-cargo-bench: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@next/swc:test-cargo-bench: |
@next/swc:test-cargo-bench: = note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)
@next/swc:test-cargo-bench:
@next/swc:test-cargo-bench: error: could not compile `next-dev` (bench "mod") due to previous error
@next/swc:test-cargo-bench: warning: build failed, waiting for other jobs to finish...
@next/swc:test-cargo-bench: ELIFECYCLE Command failed with exit code 101.
```
https://github.com/vercel/next.js/actions/runs/5476339074/jobs/9973767389#step:27:959
Depends on https://github.com/vercel/turbo/pull/5398 (and downstack) on
the Turbo side.
This PR makes it so the output path of Node.js entry chunks for pages is
determined solely from the pathname. This isn't actually necessary for
pages, but it makes for easier debugging anyway. See the Turbo PR for
more details.
This also changes the page structure so it also works if there is no
`pages` directory. In this case, we still want to use pages' default
_app, _document, and _error pages, even with existing app routes. So an
empty page structure should still have an effect.
## Turbopack updates
* https://github.com/vercel/turbo/pull/5415 <!-- Will Binns-Smith -
Turbopack: Execution tests in node.js -->
* https://github.com/vercel/turbo/pull/5461 <!-- Tobias Koppers - use a
lock to ensure atomic invalidation from file changes -->
* https://github.com/vercel/turbo/pull/5398 <!-- Alex Kirszenberg -
Configure the path of the Node.js entry chunk -->
* https://github.com/vercel/turbo/pull/5469 <!-- Tobias Koppers - allow
hmr tests to correctly detect hmr event -->
### What?
I've found 2 more spots that didn't properly cancel the streaming response when the client disconnects. This fixes `RenderResult.pipe()` (called during dynamic render results) and `sendResponse()` (used during Route Handlers using `nodejs` runtime).
It also (finally) adds tests for all the cases I'm aware of.
### Why?
To allow disconnecting from an AI service when a client disconnects. $$$
### How?
Just checks for `response.closed`, which will be closed when the client's connection disconnects.
The current message isn't very clear about `"use server" function` and
`"use server" file`, and there's no link to docs to explain it further:
```
"use server" functions are not allowed in client components.
You can import them from a "use server" file instead.
```
This PR makes it a bit more verbose:
```
It is not allowed to define inline "use server" annotated Server Actions in Client Components.
You can either mark the entire file by putting "use server" at the top, or pass them down through props from a Server Component.
Read more: https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions#with-client-components
```
### What?
Add all passing tests
### Why?
We enforced that they all run turbopack and these are passing.
If any of these tests would become red, that indicated a problem.
Currently we are validating the `experimental.serverActions` flag when creating the actual entries for Server Actions, this causes two problems. One is that syntax errors caught at compilation time are still shown, even if you don't have this flag enabled. Another problem is we still traverse the client graph to collect these action modules even if the flag isn't enabled.
This PR moves that check to be happening at compilation time, which addresses the two above but also brings the extra benefit of showing the exact span and module trace that errors:
<img width="974" alt="CleanShot 2023-07-03 at 20 26 34@2x" src="https://github.com/vercel/next.js/assets/3676859/1676b1f6-e205-4963-bce4-5b515a698e9c">
When looking at [some sites](https://rsc-llm-on-the-edge.vercel.app/) with a large amount of chunks streamed, I noticed that the inlined Flight data array can be optimized quite a lot. Currently we do:
```js
self.__next_f.push([1,"d5:[\"4\",[\"$\",\"$a\",null,..."])
```
1. The `self.` isn't needed (except for the initial bootstrap tag) as React itself has `<script>$RC("B:f","S:f")</script>` too.
2. After the bootstrap script tag, all items are an array with `[1, flight_data]` and `flight_data` is always a string. We can just push only these strings.
3. We use `JSON.stringify(flight_payload)` to inline the payload where the payload itself is a string with a lot of double quotes (`"`), this results in a huge amount of backslashes (`\`). Here we can instead replace it to use a pair of single quotes on the outside and un-escape the double quotes inside.
Here's a side-by-side comparison of a small page:
<img width="1710" alt="CleanShot 2023-06-30 at 11 41 02@2x" src="https://github.com/vercel/next.js/assets/3676859/398356ec-91d5-435c-892d-16fb996029e8">
For a real production page I saw the HTML payload reduced by 11,031 bytes, a 3% improvement.
Note that all the tests are not considering gzip here, so the actual traffic impact will be smaller.
## Background
Currently we're track all imported CSS modules by an entry, in the
client manifest:
```js
// client manifest
{
entryCSSFiles: {
'app/entry_name': {
modules: ['app/foo.css', ...]
}
}
}
```
And then, in the font manifest we track all emitted assets (fonts) by
each CSS module:
```js
// font manifest
{
app: {
'app/foo.css': [
'static/font/abc.woff', ...
]
}
}
```
These two fields are only used together by `get-preloadable-fonts.tsx`,
so it can know which font files need to be preloaded for this entry.
(Although previously we use `.modules` for something else, but it's gone
now)
## Changes
Since we only need the font assets per entry, it's unnecessary to track
these in the module level and then join them together. This PR removes
the `modules` field from the client manifest, and changes the font
manifest to directly keep the entry—font mapping.
This gets rid of one module traversal from the client manifest plugin.
Small QOL improvements to `RenderResult` and `RouteModule` setup. This
also adds a test to verify that headers aren't sent on responses that
have already been sent.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
### What?
this forces all tests to use turbopack independent of the way they invoke next dev
### Why?
some tests were not running turbopack
fixes WEB-1187
### What?
* allow to use `runtime = "edge"` for app dir in turbopack
* move common imports from next-app-loader to
`packages/next/src/server/app-render/entry-base.ts`
* move common turbopack code to communicate between JS and Rust into
separate files
### Why?
A lot test cases depend on edge rendering
### How?
---------
Co-authored-by: Alex Kirszenberg <alex.kirszenberg@vercel.com>
This contains the original POC for `next build --turbo`. The implementation is _just enough_ to get pages building, and doesn't support the app router yet.
I'll write more details here on the implementation and what the next steps are next week.
Necessary changes on the Turbo side: https://github.com/vercel/turbo/pull/4998
Make sure we are using `var` instead of `const` as we always put the new
appended statements to the end of the module body, but they can still be
referenced before in the HOC case in the runtime. This causes a runtime
error.
Tl;dr: `a = 1; var a` is fine, but `a; const a = ...` will result in a
compilation error.
Closes#49344.
Remove the Server CSS manifest and related logic, and use the chunkGroup
CSS files instead for each entry to inject stylesheet links.
Why was that manifest needed in the first place? When implementing CSS
collection for RSC and nested layout initially, we collect CSS imports
at each layer and create corresponding client entries. But then soon got
hit by the problem of duplication and improper tree-shaking. Two layers
can have the same CSS imported so we solved it in a way that "if an
upper layer imports this module, skip it in child layers". Note that
this is deduped by module, so we need to keep the information of "layer
(entry) → CSS modules" somewhere, in a manifest.
Another reason is that we create the client entry before Webpack
optimizes modules, so we can inject the client entry into the same
compilation. But that means the collected client modules from the server
layer are not properly optimized (DCE). **This is a general issue at the
moment that's not specifically related to CSS, although using that
manifest to collect DCE'd info and join the original collected CSS files
with that info temporarily solved it.** That's why I disabled some tests
related to font CSS collection and want to improve it in a more general
way.
Why is that not needed anymore? Main reason is to keep a good balance
between duplication and number of chunks, and delegate the decision to
Webpack's splitChunks plugin. It is not possible to get to a point of 0
duplication unless we ship every CSS module as a single chunk. And since
in #50406 we made the duplication better but at the chunk asset level,
instead of the original module level.
Prior work: #50406, #50610.
This removes the `__internal_nextjs_integration_test` compile-time feature. This would end up requiring an additional set of build targets for running integration tests was only used for two cases:
* Determining whether to read from an env var to load mocked Google fonts responses in next/font/google tests. This now uses the presence of this variable to determine whether to use mocked responses.
* Enabling the `next-dev/serializable` feature. This is now set directly instead.
Test Plan:
* Integration tests with Turbopack.
* Manual test of next/font/google with Turbopack
### What?
* enable turbopack tests in new CI
* split pre-build step into native and JS builds to allow to start
native tests faster
* update swc_core to sync with turbo
* update turbopack
### Why?
Running our test suite is still important as long we don't have the full
integration test suite enabled.
### Turbopack updates
* https://github.com/vercel/turbo/pull/5215 <!-- OJ Kwon - ci(workflow):
upload benchmark results to datadog -->
* https://github.com/vercel/turbo/pull/5239 <!-- OJ Kwon -
fix(swc_plugin): use shared runtime -->
* https://github.com/vercel/turbo/pull/3090 <!-- CHEN Yuan - Docs: prior
to run testcases, add guides to install dependencies for testcases. -->
* https://github.com/vercel/turbo/pull/5264 <!-- Tobias Koppers - update
test runner -->
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Instead of using azAvgWidth for next/font/google, use xWidthAvg as the webpack implementation does. For now, next/font/local in both the webpack and Turbopack implementations continue to use azAvgWidth. We should align these in the future.
Test Plan: We now pass 26 next/font tests in Turbopack, up from 22.
For RSC server layer so far we bundle all dependencies, ESM format is the better one rather than CJS to analyze and tree-shake out the unused parts. This PR changes pick the condition names that are in ESM format first for server layer.
Also fixes the misorder of condition names of edge runtime, `conditionNames` should only contain either ESM or CJS, previously the main fields are mixed with conditon names which is not expected for webpack, we separate them now.
Since we're picking ESM instead CJS now, the error of require `exports * from` doesn't exist anymore, but if you're using a CJS dependency which require a ESM package, it will error. This is the existing behavior for our webpack configuration but could happen on server layer bundling
Other related changes:
* Imports are hoisted in ESM, so migrate`enhanceGlobals` to a imported module
* Use `...` to pick the proper imports by import expression, and prefer the `react-server` / `edge-light` condition names for corresponding cases
* Remove edge SSR duplicated `middleware` export checking
Move as much of codeowners as possible to use Vercel Spaces.
1. Makes `@timneutkens @ijjk @shuding @huozhi @feedthejim` global owners
2. Make the `@vercel/next-js` team _optional_ owners of **/docs**,
**/errors**, and **/contributing**, makes team owners of a few packages
as per old config.
3. Make `@vercel/devex` (docs and devrel) owners of **/docs**,
**/errors**, and **/contributing**
4. Make `@vercel/devrel` (devrel only) owners of **/examples**
5. Make `@vercel/web-tooling` owners of specific files and folders (as
per old config)
Leaves @styfle as owner of **image** files on the old config since this
pattern `/**/*image*/** ` can't be used with Vercel Spaces.
Note: We cannot add * or / at the end of files.
[Docs](https://spaces-docs.vercel.sh/docs/code-owners#:~:text=Code%20Owners%20files%20are%20meant%20to%20encourage%20distributed%20ownership%20definitions%20across%20a%20codebase.)
This PR fixes the case of having a shorthand prop `{ id }` referenced from a non-top level closure in the Server Actions compiler. The main problem is that an `Ident` in `PropOrSpread` is not considered as an `Expr`.
See corresponding issue report and test case for more details.
fix#50445
fix NEXT-1254
With vercel/turbo#5156, we'll be able to install our the locally built `next` package. Before, we'd test the tip-of-branch `next-dev` binary against the last cut `next` canary, which causes headaches when we make breaking changes.
With this PR, we'll now test tip-of-branch `next-dev` binary against a tip-of-branch `next` package, and breaking changes can be properly benched.
Fixes WEB-1133
This starts the process of moving the rendering logic for pages into the
bundle. The `render` method on the module now provides the same
capability as the underlying `renderHTML` method available on the
`NextNodeServer`. In the next few change sets, more layers of the
rendering pipeline will shift into the bundled code, removing them from
the main boot path.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Some font entries in capsize do not include fields like `cap_height` and `x_height`. Since these are unused by next/font, remove these from the serde structure entirely.
Additionally, this makes it so that failing to read or deserialize the capsize map results in an `Error` rather than a `FontFallbackError`, which causes a build failure rather than silently omitting a fallback font. `next/font` should always include this map, and it obscured the real issue here. However, a missing entry in this map _should_ result in omitting a fallback.
This field is no longer needed. Although removing it might cause
unnecessary styles being included that could be tree-shaken potentially,
it solves a more critical issue of missing CSS resources: since now CSS
import can be "hoisted" to its parent entry (e.g. a layout), we can't
simply get all used CSS resources from the "leaf entry" (e.g. a page).
I noticed while testing that we're getting a bunch of 500 errors after #50241 merged. Turns out that `fetchNextData` will [fetch `HEAD` requests](https://github.com/vercel/next.js/blob/cf9591cd/packages/next/src/shared/lib/router/router.ts#L619-L621) for background priority. The problem is that, somewhere, the Next router is draining the body from HEAD responses, leading us to trying to `JSON.parse` an empty string.
This changes the way we return results to the Turbopack router. Instead of `JSON.stringify`ing the result into the body (which will be drained by something), we directly return the result. And it saves us a `stringify` -> `parse` -> `stringify` round trip, so that's nice.
I took the chance to clean up some of our boilerplate code, too.
Bumping `@edge-runtime/*` <picture data-single-emoji=":edge-runtime:" title=":edge-runtime:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/edge-runtime/b940e917443aa49f.png" alt=":edge-runtime:" align="absmiddle"></picture> packages to their latest beta, as we had some huge improvements and big changes on how it works internally.
* Using the latest `undici` which provides a WebSocket implementation
* Less polyfill usage as we load modules in the Node.js realm, and can reuse modules from Node.js as time goes on <picture data-single-emoji=":just-right2:" title=":just-right2:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/just-right2/588cf34d02f4b3bd.png" alt=":just-right2:" align="absmiddle"></picture>
* `instanceof` checks now work within the realm <picture data-single-emoji=":mind_blown:" title=":mind_blown:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/mind_blown/0186b6f181040126.gif" alt=":mind_blown:" align="absmiddle"></picture>
This breaks up some of our longest running tests which allows more
parallelizing of the tests. This also moves turbopack tests filtering
back to an allow list as it is running a lot of unrelated tests
currently which isn't ideal. We should only be running against tests
that are explicitly testing turbopack e.g. build tests should not be
duplicated in the turbopack group.
```sh
test/integration/css/test/group-1.test.js: 762.655
test/integration/edge-runtime-module-errors/test/index.test.js: 695.309
test/integration/css/test/group-2.test.js: 671.848
test/integration/i18n-support/test/index.test.js: 518.173
test/integration/scss-modules/test/index.test.js: 451.704
test/integration/css-features/test/index.test.js: 417.318
test/integration/css-modules/test/index.test.js: 403.405
test/integration/eslint/test/index.test.js: 381.563
test/integration/500-page/test/index.test.js: 371.134
test/integration/telemetry/test/index.test.js: 367.691
test/development/acceptance-app/ReactRefreshLogBox.test.ts: 335.878
test/integration/create-next-app/templates.test.ts: 334.01
test/integration/scss/test/group-2.test.js: 327.255
test/integration/scss/test/group-1.test.js: 318.574
test/integration/edge-runtime-configurable-guards/test/index.test.js: 313.834
test/e2e/instrumentation-hook/instrumentation-hook.test.ts: 294.618
test/development/acceptance-app/error-recovery.test.ts: 283.355
test/e2e/app-dir/app/vercel-speed-insights.test.ts: 278.242
test/integration/create-next-app/index.test.ts: 272.442
```
This adds new `build and test` and `build and deploy` workflows in favor
of the existing massive `build, test, and deploy` workflow. Since the
new workflows will use `pull_request_target` this waits to remove the
existing workflow until the new one is tested.
While testing this new workflow flakey behavior in tests have also been
addressed. Along with the new workflow we will also be leveraging new
runners which allow us to run tests against the production binary of
`next-swc` so this avoids slight differences in tests we've seen due to
running against the dev binary.
Furthermore we will have a new flow for allowing workflow runs on PRs
from external forks which will either require a comment be checking a
box approving the run after each change or a label added by the team.
The new flow also no longer relies on `actions/cache` or similar which
have proven to be pretty unreliable.
Tests runs with the new workflow can be seen here
https://github.com/vercel/next.js/actions/runs/5100673508/jobs/9169416949
### What?
create internal modules via `context.process` with
`ReferenceType::Internal`
### Why?
We need this for future refactoring where we want to the internal
modules with transitions.
### Turbopack Changes
* https://github.com/vercel/turbo/pull/5095 <!-- Tobias Koppers - allow
to create internal modules via AssetContext -->
* https://github.com/vercel/turbo/pull/5104 <!-- Alex Kirszenberg -
Stable chunk list ident -->
* https://github.com/vercel/turbo/pull/5106 <!-- Tobias Koppers -
followup fix -->
---------
Co-authored-by: Alex Kirszenberg <alex.kirszenberg@vercel.com>
This exports the named exports from the user's page route file and
exports a new `routeModule` stub. This stub will be the new rendering
mechanism used for pages within `pages/` and will serve to allow us to
deprecate large chunks of the server code to instead live within the
pages entrypoints.
TLDR;
- Route Modules now accept a `definition` so that they have their full
route definition
- Exports are now hoisted from the userland module by name to prevent
name collisions
- A new `PagesRouteModule` stub was added that will in the future
contain all the pages rendering logic
---------
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
This PR is extracted from https://github.com/vercel/next.js/pull/49942
and mostly contains changes necessary after the Turbopack PR adding the
Node.js production runtime https://github.com/vercel/turbo/pull/4998,
without any of the actual Next Build stuff, in order to be able to merge
both quickly.
* ChunkData moved from tp-dev to tp-core, the ES-serializable part moved
to tp-ecmascript;
* all runtime types moved to tp-ecmascript-runtime
This also upgrades Turbopack to turbopack-230526.2:
* https://github.com/vercel/turbo/pull/5102 <!-- Donny/강동윤 - refactor:
Fix binary bloat caused by `ValueDebugFormat` impl -->
* https://github.com/vercel/turbo/pull/4998 <!-- Alex Kirszenberg -
Node.js production runtime POC -->
This change allows the Turbopack's node router to serve
`_next/static/development/_devMiddlewareManifest.json` file, which is
just the matchers for middleware routes. This is usually handled by the
node dev server anyways, but we were filtering out the route so that
Turbopack could handle it.
But, Turbopack doesn't implement the route matching logic at all, it's
all handled by our node router. So Turbopack can't know what the
middleware matchers are. Having the router just handle this like the
normal dev server is both easy and efficient (I don't want to spin up
_another_ node instance just to transform the matchers into the expected
format).
Fixes WEB-979
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
### What?
This reverts commit 6ebc725fe6 / #50247.
### Why?
#49972 is reverted due to bugs, and I'm retrying it.
### How?
Closes WEB-1072
Closes WEB-1097
Closes NEXT-1156 (as it's reopened by the revert PR)
fix#48469
---------
Co-authored-by: Shu Ding <g@shud.in>
### What?
Implement a CJS optimizer for next-swc
### Why?
x-ref: https://vercel.slack.com/archives/C02HY34AKME/p1684341093462309
Assuming most CJS files are transpiled from ESM, we can mimic
tree-shaking using an AST transform.
### How?
Closes WEB-1072
Fixes#48469
fix NEXT-1156
---------
Co-authored-by: Shu Ding <g@shud.in>
