After the changes in #58669, it's now possible for these `DynamicUsageErrors` to be caught in the streaming renderer. We didn't have any logic to handle this happening inside the existing catch block, so this adds a check for that specific error and re-throws it so that it properly bubbles to `exportPage`. `exportPage` checks the error to see if it's a `DynamicUsageError` and if so, marks the page as dynamic.
(This resolves an issue with particular scenarios hanging the build, caught by a few of our tests: [example](https://github.com/vercel/next.js/actions/runs/6949079695/job/18907597751?pr=58744), [example](https://github.com/vercel/next.js/actions/runs/6948412805/job/18904387924))
### What?
When a layout segment forces dynamic rendering (such as with
`force-dynamic` or `revalidate: 0`), navigating to sub-pages of that
layout will attempt to re-render the layout, also resulting in side
effects re-running. This means if your layout relies on a data fetch and
you render the result of that data in the layout, it will unexpectedly
change when navigating between sub-paths, as described in #57326.
As a separate issue (but caused by the same underlying mechanism), when
using `searchParams` on a dynamic page, changes to those search params
will be erroneously ignored when navigating, as described in #57075
### Why?
As a performance optimization we generate static prefetch files for
dynamic segments ([original
PR](https://github.com/vercel/next.js/pull/54403)). This makes it so
that when prefetching is turned on, the prefetch can be served quickly
from the edge without needing to invoke unnecessarily. We're able to
eagerly serve things that can be safely prefetched. This is nice for
cases where a path has a `loading.js` that we can eagerly render while
waiting for the dynamic data to be loaded.
This causes a problem with layouts that opt into dynamic rendering: when
the page loads and a prefetch is kicked off for the sub-page, it'll load
the static prefetch, which won't be generated with the same router state
as the dynamically rendered page. This causes a mismatch between the two
trees, and when navigating within the same segment, a refetch will be
added to the router because it thinks that it's navigating to a new
layout.
This also causes issues for dynamic pages that use `searchParams`. The
static prefetch will be generated without any knowledge of search
params, and when the prefetch occurs, we still match to the prefetch
generated without search params. This will make the router think that no
change occurs, and the UI will not update to reflect the change.
### How?
There's ongoing work by @acdlite to refactor the client router.
Hopefully it will be easier to re-land this once that work is finished.
For now, I've reverted the behavior as it doesn't seem to be worth the
bugs it currently causes. I've also added tests so that when we do
re-land this behavior, we can catch these subtleties.
Fixes#57326Fixes#57075
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### What?
Various fixes in handling of externals in node.js
* add extensions to builtin externals to allow node.js ESM to work
* improve the auto-externals logic to detect more edge cases
* prepare for `esmExternals` support, but that's blocked by client
manifest missing the async flag
* currently only `esmExternals: false` is supported
### Why?
### Turbopack Changes
* https://github.com/vercel/turbo/pull/6531 <!-- Leah - fix(turbopack):
postcss should be applied to `@import`ed CSS files -->
* https://github.com/vercel/turbo/pull/6530 <!-- Tobias Koppers - fix
node.js externals -->
---------
Co-authored-by: Leah <github.leah@hrmny.sh>
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
### What?
* include path and title in errors
* don't block the page for errors in node_modules
### Why?
### How?
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
## Based on #58666
*I submitted this stack as separate PRs so I could run CI but I want to
land them as a single unit. So I've left all but this last one in draft
mode. You can review them commit-by-commit.*
We render nested layouts in parallel on the server. This means we should
be able to create a CacheNode entry for every layout in the tree as soon
as the Flight response is received by the client, even before the nested
layouts have streamed in. Currently, we wait until the nested layouts
start rendering before we write them into the cache, which prevents us
from performing certain optimizations. That's because the CacheNodes are
sent as a prop to LayoutRouter; the only way to unwrap the CacheNode is
to wait for LayoutRouter to render.
In previous PRs, I updated the server to create a top-level data
structure that contains all the CacheNodes for the entire tree. This PR
updates the client side to receive the nodes and write them into the
cache.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Currently to make inline-defined Server Actions work, the compiler hoists the actual `"use server"` function to the module level and convert the inlined function to a parentheses expression that creates a noop wrapper function and wraps it with the proxy. This works fine however expressions are still different from declarations (#57392). So there're some details that can't be aligned well.
With this change, we're going to make the compilation for the two types of inline-defined Server Actions more robust and more lightweight:
#### 1. Expressions
```jsx
const action = async () => { "use server" ... }
const action = async function () { "use server" ... }
const action = async function named () { "use server" ... }
foo={async function () { "use server" ... }}
...
```
These expressions can directly be replaced with a new expression `createActionProxy("id", hoisted_action)`. A `.bind(...)` member call can be followed if it needs to bind any variables from the closure level.
#### 2. Declarations
```js
async function named () { "use server" ... }
```
In this case, we replace all the same `named` idents to be the expression `createActionProxy("id", hoisted_action)`, and removed that function declaration.
With these changes, these will be fewer structural changes to the AST and the code is more performant.
The PR also changes it to use React's `registerServerReference` method directly instead of our in-house implementation inside `createActionProxy`.
Another small change is to stabilize the comment header to use `BTreeMap` inside the SWC transform. Otherwise the test snapshots will randomly mismatch.
Closes#57392.
## What?
Enables `experimental.windowHistorySupport` support for Turbopack.
## How?
Turbopack has a guard for unknown options (allow-list). Added the option
to the list of supported configurations.
No additional changes are needed as Turbopack already uses the
define-env config.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
### What?
I noticed that code that was responsible for download the SWC Wasm
fallback wasn't bullet proof and there was a chance for a race
condition. The reason was that both `write` and `close` from a write
stream are async operations and it's best to wait for them to complete,
otherwise the promise returned from `body.pipeTo` could resolve before
all data has been written and the stream was closed. Right after that it
calls `rename` and it could happen that, at that point, `tempFile`
doesn't contain all the data yet which means an empty file may be
renamed into another file ending up with `path.join(cacheDirectory,
tarFileName)` being empty as well.
The fix is to wait for both `write` and `close` to be completed which
eliminates the potential race condition between the fetch and the
rename.
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
### What?
We are experimenting with `lightningcss`. This is about replacing
`swc_css` with `lightningcss` in turbopack, and the main reason for this
is to reduce the maintenance burden.
But when I tried, it introduced several regressions, so I'm putting it
behind an experimental flag.
You can enable `lightningcss` mode for **turbopack** by adding a flag to
the next config file.
```js
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {
experimental: {
useLightningcss: true,
},
}
module.exports = nextConfig
```
Note that this is only for turbopack because we were not using `swc_css`
for non-turbopack mode of next.js
x-ref:
https://vercel.slack.com/archives/C03EWR7LGEN/p1700025496732229?thread_ts=1700019629.866549&cid=C03EWR7LGEN
### Why?
We should avoid regressions.
### How?
---
turbopack PR: https://github.com/vercel/turbo/pull/6456
Closes PACK-1966
---------
Co-authored-by: Leah <github.leah@hrmny.sh>
This updates the rust toolchain to 2023-11-16 and:
- Removes now-unnecessary `#![feature(async_fn_in_trait)]`
- Applies auto-fixable lint fixes
- Uses `Cargo.toml` new `lint` section instead of command line rustc flags
Test Plan: Tested with updated turbo in a create-next-app
Closes PACK-1979
Co-authored-by: OJ Kwon <1210596+kwonoj@users.noreply.github.com>
Refactors createComponentTree to return a top-level tree of all the
subtree data in the entire response. Although we were already collecting
this data, it's passed to the client as a prop LayoutRouter, which means
it can only be unwrapped by rendering the React tree.
Instead, we will hoist all the subtree data (i.e. the React nodes that
represent the nested layouts) into a top-level object that can be
immediately unwrapped by the client when it processes the response.
Then, the client will use this tree to eagerly populate the cache nodes,
rather than waiting for the LayoutRouter to lazily populate the cache
during render, like we do today.
This PR does not implement the client-side changes yet; it only creates
the new data structure, which I've named CacheNodeSeedData. The rest
will come in subsequent PRs.
---------
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This adds an additional item to the FlightData type. The type of the
field is currently `null`, but eventually this slot will represent a
tree of data that is used to seed the cache nodes.
This is a fragile change because the FlightData type is not covered by
TypeScript. This was an intentional decision to optimize the size of the
Flight payload. It's made more tricky because there are many places in
the codebase that access the fields of FlightData using direct indexing,
e.g. `flightData[0]`.
To minimze the number of places I needed to update, I added the new
field to the end of the array. However, many places access the fields
using negative indexing (via the `slice` method), so I needed to update
all of those. I also had to change any place that checked the length of
the array.
In the future, when we introduce clever types like this that are
intentionally unsound, we should contain the unsoundness to a single
module by only accessing the type with getter functions. Something like:
```js
const treePath = getTreePatch(flightData);
const seedData = getCacheNodeSeedData(flightData);
```
and so on.
That way when we add a new field like this, we don't have to carefully
update every single place that accesses the type. (TypeScript lacks the
ability to mark a type as opaque, unfortunately, but I believe you can
simulate it in other ways. This is one feature I miss from Flow.)
I considered adding these getter methods as part of this PR, but since
we're in the middle of a larger refactor of the Flight response type,
I'd prefer to change as little as possible for now until we can land an
MVP of PPR for client navigations. (That being said, if we struggle to
land this, I'll reconsider.)
My strategy for finding the places that needed to update was to change
the type of FlightDataPath to a nonsense type (e.g. number) and track
down all the TypeScript errors. I also searched for all references to
variables named `flightDataPath` or similar. I also referred to a prior
PR, #42791, which added the `head` field to the FlightDataPath type.
Because of the nature of the change, there's a moderately elevated risk
that this will break something. However, if something does slip through
the cracks, it's likely to fail fast, given how common most of these
paths are; I expect any egregious oversights would be caught by our e2e
test suite.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
We've heard confusion around Route Handlers being cached by default for
`GET`s. While we do have a section on this in the docs, I've made this
more explicit by making the examples default to dynamic, while
mentioning the defaults.
### What?
I changed the behavior of fetch() when 'force-dynamic' is specified in the `dynamic` of Route Segment Config to be similar to when 'force-no-store' is specified in `fetchCache`.
### Why?
The document (https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config#dynamic) contains an explanation that when 'force-dynamic' is specified for `dynamic`, it will behave equivalently to the following:
> Setting the segment config to export const fetchCache = 'force-no-store'
I tried to correct it because it was not actually behaving this way.
### How?
When determining if `fetchCache` is 'force-no-store', I have modified the code to also check the `dynamic` setting.
Fixes#47033
Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
### What?
When using the file system cache with `isrMemoryCacheSize: 0`, time-based revalidation is not working, and the file is constantly updated. I have also added some debug logging to mirror that in the `fetch-cache` handler
Detailed explanation in #58507
### Why?
The cached object's tags are incorrectly accessed, causing the cache to be rewritten every hit. This is catastrophic for a caching system that relies on file modification timestamps. The tags are one level up in the object from where [they are currently being accessed](9ab8828f72/packages/next/src/server/lib/incremental-cache/file-system-cache.ts (L178)).
Below shows a cached fetch representation on disk. When written, the tags reside at `obj.tags` instead of `obj.data.tags`
```json
{
"kind": "FETCH",
"data": {
"headers": {
"connection": "keep-alive",
"content-encoding": "br",
"content-type": "application/json; charset=utf-8",
"date": "Wed, 15 Nov 2023 21:17:42 GMT",
"server": "nginx/1.18.0 (Ubuntu)",
"transfer-encoding": "chunked",
"vary": "Accept-Encoding"
},
"body": "[SNIP]",
"status": 200,
"url": "https://timeapi.io/api/Time/current/zone?timeZone=UTC"
# this is where the current code is trying to pull the tags
# "tags": [ "time-with-fetch" ]
},
"revalidate": 20,
# tags actually live here
"tags": [
"time-with-fetch"
]
}
```
Fixes#58507
This PR makes sure that the `start-dev-server` trace span is recorded to the `.next/trace` file and uploaded to the telemetry endpoint when opted in.
Before this PR, the `start-dev-server` span was recorded in `next-dev` in the CLI. However, this would never be written to the `.next/trace` file since it happens in a different process (not the child process for next server). After this PR, the trace span happens in the next-server child process and will be recorded in the `.next/trace` file along with the other events produced by the dev server.
### What?
When using a `Request` object with fetch, we'll log a warning indicating that using the `cache` property in addition to `revalidate` is unsupported.
### Why?
`Request` sets some defaults on the request init, such as `cache: "default"`. This makes the warning confusing and there's no way to avoid it aside from switching the resource argument to be a URL string instead.
### How?
This keeps existing behavior but omits the log in the case where a request object is used and no explicit cache overrides are specified.
Fixes#58109
### Issue
In the client components world, when you're using `next/dynamic` with `ssr: false` to split chunks in pages of edge runtime, you could get the dynamic imported module still bundled in the server bundle for edge runtime. This could easily hit the bundle limit on edge runtime if you're loading a large size of non-SSR module.
This is caused by the whole chunk is still being included when we're creating the client entry. Since the client entry is imported eagerily, webpack will bundle all the modules under it, unless it's explicitly marked not being included.
### Fix
For client components, SSR rendering layer of bundle, non-SSR `next/dynamic` calls, we're transform the result of `dynamic()` call from to conditional import the dynamic loaded module.
From
```js
dynamic(() => import(...))
```
To
```js
dynamic(() => {
require.resolveWeak(...)
}, { ssr: false })
```
This will only be applied to SSR layer bundle client components non-SSR `next/dynamic` calls and only when webpack is bundling since turbopack doesn't need this. In this way, the server side will be stripped but it can still enter the module graph since we need to traverse if there's SA in client modules with using webpack API `require.resolveWeak`. And for client side bundle will still include the actual module.
Close NEXT-1703
I'm working on a refactor to seed the CacheNodes as soon as the Flight
payload is received, rather than lazily during the render phase. This
means we no longer need to pass a child element prop to LayoutRouter via
childProp.
ChildProp includes two fields: a segment and a child element. The child
element is the part that will soon be removed, because we'll instead
always read from the cache nodes.
But even after this refactor, we still need to pass the segment to
LayoutRouter. So as an incremental step, I've inlined both fields into
separate props:
- childProp.current -> initialChildNode. This will be removed in a later
step in favor of reading from the cache nodes. In fact, we already
always read from the cache nodes — childProp is ignored completely once
the cache node is populated, hence the updated name.
- childProp.segment -> childPropSegment. This probably isn't the best
name anymore but I'll leave renaming until later once more of this
refactor has settled.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
