### What?
Calls to `revalidateTag` when using the data cache would sometimes be ignored. This would result in stale data being displayed even after manually triggering a revalidation.
### Why?
Revalidation calls are pushed onto a `pendingRevalidates` array and, in the case of a route handler, are awaited [here](8b11264ea9/packages/next/src/server/send-response.ts (L53-L55)).
Previously this took place as part of the former `pipeReadable` implementation ([ref](57bb52d37d/packages/next/src/server/send-response.ts (L49))) which awaited the promise before calling res.end ([ref](57bb52d37d/packages/next/src/server/pipe-readable.ts (L99-L105))). There was a subtle change in behavior here in a recent refactor that seems to cause a race between awaiting the promise and ending the response, which results in this unpredictable behavior.
### How?
This applies similar logic that used to be in the `pipeReadable` function to the new `pipeToNodeResponse` function. Namely, we plumb `waitUntil` into the writeable stream close handler and await it before calling `res.end()`.
This is a very difficult edge case to test, and as such resorted to rigorous manual testing when deployed to an environment that uses the data cache.
Fixes#52962Fixes#57632
### What?
In #57062, a deprecation warning was added for usage of `nextConfig.images.domains`. This warning was erroneously showing up for users that had an `assetPrefix` configured.
### Why?
When an `assetPrefix` specified, during assignment for `nextConfig` default values, we push to `nextConfig.images.domains` so that the user doesn't receive errors about hostnames not being configured ([Original PR](https://github.com/vercel/next.js/pull/29307)). This means people are indirectly being opted into a deprecated config, and receiving a warning because of it.
### How?
We no longer push any defaults to the deprecated config option. Additionally, this applies the same defaults handling to `remotePatterns,` as otherwise people using this new config will run into the same issue as this block was originally intended to fix.
Fixes#58074
After the changes in #58669, it's now possible for these `DynamicUsageErrors` to be caught in the streaming renderer. We didn't have any logic to handle this happening inside the existing catch block, so this adds a check for that specific error and re-throws it so that it properly bubbles to `exportPage`. `exportPage` checks the error to see if it's a `DynamicUsageError` and if so, marks the page as dynamic.
(This resolves an issue with particular scenarios hanging the build, caught by a few of our tests: [example](https://github.com/vercel/next.js/actions/runs/6949079695/job/18907597751?pr=58744), [example](https://github.com/vercel/next.js/actions/runs/6948412805/job/18904387924))
### What?
When a layout segment forces dynamic rendering (such as with
`force-dynamic` or `revalidate: 0`), navigating to sub-pages of that
layout will attempt to re-render the layout, also resulting in side
effects re-running. This means if your layout relies on a data fetch and
you render the result of that data in the layout, it will unexpectedly
change when navigating between sub-paths, as described in #57326.
As a separate issue (but caused by the same underlying mechanism), when
using `searchParams` on a dynamic page, changes to those search params
will be erroneously ignored when navigating, as described in #57075
### Why?
As a performance optimization we generate static prefetch files for
dynamic segments ([original
PR](https://github.com/vercel/next.js/pull/54403)). This makes it so
that when prefetching is turned on, the prefetch can be served quickly
from the edge without needing to invoke unnecessarily. We're able to
eagerly serve things that can be safely prefetched. This is nice for
cases where a path has a `loading.js` that we can eagerly render while
waiting for the dynamic data to be loaded.
This causes a problem with layouts that opt into dynamic rendering: when
the page loads and a prefetch is kicked off for the sub-page, it'll load
the static prefetch, which won't be generated with the same router state
as the dynamically rendered page. This causes a mismatch between the two
trees, and when navigating within the same segment, a refetch will be
added to the router because it thinks that it's navigating to a new
layout.
This also causes issues for dynamic pages that use `searchParams`. The
static prefetch will be generated without any knowledge of search
params, and when the prefetch occurs, we still match to the prefetch
generated without search params. This will make the router think that no
change occurs, and the UI will not update to reflect the change.
### How?
There's ongoing work by @acdlite to refactor the client router.
Hopefully it will be easier to re-land this once that work is finished.
For now, I've reverted the behavior as it doesn't seem to be worth the
bugs it currently causes. I've also added tests so that when we do
re-land this behavior, we can catch these subtleties.
Fixes#57326Fixes#57075
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### What?
Various fixes in handling of externals in node.js
* add extensions to builtin externals to allow node.js ESM to work
* improve the auto-externals logic to detect more edge cases
* prepare for `esmExternals` support, but that's blocked by client
manifest missing the async flag
* currently only `esmExternals: false` is supported
### Why?
### Turbopack Changes
* https://github.com/vercel/turbo/pull/6531 <!-- Leah - fix(turbopack):
postcss should be applied to `@import`ed CSS files -->
* https://github.com/vercel/turbo/pull/6530 <!-- Tobias Koppers - fix
node.js externals -->
---------
Co-authored-by: Leah <github.leah@hrmny.sh>
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
### What?
* include path and title in errors
* don't block the page for errors in node_modules
### Why?
### How?
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
## Based on #58666
*I submitted this stack as separate PRs so I could run CI but I want to
land them as a single unit. So I've left all but this last one in draft
mode. You can review them commit-by-commit.*
We render nested layouts in parallel on the server. This means we should
be able to create a CacheNode entry for every layout in the tree as soon
as the Flight response is received by the client, even before the nested
layouts have streamed in. Currently, we wait until the nested layouts
start rendering before we write them into the cache, which prevents us
from performing certain optimizations. That's because the CacheNodes are
sent as a prop to LayoutRouter; the only way to unwrap the CacheNode is
to wait for LayoutRouter to render.
In previous PRs, I updated the server to create a top-level data
structure that contains all the CacheNodes for the entire tree. This PR
updates the client side to receive the nodes and write them into the
cache.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Currently to make inline-defined Server Actions work, the compiler hoists the actual `"use server"` function to the module level and convert the inlined function to a parentheses expression that creates a noop wrapper function and wraps it with the proxy. This works fine however expressions are still different from declarations (#57392). So there're some details that can't be aligned well.
With this change, we're going to make the compilation for the two types of inline-defined Server Actions more robust and more lightweight:
#### 1. Expressions
```jsx
const action = async () => { "use server" ... }
const action = async function () { "use server" ... }
const action = async function named () { "use server" ... }
foo={async function () { "use server" ... }}
...
```
These expressions can directly be replaced with a new expression `createActionProxy("id", hoisted_action)`. A `.bind(...)` member call can be followed if it needs to bind any variables from the closure level.
#### 2. Declarations
```js
async function named () { "use server" ... }
```
In this case, we replace all the same `named` idents to be the expression `createActionProxy("id", hoisted_action)`, and removed that function declaration.
With these changes, these will be fewer structural changes to the AST and the code is more performant.
The PR also changes it to use React's `registerServerReference` method directly instead of our in-house implementation inside `createActionProxy`.
Another small change is to stabilize the comment header to use `BTreeMap` inside the SWC transform. Otherwise the test snapshots will randomly mismatch.
Closes#57392.
## What?
Enables `experimental.windowHistorySupport` support for Turbopack.
## How?
Turbopack has a guard for unknown options (allow-list). Added the option
to the list of supported configurations.
No additional changes are needed as Turbopack already uses the
define-env config.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
### What?
I noticed that code that was responsible for download the SWC Wasm
fallback wasn't bullet proof and there was a chance for a race
condition. The reason was that both `write` and `close` from a write
stream are async operations and it's best to wait for them to complete,
otherwise the promise returned from `body.pipeTo` could resolve before
all data has been written and the stream was closed. Right after that it
calls `rename` and it could happen that, at that point, `tempFile`
doesn't contain all the data yet which means an empty file may be
renamed into another file ending up with `path.join(cacheDirectory,
tarFileName)` being empty as well.
The fix is to wait for both `write` and `close` to be completed which
eliminates the potential race condition between the fetch and the
rename.
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
### What?
We are experimenting with `lightningcss`. This is about replacing
`swc_css` with `lightningcss` in turbopack, and the main reason for this
is to reduce the maintenance burden.
But when I tried, it introduced several regressions, so I'm putting it
behind an experimental flag.
You can enable `lightningcss` mode for **turbopack** by adding a flag to
the next config file.
```js
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {
experimental: {
useLightningcss: true,
},
}
module.exports = nextConfig
```
Note that this is only for turbopack because we were not using `swc_css`
for non-turbopack mode of next.js
x-ref:
https://vercel.slack.com/archives/C03EWR7LGEN/p1700025496732229?thread_ts=1700019629.866549&cid=C03EWR7LGEN
### Why?
We should avoid regressions.
### How?
---
turbopack PR: https://github.com/vercel/turbo/pull/6456
Closes PACK-1966
---------
Co-authored-by: Leah <github.leah@hrmny.sh>
This updates the rust toolchain to 2023-11-16 and:
- Removes now-unnecessary `#![feature(async_fn_in_trait)]`
- Applies auto-fixable lint fixes
- Uses `Cargo.toml` new `lint` section instead of command line rustc flags
Test Plan: Tested with updated turbo in a create-next-app
Closes PACK-1979
Co-authored-by: OJ Kwon <1210596+kwonoj@users.noreply.github.com>
Refactors createComponentTree to return a top-level tree of all the
subtree data in the entire response. Although we were already collecting
this data, it's passed to the client as a prop LayoutRouter, which means
it can only be unwrapped by rendering the React tree.
Instead, we will hoist all the subtree data (i.e. the React nodes that
represent the nested layouts) into a top-level object that can be
immediately unwrapped by the client when it processes the response.
Then, the client will use this tree to eagerly populate the cache nodes,
rather than waiting for the LayoutRouter to lazily populate the cache
during render, like we do today.
This PR does not implement the client-side changes yet; it only creates
the new data structure, which I've named CacheNodeSeedData. The rest
will come in subsequent PRs.
---------
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This adds an additional item to the FlightData type. The type of the
field is currently `null`, but eventually this slot will represent a
tree of data that is used to seed the cache nodes.
This is a fragile change because the FlightData type is not covered by
TypeScript. This was an intentional decision to optimize the size of the
Flight payload. It's made more tricky because there are many places in
the codebase that access the fields of FlightData using direct indexing,
e.g. `flightData[0]`.
To minimze the number of places I needed to update, I added the new
field to the end of the array. However, many places access the fields
using negative indexing (via the `slice` method), so I needed to update
all of those. I also had to change any place that checked the length of
the array.
In the future, when we introduce clever types like this that are
intentionally unsound, we should contain the unsoundness to a single
module by only accessing the type with getter functions. Something like:
```js
const treePath = getTreePatch(flightData);
const seedData = getCacheNodeSeedData(flightData);
```
and so on.
That way when we add a new field like this, we don't have to carefully
update every single place that accesses the type. (TypeScript lacks the
ability to mark a type as opaque, unfortunately, but I believe you can
simulate it in other ways. This is one feature I miss from Flow.)
I considered adding these getter methods as part of this PR, but since
we're in the middle of a larger refactor of the Flight response type,
I'd prefer to change as little as possible for now until we can land an
MVP of PPR for client navigations. (That being said, if we struggle to
land this, I'll reconsider.)
My strategy for finding the places that needed to update was to change
the type of FlightDataPath to a nonsense type (e.g. number) and track
down all the TypeScript errors. I also searched for all references to
variables named `flightDataPath` or similar. I also referred to a prior
PR, #42791, which added the `head` field to the FlightDataPath type.
Because of the nature of the change, there's a moderately elevated risk
that this will break something. However, if something does slip through
the cracks, it's likely to fail fast, given how common most of these
paths are; I expect any egregious oversights would be caught by our e2e
test suite.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
We've heard confusion around Route Handlers being cached by default for
`GET`s. While we do have a section on this in the docs, I've made this
more explicit by making the examples default to dynamic, while
mentioning the defaults.
### What?
I changed the behavior of fetch() when 'force-dynamic' is specified in the `dynamic` of Route Segment Config to be similar to when 'force-no-store' is specified in `fetchCache`.
### Why?
The document (https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config#dynamic) contains an explanation that when 'force-dynamic' is specified for `dynamic`, it will behave equivalently to the following:
> Setting the segment config to export const fetchCache = 'force-no-store'
I tried to correct it because it was not actually behaving this way.
### How?
When determining if `fetchCache` is 'force-no-store', I have modified the code to also check the `dynamic` setting.
Fixes#47033
Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
### What?
When using the file system cache with `isrMemoryCacheSize: 0`, time-based revalidation is not working, and the file is constantly updated. I have also added some debug logging to mirror that in the `fetch-cache` handler
Detailed explanation in #58507
### Why?
The cached object's tags are incorrectly accessed, causing the cache to be rewritten every hit. This is catastrophic for a caching system that relies on file modification timestamps. The tags are one level up in the object from where [they are currently being accessed](9ab8828f72/packages/next/src/server/lib/incremental-cache/file-system-cache.ts (L178)).
Below shows a cached fetch representation on disk. When written, the tags reside at `obj.tags` instead of `obj.data.tags`
```json
{
"kind": "FETCH",
"data": {
"headers": {
"connection": "keep-alive",
"content-encoding": "br",
"content-type": "application/json; charset=utf-8",
"date": "Wed, 15 Nov 2023 21:17:42 GMT",
"server": "nginx/1.18.0 (Ubuntu)",
"transfer-encoding": "chunked",
"vary": "Accept-Encoding"
},
"body": "[SNIP]",
"status": 200,
"url": "https://timeapi.io/api/Time/current/zone?timeZone=UTC"
# this is where the current code is trying to pull the tags
# "tags": [ "time-with-fetch" ]
},
"revalidate": 20,
# tags actually live here
"tags": [
"time-with-fetch"
]
}
```
Fixes#58507
This PR makes sure that the `start-dev-server` trace span is recorded to the `.next/trace` file and uploaded to the telemetry endpoint when opted in.
Before this PR, the `start-dev-server` span was recorded in `next-dev` in the CLI. However, this would never be written to the `.next/trace` file since it happens in a different process (not the child process for next server). After this PR, the trace span happens in the next-server child process and will be recorded in the `.next/trace` file along with the other events produced by the dev server.