For a more detailed explanation of the algorithm, refer to the comments
in ppr-navigations.ts. Below is a high-level overview.
### Step 1: Render the prefetched data immediately
Immediately upon navigation, we construct a new Cache Node tree (i.e.
copy-on-write) that represents the optimistic result of a navigation,
using both the current Cache Node tree and data that was prefetched
prior to navigation.
At this point, we haven't yet received the navigation response from the
server. It could send back something completely different from the tree
that was prefetched — due to rewrites, default routes, parallel routes,
etc.
But in most cases, it will return the same tree that we prefetched, just
with the dynamic holes filled in. So we optimistically assume this will
happen, and accept that the real result could be arbitrarily different.
We'll reuse anything that was already in the previous tree, since that's
what the server does.
New segments (ones that don't appear in the old tree) are assigned an
unresolved promise. The data for these promises will be fulfilled later,
when the navigation response is received.
The tree can be rendered immediately after it is created. Any new trees
that do not have prefetch data will suspend during rendering, until the
dynamic data streams in.
### Step 2: Fill in the dynamic data as it streams in
When the dynamic data is received from the server, we can start filling
in the unresolved promises in the tree. All the pending promises that
were spawned by the navigation will be resolved, either with dynamic
data from the server, or `null` to indicate that the data is missing.
A `null` value will trigger a lazy fetch during render, which will then
patch up the tree using the same mechanism as the non-PPR implementation
(serverPatchReducer).
Usually, the server will respond with exactly the subset of data that
we're waiting for — everything below the nearest shared layout. But
technically, the server can return anything it wants.
This does _not_ create a new tree; it modifies the existing one in
place. Which means it must follow the Suspense rules of cache safety.
## To Do
Not all necessarily PR-blocking, since the status quo is that
navigations don't work at all when PPR is enabled
- [x] Figure out how to handle dynamic metadata. Need to switch from
prefetched metadata to final.
- [x] Some mistake related to parallel routes, need to look into failing
tests
Closes NEXT-1894
### What?
Navigating to a layout that is part of a route group that uses route
interception currently will trigger a 404 error if the route group
doesn't define a `default` segment.
### Why?
When `next-app-loader` injects fallback defaults into the loader tree,
it does so by first seeing if a default already exists. However it does
this without ignoring route groups, meaning if you have a
`/app/default.tsx` and your interception route is at
`/app/(level1)/(level2)`, it will look for the default at
`/app/(level1)/(level2)/default.tsx`.
When a `default` isn't found, the fallback behavior is to trigger a
`notFound()` error. This means navigating to the intercepting route that
has no `default` for the `children` segment will 404.
### How?
This adjusts the fallback behavior by attempting to find the `default`
by normalizing the segment path, which will ignore route groups. That
way `/app/(level1)/(level2)/default` will first check `/app/default.tsx`
before falling back to `notFound` behavior.
Fixes#59279
Closes NEXT-1813
If the data for a segment is missing when LayoutRouter renders, it
initiates a lazy fetch to patch the cache. This is how all dynamic data
fetching works in the pre-PPR implementation.
For PPR, we won't use this mechanism anymore for regular navigations,
but (at least for now) we will still use it as a fallback behavior if
the server response does not match what we expected to receive.
This commit adds support for asynchronously triggering a lazy fetch, by
unwrapping the segment data promise inside LayoutRouter to check if it's
missing. If so, it will trigger the lazy fetch mechanism.
When PPR is not enabled this should not observably impact behavior.
Closes NEXT-1893
When uploading traces from `.next/trace`, target paths that trigger
compilations were being normalized to paths like
`[project]/../../../../../middleware`. This PR removes the normalization
logic so that the triggers appear as `/middleware` which is easier to
understand.
## What?
Moves the changeSubscription for _document into the finally block,
similar to how the page itself is handled there as well.
This should allow moving the rest of the try block into a separate
function that can be reused for builds too.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-1885
## What?
I'm working on consolidating a bunch of the file writing related pieces
in the Turbopack handling in the dev server so that it can be abstracted
out as it's needed for `next build` too.
These changes make sure that there is a single `writeManifests()`
instead of picking specific manifests to write.
We can optimize this later but for now the overhead of writing them to
disk separately is negligible.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-1884
### What?
There are a bunch of different bugs caused by the same underlying issue,
but the common thread is that performing any sort of router cache update
(either through `router.refresh()`, `revalidatePath()`, or `redirect()`)
inside of a parallel route would break the router preventing subsequent
actions, and not resolve any pending state such as from `useFormState`.
### Why?
`applyPatch` is responsible for taking an update response from the
server and merging it into the client router cache. However, there's
specific bailout logic to skip over applying the patch to a
`__DEFAULT__` segment (which corresponds with a `default.tsx` page).
When the router detects a cache node that is expected to be rendered on
the page but contains no data, the router will trigger a lazy fetch to
retrieve the data that's expected to be there
([ref](5adacb6912/packages/next/src/client/components/layout-router.tsx (L359-L370)))
and then update the router cache once the data resolves
([ref](5adacb6912/packages/next/src/client/components/layout-router.tsx (L399-L404))).
This is causing the router to get stuck in a loop: it'll fetch the data
for the cache node, send the data to the router reducer to merge it into
the existing cache nodes, skip merging that data in for `__DEFAULT__`
segments, and repeat.
### How?
We currently assign `__DEFAULT__` to have `notFound()` behavior when
there isn't a `default.tsx` component for a particular segment. This
makes it so that when loading a page that renders a slot without slot
content / a `default`, it 404s. But when performing a client-side
navigation, the intended behavior is different: we keep whatever was in
the `default` slots place, until the user refreshes the page, which
would then 404.
However, this logic is incorrect when triggering any of the above
mentioned cache node revalidation strategies: if we always skip applying
to the `__DEFAULT__` segment, slots will never properly handle reducer
actions that rely on making changes to their cache nodes.
This splits these different `applyPatch` functions: one that will apply
to the full tree, and another that'll apply to everything except the
default segments with the existing bailout condition.
Fixes#54173Fixes#58772Fixes#54723Fixes#57665
Closes NEXT-1706
Closes NEXT-1815
Closes NEXT-1812
## What?
Ensures `Object.entries` is not called on the `Map`. Seems this only
fails in a very particular case but potentially this fixes other issues
than the one I added in the tests too.
## How?
`Object.entries()` results in an empty array when called on a `Map`.
Created a shared type declaration for the value and removed the
`Object.entries`. Benefit of this is that we can skip the loop as well.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
## What?
Small refactor to move Turbopack hotreloader interface creation to a
separate function: `createHotReloaderTurbopack`.
Renamed `HotReloader` to `HotReloaderWebpack`.
Initially wanted to move `createHotReloaderTurbopack` to a separate file
but it relies on a bunch of in-scope variables so that is not
straightforward. Will do that later.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-1881
This fixes some of headers (and adds associated tests) for pages when
PPR is enabled. Namely, the `Cache-Control` headers are now returning
correctly, reflecting the non-cachability of some requests:
- Requests that postpone (dynamic data is streamed after the initial
static shell is streamed)
- Requests for the Dynamic RSC payload
Additionally, the `X-NextJS-Cache` header has been updated for better
support for PPR:
- Requests that postpone no longer return this header as it doesn't
reflect the cache state of the request (because it streams)
- Requests for the Prefetch RSC now returns the correct cache headers
depending on the segment and pre-postpone state
This also enables the other pathnames in the test suites 🙌🏻
Closes NEXT-1840
When PPR is enabled, the CacheNodeSeedData and FlightRouterState
contained in an RSC payload (whether it's static or dynamic) should
always have the same tree structure.
The only reason the two trees would ever be different is if the server
partially renders with `loading.tsx`, but that behavior doesn't exist
when PPR is enabled. (In that case, the CacheNodeSeedData would describe
only a subset of FlightRouterState.)
The behavior is already correct but to make the types a bit closer, this
changes the `parallelRoutes` slot of CacheNodeSeedData to be
non-nullable; if there are no children, it should be an empty object.
It's not semantically important, I'm only changing it to make the types
more consistent, since we typically traverse both trees in parallel.
Eventually we will probably merge these into a single transport type.
Closes NEXT-1868
The PPR implementation of navigateReducer is expected to diverge
significantly from the existing, non-PPR implementation. So this forks
them into two separate functions. This will be easier to maintain than
two different implementations inside the same function, especially
considering we don't expect any more changes to the non-PPR
implementation.
This also reduces the chances we'll introduce an accidental regression
into the non-PPR version, which is the stable one that all users (except
for the ones dogfooding PPR) are currently using.
For now, the two implementations are identical. I'll start making
changes in subsequent PRs.
Only one implementation will be included in the final build; the other
one will be dead code eliminated because the feature check is statically
inlined at build time:
```js
export const navigateReducer = process.env.__NEXT_PPR
? navigateReducer_PPR
: navigateReducer_noPPR
```
Closes NEXT-1856
This ensures that `export const dynamic = 'force-static'` is properly
honored when a page contains fetches with `cache: 'no-store'`, `cache:
'no-cache'` or `next: { revalidate: 0 }`.
Closes NEXT-1858
Implementation of feature request opened here -
https://github.com/vercel/next.js/discussions/59427
Approach:
~~We are using micromatch in the csrf protection step of actionHandler
to allow for wildcard domains passed in allowedDomains. This is the same
library used for matching domains for remote images.~~
If any of the allowed domains match the origin of the request, we skip
the downstream error thrown for csrf protection.
Edit:
Micromatch is not available in this context as it is only compatible
with Node. This codepath can be run from the edge, so we need to rely on
vanilla js compatible code only.
Instead of falling back to allowing the user to pass in a regex, which
can be somewhat insecure, we opt into continuing to use a wildcard
pattern from a configuration standpoint and instead use a simple
function that matches on wildcards using string comparison and
iteration.
Ideally, Micromatch can be retrofitted to work in non-Node settings and
this piece of code can be replaced in the future, without deprecating or
changing the next.config interface.
---------
Co-authored-by: Josh Story <story@hey.com>
Adds a new field `prefetchRsc` to CacheNode that will be used by the PPR
implementation. It represents a static version of the segment that can
be showed immediately, and may or may not contain dynamic holes. It's
prefetched before a navigation occurs. During rendering, we will choose
whether to render `rsc` or `prefetchRsc` with `useDeferredValue`.
As with the `rsc` field, a value of `null` means no value was provided.
In this case, the LayoutRouter will go straight to rendering the `rsc`
value; if that one is also missing, it will suspend and trigger a lazy
fetch.
The non-PPR implementation will never set this value.
This PR adds the field to the CacheNode type but doesn't implement any
of the behavior yet. Mostly this involves updating the router reducer
unit tests.
Closes NEXT-1855
Passing `NODE_OPTIONS='--inspect'` was failing because a sub-process was
getting created (`getRegistry()`) which would trigger a "address already
in use" error when the process inherited the same debugger port from the
parent process.
I had to disable this test in
https://github.com/vercel/next.js/pull/59508 because it wasn't passing,
CI was just skipping it. This PR fixes the behavior and re-enables the
test.
Fixes#55862
Closes NEXT-1854
I want to give this field a name that's bit less generic and
distinguishes it from `lazyData` (because that one has a different type
and is a special case we want to eventually remove).
I'm also about to add an optional `prefetchRsc` field that represents a
prefetched version of the same value. The common suffix is meant to
communicate how they are related.
Doing this rename in its own PR because it's a pure find-and-replace,
whereas the later steps are not.
Closes NEXT-1846
This updates the `collectGenerateParams` method to use a loop rather
than being recursive as well as updating some of the Typescript types.
This is a follow up of #59420.
Closes NEXT-1839
`CacheNode.data` is used to lazily kick off a request during render, and
represents the result of the entire Flight response. It doesn't
correspond directly to the RSC data of the cache node itself — that's
`subTreeData`. To complicate things further, I'm about to add another
field to CacheNode that represents prefetched RSC data.
To make it a little less confusing, I've renamed the `data` field to
`lazyData`. Still not perfectly clear on first glance, but it's at least
more specific. With PPR, the goal is to remove the lazy data fetching
mechanism in favor of initiating the request immediately upon
navigation. So this field will eventually go away.
In the next PR, I will rename `subTreeData`, too. Perhaps something with
"rsc" in the name so it's less generic than "data".
Closes NEXT-1843