This updates the rust toolchain to 2023-11-16 and:
- Removes now-unnecessary `#![feature(async_fn_in_trait)]`
- Applies auto-fixable lint fixes
- Uses `Cargo.toml` new `lint` section instead of command line rustc flags
Test Plan: Tested with updated turbo in a create-next-app
Closes PACK-1979
Co-authored-by: OJ Kwon <1210596+kwonoj@users.noreply.github.com>
Refactors createComponentTree to return a top-level tree of all the
subtree data in the entire response. Although we were already collecting
this data, it's passed to the client as a prop LayoutRouter, which means
it can only be unwrapped by rendering the React tree.
Instead, we will hoist all the subtree data (i.e. the React nodes that
represent the nested layouts) into a top-level object that can be
immediately unwrapped by the client when it processes the response.
Then, the client will use this tree to eagerly populate the cache nodes,
rather than waiting for the LayoutRouter to lazily populate the cache
during render, like we do today.
This PR does not implement the client-side changes yet; it only creates
the new data structure, which I've named CacheNodeSeedData. The rest
will come in subsequent PRs.
---------
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This adds an additional item to the FlightData type. The type of the
field is currently `null`, but eventually this slot will represent a
tree of data that is used to seed the cache nodes.
This is a fragile change because the FlightData type is not covered by
TypeScript. This was an intentional decision to optimize the size of the
Flight payload. It's made more tricky because there are many places in
the codebase that access the fields of FlightData using direct indexing,
e.g. `flightData[0]`.
To minimze the number of places I needed to update, I added the new
field to the end of the array. However, many places access the fields
using negative indexing (via the `slice` method), so I needed to update
all of those. I also had to change any place that checked the length of
the array.
In the future, when we introduce clever types like this that are
intentionally unsound, we should contain the unsoundness to a single
module by only accessing the type with getter functions. Something like:
```js
const treePath = getTreePatch(flightData);
const seedData = getCacheNodeSeedData(flightData);
```
and so on.
That way when we add a new field like this, we don't have to carefully
update every single place that accesses the type. (TypeScript lacks the
ability to mark a type as opaque, unfortunately, but I believe you can
simulate it in other ways. This is one feature I miss from Flow.)
I considered adding these getter methods as part of this PR, but since
we're in the middle of a larger refactor of the Flight response type,
I'd prefer to change as little as possible for now until we can land an
MVP of PPR for client navigations. (That being said, if we struggle to
land this, I'll reconsider.)
My strategy for finding the places that needed to update was to change
the type of FlightDataPath to a nonsense type (e.g. number) and track
down all the TypeScript errors. I also searched for all references to
variables named `flightDataPath` or similar. I also referred to a prior
PR, #42791, which added the `head` field to the FlightDataPath type.
Because of the nature of the change, there's a moderately elevated risk
that this will break something. However, if something does slip through
the cracks, it's likely to fail fast, given how common most of these
paths are; I expect any egregious oversights would be caught by our e2e
test suite.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
We've heard confusion around Route Handlers being cached by default for
`GET`s. While we do have a section on this in the docs, I've made this
more explicit by making the examples default to dynamic, while
mentioning the defaults.
### What?
I changed the behavior of fetch() when 'force-dynamic' is specified in the `dynamic` of Route Segment Config to be similar to when 'force-no-store' is specified in `fetchCache`.
### Why?
The document (https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config#dynamic) contains an explanation that when 'force-dynamic' is specified for `dynamic`, it will behave equivalently to the following:
> Setting the segment config to export const fetchCache = 'force-no-store'
I tried to correct it because it was not actually behaving this way.
### How?
When determining if `fetchCache` is 'force-no-store', I have modified the code to also check the `dynamic` setting.
Fixes#47033
Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
### What?
When using the file system cache with `isrMemoryCacheSize: 0`, time-based revalidation is not working, and the file is constantly updated. I have also added some debug logging to mirror that in the `fetch-cache` handler
Detailed explanation in #58507
### Why?
The cached object's tags are incorrectly accessed, causing the cache to be rewritten every hit. This is catastrophic for a caching system that relies on file modification timestamps. The tags are one level up in the object from where [they are currently being accessed](9ab8828f72/packages/next/src/server/lib/incremental-cache/file-system-cache.ts (L178)).
Below shows a cached fetch representation on disk. When written, the tags reside at `obj.tags` instead of `obj.data.tags`
```json
{
"kind": "FETCH",
"data": {
"headers": {
"connection": "keep-alive",
"content-encoding": "br",
"content-type": "application/json; charset=utf-8",
"date": "Wed, 15 Nov 2023 21:17:42 GMT",
"server": "nginx/1.18.0 (Ubuntu)",
"transfer-encoding": "chunked",
"vary": "Accept-Encoding"
},
"body": "[SNIP]",
"status": 200,
"url": "https://timeapi.io/api/Time/current/zone?timeZone=UTC"
# this is where the current code is trying to pull the tags
# "tags": [ "time-with-fetch" ]
},
"revalidate": 20,
# tags actually live here
"tags": [
"time-with-fetch"
]
}
```
Fixes#58507
This PR makes sure that the `start-dev-server` trace span is recorded to the `.next/trace` file and uploaded to the telemetry endpoint when opted in.
Before this PR, the `start-dev-server` span was recorded in `next-dev` in the CLI. However, this would never be written to the `.next/trace` file since it happens in a different process (not the child process for next server). After this PR, the trace span happens in the next-server child process and will be recorded in the `.next/trace` file along with the other events produced by the dev server.
### What?
When using a `Request` object with fetch, we'll log a warning indicating that using the `cache` property in addition to `revalidate` is unsupported.
### Why?
`Request` sets some defaults on the request init, such as `cache: "default"`. This makes the warning confusing and there's no way to avoid it aside from switching the resource argument to be a URL string instead.
### How?
This keeps existing behavior but omits the log in the case where a request object is used and no explicit cache overrides are specified.
Fixes#58109
### Issue
In the client components world, when you're using `next/dynamic` with `ssr: false` to split chunks in pages of edge runtime, you could get the dynamic imported module still bundled in the server bundle for edge runtime. This could easily hit the bundle limit on edge runtime if you're loading a large size of non-SSR module.
This is caused by the whole chunk is still being included when we're creating the client entry. Since the client entry is imported eagerily, webpack will bundle all the modules under it, unless it's explicitly marked not being included.
### Fix
For client components, SSR rendering layer of bundle, non-SSR `next/dynamic` calls, we're transform the result of `dynamic()` call from to conditional import the dynamic loaded module.
From
```js
dynamic(() => import(...))
```
To
```js
dynamic(() => {
require.resolveWeak(...)
}, { ssr: false })
```
This will only be applied to SSR layer bundle client components non-SSR `next/dynamic` calls and only when webpack is bundling since turbopack doesn't need this. In this way, the server side will be stripped but it can still enter the module graph since we need to traverse if there's SA in client modules with using webpack API `require.resolveWeak`. And for client side bundle will still include the actual module.
Close NEXT-1703
I'm working on a refactor to seed the CacheNodes as soon as the Flight
payload is received, rather than lazily during the render phase. This
means we no longer need to pass a child element prop to LayoutRouter via
childProp.
ChildProp includes two fields: a segment and a child element. The child
element is the part that will soon be removed, because we'll instead
always read from the cache nodes.
But even after this refactor, we still need to pass the segment to
LayoutRouter. So as an incremental step, I've inlined both fields into
separate props:
- childProp.current -> initialChildNode. This will be removed in a later
step in favor of reading from the cache nodes. In fact, we already
always read from the cache nodes — childProp is ignored completely once
the cache node is populated, hence the updated name.
- childProp.segment -> childPropSegment. This probably isn't the best
name anymore but I'll leave renaming until later once more of this
refactor has settled.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: @BRKalow <bryce@clerk.dev>
### What?
A number of our customers have been experiencing issues stemming from an
`x-forwarded-host` header that doesn't match the `host` header.
### Why?
[This PR](https://github.com/vercel/next.js/pull/57815) removes
functionality which sets `x-forwarded-host` to `req.headers['host']` and
relies solely on the server's hostname and port.
This can be seen locally when visiting the app via a localhost
subdomain.
The `x-forwarded-host` header will remain as `localhost:${port}` while
the actual requested host will contain the subdomain.
### Related
- https://github.com/vercel/next.js/pull/57815#issuecomment-1808496790
---------
Co-authored-by: BRKalow <bryce@clerk.dev>
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
### What?
When navigating between pages (via `prefetch: false`) within a dynamic
segment, the shared layout is re-rendered. This can cause unexpected
behavior like layout data changing when navigating between child
segments.
### Why?
When prefetch is false, we're currently opting into an "optimistic
navigation" codepath, which will optimistically render layout-routers up
to the point where data is missing, while kicking off data fetches. It
attempts to determine where refetching needs to happen by traversing the
router cache nodes and checking where data is missing. However, it
locates these cache nodes by using "segments" obtained by
[deconstructing the
URL](https://github.com/vercel/next.js/blob/fix/optimistic-bailout/packages/next/src/client/components/router-reducer/reducers/navigate-reducer.ts#L142),
which won't accurately contain dynamic segment data. For ex, `/en` which
corresponds with `/app/[lang]/page.tsx` will have a cache node key of
`lang|en|d`, not `en`. Similarly, the optimistic tree that gets
constructed will also be incorrect, since it uses the URL segment.
### How?
My initial fix was to match the dynamic segment against the segment
constructed by the URL. But after discussion with @sebmarkbage and the
team, it seems more correct to remove the optimistic case all together
as there's no guarantee that the url will actually match to that
segment.
Fixes#50670
---------
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### What?
Using `revalidateTag` or `revalidatePath` in a route handler will not
currently opt the handler into dynamic behavior. This means that if you
use these APIs and don't opt into dynamic behavior by some other means,
the revalidation call won't do anything as the route handler will be
served statically.
### Why?
During static generation, we do not currently indicate that usage of
these APIs should opt into dynamic usage.
### How?
This updates `revalidateTag` to throw a `DynamicUsageError` (similar to
our other scenarios, such as search params bailout, headers/cookies, or
fetch + revalidate/no-store)
Closes NEXT-1712
This serves to address a few problems as it relates to headers handling within the App Page render pipeline.
1. During static generation, we should not write to the response directly as the revalidation for static responses is performed out of band from the actual request/response as a stale version can be served early. This is partially addressed by modifying the `onHeaders` to only write when there it is not during static generation, but long term I'd hope to see that we can instead pass a immutable request and `null` for the response to help indicate to the render pipeline that it should instead persist the data into the `RenderResult` objects. Alternatively, the response could use a mocked version that would then supersede the `RenderResult` and be used instead.
2. The types for rendering App Pages were merged incorrectly, so this additionally restructures the way that we initialize those renderers such that only the options that are applicable to each renderer is passed down.
