Addition to docs (robots.mdx) - Customize user-agent rules
For issue #59178
Docs: Guidance on allowing multiple user agent rules with varying
allow/disallow.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: Delba de Oliveira <32464864+delbaoliveira@users.noreply.github.com>
Co-authored-by: Sam Ko <sam@vercel.com>
Co-authored-by: Delba de Oliveira <delbabrown@gmail.com>
This pull request updates the dependencies in the cache-handler-redis
example. It updates the versions of the `@neshca/cache-handler`, which
introduces new features.
---------
Co-authored-by: Delba de Oliveira <32464864+delbaoliveira@users.noreply.github.com>
This PR fix `image-component` example viewsource paths and shimmer page
filename. Which would result a Github 404 page in the current live
example.
A flow up fix on this PR #60289
### What?
Adds the name of the test that's running when the browser is started to
the recording.
Also makes `RECORD_REPLAY=1` work without `run-tests.js`
Closes PACK-2206
### What?
This PR adds a new flag called
`experimental.missingSuspenseWithCSRBailout`.
### Why?
Via this PR we can break a build when calling `useSearchParams` without
wrapping it in a suspense boundary.
If no suspense boundaries are present, Next.js must avoid doing SSR and
defer the entire page's rendering to the client. This is not a great
default. Instead, we will now break the build so that you are forced to
add a boundary.
### How?
Add an experimental flag. If a `BailoutToCSRError` error is thrown and
this flag is enabled, the build should fail and log an error, instead of
showing a warning and bail the entire page to client-side rendering.
Closes NEXT-1770
---------
Co-authored-by: Balázs Orbán <info@balazsorban.com>
Co-authored-by: Wyatt Johnson <accounts+github@wyattjoh.ca>
### Fixing a bug
### What?
Custom cache handler doesn't work on Windows
### Why?
It broke in a recent fix, when adding ESM support - #59863. The problem
is not new - dynamic imports consider an absolute path in Windows as a
protocol:
`ERR! Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme
in: file, data are supported by the default ESM loader. On Windows,
absolute paths must be valid file:// URLs. Received protocol 'C:'`
### How?
As a solution, it is necessary to explicitly indicate that it is indeed
an absolute path, for example by adding a / at the beginning, but the
most reliable way is to use pathToFileURL.
Since the logic is repeated in 4 places - I created a common function.
Fixes#58509
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
-->
### What?
This PR refactors organization for the rust side packages to build
`next-swc`.
### Why?
We had some historical legacy around package structures, have ambiguous
name for `core` / `next-core`. One contains swc transform visitor for
the next.js, and the other one is new for turbopack's core next.js
features. In addition to that, there was a package dependency chain
prevents to use `core` in the turobpack / next-swc both, so each time
porting a transformer into turbopack it requires to extract new
dependency to be imported in the both place.
PR touches its organization - while PR is large to touch various files,
the crux is summarized at
2cedd06ea5
:
1. `core` becomes `next-custom-transforms`, also this becomes an
agnostic pkg can be imported in turbopack / wasm / next-swc
2. simplify dependency chain to import next-custom-transforms, organized
as below
```mermaid
flowchart TD
C(next-custom-transforms) --> A(napi)
C(next-custom-transforms) --> B(wasm)
D(next-core) --> A(napi)
E(next-build) --> A(napi)
F(next-api) --> A(napi)
C(next-custom-transforms) --> D
D(next-core) --> F(next-api)
D(next-core) --> E(next-build)
```
`impl CustomTransformer` for the each transform still lives in
`next-core`, so turbopack specific dependency is isolated under
`next-core/build/api`.
Closes PACK-2201
Closes PACK-2202
---------
Co-authored-by: hrmny <8845940+ForsakenHarmony@users.noreply.github.com>
### What?
Previously, if an error such as the client side rendering bailout (not
to be confused with the static rendering bailout which PPR supercedes)
occurs during render, and the `postpone` function was invoked during the
original render, then the `staticGenerationStore` would incorrectly
report that the render did call `postpone`, because the value is not
reset on the render for the error page.
### How?
Returning no error when the standard render is used and returning the
error when the error render was used ensures that we don't warn about
missing postpone data when a client side rendering bailout occurs.
### Looking Ahead
A refactor of the `AsyncLocalStorage` should be done such that the
stores are:
1. Returned by the calling function so we aren't reaching into store
properties at different parts
2. Reorganizing the stores so that they're tied to the invocation
lifetime, not the entire request lifetime, so that operations (like
postpone) should only be available during renders that support postpone,
not all renders during a request.
Closes NEXT-1927
### What?
In the previous PR (#60249), it was found that the case when the
response size is larger than 2MB is not sufficiently covered by tests
[[comment](https://github.com/vercel/next.js/pull/60249#pullrequestreview-1807140518)]
### How?
I added a few more checks.
Just checking the number of API calls is not enough - I believe it is
important to additionally verify what exactly the page received.
If I'm honest, I couldn't find the exact reason why data is loaded after
application start when using `customCacheHandler`, but without it -
during build. It seems like something in the butcher. Therefore, in this
part, I simply configured it for the current version.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
The original motivation of this PR is to get `unstable_cache` into a
state where I can more easily change the implementation of postpone to
be more akin to "dynamic rendering APIs". the existing approach made
typing the staticGenerationStore in a way I wanted to for the planned
changes would have been hard to implement with the current approach. At
the same time this was an opportunity to make the implementation more
efficient and easier to reason about.
reorganizes unstable_cache to improve performance and potentially fix
latent bugs related to nest cache calls
In the original implementation there are repeated defined checks for the
store and relatively complex logic around gathering the cache entry. In
my refactor I fork the implementation based on whether we have a store
or not. Loosely this translates to whether the cache call is for App
Router vs Pages Router however due to a quirk in how we scope inner
cache calls there is an existing and unchanged case where a Pages Router
cached callback runs with a "fake" store that is used to scope some
cache values to prevent inner caching when one cached function calls
another. It should be noted that this fake store technique means that
inner cache calls inside Pages Router will hit the App Router pathway in
unstable cache. This is not great but it is the current behavior and
while I have made some changes that might fix some bugs changing this
felt like a much bigger lift to do in a primarily refactor PR.
This "fake" store can be replaced by a different async store for Pages
Router which we can use to scope the inner environment to not be cached
eventually though it may make more sense to just generalize the
staticGenerationStore into a kind of RenderStore and have it run for
Pages Router too.
I moved as much computation that can be done in the closure around the
cached function out of the cached function and I narrowed the scope of
the run call to make it clear that we really only need to scope the
callback.
I removed function allocations per invocation
I probably fixed a bug in how the revalidate property was refined on the
static generation store. Previously it would be possible to go from
number to false and back again but this doesn't make sense as false is
more like INFINITY in terms of refining to shorter values. This PR
updates this logic to be apparently
Closes NEXT-2028
### What?
When accessing `params` on a `RootLayout`, while also using parallel
routes, two potential errors would occur:
- A `Warning: React.createElement: type is invalid` error when
attempting to render a `NotFound` component that doesn't exist
- A `TypeError: Cannot read properties of undefined` error when
attempting to access params in the root layout.
### Why?
`createComponentTree` will render a duplicate `RootLayout` (to ensure
the `notFound()` fallback in unmatched parallel slots have a
`NotFoundBoundary` to catch them) but it currently doesn't ensure a
`NotFound` component exists nor does it forward `params` to the layout.
### How?
This forwards the params to the `RootLayout` and doesn't render a
`NotFoundComponent` if one doesn't exist. This replaces a few `any`
types with more sound types that would have helped catch these mistakes.
There's still a lot more typing that needs to be done (left a comment
below with some additional details) but I opted to make the minimal
changes related to this issue.
Longer term we should remove this duplicate `RootLayout` (see #60220)
which will require special UI to show unmatched slots (similar to the
error overlay, but less harsh)
Closes NEXT-1909
Fixes#59711
When you have a gaint code base of next.js app, the output client side
loaded js bundle size could be large since most of them are functional,
it might not make sense to display yellow or red color to warn you that
your bundle is too large since they conatin the basic functionality.
We display the previous colored ones with opinionless bold white color
to highlight them
Closes NEXT-2017
Closes NEXT-2015
## What?
Adds instructions for how to run tests with Turbopack and how to run
tests while recording using Replay.io.
I've also updated the instructions to mention `test-dev` and
`test-start` which are the commands we generally use.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-2031
This includes the list of changed modules in the `client-hmr-latency`
event for changes in app router pages as well as when using Turbopack.
For Turbopack, this list is derived from the data structure included in
the `turbopack-message` event. Each of these is originally a module id,
which includes additional context information. For analytics purposes
this is removed (and could maybe surfaced in the future once the webpack
data is also normalized).
Closes PACK-2167
### What?
When running a
[multi-zone](https://github.com/vercel/next.js/tree/canary/examples/with-zones)
app in dev, guest app pages would infinitely reload if you change the
basePath of the host app to the default one (omit basePath settings in
next.config.js) (empty string `""` as per Next.js docs).
### Why?
The HMR upgrade request would fail and get caught into a retry loop. In
the multi-zone case, they fail because the upgrade request would be sent
again for a request that had already been upgraded. This resulted in a
"server.handleUpgrade() was called more than once with the same socket"
error, causing the upgrade request to fail.
Every time a retry occurred, the page would trigger a full refresh since
certain HMR errors cause the browser to reload.
### How?
This ensures the upgrade handler only responds to requests that match
the configured basePath (considering when there is no basePath). Default
basePath for Next.js applications it's an empty string `""`.
Ref: https://nextjs.org/docs/app/api-reference/next-config-js/basePath
Other fixes & updates related to the bug:
- Updated test apps to avoid having issues regarding client & server
mismatch for dates
- Added default use case in e2e tests, where you have a default Next.js
application where the basePath it's the default one and a guest app that
it's being routed by the main one through Next.js rewrites.
Closes NEXT-1797
Fixes#59161Fixes#56615Fixes#54454
---------
Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Remove `chalk` inside `@next/react-dev-overlay`
- `@next/react-dev-overlay` is bundled with Next.js. The usage of
`chalk` inside `@next/react-dev-overlay` was not removed in
https://github.com/vercel/next.js/pull/55992, thus the `chalk` is still
being shipped.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
At the moment, `@next/third-parties` is only working on canary. This can
be a blocker for those trying it out on `latest`. This updates the
experimental note to recommend installing canary.
The note should be removed once the package is stable.
Closes: https://github.com/vercel/feedback/issues/51035
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Re-add precompile `watchpack`
- Next.js has stopped bundling the `watchpack` since
https://github.com/vercel/next.js/pull/50792. Currently, `watchpack` is
a direct dependency of Next.js. The PR adds the pre-compile back.
Co-authored-by: Steven <steven@ceriously.com>
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Remove `find-cache-dir`
- The usage of `find-cache-dir` was removed **4 years ago**
(https://github.com/vercel/next.js/pull/7013) but the dist and the build
script were never removed.
- Remove `@segment/ajv-human-errors`
- The usage of `@segment/ajv-human-errors` was removed in
https://github.com/vercel/next.js/pull/56383 by me. Though the build
script was also removed in that PR, the dist never got removed.
## What?
Adds the missing dependencies for recording the Next.js test suite in
Replay that @jaril added.
### Steps to use it
1. Clear all local replays using `pnpm replay rm-all`
1. Run the test locally using the `RECORD_REPLAY=1` environment
variables. I.e. `RECORD_REPLAY=1 pnpm test-dev
test/e2e/app-dir/app/index.test.ts`
1. Upload all the replays to your workspace using your the API key:
`RECORD_REPLAY_API_KEY=keyhere pnpm replay upload-all`
1. Check the uploaded replays in your workspace, while uploading it
provides the URLs.
You can check if the replay was recorded correctly using `pnpm replay
ls`
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-2014
Per @styfle suggestion, #58038 has been split into multiple PRs for
easier review.
- Enable minification of `@next/font`
- Currently, the `@next/font` is built separately (without minification)
and then copied to the `dist` folder of the Next.js.
### What
Fix bad detection of dynamic route of sitemap metadata route, the swc
AST check should process when the text are detected. But prevuious if
there's text with `generateSitemap` such as comment but not the actual
export it will fail.
### How
Add both checks on metadata loader side and detection helper side
* Only call `generateSitemaps` helper when the export existed
* Fix the helper detection logic (major part of this PR)
Fixes#59698Closes#60344
Closes NEXT-2007
ts version has been identical to the js version
---------
Co-authored-by: Lee Robinson <me@leerob.io>
Co-authored-by: Delba de Oliveira <32464864+delbaoliveira@users.noreply.github.com>
Co-authored-by: Delba de Oliveira <delbabrown@gmail.com>
