Commit graph

100 commits

Author SHA1 Message Date
Tim Neutkens
2916042fca
Change server actions cache default to no-store (#60170)
## What?


Currently there is a bug in Server Actions when you `fetch` as it uses
the same defaults (caching when not specified) as rendering, this causes
some issues as you want to read your writes in Server Actions.

This change adds the `no-store` default for Server Actions, you can
still override it by specifying `cache: 'force-cache'` for example, but
it defaults to `cache: 'no-store'`.

Fixes NEXT-1926

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Adding or Updating Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md


## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->

---------

Co-authored-by: Zack Tanner <zacktanner@gmail.com>
2024-01-04 14:01:50 +01:00
Tim Neutkens
002af4eb3a
Add test for importing client components from server actions (#59615)
## What?

Adds two tests for server actions returning client components:

- (supported) Importing a server action from a server component. That
server action imports a client component and returns it.
- (not supported yet) Importing a server action from a client component.
The server action imports a client component and returns it.

The second case is not supported yet as it would effectively mean a
compilation loop:

`server` -> `client` -> `server` -> `client`

and if that last client component includes another server action it goes
even further:

`server` -> `client` -> `server` -> `client` -> `server` (and if that
server action includes anothe client component it goes further and
further)


Whereas currently it's only `server` -> `client` -> `server`, so it's
limited to that.

In the future we should be able to support this
server->client->server->client loop in Turbopack specifically because
Turbopack has a single module graph.

Importing the client component in a server action that is defined in the
server compiler (i.e. when created inline or when imported from a server
component) it does work correctly already.

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Adding or Updating Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md


## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->


Closes NEXT-1873
2023-12-14 13:23:55 +01:00
Zack Tanner
363c2e8eb7
fix server actions behavior on intercepted routes (#59175)
### What?
When using a server action on an intercepted route, when submitting that
action, you'd expect it to correspond with the page you're currently on.
However if you have route interception set up, and you load the page
rather than the intercepted page, submitting the action would `POST` to
the intercepted page. This would result in a 404 error because the
action ID you're attempting to submit wouldn't be found on the requested
page.

### Why?
Interception routes rely on the `Next-Url` request header to determine
if an interception should occur via a rewrite. However, server actions
are submitted with this header as well, so the rewrite will be applied
to the `POST` request corresponding with a non-existent action, or an
action on the intercepted page.

### How?
When loading a page that has an intercepted route, `nextUrl` should be
consistent with URL derived from the flight router state tree. But when
an interception occurs via navigation, `nextUrl` will now deviate. I'm
using this to determine whether or not `Next-Url` should be forwarded
along in the `POST` request.

Closes NEXT-1436
Fixes #52591
Fixes #49934
2023-12-01 14:45:00 -08:00
Zack Tanner
8395059d33
verify action id before parsing body (#58977)
### What?
When handling a server action, in the non-progressive enhanced case,
React will attempt to parse the request body before verifying if a valid
server action is received. This results in an "Error: Connection Closed"
error being thrown, rather than ignoring the action and failing more
gracefully

### Why?
To support progressive enhancement with form actions, the `actionId`
value is added as a hidden input in the form, so the action ID from the
header shouldn't be verified until determining that we've reached the
non-PE case. ([React
ref](https://github.com/facebook/react/pull/26774)). However, in
https://github.com/vercel/next.js/pull/49187, support was added for a
URL encoded form (which is not currently used, as indicated on the PR).

Despite it not being used for server actions, it's currently possible to
trigger this codepath, ie by calling redirect in an action handler with
a 307/308 status code with some data in the URL. This would result in a
500 error.

### How?
React should not attempt to parse the URL encoded form data until after
we've verified the server action header for the non-PE case.

x-ref NEXT-1733
[Slack
context](https://vercel.slack.com/archives/C03S8ED1DKM/p1700674895218399?thread_ts=1700060786.749079&cid=C03S8ED1DKM)
2023-11-29 19:55:00 +00:00
Zack Tanner
77f8889b7c
use 303 status code for redirects in fetch actions (#59017)
### What?
A `redirect` that occurs during a fetch action will get a status code of
200, while the redirection logic is handled client-side.

### Why?
In this scenario, the redirect is handled by the client router, so no
`Location` is set on the action response. However for debugging /
logging purposes, it'd be useful to still return the same status code
used in other cases (see #58885)

### How?
Rather than selectively setting the status to 303 in the non-fetch
action case, this always applies it.

Closes NEXT-1745
2023-11-29 11:45:06 -08:00
Zack Tanner
eafaba39cb
update status codes for redirect and permanentRedirect in action handlers (#58885)
### What?
Calling `redirect` or `permanentRedirect` with a route handler used by a server action will result in that POST request following the redirect. This could result in unexpected behavior, such as re-submitting an action (in the case where the redirected URL makes use of the same server action).

### Why?
By spec, 307 and 308 status codes will attempt to reuse the original request method & body on the redirected URL.

### How?
In all cases when calling a `redirect` handler inside of an action, we'll return a `303 See Other` response which is a typical status code when redirecting to a success / confirmation page as a result of a POST/PUT.

The other option would be to use 301 / 302 status codes, but since we're already doing a 303 status code [here](https://github.com/vercel/next.js/blob/canary/packages/next/src/server/app-render/action-handler.ts#L603), this aligns the behavior for the route handler case. 

Closes NEXT-1733
See also: https://github.com/vercel/next.js/issues/51592#issuecomment-1810212676
[Slack x-ref](https://vercel.slack.com/archives/C03S8ED1DKM/p1700060786749079)
2023-11-29 08:35:50 +00:00
Zack Tanner
298bbe5489
fix async action queue behavior (#59038)
### What?
When the router action queue receives a bunch of async actions in quick succession, some of those requests are dropped, and as a result, anything observing pending transitions will be stuck in a pending state.

### Why?
When adding items to the action queue, the intended behavior is for new actions to be added to the end of the action queue, to be picked up by `runRemainingActions` once the in-flight action is processed. However, new actions are erroneously overwriting pending actions in the queue rather than appending them, as `actionQueue.last` might have a pending action attached to it. 

### How?
This moves the assignment of `actionQueue.last` to always be in `dispatchAction`, rather than the function that processes the action, so that we always have a single spot where `last` is assigned and to prevent it from erroneously omitted/overwritten. 

Fixes #59011
2023-11-28 22:54:04 +00:00
Shu Ding
c85caae8d6
Fix encoding in encryption of Server Actions (#59000)
Utils `stringToUint8Array` and `arrayBufferToString` assume that the values are just arbitrary fixed width data. However that doesn't work when we do unicode concatenation (`actionId + arg`) which requires Text encoder/decoder to be used.

Closes #58463, closes #58579. In general any complex unicode characters will cause the same issue, for example emojis.
2023-11-28 10:20:47 +00:00
Jiachi Liu
d6d6d56133
Remove client only dynamic chunks from edge bundle (#56761)
### Issue

In the client components world, when you're using `next/dynamic` with `ssr: false` to split chunks in pages of edge runtime, you could get the dynamic imported module still bundled in the server bundle for edge runtime. This could easily hit the bundle limit on edge runtime if you're loading a large size of non-SSR module.

This is caused by the whole chunk is still being included when we're creating the client entry. Since the client entry is imported eagerily, webpack will bundle all the modules under it, unless it's explicitly marked not being included.

### Fix

For client components, SSR rendering layer of bundle, non-SSR `next/dynamic` calls, we're transform the result of `dynamic()` call from to conditional import the dynamic loaded module.

From
```js
dynamic(() => import(...))
```
To
```js
dynamic(() => {
  require.resolveWeak(...)
}, { ssr: false })
```

This will only be applied to SSR layer bundle client components non-SSR `next/dynamic` calls and only when webpack is bundling since turbopack doesn't need this. In this way, the server side will be stripped but it can still enter the module graph since we need to traverse if there's SA in client modules with using webpack API `require.resolveWeak`. And for client side bundle will still include the actual module.

Close NEXT-1703
2023-11-16 15:10:28 +00:00
Shu Ding
24a617c24f
Change allowed forwarded hosts to be allowed origins for Server Actions (#58023)
The allowlist should be origin domains that are allowed to send the
requests, not the list of forwarded hosts (i.e. reverse proxies).
2023-11-08 11:20:32 +01:00
Zack Tanner
85abc48c90
suspend in render, not in reducers (#56497)
This removes our current convention of throwing promises in reducers in
favor of returning promises that can be consumed by `use` instead. This
will help unblock some future improvements (batching, PPR)

Reducers that would typically throw a promise now return their promise.
This gets maintained by a mutable queue (initialized in hydrate) to
ensure actions are processed in-order. The queue is also responsible for
mutating state and passing it as an input to subsequent actions.

This PR does not modify reducer behavior to keep changes minimal, but
there's more cleanup that we can do in a follow-up PR to remove things
that previously assumed reducers would be replayed.

(I recommend reviewing with whitespace turned off)

---------

Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
2023-11-02 11:53:50 +01:00
Shu Ding
937254edf5
Add serverActions.allowedForwardedHosts option (#57529)
This new option specifies a list of host names that are considered safe, to accept as Server Action requests if they're different from the initial request origin. It can be very helpful when the hosted app has many layers of reverse proxies ahead.

Closes #57397.
2023-11-01 12:20:00 +00:00
Jiachi Liu
07483d4b4c
Move serverActionsBodySizeLimit to serverActions.bodySizeLimit (#57433)
Scope all `serverActions` config in one group "serverActions" to make it
more semantics

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-10-26 08:51:32 +02:00
Justin Ridgewell
df0fb70c8c
turbopack: Support Actions in both RSC and Client layers (#57475)
### What?

Adds support for Server Actions imported by both server and client.

### Why?

If an Action is imported by both the Client and RSC layers, we need to
support them as if they're the same action.

### How?

First, we need to ensure both layers create the same action hash ids,
which we can then use to deduplicate actions imported by both layers. If
a conflict is found, we prefer the RSC layer's action.

Closes WEB-1879

---------

Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-10-26 08:32:04 +02:00
Shu Ding
59ebfbea9e
Fix request body hanging when middleware is preset (#57381)
Instead of `Readable.toWeb` we're gonna manually convert the Node.js stream to a Web stream. `toWeb` is either having a bug, or not compatible with middleware-cloned `PassThrough` streams.

Closes #56286. The case should be already covered with existing tests.
2023-10-25 02:44:10 +00:00
Jiachi Liu
4e1429182f
Move logging option to stable (#56791)
We introduced a data fetching logging before, and the control option was under experimental. After a bit experiments turns out users really loves it. We decide to move it to a stable option.

### Changes
We're going to move the `logging` option outside of `experimental`, and scope the `fetches` related config under `logging.fetches`.

```js
// next.config.js
logging?: {
    fetches?: {
      fullUrl?: boolean
    }
}
```
2023-10-23 18:45:00 +00:00
Shu Ding
a383b93556
Encoding and decoding variables of Server Actions from the closure (#56377)
This PR implements encryption and decryption for Server Action bound values that are from the closure level. Explicit `.bind` values, function arguments and module-level values are NOT handled.

### Compiler

The compiler now groups all closure bound values to an array which gets wrapped with `encrypt`. And then inside the action body, it prepends an expression to recreate the values via `await decrypt`.

Since closure-closed variables will only exist on the server layer, the encryption utility has `"server-only"` annotated.

### Encryption

During build time, a private AES-GCM encryption key is randomly generated and stored in the built server manifest. Before encrypting/decrypting, an extra round of Flight server and client will be used to serialize/deserialize the value.

When encrypting, a salt that contains the action ID is provided to prevent replay attack towards different API endpoints. The encryption key can be overridden via the `NEXT_SERVER_ACTIONS_ENCRYPTION_KEY` env variable so it can be built on multiple machines on scale.

A global singleton for storing the client reference manifest was made for Flight's serialization/deserialization as that might happen outside of rendering.

After encryption, we then serialize the ArrayBuffer as Base64 to send it to the client.
2023-10-21 17:52:06 +00:00
Josh Story
5528cc6d4e
Remove the experimental serverActions flag (#57145)
Remove the experimental `serverActions` flag

Co-authored-by: Shu Ding <3676859+shuding@users.noreply.github.com>
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
2023-10-20 20:45:25 +00:00
Quentin
abe8b1e0a8
Improve performance of String.prototype.split uses (#56746)
This PR adds the optional `limit` parameter on String.prototype.split uses.

> If provided, splits the string at each occurrence of the specified separator, but stops when limit entries have been placed in the array. Any leftover text is not included in the array at all.

[MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split#syntax)

While the performance gain may not be significant for small texts, it can be huge for large ones.

I made a benchmark on the following repository : https://github.com/Yovach/benchmark-nodejs

On my machine, I get the following results:
`node index.js`
> normal 1: 570.092ms
> normal 50: 2.284s
> normal 100: 3.543s

`node index-optimized.js`
> optmized 1: 644.301ms
> optmized 50: 929.39ms
> optmized 100: 1.020s

The "benchmarks" numbers are : 
- "lorem-1" file contains 1 paragraph of "lorem ipsum"
- "lorem-50" file contains 50 paragraphes of "lorem ipsum"
- "lorem-100" file contains 100 paragraphes of "lorem ipsum"
2023-10-19 00:25:15 +00:00
Josh Story
0a80017d03
Update React from d900fadbf to 09fbee89d. Removes server context and experimental prefix for server action APIs (#56809)
The latest React canary builds have a few changes that need to be
adopted for compatability.

1. the `useFormState` and `useFormStatus` hooks in `react-dom` and the
`formData` opiont in `react-dom/server` are no longer prefixed with
`experimental_`
2. server content (an undocumented React feature) has been removed. Next
only had trivial intenral use of this API and did not expose a coherent
feature to Next users (no ability to seed context on refetches). It is
still possible that some users used the React server context APIs which
is why this should go into Next 14.

### React upstream changes

- https://github.com/facebook/react/pull/27513
- https://github.com/facebook/react/pull/27514
- https://github.com/facebook/react/pull/27511
- https://github.com/facebook/react/pull/27508
- https://github.com/facebook/react/pull/27502
- https://github.com/facebook/react/pull/27474
- https://github.com/facebook/react/pull/26789
- https://github.com/facebook/react/pull/27500
- https://github.com/facebook/react/pull/27488
- https://github.com/facebook/react/pull/27458
- https://github.com/facebook/react/pull/27471
- https://github.com/facebook/react/pull/27470
- https://github.com/facebook/react/pull/27464
- https://github.com/facebook/react/pull/27456
- https://github.com/facebook/react/pull/27462
- https://github.com/facebook/react/pull/27461
- https://github.com/facebook/react/pull/27460
- https://github.com/facebook/react/pull/27459
- https://github.com/facebook/react/pull/27454
- https://github.com/facebook/react/pull/27457
- https://github.com/facebook/react/pull/27453
- https://github.com/facebook/react/pull/27401
- https://github.com/facebook/react/pull/27443
- https://github.com/facebook/react/pull/27445
- https://github.com/facebook/react/pull/27364
- https://github.com/facebook/react/pull/27440
- https://github.com/facebook/react/pull/27436

---------

Co-authored-by: Zack Tanner <zacktanner@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
2023-10-16 15:46:10 -07:00
Balázs Orbán
1ff7f07875
feat: drop Node.js 16 (#56896)
### What?

BREAKING CHANGE: Bump the minimum required Node.js version.

### Why?

Node.js 16 has reached end-of-life in September.

Bumped to `18.18.2` since it contained some security-related patches: https://nodejs.org/en/blog/vulnerability/october-2023-security-releases

### How?

Bumped `engines` where needed, upgraded the workflows.

This will allow us to remove quite a few polyfills, I'll open separate PRs for those.
2023-10-16 21:41:38 +00:00
Tim Neutkens
524b31513a
Fix logging level in actions test (#56473)
This test was using the old format and because of that failed in
Turbopack. This fixes that.

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Adding or Updating Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md


## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->
2023-10-05 12:30:39 +02:00
Shu Ding
8469d0ac27
Add test case for the permalink option of useFormState (#56329)
Ref: https://github.com/facebook/react/pull/27372.

Co-authored-by: JJ Kasper <jj@jjsweb.site>
2023-10-03 13:47:03 +02:00
Shu Ding
4e44598249
Progressive enhancement for form state (#55584)
This PR implements progressive enhancement for `useFormState`:
1. Inline the form state in `__next_f` and use it for hydration.
2. Encode the new form state based on the action return value, and pass
it to Flight/Fizz.

Also fixed a problem caused by inconsistent React tree structure between
static-generation and non-static-generation.
2023-09-26 18:03:31 +02:00
Zack Tanner
c2e0213fa5
fix bugs pertaining to server actions + navigation (#55853)
This fixes some scenarios where executing a server action after navigation can cause the action to behave incorrectly (double submitting, not resolving). There are two separate issues:

- `canonicalUrl` and `pendingNavigatePath` were not constructed using the same function (`createHrefFromUrl`) so in certain situations they'd be comparing different values
- a fulfilled inFlightServerAction should not be invoked again

Closes NEXT-1655
Closes NEXT-1654
Fixes #55845
Fixes #55814
Fixes #55805
2023-09-23 01:42:39 +00:00
Zack Tanner
a73abad609
fix: server actions initiated from static pages (#51534)
### What?
Pages marked with `generateStaticParams` don't currently support server actions, and instead return a 405 Method Not Allowed, with no action being taken on the client. Additionally, pages that are marked static & use server actions are opted into dynamic rendering.

### Why?
The page that has `generateStaticParams` is marked as `isSSG` [here](ee2ec3dd1d/packages/next/src/server/base-server.ts (L1337)).

As a result, the request is short-circuited because a POST request isn't supported on static pages. Upon detecting a server action on a page marked SSG, we bypass the static cache and go straight to the lambda. 

This PR introduces an experimental option to the prerender manifest that will allow for selectively bypassing the static cache

This also removes the need to bail out of static generation

Closes NEXT-1167
Closes NEXT-1453
Fixes #49408
Fixes #52840
Fixes #50932
2023-09-14 21:22:19 +00:00
Jimmy Lai
5217e7eb06
server: re-land bundled runtimes (#55139)
see https://github.com/vercel/next.js/pull/52997

also added a fix by @jridgewell to fix turbopack





Co-authored-by: Justin Ridgewell <112982+jridgewell@users.noreply.github.com>
2023-09-08 16:05:29 +00:00
JJ Kasper
7267538e00
Revert "perf: add bundled rendering runtimes (#52997)" (#55117)
This reverts commit a5b7c77c1f.

Our E2E tests are failing with this change this reverts to allow investigating async 

x-ref: https://github.com/vercel/next.js/actions/runs/6112149126/job/16589769954
2023-09-07 21:07:53 +00:00
Jimmy Lai
a5b7c77c1f
perf: add bundled rendering runtimes (#52997)
## What?

In Next, rendering a route involves 3 layers:
- the routing layer, which will direct the request to the correct route to render
- the rendering layer, which will take a route and render it appropriately
- the user layer, which contains the user code 

In #51831, in order to optimise the boot time of Next.js, I introduced a change that allowed the routing layer to be bundled. In this PR, I'm doing the same for the rendering layer. This is building up on @wyattjoh's work that initially split the routing and the rendering layer into separate entry-points.

The benefits of having this approach is that this allows us to compartmentalise the different part of Next, optimise them individually and making sure that serving a request is as efficient as possible, e.g. rendering a `pages` route should not need code from the `app router` to be used.

There are now 4 different rendering runtimes, depending on the route type:
- app pages: for App Router pages
- app routes: for App Router route handlers
- pages: for legacy pages
- pages api: for legacy API routes

This change should be transparent to the end user, beside faster cold boots.

## Notable changes

Doing this change required a lot of changes for Next.js under the hood in order to make the different layers play well together.

### New conventions for externals/shared modules

The big issue of bundling the rendering runtimes is that the user code needs to be able to reference an instance of a module/value created in Next during the render. This is the case when the user wants to access the router context during SSR via `next/link` for example; when you call `useContext(value)` the value needs to be the exact same reference to one as the one created by `createContext` earlier.

Previously, we were handling this case by making all files from Next that were affected by this `externals`, meaning that we were marking them not to be bundled.

**Why not keep it this way?**

The goal of this PR as stated previously was to make the rendering process as efficient as possible, so I really wanted to avoid extraneous fs reads to unoptimised code. 

In order to "fix" it, I introduced two new conventions to the codebase:
- all files that explicitly need to be shared between a rendering runtime and the user code must be suffixed by `.shared-runtime` and exposed via adding a reference in the relevant `externals` file. At compilation time, a reference to a file ending with this will get re-written to the appropriate runtime.
- all files that need to be truly externals need to be suffixed by `.external`. At compilation time, a reference to it will stay as-is. This special case is needed mostly only for the async local storages that need to be shared with all three layers of Next.

As a side effect, we should be bundling more of the Next code in the user bundles, so it should be slightly more efficient.

### App route handlers are compiled on their own layer

App route handlers should be compiled in their own layer, this allows us to separate more cleanly the compilation logic here (we don't need to run the RSC logic for example).

### New rendering bundles

We now generate a prod and a dev bundle for:
- the routing server
- the app/pages SSR rendering process
- the API routes process

The development bundle is needed because:
- there is code in Next that relies on NODE_ENV
- because we opt out of the logic referencing the correct rendering runtime in dev for a `shared-runtime` file. This is because we don't need to and that Turbopack does not support rewriting an external to something that looks like this `require('foo').bar.baz` yet. We will need to fix that when Turbopack build ships.

### New development pipeline

Bundling Next is now required when developing on the repo so I extended the taskfile setup to account for that. The webpack config for Next itself lives in `webpack.config.js` and contains the logic for all the new bundles generated.

### Misc changes

There are some misc reshuffling in the code to better use the tree shaking abilities that we can now use.

fixes NEXT-1573

Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
2023-09-07 15:51:49 +00:00
Jiachi Liu
e117c000e4
Redesign nextjs logging (#54713)
The current logging styles has been existed for a while, this PR gives a fresh impression for the logging output from Next.js.
We want to achieve few new goals that makes the output clean, modernized, sweet 🍫 .

Few goals are addressed with this redesign:

## Refresh Impression & Simplification

The new design of logging is much more information centralized and streamlined.

* Given a `ready` message at the begining when compilers are bootstrapped.
* Only show `compiled` event with green check mark indicating succesful compilation, this will merge the unclear `compiling` event which shows `(client and server)` before, now tell you the route compilation info in one line.

hello world app

### `next dev`

#### After vs Before


<img src="https://github.com/vercel/next.js/assets/4800338/9649b340-8241-4756-a2b3-a989f0b74003" height="120"> 
<img src="https://github.com/vercel/next.js/assets/4800338/ee181263-3dd4-40d0-9ffc-819a56b45900" height="120">  

 


 

### `next build`

#### After vs Before


<img src="https://github.com/vercel/next.js/assets/4800338/5db9829a-9ffc-49f0-b030-93ee92f5c248" width="360"> 
<img src="https://github.com/vercel/next.js/assets/4800338/b9527b83-27c8-4426-9c0d-c0d4072b7d58" width="360">





### error status

#### After vs Before

<img src="https://github.com/vercel/next.js/assets/4800338/00455226-ace7-468b-8d90-0d36bf038489" height="120"> 
<img src="https://github.com/vercel/next.js/assets/4800338/1be8c451-d3f0-465c-9ef7-6b0dde7cff85" height="120"> 



## Streamlization

If you have customized envs and experiments Next.js will give the brief in the early summary about your network information, env vars, and enabled experimental features

<img src="https://github.com/vercel/next.js/assets/4800338/ca1a7409-1532-46cb-850f-687e61e587b2" width="400">


## Polish

### fetching logging structure 

#### After vs Before
<img src="https://github.com/vercel/next.js/assets/4800338/97526397-dffe-4736-88ed-e5cbe5e945bd" width="400">
<img src="https://github.com/vercel/next.js/assets/4800338/ab77c907-5ab5-48bb-8347-6146d2e60932" width="400">


### Dedupe Duplicates

The logging is moved from `@next/env` to `next` itself, `@next/env` will only notify the invoker that the env is reloaded. Then the duplicated logs for the env reloading cases can be avoid.

#### After vs Before
<img src="https://github.com/vercel/next.js/assets/4800338/04799295-e739-4035-87aa-61cec962fc39" width="400">
<img src="https://github.com/vercel/next.js/assets/4800338/e29020c9-0031-4bf3-a21b-8b64633f43a2" width="400"> 


### Different indicators

Use unicode text icons for different situation: 
* passed -> check mark
* warning -> warning
* error -> red cross
* loading -> circle

<img src="https://github.com/vercel/next.js/assets/4800338/715c34bd-298f-4990-a5d7-e12e455ead44" width="400">



Co-authored-by: Tim Neutkens <6324199+timneutkens@users.noreply.github.com>
2023-09-05 11:40:00 +00:00
Zack Tanner
c013e98fa4
fix: server actions firing twice after navigation (#54948)
The original logic here was introduced to unblock client side navigations if a server action was in flight, however this introduced a bug where subsequent actions would fetch twice after navigation. 

This was happening because the promise handling was in the wrong spot: previously this would potentially cause both the `then` callback to fire while simultaneously the action reducer would handle the result. Moving it to where we're first checking if there's a pending navigation will more reliably indicate if the action was resolved after we discarded it in the reducer.

Closes NEXT-1589
Fixes #54746
2023-09-03 20:31:48 +00:00
JJ Kasper
2eef775472
Update tag handling for app cache (#53321)
Optimizes how we handle cache tags for soft tags (auto-added by Next.js)
and normal tags (added manually) and adds differentiating between
`revalidatePath('/blog/first')` and page/layout.

Soft tags are not stored across cache entry and instead auto sent along
when checking cache entries. This allows us to prevent storing
exponential amounts of tags across cache entries while still having the
relationship between them so that single path revalidation can work
properly.

x-ref: [slack
thread](https://vercel.slack.com/archives/C042LHPJ1NX/p1690586837903309)
2023-08-31 15:17:40 -07:00
Shu Ding
758b9d251a
Use push for Server Action redirections (#54458)
Closes #53911. When calling `redirect()` instead a Server Action, the previous route should exist in the history when it's handled by the framework.
2023-08-23 19:47:46 +00:00
Shu Ding
ad556aee1b
Fix compilation of next/dynamic with ssr: false in App Router (#54411)
For the server compilation, we currently transpile the dynamic import
expression to `null` if `ssr` is disabled. However to make the Server
Actions layer work (as it can be created again from a Client Component),
we can't do that optimization.

This PR changes it to always keep that import expression when
`react_server_components` (App Router) is enabled, no matter which layer
it's on.

Closes #52672.
2023-08-23 14:15:31 +02:00
Zack Tanner
892839ff83
fix: server actions blocking navigation events (#54307)
A long-running server action shouldn't block page navigation. This makes use of a global mutable to detect when a navigation event occurs -- this change will unblock other reducers (such as navigation).

If we bailed on the action, we trigger a `router.refresh()` to ensure any side effects from the action are refetched.

Closes NEXT-1131
Fixes #49425
2023-08-21 20:34:42 +00:00
Shu Ding
0e78798f37
Fix renamed export of Server Actions (#54241)
This fixes the compilation of `export { action as renamed }` syntax. Previously it's compiled as `export var action = ...` and with this fix, it will be `export var renamed = ...`.

Closes #54229.
2023-08-18 20:41:32 +00:00
JJ Kasper
4b4533787b
Add warnings for static generation bail outs (#53761)
Currently using server actions on a page or using edge runtime causes
that page to bail out of ISR or static generation so this adds warnings
to make users aware of this.

x-ref: [slack
thread](https://vercel.slack.com/archives/C03KAR5DCKC/p1690816539472449)

---------

Co-authored-by: Zack Tanner <zacktanner@gmail.com>
2023-08-08 20:09:34 -07:00
yudai yamamoto
b993afbf7c
Fix action failures due to state tree encoding (#53655)
fixes #53654

### Related PRs
- #51017
2023-08-07 10:02:48 +00:00
Shu Ding
0ecde6bd32
Add test for client router state invalidation caused by cookie mutations (#53494)
Closes #53261. Closes NEXT-1478.
2023-08-03 11:02:19 +00:00
Tim Neutkens
a721a749d8
router: apply server actions in a similar way to router.refresh() (#53373)
## What?

I was investigating reports of server actions with `revalidatePath` /
`revalidateTag` not invalidating the client-side router cache. Managed
to reproduce the issue here:
https://github.com/timneutkens/server-actions-test (requires Vercel KV
to run).

While looking at the reducer I noticed a few things that seemed off:

- setTimeout to trigger another `dispatch` on the same reducer with the
fetched data. This means that it would not be part of the same React
Transition.
- redirects weren't applying the data that comes back from the server
(that allows for single hop navigation)
- prefetchCache was invalidated, but the router cache which is used for
back/forward navigation was not invalidated, causing back/forward to not
get the new data.
- all changes in the action-reducer mutate. The part that shouldn't
mutate was part of that setTimeout dispatch.

This PR aims to solve all of these by reworking the actions reducer to
be handled similarly to `router.refresh()`. Since `router.refresh()` was
already modeled to be similar to `revalidatePath('/')`.

The flow is now more like the other reducers:
- Fetch data
- Wait for the data to come back
- Apply the data to the cache and keep it in a mutable variable
- Return the new cache and otherwise values like the url

In particular the actions reducer handles a few extra specifics:
- Resolving the server action promise, so that the value is passed to
the application code waiting for the result.
- Applying redirects from `redirect()` calls.
- Invalidating the router cache

## Followup changes

- Currently when calling `revalidatePath('/dashboard')` the entire
router cache is invalidated instead of only `/dashboard` and further
down, this is not in scope for this PR but still has to be added.
- `notFound()`, I'm not sure how this is supposed to work exactly, it
doesn't really make sense to me to allow it in server actions.

Kudos @feedthejim for helping investigate for this PR 😌 

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

### Adding or Updating Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md


## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->
2023-07-31 21:52:31 +02:00
Jimmy Lai
f2c5eb840f
actions: fix revalidate after redirect (#53368)
This PR reverts a change that removed the `content-length` header filtering from the req made by the actions when redirecting. This change made some tests flaky and presumably also broke server actions in subtle ways.

There's still one other bug when redirecting after revalidating that will happen in you revalidate a page that was already rendered before where we will still show stale content. @timneutkens is fixing that one.

NEXT-1483
2023-07-31 10:08:52 +00:00
JJ Kasper
4a926efb83
Update flakey app-action and prerender-prefetch tests (#53340)
x-ref:
https://github.com/vercel/next.js/actions/runs/5699334417/job/15448149485#step:27:600
x-ref:
https://github.com/vercel/next.js/actions/runs/5702064037/job/15453403529?pr=53340
2023-07-30 12:51:24 -07:00
Zack Tanner
22ca85946e
Wrap incremental cache in an IPC server (#53030)
This uses an IPC server (if available) for incremental cache methods to help prevent race conditions when reading/writing from cache and also to dedupe requests in cases where multiple cache reads are in flight. This cuts down on data fetching across the different build-time workers.

Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
2023-07-26 23:19:46 +00:00
JJ Kasper
1c5c0ba3ff
Temporarily skip flakey action revalidate (#53134)
These have been flaking for quite a while so skips them more
consistently until they are investigated.

x-ref:
https://github.com/vercel/next.js/actions/runs/5650048471/job/15305721826#step:27:590
2023-07-24 15:37:30 -07:00
Jiachi Liu
1fefb4a8d2
Reland "Refine the not-found rendering process for app router" (#52985)
Reland #52790
Reverts vercel/next.js#52977

was failed due to failed job
[vercel/next.js/actions/runs/5616458194/job/15220295829](https://github.com/vercel/next.js/actions/runs/5616458194/job/15220295829)

Should be fine to resolve with
https://github.com/vercel/next.js/pull/52979 now

Fixes #52718
Fixes #52739

---------

Co-authored-by: Alex Kirszenberg <alex.kirszenberg@vercel.com>
Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
2023-07-21 10:09:30 -07:00
JJ Kasper
d7335b75d1
Revert "Refine the not-found rendering process for app router" (#52977)
Reverts vercel/next.js#52790

Reverting temporarily as this breaks turbopack's not found handling due
to the app tree being generated there not having the necessary parallel
routes in the `_not-found` entry x-ref:
0df8aac935/packages/next-swc/crates/next-core/src/app_structure.rs (L677-L681)

x-ref:
https://github.com/vercel/next.js/actions/runs/5616458194/job/15220295829
2023-07-20 20:39:05 -07:00
Jimmy Lai
55eebefbab
app-router: prefetching tweaks (#52949)
This PR tries to address some feedback around prefetching, like in #49607, where they were some warnings because we were over prefetching.

The tweaks in this PR:
- if there are no loading boundary below, we don't prefetch the full page anymore. I made that change a while ago but I think it wasn't the original intent from @sebmarkbage. Fixing that now. So now, we will prefetch the page content until the nearest loading boundary, only if there is any.
- there's now a queue for limiting the number of concurrent prefetches. This is to not ruin the bandwidth for complex pages. Thanks @alvarlagerlof for that one.
- also, if the prefetch is in the queue when navigating, it will get bumped.
- bonus: fixes a bug where we were wrongly stripping headers in dev for static pages

Test plan:
<img width="976" alt="CleanShot 2023-07-20 at 17 53 43@2x" src="https://github.com/vercel/next.js/assets/11064311/2ea34419-c002-4aea-94df-57576e3ecb2e">
In the screenshot you can see that:
- only 5 requests at a time are authorised
- when I clicked on 15, it got prioritised in the queue (did not cancel the rest)
- the prefetch only included the content until the loading boundary




Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
2023-07-20 21:46:38 +00:00
Jiachi Liu
cb24c555a6
Refine the not-found rendering process for app router (#52790)
### What

This PR changes the flow of not-found rendering process. 

### Why

`not-found.js` was rendered in two ways before:
* 1 is SSR rendering the not-found as 404
* 2 is triggering the error on RSC rendering then the error will be
preserved in inline flight data, on the client it will recover the error
and trigger the proper error boundary.

The solution has been through a jounery:
No top-level not found boundary -> introduce metadata API -> then we
create a top level root not found boundary -> then we delete it due to
duplicated rendering of root layout -> now this

So the solution before this PR is still having a root not found boundary
wrapped in the `AppRouter`, it's being used in a lot of places including
HMR. As we discovered it's doing duplicated rendering of root layout,
then we removed it and it started failing with rendering `not-found` but
missing root layout. In this PR we redesign the process.

### How

Now the rendering architecture looks like:

* For normal root not-found and certain level of not-found boundary
they're still covered by `LayoutRouter`
* For other error renderings including not-found
* Fully remove the top level not-found boundary, when it renders with
404 error it goes to render the fallback page
* During rendering the fallback page it will check if it should just
renders a 404 error page or render nothing and let the error from inline
flight data to trigger the error boundary

pseudo code
```
try {
  render AppRouter > PageComponent
} catch (err) {
  create ErrorComponent by determine err
  render AppRouter > ErrorComponent
}
```

In this way if the error is thrown from top-level like the page itself
or even from metadata, we can still catch them and render the proper
error page based on the error type.

The problematic is the HMR: introduces a new development mode meta tag
`<meta name="next-error">` to indicate it's 404 so that we don't do
refresh. This reverts the change brougt in #51637 as it will also has
the duplicated rendering problem for root layout if it's included in the
top level not found boundary.

Also fixes the root layout missing issue:

Fixes #52718
Fixes #52739

---------

Co-authored-by: Shu Ding <g@shud.in>
2023-07-20 14:12:06 -07:00
Sebastian Markbåge
b095e9e980
Test Progressive Enhancement of Server Actions (#52062)
Adds a regression test for testing progressive enhancement of Server
Actions. Both when passed from a Server Component and when imported into
a Client Component.

#51723 landed a bit too early which broke this but it'll be fixed again
once React is upgraded.

Co-authored-by: Shu Ding <g@shud.in>
2023-07-06 18:21:59 +02:00
Shu Ding
3a87f0005e
Change the Server Actions feature flag to be validated at compile time (#52147)
Currently we are validating the `experimental.serverActions` flag when creating the actual entries for Server Actions, this causes two problems. One is that syntax errors caught at compilation time are still shown, even if you don't have this flag enabled. Another problem is we still traverse the client graph to collect these action modules even if the flag isn't enabled.

This PR moves that check to be happening at compilation time, which addresses the two above but also brings the extra benefit of showing the exact span and module trace that errors:

<img width="974" alt="CleanShot 2023-07-03 at 20 26 34@2x" src="https://github.com/vercel/next.js/assets/3676859/1676b1f6-e205-4963-bce4-5b515a698e9c">
2023-07-03 20:29:57 +00:00