This PR will enable minifying the *server* part of the user code by default when running `next build`.
## Explanation
Next.js compiles two versions of your code: the client version for the app that runs in the browser, and the server for the code that will run on the server. Whilst the client code has always been minified and optimised, we haven't done so historically for the server side.
## How does this impact me?
There are several consequences to this change:
- minified code makes error stacks less readable. To fix that, you can use source maps, which you can enable with `experimental.serverSourceMaps`. This is not enabled by default as this is an expensive operation.
- however the server code will be optimised for size so as a result, cold boots should improve
## opting out
you can opt out via specifying `experimental.serverMinification: false` in `next.config.js`
In dev mode, we're using a catch-all route for dynamic metadata routes, e.g. page path `/twitter-image` would become `/twitter-image/[[...metadata_id...]]/route` as a dynamic custom app route.
But we're missing to convert it in filesystem scanning for routing purpose, adding the metadata related normalizing logic for app page to align with other places.
This PR introduces a change in the Next.js server that should improve memory usage nicely during dev.
## How
While investigating the repro cases in #54708, @timneutkens and I noticed that the high memory usage often involved `googleapis`. Digging a bit more, I also saw that in dev, the bundle generated for a page using `googleapis` was around 80MB and that requiring it in Node.js increased memory by 160MB 🥲 and that requiring another page that also used this also increased the memory by 160MB.
The problem is that Next.js, on new navigations and hot reloads, might need to load/reload all the code required for the page *all the time*. This issue is also exacerbated by the fact that there's a nasty Node.js bug that makes it impossible to clean that memory up, leading to memory bloat and overall a pretty bad DX.
So if we can't clean this up, what can we do about it?
The change I'm introducing in this PR is that I'm changing Next.js in order to split the code you're using from `node_modules` from the code you've written in different chunks. The idea is that the heavy code you're loading from `node_modules` is only gonna be loaded once per session this time.
This should make a navigation/page reload only load the user bundle now. On my simple test case, the cost of navigation went from ~200MB to ~40MB.
A few notes on the implementation:
- The chunks for the `node_modules` are split at the module level, this is to ensure that each of the node_modules dependencies is split in the most memory efficient manner. If it was a big monolithic chunk, it would potentially be reloaded again and again whenever reloaded, leading to leakage.
- I'm guessing we could do the same for the Edge server
- the first load for a page will still be fairly heavy memory wise, there's probably something else we can do there
- there's also an issue with the webpack require cache being flushed, whereas it could be reused
## comparisons
### navigating a page
before
<img width="284" alt="CleanShot 2023-09-04 at 21 00 46@2x" src="https://github.com/vercel/next.js/assets/11064311/44e37df8-4414-4ca1-b6bf-fb0fb11751ea">
after
<img width="392" alt="CleanShot 2023-09-04 at 20 58 53@2x" src="https://github.com/vercel/next.js/assets/11064311/46226123-a73a-4132-a99d-fb812e59df46">
When using page routes we need to pass the contents of the `export const config` to the `extraConfig` populating the functions config manifest rather then relying on named exports like in app router.
### 🧐 What's in there?
As discussed with @huozhi, it's restoring code from #52393 which I forgot to restore in #54891.
It is unlikely to make a difference, but we'll set globalWindow based on the desired test environment. This will not make any difference for most people.
### 🧪 How to test?
Once you've rebuilt your local version of Next, `pnpm testheadless jest/rsc`
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
This bumps `undici` to the latest version to match the one used in newer Node.js versions', potentially fixing compatibility issues.
I also cleaned up some related `global as any` usage.
While investigating the HMR event types I noticed `pong` is not being used in Pages Router nor in App Router.
This PR removes the code that sends it as it's essentially dead code.
Adds `serverError` to the list of HMR actions, missed this in the earlier PRs as it wasn't using `hotReloader.send`.
Also ensures on-demand-entry-handler reuses the hotReloader.send method.
Ensures only one name is used for the event, makes it easier to search for, going to replace this with the constant like the other events in a follow-up PR but this can be landed as-is.
The original logic here was introduced to unblock client side navigations if a server action was in flight, however this introduced a bug where subsequent actions would fetch twice after navigation.
This was happening because the promise handling was in the wrong spot: previously this would potentially cause both the `then` callback to fire while simultaneously the action reducer would handle the result. Moving it to where we're first checking if there's a pending navigation will more reliably indicate if the action was resolved after we discarded it in the reducer.
Closes NEXT-1589
Fixes#54746
For group routes, as their custom layout show up at the 1st level of the loader tree, so previously we tried to add the default not-found to that layer, but if they have custom root not-found, we should use that as the root not-found componeny, instead of using default not-found component to form the 404 page
Closes NEXT-1588
Fixes#51075Fixes#54892
This fix ensures the case that if you `import { foo } from './index'`,
where:
index.js:
```js
export * from './subpkg'
```
subpkg.js:
```js
const unrelatedExpressions = ...
export { foo } from './foo'
export { bar } from './bar'
```
Previously we'll transform the proxy to the second module to `export {
foo } from './subpkg'`. With this fix it will be correctly optimized as
`export { foo } from './foo'` which is a shorter path.
First part adding additional types for the various events that are sent through the websocket to the browser.
Will add additional details / comments on the types after landing these changes in the browser-side code too.
This:
- Profiles dev server startup by wrapping most logic in a debug tracing Span
- Sets tracing globals for this case earlier, as they would otherwise be unavailable
- Allows Turbopack traces to be uploaded, as while currently incomplete, they are accurate
Closes WEB-1440
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
Simplified some logic around `getPagePaths` as well as another normalization function. This instead utilized some flat arrays using push instead of relying on a separate `flatten` method (allowing us to remove it). This also reduced the number of logic checks for this generation.
Depends on https://github.com/vercel/turbo/pull/5852
This removes references to the `source_maps` module from Turbopack as it no longer exists in https://github.com/vercel/turbo/pull/5852
Test Plan:
- Opened a Next.js app with `--turbo-experimental`
- Change a file, verify the page is updated with HMR
- Confirm the source map for the updated file is correctly loaded in DevTools
Closes WEB-1483
Co-authored-by: Justin Ridgewell <112982+jridgewell@users.noreply.github.com>
This updates the check to see if a request is made in middleware to be a bit less brittle so it doesn't break if middleware is located in `/src/middleware` (`params.page` would be `/src/middleware` rather than `/middleware` in that case)
fixes https://github.com/vercel/next.js/discussions/54853
It's a bad pattern to do things like `(renderOpts as any).fetchMetrics = staticGenerationStore.fetchMetrics` and receive the updated value in the outside. This PR updates it to use `RenderResults`'s metadata instead.
In the future we should use `Reaedonly<RenderOpts>` and ban `as any` <picture data-single-emoji=":lolsob:" title=":lolsob:"><img class="emoji" width="20" height="auto" src="https://emoji.slack-edge.com/T0CAQ00TU/lolsob/2e754996aebc14fa.png" alt=":lolsob:" align="absmiddle"></picture>
There're various of cases would cause the `beforeRender` performance mark is not existed in the performance entries, learned from the issues description. We have to check if that mark is existed.
This PR also refactors all the mark names and measurement names into constants so that we won't easily mistype them
Fixes#20743Fixes#40903Fixes#47560
Co-authored-by: Balázs Orbán <18369201+balazsorban44@users.noreply.github.com>
### What?
This hooks up Turbopack's `TurbopackResult` issues into Next's HMR
messages.
### Why?
We need to deliver good error messages to devs so that they can quickly
fix any issues.
### How?
This builds up a debounced HMR queue (necessary because of the Turbopack
architecture delivering multiple messages via different async
iterators). If any HRM message contains an issue, we'll send down a
`built` message containing the issues and delay sending the real
messages. Once everything has been resolved, we'll resume sending down
HMR messages.
<img width="990" alt="Screenshot 2023-08-31 at 9 01 20 PM"
src="https://github.com/vercel/next.js/assets/112982/d5421779-2415-4a60-8d95-94f0e1147d6d">
Closes WEB-1484
---------
Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
This adds logging to indicate build progress while using Turbopack in the same format that webpack uses. It uses the same store object and sets state to log messages like so:
```
- wait compiling /page (client and server)...
- event compiled client and server successfully in 1366 ms
```
Given:
- An app router page `/`, `compiling /page` is logged
- An app router page `/bar`, `compiling /bar/page` is logged
- An app router route `/api/bar`, `compiling /api/bar/route` is logged
- A page router page `/foo`, `compiling /foo` is logged
- A page router api route `/api/foo`, `compiling /api/foo` is logged
Closes WEB-1476
Optimizes how we handle cache tags for soft tags (auto-added by Next.js)
and normal tags (added manually) and adds differentiating between
`revalidatePath('/blog/first')` and page/layout.
Soft tags are not stored across cache entry and instead auto sent along
when checking cache entries. This allows us to prevent storing
exponential amounts of tags across cache entries while still having the
relationship between them so that single path revalidation can work
properly.
x-ref: [slack
thread](https://vercel.slack.com/archives/C042LHPJ1NX/p1690586837903309)
### What?
Fixes a bug introduced in #54668, where the `/` path would map to the build manifest `.next/server/page/app-build-manifest.json` in App router instead of the correct `.next/server/index/page/app-build-manifest.json`
It also addresses an issue where `not-found` manifests would map to `.next/server/not-found/app-build-manifest.json` instead of the correct `.next/server/not-found/page/app-build-manifest.json`.
### Why?
The `&& type === 'pages'` mean we only rewrote `/` to `/index` in Pages router, where it needs to apply to both App and Pages. Likewise, special-casing `not-found` led to it not finding the manifest.
### How?
Duplicate the logic from d3a107991c/packages/next-swc/crates/next-core/src/util.rs (L65-L76)
Closes WEB-1471
Co-authored-by: Will Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
### What?
There were a few issues with the initial implementation of next-api HMR:
1. We incorrectly errored out when we received a Next.js WebSocket message
2. We didn't handle Next's `span-end` message, leading to another error
3. We listened to the `htmlEndpoint` change events instead of the `dataEndpoint`/`rscEndpoint`, leading to us detecting client-side changes and causing full page reloads
### Why?
HMR is the life-blood of development
### How?
Small fixes to our Turbopack reimplementation of the server-side HRM handlers
Closes WEB-1475
