This ensures our dynamic routes that have the same specificity as
`_next/static/:path*` don't get matched unexpectedly when the
`_next/static` asset doesn't exist. We were holding off on making this
change explicit due to compatibility concerns but these are no longer a
concern and the unexpected matching is more of a concern.
Closes: CSM-11
Fixes: https://github.com/vercel/next.js/issues/19270
Closes NEXT-2613
## What?
In Next, rendering a route involves 3 layers:
- the routing layer, which will direct the request to the correct route to render
- the rendering layer, which will take a route and render it appropriately
- the user layer, which contains the user code
In #51831, in order to optimise the boot time of Next.js, I introduced a change that allowed the routing layer to be bundled. In this PR, I'm doing the same for the rendering layer. This is building up on @wyattjoh's work that initially split the routing and the rendering layer into separate entry-points.
The benefits of having this approach is that this allows us to compartmentalise the different part of Next, optimise them individually and making sure that serving a request is as efficient as possible, e.g. rendering a `pages` route should not need code from the `app router` to be used.
There are now 4 different rendering runtimes, depending on the route type:
- app pages: for App Router pages
- app routes: for App Router route handlers
- pages: for legacy pages
- pages api: for legacy API routes
This change should be transparent to the end user, beside faster cold boots.
## Notable changes
Doing this change required a lot of changes for Next.js under the hood in order to make the different layers play well together.
### New conventions for externals/shared modules
The big issue of bundling the rendering runtimes is that the user code needs to be able to reference an instance of a module/value created in Next during the render. This is the case when the user wants to access the router context during SSR via `next/link` for example; when you call `useContext(value)` the value needs to be the exact same reference to one as the one created by `createContext` earlier.
Previously, we were handling this case by making all files from Next that were affected by this `externals`, meaning that we were marking them not to be bundled.
**Why not keep it this way?**
The goal of this PR as stated previously was to make the rendering process as efficient as possible, so I really wanted to avoid extraneous fs reads to unoptimised code.
In order to "fix" it, I introduced two new conventions to the codebase:
- all files that explicitly need to be shared between a rendering runtime and the user code must be suffixed by `.shared-runtime` and exposed via adding a reference in the relevant `externals` file. At compilation time, a reference to a file ending with this will get re-written to the appropriate runtime.
- all files that need to be truly externals need to be suffixed by `.external`. At compilation time, a reference to it will stay as-is. This special case is needed mostly only for the async local storages that need to be shared with all three layers of Next.
As a side effect, we should be bundling more of the Next code in the user bundles, so it should be slightly more efficient.
### App route handlers are compiled on their own layer
App route handlers should be compiled in their own layer, this allows us to separate more cleanly the compilation logic here (we don't need to run the RSC logic for example).
### New rendering bundles
We now generate a prod and a dev bundle for:
- the routing server
- the app/pages SSR rendering process
- the API routes process
The development bundle is needed because:
- there is code in Next that relies on NODE_ENV
- because we opt out of the logic referencing the correct rendering runtime in dev for a `shared-runtime` file. This is because we don't need to and that Turbopack does not support rewriting an external to something that looks like this `require('foo').bar.baz` yet. We will need to fix that when Turbopack build ships.
### New development pipeline
Bundling Next is now required when developing on the repo so I extended the taskfile setup to account for that. The webpack config for Next itself lives in `webpack.config.js` and contains the logic for all the new bundles generated.
### Misc changes
There are some misc reshuffling in the code to better use the tree shaking abilities that we can now use.
fixes NEXT-1573
Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
This adds new `build and test` and `build and deploy` workflows in favor
of the existing massive `build, test, and deploy` workflow. Since the
new workflows will use `pull_request_target` this waits to remove the
existing workflow until the new one is tested.
While testing this new workflow flakey behavior in tests have also been
addressed. Along with the new workflow we will also be leveraging new
runners which allow us to run tests against the production binary of
`next-swc` so this avoids slight differences in tests we've seen due to
running against the dev binary.
Furthermore we will have a new flow for allowing workflow runs on PRs
from external forks which will either require a comment be checking a
box approving the run after each change or a label added by the team.
The new flow also no longer relies on `actions/cache` or similar which
have proven to be pretty unreliable.
Tests runs with the new workflow can be seen here
https://github.com/vercel/next.js/actions/runs/5100673508/jobs/9169416949
With the addition of the query prefix we can hit the max length for PCRE
named matches so this reduces the prefix length and ensures we go
through the param name validation still
x-ref: https://twitter.com/simonecervini/status/1644123851003928579
This ensures we prefix the dynamic route params in the query so that
they can be kept separate from actual query params from the initial
request.
Fixes: https://github.com/vercel/next.js/issues/43139
This ensures we properly honor the `export const fetchCache` config and
also ensures we properly bypass fetch-cache when an On-Demand
Revalidation is occurring.
The `export const dynamic` handling is not changed here as that was
behaving correctly and should not influence fetch cache handling only
whether a page is prerendered fully or treated as SSR.
Fixes: https://github.com/vercel/next.js/issues/47273
x-ref: [slack
thread](https://vercel.slack.com/archives/C042LHPJ1NX/p1679078572123979)
<!--
Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
-->
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Errors have a helpful link attached, see
[`contributing.md`](https://github.com/vercel/next.js/blob/canary/contributing.md)
## Related
* Closes https://github.com/vercel/next.js/pull/39146 - PR abandoned.
## Details
I was having the same issue at work, as a workaround, we've changed the
`largePageDataBytes` setting as advised but we'd like to keep the
warning ideally. It's something we need to address, but just don't want
to spam our log aggregators while the large data issue isn't resolved.
I believe I've more or less copied what was in the original PR, some
small differences:
* I use `Set` instead of `Map`, please let me know if there is an
advantage to using Map!
* Just to keep code a bit tidier (subjective) put an early return
instead of nesting all the code in an if.
* Added a test to verify only one log appears even when the page is
accessed twice [as requested
here](https://github.com/vercel/next.js/pull/39146#pullrequestreview-1064486985).
Cases like `next/link` and `next/router` imports are not alias since
they're not matching the existing alias pattern setting for edge
runtime, which causes router-context being bundled twice (both with cjs
and esm) into edge worker bundle.
so we resolve their paths and alias them to esm bundle for webpack
bundling.
Other minor changes:
* update `require` calls to `import` expressions in edge ssr loaders
* remove client layer for apps without `appDir` enabled
* export `type` for ts typings in next/image to avoid alias to break
resolving
## Bug
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have a helpful link attached, see `contributing.md`
Fixes#39057
Will now show:
```
Warning: data for page "/[[...slug]]" (path "/some-page") is 256 kB which exceeds the threshold of 128 kB, this amount of data can reduce performance.
See more info here: https://nextjs.org/docs/messages/large-page-data
```
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
This ensures we create any nested folders necessary when setting initial ISR cache entries with the filesystem cache. Also adds a regression test for this.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
Fixes: https://github.com/vercel/next.js/issues/38130
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using fixes#33829
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
This continues off of https://github.com/vercel/next.js/pull/36285 fixing some of the failing test cases noticed when running the E2E tests against deployments. After these are resolved the tests will be added to our CI flow after each canary release.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
x-ref: https://github.com/vercel/next.js/pull/36285
This adds a new test mode `next-deploy` which allows testing against deployments using the Vercel CLI. After running these tests they uncovered some bugs we need to correct before fully enabling. Patching the uncovered issues will be done in follow-up PRs and then after resolved this will be enabled to run after new publishes.
Tests that uncovered bugs to patch in follow-ups:
- [ ] test/e2e/getserversideprops/test/index.test.ts (req.url normalizing)
- [ ] test/e2e/i18n-api-support/index.test.ts (locale prefixed API routes matching)
- [ ] test/e2e/prerender.test.ts (/_next/data/build-id/ does not 404)
* Add experimental ifGenereated flag for unstable_revalidate
* Apply suggestions from code review
Co-authored-by: Steven <steven@ceriously.com>
* update ifGenerated -> onlyGenerated
* rename const as well
Co-authored-by: Steven <steven@ceriously.com>
## Bug
Fixes a TypeError when accessing url with no path params `_next/data/<BUILD_ID>`
```
TypeError: Cannot read property 'endsWith' of undefined
at Object.fn (/app/node_modules/next/server/next-server.ts:644:52)
at Router.execute (/app/node_modules/next/server/router.ts:346:40)
at Server.run (/app/node_modules/next/server/next-server.ts:1229:41)
at Server.handleRequest (/app/node_modules/next/server/next-server.ts:489:25)
at /app/node_modules/next/server/next.ts:47:14
```
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
- [ ]
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [x] Make sure the linting passes
