<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
-->
### What?
This PR implements a feature for the turbpack to support (partial)
transform configuration inherited from ts/jsconfigs, notably for the
legacy decorators and jsx runtimes.
### Why?
### How?
- Closes WEB-667
Continue the work to improve #32314.
This PR splits `webpack-build` into an `index` module and an `impl`
module. The `index` module invokes `impl` based on that if it should use
workers or in the same process.
Another notable change is that I modified the number of static workers,
to make it also based on remaining memory. I did some benchmarks and
found that using the number of CPUs might not be ideal. After the build
process (if `webpackBuildWorker` **isn't** enabled), the memory usage
was already very high. While the memory usage of the main process didn't
increase in the static optimization step, the overall number went high
again because we created 10 static workers:
As of today, a MacBook Pro has 10 CPU cores and 16 GB memory. The idle
memory of the system might be 8 GB, and the remaining memory when
starting the static workers might be 4~6 GB only. Here's a benchmark of
`next build` on `vercel-site`:
The memory dropped there quickly because it has `webpackBuildWorker`
enabled, but then it increased again with static workers (note that we
already have `cpus: Math.round(require('node:os').cpus().length / 2) ||
1` configured in that repo). It makes more sense to limit that inside
Next.js natively.
- Add links to the error warning
- Log with warning prefix for every url that missing `metadataBase`, so users can easily locate where it's from
Closes NEXT-931
### What?
Refactors the resolving logic for the `app` loader tree. This PR ensures
it's used to create entrypoints in turbopack. Next up is integrating it
into the webpack build too.
These changes also ensure that parallel routes resolving is applied,
which previously wasn't supported in turbopack.
### Why?
Part of the effort to deduplicate critical logic between
turbopack/webpack in Next.js, this will help land features in
turbopack/webpack at the same time.
### How?
Quite a few changes. @sokra helped a ton on this PR. `app_structure.rs`
was changed to the new resolving logic so most of the logic is there.
Additionally we added support for calling the same function in two ways
from Node.js: `turbo.entrypoints.get` and `turbo.entrypoints.getStream`.
`get` can be used by `next build` to get the full list of
entrypoints/loadertrees once. `getStream` has watching built-in and
calls a callback function with the new list anytime a file is added that
would change the loadertree.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Should fill dynamic routes url with params when generate the metadata
image urls.
For `/(group)`, we need to normalize the path first;
For dynamic routes `/[slug]`, we need to fill the params.
Change the image module from `() => image` to `(prop) => image(proop)`
so we can fill the runtime `params` into url
Closes NEXT-932
---------
This updates to have a separate routing process and separate rendering
processes for `pages` and `app` so that we can properly isolate the two
since they rely on different react versions.
Besides allowing the above mentioned isolation this also helps us
control recovering from process crashes easier as pieces are more
isolated from one another e.g. an infinite loop during rendering will no
longer block the compiler and can be stopped/restarted as needed.
In follow-up PRs we will continue to separate out the routing logic from
the rendering logic so that each process only loads what is relevant to
it helping simplify the flow for requests regardless of type.
---------
Co-authored-by: Shu Ding <g@shud.in>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### What / why?
Currently there is a bug because the `subTreeData` can only be applied
once, however, we reuse the same response when a url was prefetched.
During the first render of the response we apply the cacheNodes and then
these should be reused for subsequent navigations. There's currently a
bug because `fillLazyItemsTillLeafWithHead` creates `LAZY_INITIALIZED`
cache nodes that don't copy the previous cache node `parallelRoutes` and
`subTreeData` value, causing the cache nodes below where the
`subTreeData` is applied to be deleted from the cache, even when they're
not affected by the navigation.
### How?
This PR adds copying of the cache node when it exists already, I've
added a marker for `wasPrefetched` to trigger that behavior but in
talking to @feedthejim about this it seems we can swap to this behavior
always when applying the server response, that needs to be discussed
further though, this is temporary implementation with the fix for routes
that were prefetched.
Fixes NEXT-402
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Since both `exportPage` and `isPageStatic` methods in the static
optimization worker involve loading the actual page component, it's
better to assign the same page to the same worker for these 2 tasks to
avoid unnecessary memory allocation.
Co-authored-by: JJ Kasper <jj@jjsweb.site>
This ensures we properly honor the `export const fetchCache` config and
also ensures we properly bypass fetch-cache when an On-Demand
Revalidation is occurring.
The `export const dynamic` handling is not changed here as that was
behaving correctly and should not influence fetch cache handling only
whether a page is prerendered fully or treated as SSR.
Fixes: https://github.com/vercel/next.js/issues/47273
x-ref: [slack
thread](https://vercel.slack.com/archives/C042LHPJ1NX/p1679078572123979)
Fix the module resolving error found during development, because now few
app route code is bundled into RSC server layer, it can't resolve
opentelementry properly and will throw the error blow due to the check
of `require('@opentelemetry/api')` to see if it's installed.
```
Module not found: Can't resolve '@opentelemetry/api'
```
Thus I alias it to default `next/dist/compiled/@opentelemetry/api` when
it doesn't have external module
---------
* use `check` to wait for css change
* remove unused swr deps and update them to latest
* remove swr esm test in `app-dir/app/` since it's already covered in
`app-dir/app-external/`
---------
For backward compatibility, we only handle `favicon.ico` file to
generate `/favicon.ico` route and link tag. If you want to use other
extension such as `png`, use `icon(\d)?.[ext]`
Fixes#46993.
The issue is introduced in #46705 by @ijjk
Next.js uses `jest-worker` for multiprocessing. `jest-worker` supports both `child_process` and `worker_threads` modes. Next.js use `child_process` mode by default, but can also switch to using `worker_threads` mode with an experimental flag `config.experimental.workerThreads`.
In #46705, @ijjk applies a fix that works for the `child_process` mode, which breaks the `worker_threads` mode (`jest-worker`'s `NodeThreadWorker` interface doesn't have the private property `_child`), causing static HTML export to fail with the following error (#46993):
```
> Build error occurred
TypeError: Cannot read properties of undefined (reading 'on')
at createWorker (/Users/[redacted]/node_modules/next/dist/lib/worker.js:32:31)
at new Worker (/Users/[redacted]/node_modules/next/dist/lib/worker.js:42:9)
at /Users/[redacted]/node_modules/next/dist/build/index.js:629:35
at async Span.traceAsyncFn (/Users/[redacted]/node_modules/next/dist/trace/trace.js:79:20)
at async Object.build [as default] (/Users/[redacted]/node_modules/next/dist/build/index.js:74:29)
```
The PR fixes that.
### What?
see also https://github.com/vercel/turbo/pull/4415
We need a full process polyfill to fix some edge cases
Also disables SourceMap for edge chunks, since they are not used
### Why?
Some packages do weird things, e. g.
```
var isBrowser = process.title === 'browser'
```
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
- closes WEB-766.
This PR enables a new job for running tests, against turbopack. Since we
have observed some flakiness across turbopack test execution, it starts
from absolute minimum set of tests to not block any CI due to unexpected
failure.
Ran manual workflow roughly ~50 times and looks like these set of tests
are fine to make it as blocking check for the PR. In the future, depends
on the stability we'll increase number of test gradually.
This ensures we don't normalize locales in the URL with the
skipMiddlewareUrlNormalize flag enabled so that casing redirects can be
applied correctly for locales.
Turbopack starts up the router process with all ENV values, but the edge
function definition didn't list any `env` keys for the function
invocation. So, middleware couldn't access any ENV values.
Turbopack doesn't currently have a way to determine what ENV keys are
actually used by the source program, so I'm just passing everything
defined. I'm not sure if that's an issue during dev (I could see it
being one for the build process, but that doesn't matter for this case).
Fixes#47766
Fixes WEB-831
Fixes WEB-834
This ensures we check if a path is `appRouter` during prefetching so
that we can trigger hard navigations quicker when routing from pages ->
app. An additional config is also exposed to allow configuring the
potential false positive rate for the client filter.
x-ref: [slack
thread](https://vercel.slack.com/archives/C017QMYC5FB/p1680225393243459)
Paired with https://github.com/vercel/turbo/pull/4329, this implements
streaming responses for App and API renders. This is accomplished by
sending an initial `headers` message (carrying the status code and
headers list), N `body-chunk` messages of bytes, and a final `body-end`
message to signal completion.
Once sent to Turbopack, these chunk messages will be streamed out of the
node rendering process directly into the HTTP server's response.
Closes WEB-27
---------
Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
