### What?
This PR improves turbopack documentation, mostly collects existing
documents into a single place.
Also, adds a support to render mermaid diagram.
Closes PACK-2488
### What?
A follow up for #58267 and #58394.
We no longer need the git branch for history it's stored in datadog now
and we also decided that we no longer need slack reporting.
Closes PACK-2039
---------
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
### What?
Moving maintenance to a separate repository.
### Why?
I want to make these actions reusable in other projects as they seem to work well in the Next.js repository.
### How?
The code is moved to https://github.com/balazsorban44/nissuer
I tested it on my fork, and all the following functionality is preserved: https://github.com/balazsorban44/next.js/issues/56
- [x] Close/lock/comment without a valid reproduction link
- [x] Minimize "+1"
- [x] Add labels based on user selection
- [x] Add comment based on the maintainer's label (eg.: "please add a complete reproduction")
## What
Adds a reworked version of webpack's [built-in memory cache garbage
collection
plugin](853bfda35a/lib/cache/MemoryWithGcCachePlugin.js (L15))
that is more aggressive in cleaning up unused modules than the default.
The default marks 1/5th of the modules as "up for potentially being
garbage collected". The new plugin always checks all modules. In my
testing this does not cause much overhead compared to the current
approach which leverages writing to two separate maps. The change also
makes the memory cache eviction more predictable: when an item has not
been accessed for 5 compilations it is evicted from the memory cache, it
could still be in the disk cache.
In order to test this change I had to spin up the benchmarks but these
were a bit outdated so I've cleaned up the benchmark applications.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
## Vendoring
Updates all module resolvers (node, webpack, nft for entrypoints, and nft for next-server) to consider whether vendored packages are suitable for a given resolve request and resolves them in an import semantics preserving way.
### Problem
Prior to the proposed change, vendoring has been accomplished but aliasing module requests from one specifier to a different specifier. For instance if we are using the built-in react packages for a build/runtime we might replace `require('react')` with `require('next/dist/compiled/react')`.
However this aliasing introduces a subtle bug. The React package has an export map that considers the condition `react-server` and when you require/import `'react'` the conditions should be considered and the underlying implementation of react may differ from one environment to another. In particular if you are resolving with the `react-server` condition you will be resolving the `react.shared-subset.js` implementation of React. This aliasing however breaks these semantics because it turns a bare specifier resolution of `react` with path `'.'` into a resolution with bare specifier `next` with path `'/dist/compiled/react'`. Module resolvers consider export maps of the package being imported from but in the case of `next` there is no consideration for the condition `react-server` and this resolution ends up pulling in the `index.js` implementation inside the React package by doing a simple path resolution to that package folder.
To work around this bug there is a prevalence of encoding the "right" resolution into the import itself. We for instance directly alias `react` to `next/dist/compiled/react/react.shared-subset.js` in certain cases. Other times we directly specify the runtime variant for instance `react-server-dom-webpack/server.edge` rather than `react-server-dom-wegbpack/server`, bypassing the export map altogether by selecting the runtime specific variant. However some code is meant to run in more than one runtime, for instance anything that is part of the client bundle which executes on the server during SSR and in the browser. There are workaround like using `require` conditionally or `import(...)` dynamically but these all have consequences for bundling and treeshaking and they still require careful consideration of the environment you are running in and which variant needs to load.
The result is that there is a large amount of manual pinning of aliases and additional complexity in the code and an inability to trust the package to specify the right resolution potentially causing conflicts in future versions as packages are updated.
It should be noted that aliasing is not in and of itself problematic when we are trying to implement a sort of lightweight forking based on build or runtime conditions. We have good examples of this for instance with the `next/head` package which within App Router should export a noop function. The problem is when we are trying to vendor an entire package and have the package behave semantically the same as if you had installed it yourself via node_modules
### Solution
The fix is seemingly straight forward. We need to stop aliasing these module specifiers and instead customize the resolution process to resolve from a location that will contain the desired vendored packages. We can then start simplifying our imports to use top level package resources and generally and let import conditions control the process of providing the right variant in the right context.
It should be said that vendoring is conditional. Currently we only vendor react pacakges for App Router runtimes. The implementation needs to be able to conditionally determine where a package resolves based on whether we're in an App Router context vs a Page Router one.
Additionally the implementation needs to support alternate packages such as supporting the experimental channel for React when using features that require this version.
### Implementation
The first step is to put the vendored packages inside a node_modules folder. This is essential to the correct resolving of packages by most tools that implement module resolution. For packages that are meant to be vendored, meaning whole package substitution we move the from `next/(src|dist)/compiled/...` to `next/(src|dist)/vendored/node_modules`. The purpose of this move is to clarify that vendored packages operate with a different implementation. This initial PR moves the react dependencies for App Router and `client-only` and `server-only` packages into this folder. In the future we can decide which other precompiled dependencies are best implemented as vendored packages and move them over.
It should be noted that because of our use of `JestWorker` we can get warnings for duplicate package names so we modify the vendored pacakges for react adding either `-vendored` or `-experimental-vendored` depending on which release channel the package came from. While this will require us to alter the request string for a module specifier it will still be treating the react package as the bare specifier and thus use the export map as required.
#### module resolvers
The next thing we need to do is have all systems that do module resolution implement an custom module resolution step. There are five different resolvers that need to be considered
##### node runtime
Updated the require-hook to resolve from the vendored directory without rewriting the request string to alter which package is identified in the bare specifier. For react packages we only do this vendoring if the `process.env.__NEXT_PRIVATE_PREBUNDLED_REACT` envvar is set indicating the runtime is server App Router builds. If we need a single node runtime to be able to conditionally resolve to both vendored and non vendored versions we will need to combine this with aliasing and encode whether the request is for the vendored version in the request string. Our current architecture does not require this though so we will just rely on the envvar for now
##### webpack runtime
Removed all aliases configured for react packages. Rely on the node-runtime to properly alias external react dependencies. Add a resolver plugin `NextAppResolverPlugin` to preempt perform resolution from the context of the vendored directory when encountering a vendored eligible package.
##### turbopack runtime
updated the aliasing rules for react packages to resolve from the vendored directory when in an App Router context. This implementation is all essentially config b/c the capability of doing the resolve from any position (i.e. the vendored directory) already exists
##### nft entrypoints runtime
track chunks to trace for App Router separate from Pages Router. For the trace for App Router chunks use a custom resolve hook in nft which performs the resolution from the vendored directory when appropriate.
##### nft next-server runtime
The current implementation for next-server traces both node_modules and vendored version of packages so all versions are included. This is necessary because the next server can run in either context (App vs Page router) and may depend on any possible variants. We could in theory make two traces rather than a combined one but this will require additional downstream changes so for now it is the most conservative thing to do and is correct
Once we have the correct resolution semantics for all resolvers we can start to remove instances targeting our precompiled instances for instance making `import ... from "next/dist/compiled/react-server-dom-webpack/client"` and replacing with `import ... from "react-server-dom-webpack/client"`
We can also stop requiring runtime specific variants like `import ... from "react-server-dom-webpack/client.edge"` replacing it with the generic export `"react-server-dom-webpack/client"`
There are still two special case aliases related to react
1. In profiling mode (browser only) we rewrite `react-dom` to `react-dom/profiling` and `scheduler/tracing` to `scheduler/tracing-profiling`. This can be moved to using export maps and conditions once react publishses updates that implement this on the package side.
2. When resolving `react-dom` on the server we rewrite this to `react-dom/server-rendering-stub`. This is to avoid loading the entire react-dom client bundle on the server when most of it goes unused. In the next major react will update this top level export to only contain the parts that are usable in any runtime and this alias can be dropped entirely
There are two non-react packages currently be vendored that I have maintained but think we ought to discuss the validity of vendoring. The `client-only` and `server-only` packages are vendored so you can use them without having to remember to install them into your project. This is convenient but does perhaps become surprising if you don't realize what is happening. We should consider not doing this but we can make that decision in another discussion/PR.
#### Webpack Layers
One of the things our webpack config implements for App Router is layers which allow us to have separate instances of packages for the server components graph and the client (ssr) graph. The way we were managing layer selection was a but arbitrary so in addition to the other webpack changes the way you cause a file to always end up in a specific layer is to end it with `.serverlayer`, `.clientlayer` or `.sharedlayer`. These act as layer portals so something in the server layer can import `foo.clientlayer` and that module will in fact be bundled in the client layer.
#### Packaging Changes
Most package managers are fine with this resolution redirect however yarn berry (yarn 2+ with PnP) will not resolve packages that are not defined in a package.json as a dependency. This was not a problem with the prior strategy because it was never resolving these vendored packages it was always resolving the next package and then just pointing to a file within it that happened to be from react or a related package.
To get around this issue vendored packages are both committed in src and packed as a tgz file. Then in the next package.json we define these vendored packages as `optionalDependency` pointing to these tarballs. For yarn PnP these packed versions will get used and resolved rather than the locally commited src files. For other package managers the optional dependencies may or may not get installed but the resolution will still resolve to the checked in src files. This isn't a particularly satisfying implemenation and if pnpm were to be updated to have consistent behavior installing from tarballs we could actually move the vendoring entirely to dependencies and simplify our resolvers a fair bit. But this will require an upstream chagne in pnpm and would take time to propogate in the community since many use older versions
#### Upstream Changes
As part of this work I landed some other changes upstream that were necessary. One was to make our packing use `npm` to match our publishing step. This also allows us to pack `node_modules` folders which is normally not supported but is possible if you define the folder in the package.json's files property.
See: #52563
Additionally nft did not provide a way to use the internal resolver if you were going to use the resolve hook so that is now exposed
See: https://github.com/vercel/nft/pull/354
#### additional PR trivia
* When we prepare to make an isolated next install for integration tests we exclude node_modules by default so we have a special case to allow `/vendored/node_modules`
* The webpack module rules were refactored to be a little easier to reason about and while they do work as is it would be better for some of them to be wrapped in a `oneOf` rule however there is a bug in our css loader implementation that causes these oneOf rules to get deleted. We should fix this up in a followup to make the rules a little more robuts.
## Edits
* I removed `.sharedlayer` since this concept is leaky (not really related to the client/server boundary split) and it is getting refactored anyway soon into a precompiled runtime.
### What?
Adding a new label comment for "please simplify reproduction".
### Why?
During triaging, a lot of time is gone into trying to isolate the actual bug being reported. Reproductions sometimes contain unrelated code - that sometimes can be the actual cause of an issue, reported as a bug in Next.js, or require third-party services, secrets, setting up databases, and other tasks that take up much of the resources before the issue can even be verified.
### How?
Similar to the `please add a complete reproduction` and `please verify canary` labels, if a maintainer adds the `please simplify reproduction` label, an automatic comment will appear on the issue, asking for a minimal reproduction
[Slack thread](https://vercel.slack.com/archives/C04DUD7EB1B/p1685527818400339)
This PR migrates and adapts the [validate docs links script from `next-site`](https://github.com/vercel/front/pull/23185). This script is triggered by a GitHub action whenever a file in the `/docs` folder is updated. If broken links are found in the docs, the check fails and we post a comment on the PR with a list of broken links.
Co-authored-by: Michael Novotny <446260+manovotny@users.noreply.github.com>
### What?
Add `.vscode` to `.eslintignore`.
### Why?
We ignore `.vscode` folder in our tests, but Next.js itself creates it
so manually running tests locally will causing them to be added. That
causes ESLint to fail because it checks JSON files inside `.vscode`
folder.
### What?
A fork of https://github.com/github/issue-labeler that we pull in and
maintain.
### Why?
The official issue labeler had some issues which I tried to fix in PRs
and eventually got merged upstream, but the release cycle was a bit
slow, and that GitHub action had many parts we did not really need
anyway. So it makes sense to maintain it ourselves.
### How?
This PR removes the
https://github.com/vercel/next.js/blob/canary/.github/issue-labeler.yml
config file, which in our case is just a sublist of
https://github.com/vercel/next.js/labels anyway. Instead, we can pull in
the labels from GitHub directly and filter out those that we want to
apply to issues. This will keep things more in sync.
fix NEXT-750 ([link](https://linear.app/vercel/issue/NEXT-750))
---------
This PR improves the RSC plugin for SWC to throw an error when the `"use
client"` directive is in the top level, but not before other statements
/ expressions. For example:
Code:
```js
import 'react'
'use client'
```
Error:
```
The "use client" directive must be placed before other expressions. Move it to the top of the file to resolve this issue.
,----
3 | 'use client'
: ^^^^^^^^^^^^
`----
```
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the
feature request has been accepted for implementation before opening a
PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm build && pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing
doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
Co-authored-by: JJ Kasper <jj@jjsweb.site>
I did some experimenting to see if we can grab the latest version from npm instead of GitHub releases since that's the version the user can actually use.
The npm registry sends the package metadata in a gigantic (~8mb for `next`) JSON response and parsing it with `res.json()` takes several seconds, but using `ReadableStream`, we only care about the first chunk. This way we can grab the relevant version information in a few ms. ~I added `undici` which implements `res.body.getReader()`.~ Using Node 18/native `fetch`.
## Summary
This PR adds a new `worker` strategy to the `<Script />` component that automatically relocates and executes the script in a web worker.
```jsx
<Script
strategy="worker"
...
/>
```
[Partytown](https://partytown.builder.io/) is used under the hood to provide this functionality.
## Behavior
- This will land as an experimental feature and will only work behind an opt-in flag in `next.config.js`:
```js
experimental: {
nextScriptWorkers: true
}
```
- This setup use a similar approach to how ESLint and Typescript is used in Next.js by showing an error to the user to install the dependency locally themselves if they've enabled the experimental `nextScriptWorkers` flag.
<img width="1068" alt="Screen Shot 2022-03-03 at 2 33 13 PM" src="https://user-images.githubusercontent.com/12476932/156639227-42af5353-a2a6-4126-936e-269112809651.png">
- For Partytown to work, a number of static files must be served directly from the site (see [docs](https://partytown.builder.io/copy-library-files)). In this PR, these files are automatically copied to a `~partytown` directory in `.next/static` during `next build` and `next dev` if the `nextScriptWorkers` flag is set to true.
## Checklist
- [X] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [X] Related issues linked using `fixes #number`
- [X] Integration tests added
- [X] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
This PR fixes#31517.
fixes#32913
Adds support for decorator metadata in SWC when enabled through ts/jsconfig.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [x] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
* Use recommended pattern in testing example
Since the official linter for testing library, `eslint-plugin-testing-library` recommends using `screen` to write queries, this MR updates the testing library example to follow the pattern recommended by the linter.
> DOM Testing Library (and other Testing Library frameworks built on top of it) exports a screen object which has every query (and a debug method). This works better with autocomplete and makes each test a little simpler to write and maintain.
> This rule aims to force writing tests using built-in queries directly from screen object rather than destructuring them from render result. Given the screen component does not expose utility methods such as rerender() or the container property, it is correct to use the render returned value in those scenarios.
See the `prefer-screen-queries` rules docs for more info: https://github.com/testing-library/eslint-plugin-testing-library/blob/main/docs/rules/prefer-screen-queries.md
* Update devDependencies
* Install and configure test linting
* Use recommended pattern in test
* Update test names for consistency
* Update docs
* Set jest environment in each file
* Use root true in `with-jest` eslint config
* Ensure nested .eslintrcs are not loaded for repo lint
Co-authored-by: jj@jjsweb.site <jj@jjsweb.site>
This PR re-includes ESLint with some notable changes, namely a guided setup similar to how TypeScript is instantiated in a Next.js application.
To add ESLint to a project, developers will have to create an `.eslintrc` file in the root of their project or add an empty `eslintConfig` object to their `package.json` file.
```js
touch .eslintrc
```
Then running `next build` will show instructions to install the required packages needed:
<img width="862" alt="Screen Shot 2021-04-19 at 7 38 27 PM" src="https://user-images.githubusercontent.com/12476932/115316182-dfd51b00-a146-11eb-830c-90bad20ed151.png">
Once installed and `next build` is run again, `.eslintrc` will be automatically configured to include the default config:
```json
{
"extends": "next"
}
```
In addition to this change:
- The feature is now under the experimental flag and requires opt-in. After testing and feedback, it will be switched to the top-level namespace and turned on by default.
- A new ESLint shareable configuration package is included that can be extended in any application with `{ extends: 'next' }`
- This default config extends recommended rule sets from [`eslint-plugin-react`](https://www.npmjs.com/package/eslint-plugin-react), [`eslint-plugin-react-hooks`](https://www.npmjs.com/package/eslint-plugin-react-hooks), and [`eslint-plugin-next`](https://www.npmjs.com/package/@next/eslint-plugin-next)
- All rules in [`eslint-plugin-next`](https://www.npmjs.com/package/@next/eslint-plugin-next) have been modified to include actionable links that show more information to help resolve each issue
This is a follow-up to https://github.com/vercel/next.js/pull/23588 to update to use a regex lexer to gather the named regex groups instead of attempting to gather them through executing the regex since it can fail to gather the regex groups when they are using specific matching. This also ensures we don't pass the value as a segment when value is defined and it doesn't use a capture group. Additional tests are added to cover these cases and documentation updated to reflect this.
Closes: https://github.com/vercel/next.js/issues/23415
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
## Documentation / Examples
- [x] Make sure the linting passes
This pull request **temporarily** removes ESLint, as it was not landed in accordance with our standard experimental policies. We are fully committed to landing this change again.
This is being reverted because:
- Next.js has very strict goals for its install size. This feature resulted in adding over 17MB, or a 43.6% increase.
- The feature was not first landed under the `experimental` key in `next.config.js`, rather, it was added under the stable namespace (top-level)
- Using the feature doesn't do a "guided setup" like TypeScript, it should ask you to "bring your own" dependencies for ESLint
- It uses a undesirable ESLint plugin name: `plugin:@next/next/recommended`. This should read out as strictly `next`, or as short as we can get it.
- Does not provide actionable warnings (missing link to resolve issue)
- Does not follow appropriate console output styling. We need to revisit how these are presented.
To re-land this, we need to ensure the following minimums are met:
- Very minor change in install size
- Fully experimental (i.e. flagged) with warnings
- Finalized package name and configuration shape, preferably so we can do ` { extends: 'next' } `.
For #22228
This PR:
- Adds ESLint to toolchain
- Included by default for builds (`next build`)
- Can be enabled for development (`next dev`)
- Custom formatter built for output
- Adds appropriate tests
- Adds two documentation pages
* set up with-ts-eslint-jest example app
* eslint ignore new app bc it has a conflicting eslintrc
* make eslint + husky setup manual
* clarify app README setup notes
* move page tests out of pages/ dir
* Simplifying configs
* extend "prettier"
* format fix
* Updated rules
* Added husky configs and removed debug option
* Removed notes and configuration
* Updated pages
* Added links to readme
* Added example to .prettierignore
* Updated snap
* Make the lint work
Co-authored-by: Luis Alvarez D <luis@vercel.com>
