@timneutkens it'd be great to get your input.
These changes introduce a new Babel loader that eliminates much of the existing overhead, resulting in better HMR speeds.
Multithreading is still in flight, and may be omitted if speed improvements end up being negligible. For now, the new loader is hidden behind an `experimental` flag.
Items to be completed before this PR is ready to merge:
- [x] reconfigure `ncc` to precompile the parts of `@babel/core` and `@babel/traverse` that we're accessing directly
- [x] change `@babel/core/...` imports to `ncc`ed version
- [x] ~~measure multithreading (not currently pushed) functionality, and include the functionality depending on the results~~ I'll open a separate PR for this
- [x] ensure TypeScript is happy with all imports as final step (`--no-verify` was used to bypass)
There will be two follow-up PRs:
- loader support for projects with custom `.babelrc`
- multithreaded loader (should the change we warranted after measurement)
Within the "with-reason-relay" example:
- Redirect from NextJS-routed root route to reason-compiled home
## Bug
- [x] Related issues linked using `fixes #23607 `
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using `fixes #23607 `
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [x] Make sure the linting passes
* fixes a bundle size regression from webpack 4 to 5
* fixes these warnings during resolving build dependencies for packages using `exports`
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
Fixes#23687
## Bug
- [x] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [x] Make sure the linting passes
This ensures we don't reach the invariant from trying to create the export progress with no items during `next build` or `next export` by returning immediately when we notice there are no paths being exported. Additional tests have been added to ensure these cases don't error.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
Fixes: https://github.com/vercel/next.js/issues/23724
This ensures we are checking the filesystem correctly when i18n is used to prevent fallback rewrites from being applied even when a page is matched. Additional tests have been added to ensure these cases are being handled.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
Fixes: https://github.com/vercel/next.js/issues/23577
This ensures we don't attempt prepending the `basePath` for external (http://) `getStaticProps`/`getServerSideProps` redirects. Additional tests to cover this case have been added in the `gssp-redirect` test suites to prevent regression.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
Fixes: https://github.com/vercel/next.js/issues/23623
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
Fixes: #23716
Follow up to #23614. `resolvedAsPath` isn't actually even being used as an output, we're just mutating it for convenience. If we do want to output it in the future, we can do it a different way (i.e. via return).
This ensures we don't parse the default params when auto-exporting dynamic routes with i18n since the page and path aren't equal when localized. This adds an additional test case to ensure we don't regress on this behavior in the `i18n-support` test suites.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
Fixes: https://github.com/vercel/next.js/issues/23167
I don't believe NodeJS code should receive Bash syntax highlighting :).
## Bug
- [x] Related issues linked using `fixes #number`
- [] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
## Documentation / Examples
- [x] Make sure the linting passes
***
Related to #23607
We should use `NEXT_PUBLIC_` as [stated here](https://nextjs.org/docs/basic-features/environment-variables) to expose environment variables to the browser.
The code was already using the correct variables name.
## Bug
- [x] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
The `with-glamor` package has been unmaintained for 4 years now.
* has some memory usage improvements
* after a while cache entries that are unused and already persistent to disk are removed from the memory cache
* When are read again, they are restored from disk again
* Should play well together with adding and removing pages due to on-demand-entries
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
This updates to use the new `fallback` rewrites support instead of the previous no-op rewrite workaround.
## Documentation / Examples
- [x] Make sure the linting passes
The underlying package [style-sheet](https://github.com/giuseppeg/style-sheet) has been archived, and the example doesn't currently work, so I think it's better deleted. Closes part of issue #23607
## Documentation / Examples
- [x] Make sure the linting passes
This PR updates the Squoosh PNG decoder, which fixes#22929 in GoogleChromeLabs/squoosh#971.
## Bug
- [x] Fixes#22929
- [x] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
This ensures we gather segments from the experimental has field when validating segments used in the destination to prevent the invalid segments in the destination error from showing incorrectly. This usage has been added to the custom-routes test suite to ensure the segments are passed correctly from the has field.
Fixes: https://github.com/vercel/next.js/issues/23415
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
* Mark this.router protected
Seems that some people (#23558) are using Next.js internals based on the typings that TS generates automatically even though these values are not documented / should not be used in applications. This ensures the router value on Server is not used by accident.
* Mark all variables as protected
https://github.com/vercel/next.js/issues/23349
* fix type error
Co-authored-by: JJ Kasper <jj@jjsweb.site>
The package `next-plugin-preact` [already contains](38cfb182dd/packages/next-plugin-preact/package.json (L25)) the dependency `@prefresh/next` so it shouldn't be explicitly installed.
## Documentation / Examples
- [x] Make sure the linting passes
This RP fixes the problem that the image optimization API uses a large amount of memory, and is not correctly freed afterwards. There're multiple causes of this problem:
### 1. Too many WebAssembly instances are created
We used to do all the image processing operations (decode, resize, rotate, encodeJpeg, encodePng, encodeWebp) inside each worker thread, where each operation creates at least one WASM instance, and we create `os.cpus().length - 1` workers by default. That means in the worst case, there will be `N*6` WASM instances created (N is the number of CPU cores minus one).
This PR changes it to a pipeline-like architecture: there will be at most 6 workers, and the same type of operations will always be assigned to the same worker. With this change, 6 WASM instances will be created in the worst case.
### 2. WebAssembly memory can't be deallocated
It's known that [WebAssembly can't simply deallocate its memory as of today](https://stackoverflow.com/a/51544868/2424786). And due to the implementation/design of the WASM modules that we are using, they're not very suitable for long-running cases and it's more like a one-off use. For each operation like resize, it will allocate **new memory** to store that data. So the memory will increase quickly as more images are processed.
The fix is to get rid of `execOnce` for WASM module initializations, so each time a new WASM module will be created and the old module will be GC'd entirely as there's no reference to it. That's the only and easiest way to free the memory use of a WASM module AFAIK.
### 3. WebAssembly memory isn't correctly freed after finishing the operation
`wasm-bindgen` generates code with global variables like `cachegetUint8Memory0` and `wasm` that always hold the WASM memory as a reference. We need to manually clean them up after finishing each operation.
This PR ensures that these variables will be deleted so the memory overhead can go back to 0 when an operation is finished.
### 4. Memory leak inside event listeners
`emscripten` generates code with global error listener registration (without cleaning them up): https://github.com/vercel/next.js/blob/99a4ea6/packages/next/next-server/server/lib/squoosh/webp/webp_node_dec.js#L39-L43
And the listener has references to the WASM instance directly or indirectly: https://github.com/vercel/next.js/blob/99a4ea6/packages/next/next-server/server/lib/squoosh/webp/webp_node_dec.js#L183-L192 (`e`, `y`, `r`).
That means whenever a WASM module is created (emscripten), its memory will be kept by the global scope. And when we replace the WASM module with a new one, the newer will be added again and the old will still be referenced, which causes a leak.
Since we're running them inside worker threads (which will retry on fail), this PR simply removes these listeners.
### Test
Here're some statistics showing that these changes have improved the memory usage a lot (the app I'm using to test has one page of 20 high-res PNGs):
Before this PR (`next@10.1.0`):
<img src="https://user-images.githubusercontent.com/3676859/113058480-c3496100-91e0-11eb-9e5a-b325e484adac.png" width="500">
Memory went from ~250MB to 3.2GB (peak: 3.5GB) and never decreased again.
With fix 1 applied:
<img src="https://user-images.githubusercontent.com/3676859/113059060-921d6080-91e1-11eb-8ac6-83c70c1f2f75.png" width="500">
Memory went from ~280MB to 1.5GB (peak: 2GB).
With fix 1+2 applied:
<img src="https://user-images.githubusercontent.com/3676859/113059207-bf6a0e80-91e1-11eb-845a-870944f9e116.png" width="500">
Memory went from ~280MB to 1.1GB (peak: 1.6GB).
With fix 1+2+3+4 applied:
<img src="https://user-images.githubusercontent.com/3676859/113059362-ec1e2600-91e1-11eb-8d9a-8fbce8808802.png" width="500">
It's back to normal; memory changed from ~300MB to ~480MB, peaked at 1.2GB. You can clearly see that GC is working correctly here.
---
## Bug
- [x] Related issues #23189, #23436
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
This pull request **temporarily** removes ESLint, as it was not landed in accordance with our standard experimental policies. We are fully committed to landing this change again.
This is being reverted because:
- Next.js has very strict goals for its install size. This feature resulted in adding over 17MB, or a 43.6% increase.
- The feature was not first landed under the `experimental` key in `next.config.js`, rather, it was added under the stable namespace (top-level)
- Using the feature doesn't do a "guided setup" like TypeScript, it should ask you to "bring your own" dependencies for ESLint
- It uses a undesirable ESLint plugin name: `plugin:@next/next/recommended`. This should read out as strictly `next`, or as short as we can get it.
- Does not provide actionable warnings (missing link to resolve issue)
- Does not follow appropriate console output styling. We need to revisit how these are presented.
To re-land this, we need to ensure the following minimums are met:
- Very minor change in install size
- Fully experimental (i.e. flagged) with warnings
- Finalized package name and configuration shape, preferably so we can do ` { extends: 'next' } `.
