This pull request edits the `BuildManifest` that is sent to `/_document` instead of modifying a single input array to decouple its implementation details.
Optimally, we'd eliminate the `files` key all together.
---
Related to #16182
- Using `namedChunks` where possible, this will also allow for faster access to the chunks as we no longer have to look them up like we did before using `find`
- Using the new asset hooks introduced in the latest webpack beta
- Using the new externals function signature
Initial PR to make `next build` work with webpack 5, still needs more work to make sure runtimeChunk and such are shared between pages.
- No longer needs the custom ChunkNamesPlugin as the default behavior was changed
- Dropping AMP First client page bundles is now compatible
Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)
Follow up of these PRs:
https://github.com/vercel/next.js/pull/13759https://github.com/vercel/next.js/pull/13870https://github.com/vercel/next.js/pull/13937https://github.com/vercel/next.js/pull/14130https://github.com/vercel/next.js/pull/14176https://github.com/vercel/next.js/pull/14268Fixes#6303Fixes#12087Fixes#1948Fixes#4368Fixes#4255Fixes#2548
Webpack will randomly execute script order if its runtime is not prioritized before chunks execute.
This seems to be somehow triggered in #13870 because of slightly different script ordering.
This had actually broke CSS, which is why our tests are failing 50% of the time:
Without this PR:
However, it's still problematic to use `async` in development since we rely on script execution order. So, this PR disables `async` in development.
We're exploring `defer` in the future anyway (over `async`), which will be ordered, so I don't mind diverging between dev and prod in this way.
---
Fixes#13911
Initial work to use chunkhashes instead of buildid for the page files in production. This does not change the calculation of the filename itself initially.
Disambiguate between pages/index.js and pages/index/index.js so that they resolve differently.
It all started with a bug in pagesmanifest that propagated throughout the codebase. After fixing pagesmanifest I was able to remove a few hacks here and there and more logic is shared now. especially the logic that resolves an entrypoint back into a route path. To sum up what happened:
- `getRouteFromEntrypoint` is the inverse operation of `getPageFile` that's under `pages/_document.tsx`
- `denormalizePagePath` is the inverse operation of `normalizePagePath`.
Everything is refactored in terms of these operations, that makes their behavior uniform and easier to update/patch in a central place. Before there were subtle differences between those that made `index/index.js` hard to handle.
Some potential follow up on this PR:
- [`hot-reloader`](https://github.com/vercel/next.js/pull/13699/files#diff-6161346d2c5f4b7abc87059d8768c44bR207) still has one place that does very similar behavior to `getRouteFromEntrypoint`. It can probably be rewritten in terms of `getRouteFromEntrypoint`.
- There are a few places where `denormalizePagePath(normalizePagePath(...))` is happening. This is a sign that `normalizePagePath` is doing some validation that is independent of its rewriting logic. That should probably be factored out in its own function. after that I should probably investigate whether `normalizePagePath` is even still needed at all.
- a lot of code is doing `.replace(/\\/g, '')`. If wanted, that could be replaced with `normalizePathSep`.
- It looks to me like some logic that's spread across the project can be centralized in 4 functions
- `getRouteFromEntrypoint` (part of this PR)
- its inverse `getEntrypointFromRoute` (already exists in `_document.tsx` as `getPageFile`)
- `getRouteFromPageFile`
- its inverse `getPageFileFromRoute` (already exists as `findPageFile ` in `server/lib/find-page-file.ts`)
It could be beneficial to structure the code to keep these fuctionalities close together and name them similarly.
- revise `index.amp` handling in pagesmanifest. I left it alone in this PR to keep it scoped, but it may be broken wrt nested index files as well. It might even make sense to reshape the pagesmanifest altogether to handle html/json/amp/... better
This removes remaining references to `granularChunks` in configs, error messages, and comments.
Also removed the `process.env.__NEXT_GRANULAR_CHUNKS` value.
---
Follow up to: https://github.com/vercel/next.js/pull/13663
Was going through _document and noticed some variable shadowing going on. Added a rule for it to our eslint configuration and went through all warnings with @Timer.
The experimental modern mode runs the type checking plugin twice, which **occasionally** suffers from a race condition that hangs the build.
This PR fixes type checking to only be run once.
While this test cannot 100% reproduce/capture the race condition, I don't feel strongly about the test case:
- We're planning on eliminating this type checking plugin ASAP (for a faster alternative)
- Modern mode implementation as-is will probably go away with webpack 5
Adding a conformance plugin the make sure users don't undo the benefits of the granularChunks config.
The plugin makes sure that minSize, maxInitialRequests values aren't overridden. Also ensures the cacheGroups - vendors, framework, libs, common, shared are maintained.
The warning and error messages do not break the build with this change. They only display a message.
cc - @prateekbh, @atcastle
* Add support for tsconfig/json `paths` option
* Add tests for paths in tsconfig.json
* Don't apply aliases when paths is empty
* Clean up unused methods and link to TypeScript license
* Add tests for jsconfig
* Put feature under an experimental flag
* Enable to see if tests pass
* Update types
* Add feature under an experimental flag
* Remove ts-ignore where possible
And replace by typecasts
* More accurate types
* bend cliententries in a correct shape earlier on
* comment becomes unnecessary
* add webpack overload to allow for the next.js use case
* Avoid changing public interface
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Adding native-url package
* Bumping native-url version
* Upgrading native-url
* Logging stats object for debugging
* Logging stats object for debugging
* Adding try catch to the error lines
* Experimenting with regex
* Experimenting with regex
* Experimenting with regex
* Testing regex changes
* Fixing defer-script test case to not include polyfill.js
* Meging changes with existing polyfill work
* Bumping version
* adjust webpack config
* Reduce size in size test
* Remove 1kb from legacy
* Bumping native-url version, includes fix for IE11
* Update lock file
* Updating native-url, fixes issue on IE11
* Fix sourcemap being added in document
* Adding Router as an app level dep. Fixes Router not being added as a dep to pages without Link when granularChunks is enabled
* Fix typescript error
* Fix modern + granularChunks hydration failing
* Fix TS error
* Update native-url version
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Polyfilling fetch and object-assign
* Polyfilling corejs object-assign
* Adding object-assign in polyfills.js. IE11 does not support Object.assign
* Fixing failing test
* Updating object.assign polyfill to fix aliasing
* Updating test case value to match new build stats
* Increasing the size of default build to 225kb
* Fixing defer-script test case to not include polyfill.js
* Revert README.md
* Re-design the polyfill approach based on PR feedback
* Adding comment and fixing test case
* Rename polyfills chunk
* Extract aliases into helper
* Remove extra new line
* Fix TypeScript typings
* Adding _internal_fetch alias
* Adjust build manifest plugin
* Build manifest plugin changes - adding a separate entry for polyfills
* Rename polyfills entry in build-manifest.json
* Remove old comment
* Fix TS
* Set key
* Polyfills already added
* Filtring polyfill.module.js
* Fix test
* Add __internal_fetch to alias rule
* Adjust name
* bump size
* ignore polyfills
* sigh
* Enable granular chunks config for all chunk types
Adding comment
Bug fix
* Update index.test.js
* Update index.test.js
* Fix test cases. Adding next/link to not trigger a different bug
* Removing obsolete comment
* Pass config.experimental.cpus to export during build
Currently, there is no way of specifying the number of worker threads of
`next export` when run as part of `next build`.
I suggest a sane default should be to just use the same amount of
workers that were used during the build process which currently seems to
be configured through `config.experimental.cpus`.
This setting is already respected in the two other places where
jest-workers are in use: The TerserPlugin and the staticCheckWorkers in
`next build`.
* Only enable worker threads if there is more than 1 worker
Multiple worker threads can cause problems when certain dependencies are
being used, see e.g. https://github.com/zeit/next.js/issues/7894
This patch allows disabling of worker threads by setting
`config.experimental.cpus = 1`.
The benefit of spawning 1 worker thread, if there is any at all, should
very limited anyways, so the workload can just as well be processed in
the main thread.
* Disable parallel build for firebase authentication example
* Add integration test to cover #7894
* Rename test suite and add worker_threads config
* Disable worker_threads by default
* Update index.test.js
* Use workerThreads config for TerserPlugin
* Update to use workerThreads config in
TerserPlugin for consistency
* Disable node 12 specific test
* Replace worker-farm with jest-worker
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Remove semaphores on top of jest-worker, unwind
terser worker, and remove extra error log
* Add custom profiler / tracer
This captures both client/production builds and everything before and after is cpu profiled, which makes sure that the whole process instead of just webpack is included in the final report.
* Update tests
* Update profiling-plugin.js
* Rename profiling-plugin.js to profiling-plugin.ts
* Update profiling-plugin.ts
* Update and rename profiling-plugin.ts to profiling-plugin.js
* Update webpack-config.ts
* Update profiling-plugin.js
* Add types
* Add missing type
* Add back TypeScript linting
* Remove tracing-js
* Add checking for hybrid AMP during static check
* Update to always hit server for prerender data
* Add removing of SPR code when not used
* Add checking for hybrid AMP during static check
* Update to always hit server for prerender data
* Add removing of SPR code when not used
* Update dead code elimination
* Refactor SplitChunksPlugin configs and add experimental chunking strategy
* Use typeDefs for SplitChunksConfig
* Modify build manifest plugin to create runtime build manifest
* Add support for granular chunks to page-loader
* Ensure normal behavior if experimental granularChunks flag is false
* Update client build manifest to remove iife & implicit global
* Factor out '/_next/' prepending into getDependencies
* Update packages/next/build/webpack-config.ts filepath regex
Co-Authored-By: Jason Miller <developit@users.noreply.github.com>
* Simplify dependency load ordering in page-loader.js
* Use SHA1 hash to shorten filenames for dependency modules
* Add scheduler to framework cacheGroup in webpack-config
* Update page loader to not duplicate script tags with query parameters
* Ensure no slashes end up in the file hashes
* Add prop-types to framework chunk
* Fix issue with mis-attributed events
* Increase modern build size budget--possibly decrement after consulting with @janicklasralph
* Use module.rawRequest for lib chunks
Co-Authored-By: Daniel Stockman <daniel.stockman@gmail.com>
* Dasherize lib chunk names
Co-Authored-By: Daniel Stockman <daniel.stockman@gmail.com>
* Fix typescript errors, reorganize lib name logic
* Dasherize rawRequest, short circuit name logic when rawRequest found
* Add `scheduler` package to test regex
* Fix a nit
* Adjust build manifest plugin
* Shorten key name
* Extract createPreloadLink helper
* Extract getDependencies helper
* Move method
* Minimize diff
* Minimize diff x2
* Fix Array.from polyfill
* Simplify page loader code
* Remove async=false for script tags
* Code golf `getDependencies` implementation
* Require lib chunks be in node_modules
* Update packages/next/build/webpack-config.ts
Co-Authored-By: Joe Haddad <timer150@gmail.com>
* Replace remaining missed windows compat regex
* Trim client manifest
* Prevent duplicate link preload tags
* Revert size test changes
* Squash manifest size even further
* Add comment for clarity
* Code golfing 🏌️♂️
* Correctly select modern dependencies
* Ship separate modern client manifest when module/module enabled
* Update packages/next/build/webpack/plugins/build-manifest-plugin.ts
Co-Authored-By: Joe Haddad <timer150@gmail.com>
* Remove unneccessary filter from page-loader
* Add lookbehind to file extension regex in page-loader
* v9.0.3
* Update examples for Apollo with AppTree (#8180)
* Update examples for Apollo with AppTree
* Fix apolloClient being overwritten when rendering AppTree
* Golf page-loader (#8190)
* Remove lookbehind for module replacement
* Wait for build manifest promise before page load or prefetch
* Updating modern-only chunks inside the right entry point
* Fixing ts errors
* Rename variable
* Revert "Wait for build manifest promise before page load or prefetch"
This reverts commit c370528c6888ba7fa71162a0854534ed280224ef.
* Use proper typedef for webpack chunk
* Re-enable promisified client build manifest
* Fix bug in getDependencies map
* Insert check for granularChunks in page-loader
* Increase size limit temporarily for granular chunks
* Add 50ms delay to flaky test
* Set env.__NEXT_GRANULAR_CHUNKS in webpack config
* Reset size limit to 187
* Set process.env.__NEXT_GRANULAR_CHUNKS to false if selectivePageBuilding
* Update test/integration/production/test/index.test.js
Co-Authored-By: Joe Haddad <timer150@gmail.com>
* Do not create promise if not using chunking PR
* Experimental: Serverless Trace target
The Serverless Trace target produces Serverless-handler wrapped entrypoints, but does not bundle all of `node_modules`.
This behavior increases bundling performance to be more akin to `target: 'server'`.
This mode is expected to be used with smart platforms (like [ZEIT Now](https://zeit.co/now) that can trace a program to its minimum dependencies.
* Use more generic variables
* Add asset relocator for production mode of serverless trace
* Verify Firebase compatiblity
* Revert "Add asset relocator for production mode of serverless trace"
This reverts commit 8404f1dcf28b60edab41a56c94b38dcd3fddec20.
* Add serverless trace tests
* Add _isLikeServerless helper
* Make constants
* Fix export
* Update packages/next-server/server/config.ts
Co-Authored-By: JJ Kasper <jj@jjsweb.site>
* Use a global helper for is like serverless
* Update import for isTargetLikeServerless
* Update packages/next/build/index.ts
Co-Authored-By: JJ Kasper <jj@jjsweb.site>
* Module/nomodule implementation based on RFC 7563
* Remove comment
* Fixing issue with building amp pages
* Fixing test cases for serverless mode
* Adding safari 10 nomodule fix. Preloading modern js by default
* Fixing size-limit integration test
* Bug fix
* Adding testcase for modern build
* Trigger rebuild
* Setting default crossOrigin value
* Moving modern config option inside experimental flag
* Adding nomodule attribute to safari-fix script
* Changing safari10NomoduleFix default value to true
* Removing safari-fix flag
* Changing .es6 to .module
* Disable modern default
* Removing default crossOrigin value. Setting modern flag to false by default. Fixed test cases
* Remove confusing defaults and mark required instead
* Adjust blacklist
* Move behavior of page marking
* Fixing childCompiler errors not being captured
* Tweak names
* Revert
* whoops
* Fixing bug with page-loader.js
* Changing modern babel cache name
* Rename helper
* Iterate over both bundles
* Correctly clamp bundle sizes
* Revert test
* Add modern mode tests
* Fix test
* test
* test2
