This PR adds a second experimental post-processing step for the framework introduced by @prateekbh in #14746. The image post-processing step scans the rendered document for the first few images and uses a simple heuristic to determine if the images should be automatically preloaded.
Analysis of quite a few production Next apps has shown that a lot of sites are taking a substantial hit to their [LCP](https://web.dev/lcp/) score because an image that's part of the "hero" element on the page is not preloaded and is getting downloaded with lower priority than the JavaScript bundles. This post-processor should automatically fix that for a lot of sites, without causing any real performance effects in cases where it fails to identify the hero image.
This feature is behind an experimental flag, and will be subject to quite a bit of experimentation and tweaking before it's ready to be made a default setting.
By popular request, this pull request adds support for returning `fallback: 'blocking'` from `getStaticPaths`.
This new mode will cause unknown paths to be rendered on-demand ("SSR") without the static (placeholder) fallback.
This feature is **currently experimental and should not be used in production yet**. It's currently flagged behind `unstable_`:
```
fallback: 'unstable_blocking'
```
TODO:
- [x] Next.js tests
- [ ] Add Vercel support
- [ ] Vercel tests
---
Fixes#15637
This adds handling for custom-routes with `basePath` to automatically add the `basePath` for custom-routes `source` and `destination` unless `basePath: false` is set for the route.
Closes: https://github.com/vercel/next.js/issues/14782
* avoid pulling code in the bundle for `trailingSlash` logic when it's not enabled
* avoid cloning the url an extra time if normalizing the path doesn't change it
We've been meaning to change this code for a while 👍
- Changed the name from spr to incremental
- Changed the code to be a class instead of using module scope variables
Noticed this while reviewing https://github.com/vercel/next.js/pull/14376. After having done https://github.com/vercel/next.js/pull/13699, this code didn't feel right to me:
```js
function prepareRoute(path: string) {
path = delBasePath(path || '')
// this /index rewrite is problematic, it makes pages/index.js
// and pages/index/index.js point to the same thing:
return toRoute(!path || path === '/' ? '/index' : path)
}
```
Added a nested index page to the prerender tests and found it was rendering the `/` route on navigation. This uncovered 2 more places around the dataroute where the index path was not translated correctly.
**edit:**
Just to note that there was nothing wrong with https://github.com/vercel/next.js/pull/14376, the issue was already there, I just noticed it while reading that PR
Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)
Follow up of these PRs:
https://github.com/vercel/next.js/pull/13759https://github.com/vercel/next.js/pull/13870https://github.com/vercel/next.js/pull/13937https://github.com/vercel/next.js/pull/14130https://github.com/vercel/next.js/pull/14176https://github.com/vercel/next.js/pull/14268Fixes#6303Fixes#12087Fixes#1948Fixes#4368Fixes#4255Fixes#2548
Extracted from https://github.com/vercel/next.js/pull/13333, the same exact code lives in that PR as well, but we can merge this separately if it makes reviewing https://github.com/vercel/next.js/pull/13333 easier
This PR does 3 things
- deduplicate code from build and next-dev-server that loads custom routes from next.config.js (`loadCustomRoutes`)
- in `loadCustomRoutes`, load these rewrites, headers and redirects configs concurrently instead of sequentially.
- in next-server, make `this.customRoutes` always defined, this allows us to remove the big `if` around its initialization code in `generateRoutes`, which in turn makes it possible to reuse this code for other routing than user defined routes, which is how https://github.com/vercel/next.js/pull/13333 adds its redirects.
Disambiguate between pages/index.js and pages/index/index.js so that they resolve differently.
It all started with a bug in pagesmanifest that propagated throughout the codebase. After fixing pagesmanifest I was able to remove a few hacks here and there and more logic is shared now. especially the logic that resolves an entrypoint back into a route path. To sum up what happened:
- `getRouteFromEntrypoint` is the inverse operation of `getPageFile` that's under `pages/_document.tsx`
- `denormalizePagePath` is the inverse operation of `normalizePagePath`.
Everything is refactored in terms of these operations, that makes their behavior uniform and easier to update/patch in a central place. Before there were subtle differences between those that made `index/index.js` hard to handle.
Some potential follow up on this PR:
- [`hot-reloader`](https://github.com/vercel/next.js/pull/13699/files#diff-6161346d2c5f4b7abc87059d8768c44bR207) still has one place that does very similar behavior to `getRouteFromEntrypoint`. It can probably be rewritten in terms of `getRouteFromEntrypoint`.
- There are a few places where `denormalizePagePath(normalizePagePath(...))` is happening. This is a sign that `normalizePagePath` is doing some validation that is independent of its rewriting logic. That should probably be factored out in its own function. after that I should probably investigate whether `normalizePagePath` is even still needed at all.
- a lot of code is doing `.replace(/\\/g, '')`. If wanted, that could be replaced with `normalizePathSep`.
- It looks to me like some logic that's spread across the project can be centralized in 4 functions
- `getRouteFromEntrypoint` (part of this PR)
- its inverse `getEntrypointFromRoute` (already exists in `_document.tsx` as `getPageFile`)
- `getRouteFromPageFile`
- its inverse `getPageFileFromRoute` (already exists as `findPageFile ` in `server/lib/find-page-file.ts`)
It could be beneficial to structure the code to keep these fuctionalities close together and name them similarly.
- revise `index.amp` handling in pagesmanifest. I left it alone in this PR to keep it scoped, but it may be broken wrt nested index files as well. It might even make sense to reshape the pagesmanifest altogether to handle html/json/amp/... better
In serverless mode, it's best practice to propagate an unhandled error so that the function is disposed instead of recycled. This helps fix the "bad state" problem.
Was going through _document and noticed some variable shadowing going on. Added a rule for it to our eslint configuration and went through all warnings with @Timer.
Fixes https://github.com/vercel/next.js/issues/13524
To do:
- [x] fix dev mode
- [x] there's a ~route ordering or~ trailing slash issue with top level catch-all (current tests reflect that)
- [x] in this test `/get-static-paths/whatever` should fall back to `/[[...optionalName]].js` since `fallback` is `false` in its `getStaticPaths` method. ~Currently seems to 500~ must have been a glitch
- [x] add tests for `null`, `undefined` ~and `false`~ behavior as well (if decided these are valid)
- [x] ~add tests for string params as well~ this is not allowed for catch-all routes
- [x] test behavior when fallback is enabled and a top level catch-all exists
This addresses some errors for `/_next/data` requests where encoded `/` values in dynamic route param would cause invalid behavior, a headers already sent error would be shown when sending the fallback page in development, and when rendering the `_error` page for a data request the error response would still be treated as a data request.
This also adds test cases for these errors to prevent regression
Since non-fallback pages don't rely on the URL for hydration we can allow them to be rewritten to but pages with fallback still can't be rewritten to because we won't be able to parse the correct `/_next/data` path to request the page's data from. I added a test for this behavior and ensured it works correctly on Now.
Example on with fallback false rewrite on Now:
https://tst-rewrite-cp9vge4bg.now.sh/about
* Add support for params in header key/values
* Update todo
* Update with handling for named patterns
* Use compiled package
Co-authored-by: Joe Haddad <joe.haddad@zeit.co>
* Add basePath in link component and add/remove it consistently
* Update to not use regex for delBasePath
* Expose addBasePath as router method
* Revert "Expose addBasePath as router method"
This reverts commit 40fed596195c6affabf837e42d472452768e13a3.
* Expose basePath as router field
* Apply suggestion
* Expose basePath as router field
* remove un-used vars
* Update externals
* Apply lint fix
* Update size-limit test
* Update prefetch
* Add error/warning when a page is rendered without a /
Throws an error for development and gives a warning in production
* Add tests for error when rendering without starting slash
* Update to always warn and add err.sh
* Update errors/render-no-starting-slash.md
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
* Warn when user has pages/_error but no pages/404
* Update test
* Update custom-error-no-custom-404.md
* Apply suggestions from code review
Co-Authored-By: Tim Neutkens <tim@timneutkens.nl>
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
* Add initial support for new env config file
* Fix serverless processEnv call when no env is provided
* Add missing await for test method
* Update env config to .env.json and add dotenv loading
* ncc dotenv package
* Update type
* Update with new discussed behavior removing .env.json
* Update hot-reloader createEntrypoints
* Make sure .env is loaded before next.config.js
* Add tests for all separate .env files
* Remove comments
* Add override tests
* Add test for overriding env vars based on local environment
* Add support for .env.test
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Use chalk for env loaded message
* Remove constant as it’s not needed
* Update test
* Update errsh, taskr, and CNA template ignores
* Make sure to only consider undefined missing
* Remove old .env ignore
* Update to not populate process.env with loaded env
* Add experimental flag and add loading of global env values
Co-authored-by: Tim Neutkens <timneutkens@me.com>
Co-authored-by: Joe Haddad <joe.haddad@zeit.co>
* Verify GS(S)P Serializability
* Add support for cyclic refs
* Add unit tests
* Test for error in development mode
* Fix prerender preview tests
* Fix gssp preview tests
* fix 2x test cases
* Add desired test
* fix some more tests
* Fix route manifest expect
* Fix test expects
* Test that `getServerSideProps` does not error in production
* Test that getStaticProps is not checked in production
* Test serialization check during build
* Fix export detection for serverless
* Update test/unit/is-serializable-props.test.js
Co-Authored-By: JJ Kasper <jj@jjsweb.site>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
* Rename getServerProps to getServerSideProps
* Remove unstable_ prefix from new methods
* Add error when legacy methods are detected
* Add legacy methods for babel transform
* Add unstable_getServerSideProps also
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Update types import
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Ability to Disable SSG Fallback
* Throw error when value is missing
* Fix existing tests
* Adjust error message
* Do not render fallback at build time for `fallback: false` page
* Fix existing fallback behavior
* fix build
* fix version
* fix some tests
* Fix last test
* Add docs for get static paths
* Add explicit mode tests
* test for fallback error message
* Add calling getStaticPaths in development before showing fallback
* Move staticPathsWorker to next-dev-server
* Make sure to clear require cache in worker process
* bump
* Remove staticPathsCache member
* Update numWorkers for staticPathsWorker
* Clean up landed experimental flags
* Remove check for experimental flags from build too
* Remove /_errors/404 in favor of /404
* Remove unneeded check for pathname
* Update test paths
* Fix test
* Update test
* Remove test for disabled config
* Set pages404 always to true
Co-authored-by: Joe Haddad <timer150@gmail.com>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
* Adjust SSG Loading Behavior
* Update expected preview behavior
* Rename two corrections
* Only use skeleton in production for now
* Fix "should SSR SPR page correctly" test
* fix tests
* fix trailing comment letter
* disable test for now
* checkpoint: api impl
* Add support for tryGetPreviewData
* snapshot: server(less) support
* Add X-Prerender-Bypass-Mode header support
* Pass preview data to getStaticProps call
* add TODO
* setPreviewData
* 100k iterations
* Handle jwt error
* Write out preview values
* forgot file
* set preview props
* Send preview props
* add preview props
* Pass around more data
* update yarn lock
* Fail on Invalid Prerender Manifest
* Make Missing Prerender Manifest Fatal
* fix ts errors
* fix test
* Fix setting cookies + maxage
* Secure is not needed as we encrypt necessary data
* Set on domain root
* Set cookie max ages
* Render a fallback on-demand for non-dynamic pages
* Test preview mode
* remove old build
* remove snapshots
* Add serverless tests
* use afterAll
* Remove object assigns
* fix cookie spread
* add comment
* Update next-server routes order for expected priority
* Update router to allow disabling page routes
* Fix headers having check: true behavior when they should not
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Add initial SSG fallback handling
* Remove extra changes and update fallback handling
* Remove extra timeout for testing
* Update SSG tests in dynamic-routing suite
* Add racing to decide between rendering fallback and data
* Update size-limit test
* Update comment
* Make sure to follow correct route change order
* Make comment more verbose for racing
* Revert getStaticData to only return Promise
* Make sure to update URL on fallback
* Add retrying for data, de-dupe initial fallback request, and merge fallback replace
* Update to add preload for fallback pages data
* Add test for data preload link
* Use pre-built fallback in production mode
* Remove preload link for fallback from _document
* Update to make sure fallback is rendered correctly for serverless
* Update to use existing util to de-dupe path check
* Update error message for requested/resolved mismatch
* Use correct dataRoute value for prerender manifest
* Fix pageUrl having double slash on Windows
* Add support for rewriting to external resources
* Update rewrite proxying test
Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Implement experimental pages/404.js for custom 404 page
* Make sure to show error for getInitialProps in pages/404 in dev mode also
* Update routes-manifest tests for new value
* Make sure page404 is boolean in routes-manifest
* Rename variables for consistency
* Make sure to only use 404 page for 404 error
* Add support for unstable_getServerProps
* Apply suggestions from review
* Add no-cache header and update types
* Revert sharing of load-components type
* Add catchall test and update routes-manifest field
* Update header check
* Update to pass query for getServerProps data requests
* Update to not cache getServerProps requests
* Rename server side props identifier
* Update to nest props for getServerProps
* Add no-cache header in serverless-loader also
* Update to throw error for mixed SSG/serverProps earlier
* Add comment explaining params chosing in serverless-loader
* Update invalidKeysMsg to return a string and inline throwing
* Inline throwing mixed SSG/serverProps error
* Update setting cache header in serverless-loader
* Add separate getServerData method in router
* Update checkIsSSG -> isDataIdentifier
* Refactor router getData back to ternary
* Apply suggestions to build/index.ts
* drop return
* De-dupe extra escape regex
* Add param test
* Add permanent field for redirects
* Update printing redirect status code
* Don't add permanent to routes-manifest
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Update serving of files from public folder to handle special chars
* Update tests and match handling in dev mode
* Fix windows public file handling
* Remove colon test to make git on windows happy
* Remove un-used files
* Add missing await
* Add check: true behavior to custom routes
* Update adding dev routes
* Add checking of pages and dynamic routes for check: true
* Fix hasPage binding
* Add tests for check: true behavior
* Update regex checking
* Make changes based on review
* Update to handle rewrites in serverless loader
* Update to not change req.url
* Make sure to always parse dynamic route params
* Export all of pathToRegexp from path-match
Co-authored-by: Joe Haddad <timer150@gmail.com>