* Use sharp for image transformations when available
* Refactor resizeImage and add sharp warning
* only show sharp warning once per instance
* Modify sharp error message
* Add documentation for optional sharp dependency
* Update docs/basic-features/image-optimization.md
Co-authored-by: Steven <steven@ceriously.com>
* Import types for sharp
* Update lockfile
* Add testing against sharp
* use fs-extra for node 12
* Rename test sharp path variable
* Apply suggestions from code review
Co-authored-by: Steven <steven@ceriously.com>
* update squoosh specific test
* Apply suggestions from code review
Co-authored-by: Steven <steven@ceriously.com>
* update tests
* Apply suggestions from code review
Co-authored-by: Steven <steven@ceriously.com>
Co-authored-by: Steven <steven@ceriously.com>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
It's possible for `renderToHTML` to return `null` if `res.finished || res.headersSent` is `true` after `getInitialProps` or `getServerSideProps`. In such cases, we can't generate a valid `ResponseCacheEntry` or `ResponsePayload`, so we shouldn't try.
I took the opportunity to add an invariant for when we expected a cacheable response, but didn't get one. This could happen because Next.js or an application erroneously mutated the underlying `res` during the rendering of a page with `getStaticProps`. This shouldn't normally be possible because `res` isn't exposed in such cases, but it's theoretically possible with a custom server, so it seemed worth flagging.
This allows the user to adjust the LRU cache size according to the
application need:
- optimize the cache size based on average page HTML & JSON size
- disable the memory cache by setting the size to 0
The hard-coded default of 50MB has been moved from the code to the
default configuration object.
Fixes#21535
See also #27325
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [X] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [X] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [X] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
Currently there is a lot of mutation in the Next.js Server and the checks for Locale are directly coded in the general request handler. Ideally, we should have a function where we just pass the request input (url + headers + config) and generate a bunch of metadata that analyzes it generating all metadata we might require for both the URL and i18n + basePath information.
This PR brings:
- A new parsing function `parseUrl` that joins parsing an absolute/relative URL into a data structure compatible with the Node parsing output but missing redundant properties.
- A wrapper `parseNextURL` that extends `parseUrl` analyzing `i18n` and `basePath` based on the provided configuration, url and headers. This function is pure and stateless so it can be used outside of the Next.js context.
- Types improvements and reuse.
- Refactors `next-server.ts` request handling using the above mentioned functions so that the code there just apply effects to the `req` object and the `parsedUrl.query` leaving the code much more straightforward.
- Refactors `getRouteRegex` decomposing in two different functions where `getParametrizedRoute` can be used to retrieve the serializable data that is used to generate the Regex.
when typechecking next.config.js as described [in the docs](https://github.com/vercel/next.js/blob/canary/docs/basic-features/typescript.md#type-checking-nextconfigjs), an error would be thrown if `future` and `experimental` are missing. When using `Partial<>` in the type definition instead, it works as expected.
Fixes#25498
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
Fixes: #24806Fixes: #26689Fixes: #27325Closes: #24807
@tommarshall has done us a huge favor with his PR https://github.com/vercel/next.js/pull/24807 which outlines exactly the issue and a pragmatic solution.
I'm not trying to "steal their work", however, the PR seems to have been stuck for some months. I think there's huge value in this for myself and others as essentially **ISR is broken** for people running Next.js at scale 😱
This PR has cherry-picked @tommarshall's fine fix and added some integrations tests around page revalidation and the edge case when the cache size is exhausted.
✏️ Edits are enabled, so feel free great Vercel staff and other maintainers to improve my bad tests or surrounding code 🙇
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
How a page is rendered depends on whether or not we're streaming. For example, if we're just rendering to a `string` or we're generating a response for a crawler or other robot, we don't want React 18 to dynamically flush `<script>` tags to update Suspense boundaries as they resolve. Instead, we just want to wait for the full HTML to resolve and return a result similar to `renderToString`.
This is what `RequestContext` and the new/refactored `pipe` and `getStaticHTML` methods allow. They add a `requireStaticHTML` option that gets passed down. A follow-up PR will make sure this is `true` when serving a robot, and also ensure React is invoked appropriately.
In a previous PR (#27200), we added `minimumCacheTTL` to configure the time-to-live for the cached image. However, this was setting the `max-age` header.
This PR ensures that `minimumCacheTTL` doesn't affect browser caching, only the upstream header can affect browser caching.
This is a bit safer in case the developer accidentally caches something that shouldn't be and the cache needs to be invalidated. Simply delete the `.next/cache/images` directory.
- Related to #19914
- Related to #22319
* Add x-forward headers to external rewrites
This commit configures the proxy used for external rewrites to include
x-forward headers [1]. This is particularly useful for incremental
adoption, where some routes will be handled by Next.js and others by a
different website. For example, a Rails app will use the
X-Forwarded-Host header to determine which host to use for URL
generation and redirects [2].
[1]: 91fee3e943/lib/http-proxy.js (L31)
[2]: 41139f6ba2/actionpack/lib/action_dispatch/http/url.rb (L221-L227)
* Handle image-optimizer case
Co-authored-by: JJ Kasper <jj@jjsweb.site>
#### improve export spinner
update at least once a minute in non-tty
update progress regularly when using the spinner
decrease frequency of the spinner (windows console output is expensive)
#### restart static page generation and collecting page data worker pools when hanging
when for 1 minute no activity happens on the worker pool, restart it
log a warning for hanging jobs
#### add page generation duration to summary tree
for `[+n more pages]` is will show `(avg 321 ms)` when the average is over the threshold.
It will allocate 8 lines for preview pages (instead of 4) when they contain slow pages
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
- Closes#23328
- Related to #19914
- Related to #22319
## Feature
- [x] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`
- Use SWC to compile Next.js core server files
- Ensure only @babel/runtime/helpers/interopRequireDefault helper is used
Just an initial comparison to compare size difference of this change.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
This updates redirects' regexes to not match `/_next` paths since this is currently unexpected and can easily cause a multi-match redirect to break loading client-side assets. This also fixes custom-routes not matching correctly when `trailingSlash: true/false` is used
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Errors have helpful link attached, see `contributing.md`
x-ref: https://github.com/vercel/next.js/discussions/24683
x-ref: [slack thread](https://vercel.slack.com/archives/CGU8HUTUH/p1626159845474000)
This adds some missing fields to the `NextConfig` type we expose under `next` and also adds a `NextConfigComplete` internal type that allows us to not treat all fields as optional like you would when using the type in `next.config.js`.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
This PR is a workaround for #24421 by adding an artificial delay when Apple M1 + buggy Node.js is detected.
Node.js 14 is unaffected because it M1 still reports `arch=x64`. Starting in Node.js 16, M1 reports `arch=arm64`.
V8 Bug: https://crbug.com/1224882
Node.js Issue: https://github.com/nodejs/node/issues/39327
This PR changes the implementation of `placeholder=blur` when using `next dev` so that it lazy loads on-demand.
This will improve the developer experience for web apps with many blurred images.
add `experimental.esmExternals: boolean | 'loose'` config option
remove `output.environment` configuration in favor of `target`
| | `esmExternals: false` (default) | `esmExternals: 'loose'` | `esmExternals: true` |
| ------------------------ | ------------------------------- | ----------------------- | -------------------- |
| import cjs package | `require()` | `require()` | `require()` |
| require cjs package | `require()` | `require()` | `require()` |
| import mixed package | `require()` *** | `import()` | `import()` |
| require mixed package | `require()` | `require()` | `require()` |
| import pure esm package | `import()` | `import()` | `import()` |
| require pure esm package | Error ** | `import()` * | Error ** |
| import pure cjs package | `require()` | `require()` | Resolving error |
| require pure cjs package | `require()` | `require()` | `require()` |
cjs package: Offers only CJS implementation (may not even have an `exports` field)
mixed package: Offers CJS and ESM implementation via `exports` field
pure esm package: Only offers an ESM implementation (may not even have an `exports` field)
pure cjs package: CommonJs package that prevents importing via `exports` field when `import` is used.
`*` This case will behave a bit unexpected for now, since `require` will return a Promise. So that need to be awaited. This will be fixed once the whole next.js bundle is ESM. It didn't work at all before this PR.
`**` This is a new Error when trying to require an esm package.
`***` For mixed packages we prefer the CommonJS variant to avoid a breaking change.
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
### Description
The redirect responses from the redirect function do not contain a message body. This is in conflict with the RFCs below and causes Traefik (a reverse proxy) to invalidate the responses. In this pull request, I add a response body to the redirect responses.
### References
- https://datatracker.ietf.org/doc/html/rfc7230#section-3.3
> All 1xx (Informational), 204 (No Content), and 304 (Not Modified) responses must not include a message-body. All other responses do include a message-body, although the body may be of zero length.
- https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.3
> The server's response payload usually contains a short hypertext note with a hyperlink to the different URI(s).
- traefik/traefik#4456
- https://github.com/auth0/nextjs-auth0/pull/399
Since we are no longer accepting new built-in loaders, users may wish to use a different cloud provider.
So this PR renames `dangerously-unoptimized` to `custom` to handle this case as well as the intention of `next export`.
If the user doesn't add a `loader` prop, we throw an error.
If the user adds a `loader` prop but it doesn't return the width, we print a warning.
- Follow up to #26847
- Fixes#21079
- Fixes#19612
- Related to #26850
Make the typing for `IncrementalCache` more explicit. With streaming, we’ll want to stream page data as well as HTML. This is a bit complicated now because we’re overloading `pageData` for both redirects and pages.
This PR makes the different types explicit. With streaming, the data for redirects is synchronously available, while the data for pages will become a stream.
A follow up PR will add a “stream through” cache in front of `IncrementalCache`
Refactors the internals of `next-server` to use `ResponsePayload` instead of `string | null` and manual `sendPayload` calls. This is the first step toward streaming support.
I split `renderToResponseWithComponents` into a separate `renderToResponseWithComponentsInternal` function for ease of review: GitHub's diff rendering was highly misleading, making it seem as though more of the function had changed. The separate function just makes the actual change clearer: we split `renderToHTMLWithComponents` into two promises; one that represents the actual render result, and one that represents all of the work (including background work for e.g. revalidation) that needs to be done as part of generating the result.
These changes make it easier to bubble up a `ResponsePayload`, instead of sometimes calling `sendPayload` out-of-band, centralizing all payload handling in `sendResponse` and eventually a similar function for public APIs that return a string. This centralization will make it much easier to handle a response that needs to be streamed, which is coming soon in another PR.
This decreases the body size limit that triggers a warning from 5MB -> 4MB, which provides a little more wiggle room. Certain things like using base64 on body, headers, path, etc can cause the response to be larger than initially calculated.
Initial PR: https://github.com/vercel/next.js/pull/26831
This ensures when an error occurs in an API route while using minimal mode the error is bubbled so it can be handled at the top-level
## Bug
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
This PR adds a warning when api responses exceed 5mb since this will end up failing once deployed. In a future version this scenario will throw an error.
## Bug
- [x] Integration tests added
## Documentation / Examples
- [x] Make sure the linting passes
This pull request adds "Vary: Accept" header to responses from the image optimizer (i.e. the /_next/image endpoint).
The image optimizer prefers re-encoding JPG files to WebP, but some browsers (such as Safari 14 on Catalina) do not yet support WebP. In such cases the optimizer uses the Accept header sent by the browser to send out a JPG response. Thus the optimizer's response may depend on the Accept header.
Potential caching proxies can be informed of this fact by adding "Vary: Accept" to the response headers. Otherwise WebP data may be served to browsers that do not support it, for example in the following scenario:
* A browser that supports WebP requests the JPG. The optimizer re-encodes it to WebP. The proxy caches the WebP data.
* After this another browser that doesn't support WebP requests the JPG. The proxy sends the WebP data to the browser.
- [x] Integration tests added
- [x] Make sure the linting passes
This solves the main use case from Issue #19914.
Previously, we would set the `Cache-Control` header to a constant and rely on the server cache. This would mean the browser would always request the image and the server could response with 304 Not Modified to omit the response body.
This PR changes the behavior such that the `max-age` will propagate from the upstream server to the Next.js Image Optimization Server and allow browser caching. ("upstream" meaning external server or just an internal route to an image)
This PR does not change the `max-age` for static imports which will remain `public, max-age=315360000, immutable`.
#### Pros:
- Fewer HTTP requests after initial browser visit
- User configurable `max-age` via the upstream image `Cache-Control` header
#### Cons:
- ~~Might be annoying for `next dev` when modifying a source image~~ (solved: use `max-age=0` for dev)
- Might cause browser to cache longer than expected (up to 2x longer than the server cache if requested in the last second before expiration)
## Bug
- [x] Related issues linked using `fixes #number`
`next-dev-server` having its own implementations of `renderToHTML` and `renderErrorToHTML` has historically made reasoning about streaming hard, as it adds additional places where status codes are explicitly set and the full HTML is blocked on.
Instead, this PR simplifies things considerably by moving the majority of the custom logic for e.g. hot reloading and on-demand compilation to when we're resolving the page to be loaded, rather than upfront when handling the request. It also cleans up a few other details (e.g. default error page rendering) that managed to creep into the base implementation over time.
One unfortunate side effect is that this makes compilation errors slightly more difficult. Previously, we'd render them directly. Now, we have to rethrow them. But since they've already been logged (by the watch pipeline), we have to make sure they don't get logged again.
Fixes#24056.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
This expands on https://github.com/vercel/next.js/issues/24070 and ensures we show the dev overlay for additional cases like where `_app` or `_document` have syntax errors causing compilation to not be able to complete. This achieves showing the dev overlay even when compilation fails from a syntax error by doing a third minimal compilation in development with the needed client-side assets to render the dev overlay.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
x-ref: https://github.com/vercel/next.js/issues/24070
* import next-server logic during the time the configuration is loaded
* load minimizer plugins only when used
* load ReactDevOverlay only when used
* load only meta information of tsconfig for validation
* make worker for configuration loading lighter
* only load runTypeCheck when used
* load postcss config only when used
Tobias has fixed the `compiler.hooks.invalid.call()` bug in webpack 5 so this is no longer needed
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
## Documentation / Examples
- [ ] Make sure the linting passes
