Commit graph

330 commits

Author SHA1 Message Date
Aurélien MANCA
97a140f733
fix(critical CSS): use the assetPrefix in the critters config (#27506)
This PR fixes #25927 bug which prevents to use `assetPrefix` config with the experimental critical CSS feature.

Fixes: #25927



## Bug

- [x] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
2021-07-26 21:00:47 +00:00
Sojin Park
23ac4351f4
fix: Make router ready in case of custom _app getInitialProps (#27473)
From https://github.com/vercel/next.js/pull/20628,  when the page is rendered server-side, `Router`'s `isReady` field needs to be initially set to `true`. However, when `_app` has custom `getInitialProps`, it seems that it is not the case, even though the page is rendered on the server. 

This leads to a bug that `Router.isReady` is never set to `true`.

This pull request fixes the problem by fixing the initial calculation logic of `isReady` of `Router`.

## Bug

- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [x] Make sure the linting passes
2021-07-26 20:25:55 +00:00
stefanprobst
622f63867a
Update eslint doc comment (#27478)
the behavior of the `eslint.dirs` config option was changed in #26401. this pr adjusts the doc comment accordingly.

## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-25 13:48:46 +00:00
JJ Kasper
c54f168d9a
Add config check for invalid duplicate locale domains (#27451) 2021-07-23 17:16:14 -05:00
stefanprobst
046f7164db
Add eslint config options to NextConfig type (#27446)
This adds missing eslint config options to the public `NextConfig` type.

## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-23 19:15:26 +00:00
JJ Kasper
a8b9c3a3ed
Ensure generateBuildId is not required in config type (#27443) 2021-07-23 12:14:25 -05:00
Steven
1d347ba5dc
Fix typo in error message (#27438)
Fixes typo mentioned in https://github.com/vercel/next.js/pull/27346#discussion_r675477904
2021-07-23 14:07:09 +00:00
Alex Castle
1b733423d6
Use Sharp if available for Image Optimization (#27346)
* Use sharp for image transformations when available

* Refactor resizeImage and add sharp warning

* only show sharp warning once per instance

* Modify sharp error message

* Add documentation for optional sharp dependency

* Update docs/basic-features/image-optimization.md

Co-authored-by: Steven <steven@ceriously.com>

* Import types for sharp

* Update lockfile

* Add testing against sharp

* use fs-extra for node 12

* Rename test sharp path variable

* Apply suggestions from code review

Co-authored-by: Steven <steven@ceriously.com>

* update squoosh specific test

* Apply suggestions from code review

Co-authored-by: Steven <steven@ceriously.com>

* update tests

* Apply suggestions from code review

Co-authored-by: Steven <steven@ceriously.com>

Co-authored-by: Steven <steven@ceriously.com>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
2021-07-22 18:11:17 -05:00
Gerald Monaco
1651129429
Don't cache null responses (#27403)
It's possible for `renderToHTML` to return `null` if `res.finished || res.headersSent` is `true` after `getInitialProps` or `getServerSideProps`. In such cases, we can't generate a valid `ResponseCacheEntry` or `ResponsePayload`, so we shouldn't try.

I took the opportunity to add an invariant for when we expected a cacheable response, but didn't get one. This could happen because Next.js or an application erroneously mutated the underlying `res` during the rendering of a page with `getStaticProps`. This shouldn't normally be possible because `res` isn't exposed in such cases, but it's theoretically possible with a custom server, so it seemed worth flagging.
2021-07-22 21:04:58 +00:00
Steven
79ed8e13b0
Add keepAlive to node-fetch polyfill (#27376)
Fixes #27109 

This PR adds a default `agent` as described in the [`node-fetch` docs](https://github.com/node-fetch/node-fetch#custom-agent).

We should see about 2x perf according to some [benchmarks](https://github.com/Ethan-Arrowood/undici-fetch/blob/main/benchmarks.md#fetch).
2021-07-22 14:34:33 +00:00
JJ Kasper
52ae69ac33
Rename spr config to ISR (#27379) 2021-07-21 21:42:33 -05:00
Stefan Becker
dbfce34ead
Add configuration for ISR Memory Cache Size (#21535) (#27337)
This allows the user to adjust the LRU cache size according to the
application need:

- optimize the cache size based on average page HTML & JSON size
- disable the memory cache by setting the size to 0

The hard-coded default of 50MB has been moved from the code to the
default configuration object.

Fixes #21535
See also #27325



## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [X] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [X] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [X] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-21 17:39:38 +00:00
Javi Velasco
e65c56e7e6
Refactor i18n checks on request handling (#27328)
Currently there is a lot of mutation in the Next.js Server and the checks for Locale are directly coded in the general request handler. Ideally, we should have a function where we just pass the request input (url + headers + config) and generate a bunch of metadata that analyzes it generating all metadata we might require for both the URL and i18n + basePath information.

This PR brings:
- A new parsing function `parseUrl` that joins parsing an absolute/relative URL into a data structure compatible with the Node parsing output but missing redundant properties.
- A wrapper `parseNextURL` that extends `parseUrl` analyzing `i18n` and `basePath` based on the provided configuration, url and headers. This function is pure and stateless so it can be used outside of the Next.js context.
- Types improvements and reuse.
- Refactors `next-server.ts` request handling using the above mentioned functions so that the code there just apply effects to the `req` object and the `parsedUrl.query` leaving the code much more straightforward.
- Refactors `getRouteRegex` decomposing in two different functions where `getParametrizedRoute` can be used to retrieve the serializable data that is used to generate the Regex.
2021-07-21 16:12:33 +00:00
Andres Barbaro
bdcb8b0a71
Allow enabling worker threads on dev server (#23027)
Co-authored-by: JJ Kasper <jj@jjsweb.site>
2021-07-20 19:12:57 -05:00
fabb
8e34902fb9
Update NextConfig type to not require experimental or future fields (#25517)
when typechecking next.config.js as described [in the docs](https://github.com/vercel/next.js/blob/canary/docs/basic-features/typescript.md#type-checking-nextconfigjs), an error would be thrown if `future` and `experimental` are missing. When using `Partial<>` in the type definition instead, it works as expected.

Fixes #25498

## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-20 18:42:29 +00:00
Jamie
7c408e7a0f
Fix ISR page re-rendering after revalidate expiry (Fork of #24807) (#27335)
Fixes: #24806 
Fixes: #26689
Fixes: #27325
Closes: #24807

@tommarshall has done us a huge favor with his PR https://github.com/vercel/next.js/pull/24807 which outlines exactly the issue and a pragmatic solution.

I'm not trying to "steal their work", however, the PR seems to have been stuck for some months. I think there's huge value in this for myself and others as essentially **ISR is broken** for people running Next.js at scale 😱 

This PR has cherry-picked @tommarshall's fine fix and added some integrations tests around page revalidation and the edge case when the cache size is exhausted.

✏️ Edits are enabled, so feel free great Vercel staff and other maintainers to improve my bad tests or surrounding code 🙇 

## Bug

- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
2021-07-20 17:01:42 +00:00
Gerald Monaco
ade52553b6
Add RequestContext (#27303)
How a page is rendered depends on whether or not we're streaming. For example, if we're just rendering to a `string` or we're generating a response for a crawler or other robot, we don't want React 18 to dynamically flush `<script>` tags to update Suspense boundaries as they resolve. Instead, we just want to wait for the full HTML to resolve and return a result similar to `renderToString`.

This is what `RequestContext` and the new/refactored `pipe` and `getStaticHTML` methods allow.  They add a `requireStaticHTML` option that gets passed down. A follow-up PR will make sure this is `true` when serving a robot, and also ensure React is invoked appropriately.
2021-07-20 16:04:31 +00:00
Steven
b123942694
Fix minimumCacheTTL so it doesn't affect browser caching (#27307)
In a previous PR (#27200), we added `minimumCacheTTL` to configure the time-to-live for the cached image. However, this was setting the `max-age` header.

This PR ensures that `minimumCacheTTL` doesn't affect browser caching, only the upstream header can affect browser caching.

This is a bit safer in case the developer accidentally caches something that shouldn't be and the cache needs to be invalidated. Simply delete the `.next/cache/images` directory.

- Related to #19914
- Related to #22319
2021-07-19 22:38:03 +00:00
Thomas Marshall
ccb62f8636
Add x-forward headers to external rewrites (#17557)
* Add x-forward headers to external rewrites

This commit configures the proxy used for external rewrites to include
x-forward headers [1]. This is particularly useful for incremental
adoption, where some routes will be handled by Next.js and others by a
different website. For example, a Rails app will use the
X-Forwarded-Host header to determine which host to use for URL
generation and redirects [2].

[1]: 91fee3e943/lib/http-proxy.js (L31)
[2]: 41139f6ba2/actionpack/lib/action_dispatch/http/url.rb (L221-L227)

* Handle image-optimizer case

Co-authored-by: JJ Kasper <jj@jjsweb.site>
2021-07-16 10:11:12 -05:00
Tobias Koppers
f9795fdd26
improve static generation UX (#27171)
#### improve export spinner

update at least once a minute in non-tty
update progress regularly when using the spinner
decrease frequency of the spinner (windows console output is expensive)

#### restart static page generation and collecting page data worker pools when hanging

when for 1 minute no activity happens on the worker pool, restart it
log a warning for hanging jobs

#### add page generation duration to summary tree

![image](https://user-images.githubusercontent.com/1365881/125750454-8845f1b1-faf0-4598-b7a4-ea796b884691.png)

for `[+n more pages]` is will show `(avg 321 ms)` when the average is over the threshold.
It will allocate 8 lines for preview pages (instead of 4) when they contain slow pages

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-16 09:21:44 +00:00
Steven
8151a7e0dc
Add minimumCacheTTL config for Image Optimization (#27200)
- Closes #23328  
- Related to #19914 
- Related to #22319 


## Feature

- [x] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`
2021-07-15 19:55:12 +00:00
Tim Neutkens
fdabeb7098
Use SWC to compile Next.js core server files (#27167)
- Use SWC to compile Next.js core server files
- Ensure only @babel/runtime/helpers/interopRequireDefault helper is used

Just an initial comparison to compare size difference of this change.



## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-14 18:12:04 +00:00
JJ Kasper
8c3c2b7ea2
Update redirect regexes to not match _next (#27143)
This updates redirects' regexes to not match `/_next` paths since this is currently unexpected and can easily cause a multi-match redirect to break loading client-side assets. This also fixes custom-routes not matching correctly when `trailingSlash: true/false` is used

## Bug

- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Errors have helpful link attached, see `contributing.md`

x-ref: https://github.com/vercel/next.js/discussions/24683
x-ref: [slack thread](https://vercel.slack.com/archives/CGU8HUTUH/p1626159845474000)
2021-07-13 19:38:14 +00:00
JJ Kasper
9c93c0543f
Add some missing fields to the NextConfig type (#27126)
This adds some missing fields to the `NextConfig` type we expose under `next` and also adds a `NextConfigComplete` internal type that allows us to not treat all fields as optional like you would when using the type in `next.config.js`. 

## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-12 21:38:57 +00:00
Gerald Monaco
cb8da5778e
Replace withCoalescedInvoke with ResponseCache (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.

This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.

As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 19:47:39 +00:00
Steven
ed19335d6e
Workaround for Node.js 16+ on Apple Silicon M1 (#27031)
This PR is a workaround for #24421 by adding an artificial delay when Apple M1 + buggy Node.js is detected.

Node.js 14 is unaffected because it M1 still reports `arch=x64`. Starting in Node.js 16, M1 reports `arch=arm64`.

V8 Bug: https://crbug.com/1224882

Node.js Issue: https://github.com/nodejs/node/issues/39327
2021-07-12 17:05:41 +00:00
Steven
31c3f33639
Enhance next dev performance with placeholder=blur (#27061)
This PR changes the implementation of `placeholder=blur` when using `next dev` so that it lazy loads on-demand.

This will improve the developer experience for web apps with many blurred images.
2021-07-10 20:27:14 +00:00
Tobias Koppers
7a8da9741d
add support for esm externals (#27069)
add `experimental.esmExternals: boolean | 'loose'` config option

remove `output.environment` configuration in favor of `target`

|                          | `esmExternals: false` (default) | `esmExternals: 'loose'` | `esmExternals: true` |
| ------------------------ | ------------------------------- | ----------------------- | -------------------- |
| import cjs package       | `require()`                     | `require()`             | `require()`          |
| require cjs package      | `require()`                     | `require()`             | `require()`          |
| import mixed package     | `require()` ***                 | `import()`              | `import()`           |
| require mixed package    | `require()`                     | `require()`             | `require()`          |
| import pure esm package  | `import()`                      | `import()`              | `import()`           |
| require pure esm package | Error **                        | `import()` *            | Error **             |
| import pure cjs package  | `require()`                     | `require()`             | Resolving error      |
| require pure cjs package | `require()`                     | `require()`             | `require()`          |

cjs package: Offers only CJS implementation (may not even have an `exports` field)
mixed package: Offers CJS and ESM implementation via `exports` field
pure esm package: Only offers an ESM implementation (may not even have an `exports` field)
pure cjs package: CommonJs package that prevents importing via `exports` field when `import` is used.

`*` This case will behave a bit unexpected for now, since `require` will return a Promise. So that need to be awaited. This will be fixed once the whole next.js bundle is ESM. It didn't work at all before this PR.
`**` This is a new Error when trying to require an esm package.
`***` For mixed packages we prefer the CommonJS variant to avoid a breaking change.

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [x] Errors have helpful link attached, see `contributing.md`

## Documentation / Examples

- [ ] Make sure the linting passes
2021-07-10 16:49:02 +00:00
Michiel Van Gendt
d22ecd97f4
Include message body in redirect responses (#25257)
### Description
The redirect responses from the redirect function do not contain a message body. This is in conflict with the RFCs below and causes Traefik (a reverse proxy) to invalidate the responses. In this pull request, I add a response body to the redirect responses.

### References
- https://datatracker.ietf.org/doc/html/rfc7230#section-3.3
> All 1xx (Informational), 204 (No Content), and 304 (Not Modified) responses must not include a message-body. All other responses do include a message-body, although the body may be of zero length.

- https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.3
> The server's response payload usually contains a short hypertext note with a hyperlink to the different URI(s).

- traefik/traefik#4456
- https://github.com/auth0/nextjs-auth0/pull/399
2021-07-09 16:32:48 +00:00
Steven
1a8ad14089
Rename next/image dangerously-unoptimized to custom and warn when applicable (#26998)
Since we are no longer accepting new built-in loaders, users may wish to use a different cloud provider.

So this PR renames `dangerously-unoptimized` to `custom` to handle this case as well as the intention of `next export`.

If the user doesn't add a `loader` prop, we throw an error.
If the user adds a `loader` prop but it doesn't return the width, we print a warning.

- Follow up to #26847 
- Fixes #21079 
- Fixes #19612 
- Related to #26850
2021-07-08 19:35:19 +00:00
Darsh Patel
226e81c733
Add dangerously-unoptimized loader for next/image (#26847)
* add support for static loader

* update existing test

* add test

* add prettier-ignore

* update docs

* apply suggestions from review

* remove un-needed normalize

* Apply suggestions from code review

Co-authored-by: Steven <steven@ceriously.com>

* update test

Co-authored-by: JJ Kasper <jj@jjsweb.site>
Co-authored-by: Steven <steven@ceriously.com>
2021-07-07 15:20:16 -05:00
Gerald Monaco
f7bc8f4b7c
Fix merge issue and use respondWith (#26961) 2021-07-06 13:34:50 -05:00
Gerald Monaco
53123ad61f
More explicit typing for IncrementalCache API (#26941)
Make the typing for `IncrementalCache` more explicit. With streaming, we’ll want to stream page data as well as HTML. This is a bit complicated now because we’re overloading `pageData` for both redirects and pages.

This PR makes the different types explicit. With streaming, the data for redirects is synchronously available, while the data for pages will become a stream.

A follow up PR will add a “stream through” cache in front of `IncrementalCache`
2021-07-06 15:45:49 +00:00
Gerald Monaco
2795b79959
Add initial ResponsePayload support (#26938)
Refactors the internals of `next-server` to use `ResponsePayload` instead of `string | null` and manual `sendPayload` calls. This is the first step toward streaming support.

I split `renderToResponseWithComponents` into a separate `renderToResponseWithComponentsInternal` function for ease of review: GitHub's diff rendering was highly misleading, making it seem as though more of the function had changed. The separate function just makes the actual change clearer: we split `renderToHTMLWithComponents` into two promises; one that represents the actual render result, and one that represents all of the work (including background work for e.g. revalidation) that needs to be done as part of generating the result.

These changes make it easier to bubble up a `ResponsePayload`, instead of sometimes calling `sendPayload` out-of-band, centralizing all payload handling in `sendResponse` and eventually a similar function for public APIs that return a string. This centralization will make it much easier to handle a response that needs to be streamed, which is coming soon in another PR.
2021-07-06 09:41:51 +00:00
Gerald Monaco
402f0360ce
Refactor decode failures (#26899) 2021-07-05 18:31:32 +02:00
Maia Teegarden
b8f3a6d2d9
5MB -> 4MB body size limit (#26887)
This decreases the body size limit that triggers a warning from 5MB -> 4MB, which provides a little more wiggle room. Certain things like using base64 on body, headers, path, etc can cause the response to be larger than initially calculated. 

Initial PR: https://github.com/vercel/next.js/pull/26831
2021-07-02 23:17:56 +00:00
JJ Kasper
a6965961bd
Ensure API route errors are propagated in minimal mode (#26875)
This ensures when an error occurs in an API route while using minimal mode the error is bubbled so it can be handled at the top-level 

## Bug

- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
2021-07-02 20:54:53 +00:00
Maia Teegarden
538095c936
Warn when response body is larger than 5mb (#26831)
This PR adds a warning when api responses exceed 5mb since this will end up failing once deployed. In a future version this scenario will throw an error.

## Bug

- [x] Integration tests added

## Documentation / Examples

- [x] Make sure the linting passes
2021-07-02 19:27:33 +00:00
Tim Neutkens
270487d797
Make sure 404 pages do not get cached by a CDN when using next start (#24983)
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
2021-07-02 10:40:13 +02:00
Steven
03b61778ba
Fix immutable header for image with static import & unoptimized (#26836)
Fixes #26587
2021-07-01 22:14:42 +00:00
Steven
0562cc77bc
Add additional tests for image type detection (#26832)
Adding additional tests. Follow up to #26705
2021-07-01 20:53:26 +00:00
Joachim Viide
d670198e7c
Add "Vary: Accept" header to /_next/image responses (#26788)
This pull request adds "Vary: Accept" header to responses from the image optimizer (i.e. the /_next/image endpoint).

The image optimizer prefers re-encoding JPG files to WebP, but some browsers (such as Safari 14 on Catalina) do not yet support WebP. In such cases the optimizer uses the Accept header sent by the browser to send out a JPG response. Thus the optimizer's response may depend on the Accept header.

Potential caching proxies can be informed of this fact by adding "Vary: Accept" to the response headers. Otherwise WebP data may be served to browsers that do not support it, for example in the following scenario:
 * A browser that supports WebP requests the JPG. The optimizer re-encodes it to WebP. The proxy caches the WebP data.
 * After this another browser that doesn't support WebP requests the JPG. The proxy sends the WebP data to the browser.

- [x] Integration tests added
- [x] Make sure the linting passes
2021-07-01 19:59:16 +00:00
Jiachi Liu
c51598268a
Leverage blocked page for _error (#26748)
## Enhance

simplify detection for visiting `_error`

x-ref: #26610
2021-07-01 00:08:36 +00:00
Steven
2373320fc8
Add upstream max-age to optimized image (#26739)
This solves the main use case from Issue #19914.

Previously, we would set the `Cache-Control` header to a constant and rely on the server cache. This would mean the browser would always request the image and the server could response with 304 Not Modified to omit the response body.

This PR changes the behavior such that the `max-age` will propagate from the upstream server to the Next.js Image Optimization Server and allow browser caching. ("upstream" meaning external server or just an internal route to an image)

This PR does not change the `max-age` for static imports which will remain `public, max-age=315360000, immutable`.

#### Pros:
- Fewer HTTP requests after initial browser visit
- User configurable `max-age` via the upstream image `Cache-Control` header

#### Cons:
- ~~Might be annoying for `next dev` when modifying a source image~~ (solved: use `max-age=0` for dev)
- Might cause browser to cache longer than expected (up to 2x longer than the server cache if requested in the last second before expiration)

## Bug

- [x] Related issues linked using `fixes #number`
2021-06-30 21:26:20 +00:00
Tim Neutkens
5b9ad8da90
Move next-server directory files to server directory (#26756)
* Move next-server directory files to server directory

* Update tests

* Update paths in other places
2021-06-30 13:44:40 +02:00
Tim Neutkens
136b754396
Move code shared between server/client to "shared" folder (#26734) 2021-06-30 11:43:31 +02:00
Gerald Monaco
9ab916ac99
Simplify next-dev-server implementation (#26230)
`next-dev-server` having its own implementations of `renderToHTML` and `renderErrorToHTML` has historically made reasoning about streaming hard, as it adds additional places where status codes are explicitly set and the full HTML is blocked on.

Instead, this PR simplifies things considerably by moving the majority of the custom logic for e.g. hot reloading and on-demand compilation to when we're resolving the page to be loaded, rather than upfront when handling the request. It also cleans up a few other details (e.g. default error page rendering) that managed to creep into the base implementation over time.

One unfortunate side effect is that this makes compilation errors slightly more difficult. Previously, we'd render them directly. Now, we have to rethrow them. But since they've already been logged (by the watch pipeline), we have to make sure they don't get logged again.
2021-06-29 23:02:10 +00:00
JJ Kasper
917a9acc2c
Update to only add image import types when enabled (#26485)
* Update to only add image import types when enabled

* add type check to test
2021-06-22 11:56:21 -05:00
JJ Kasper
76f0b3585f
Update to latest TypeScript version and de-dupe versions (#26285)
* Update to latest TypeScript version and de-dupe versions

* Update version test
2021-06-17 17:43:25 -05:00
JJ Kasper
ec9ed57487
Remove import type syntax (#25145)
* Remove import type syntax

* Update build-output test
2021-05-14 13:42:36 -05:00
Darsh Patel
9e87596bd6
Fix: Non-writable pages/_app breaks build (#24849)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-10 10:39:18 +02:00
Shu Ding
6cd1c87451
Fix not exposing server errors in hot reloader (#24331)
Fixes #24056.

## Bug

- [x] Related issues linked using `fixes #number`
- [x] Integration tests added

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.

## Documentation / Examples

- [ ] Make sure the linting passes
2021-04-22 11:47:15 +00:00
JJ Kasper
85d87a3795
Ensure dev overlay is triggered for more _app/_document errors (#24328)
This expands on https://github.com/vercel/next.js/issues/24070 and ensures we show the dev overlay for additional cases like where `_app` or `_document` have syntax errors causing compilation to not be able to complete. This achieves showing the dev overlay even when compilation fails from a syntax error by doing a third minimal compilation in development with the needed client-side assets to render the dev overlay. 

## Bug

- [x] Related issues linked using `fixes #number`
- [x] Integration tests added

x-ref: https://github.com/vercel/next.js/issues/24070
2021-04-22 11:08:47 +00:00
Tim Neutkens
8d84b08752
Add event for webpack 5 usage (#24222) 2021-04-20 16:46:40 +02:00
Tobias Koppers
5bdef6aabe
startup performance improvements (#24129)
* import next-server logic during the time the configuration is loaded
* load minimizer plugins only when used
* load ReactDevOverlay only when used
* load only meta information of tsconfig for validation
* make worker for configuration loading lighter
* only load runTypeCheck when used
* load postcss config only when used
2021-04-18 10:28:09 +00:00
Tim Neutkens
2b48670bd6
Remove bugfix needed for webpack 4 when webpack 5 is used (#23886)
Tobias has fixed the `compiler.hooks.invalid.call()` bug in webpack 5 so this is no longer needed



## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.

## Documentation / Examples

- [ ] Make sure the linting passes
2021-04-11 15:06:35 +00:00
Joe Haddad
eecc3342cb
Temporarily remove experimental ESLint integration (#23521)
This pull request **temporarily** removes ESLint, as it was not landed in accordance with our standard experimental policies. We are fully committed to landing this change again.

This is being reverted because:

- Next.js has very strict goals for its install size. This feature resulted in adding over 17MB, or a 43.6% increase.
- The feature was not first landed under the `experimental` key in `next.config.js`, rather, it was added under the stable namespace (top-level)
- Using the feature doesn't do a "guided setup" like TypeScript, it should ask you to "bring your own" dependencies for ESLint
- It uses a undesirable ESLint plugin name: `plugin:@next/next/recommended`. This should read out as strictly `next`, or as short as we can get it.
- Does not provide actionable warnings (missing link to resolve issue)
- Does not follow appropriate console output styling. We need to revisit how these are presented.

To re-land this, we need to ensure the following minimums are met:
- Very minor change in install size
- Fully experimental (i.e. flagged) with warnings
- Finalized package name and configuration shape, preferably so we can do ` { extends: 'next' } `.
2021-03-30 08:26:35 +00:00
Tim Neutkens
b34a0c98fa
Update err.sh links to use nextjs.org/docs/messages instead (#23353) 2021-03-29 10:25:00 +02:00
JJ Kasper
d130f63c41
Add handling fo beforeFiles, afterFiles, and fallback rewrites (#23407)
This adds support for returning an object from `rewrites` in `next.config.js` with `beforeFiles`, `afterFiles`, and `fallback` to allow specifying rewrites at different stages of routing. The existing support for returning an array for rewrites is still supported and behaves the same way. The documentation has been updated to include information on these new stages that can be rewritten and removes the outdated note of rewrites not being able to override pages. 



## Bug

- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added

## Feature

- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.

## Documentation / Examples

- [ ] Make sure the linting passes
2021-03-26 15:19:48 +00:00
Houssein Djirdeh
e5ef60fecb
Add ESLint to Next.js (#22437)
For #22228

This PR:

- Adds ESLint to toolchain
  - Included by default for builds (`next build`)
  - Can be enabled for development (`next dev`)
  - Custom formatter built for output
- Adds appropriate tests
- Adds two documentation pages
2021-03-23 21:32:42 +00:00
Shu Ding
1bf4cf3e76
Upgrade jest-worker (#23077)
This PR upgrades `jest-worker` and `jest-cli` to the latest pre-release version, also removed `jest-circus` which is included in Jest by default. `jest-worker@next` includes a fix for memory leak that we need (https://github.com/facebook/jest/pull/11187). 

Fixes #22925. This will also improve the OOM issue for `next dev` #15855.
2021-03-16 21:08:35 +00:00
Dale Bustad
e27b7e996d
Telemetry-compatible tracing (#22713)
A number of changes here.  I recommend viewing the diff with the <a href="?w=1">whitespace flag enabled</a>.

- OpenTelemetry is replaced with a custom and lightweight tracing solution.
- Three trace targets are currently supported: console, Zipkin, and NextJS.
- Tracing is now governed by environment variables rather than `--require instrument.js`.
  + `TRACE_TARGET`: one of `CONSOLE`, `ZIPKIN`, or `TELEMETRY`; defaults to `TELEMETRY` if unset or invalid.
  + `TRACE_ID`: an 8-byte hex-encoded value used as the Zipkin trace ID; if not provided, this value will be randomly generated and passed down to subprocesses.

Other sundry:

- I'm missing something, probably a setup step, with the Zipkin target.  Traces are captured successfully, but you have to manually enter the Trace ID in order to view the trace - it doesn't show up in queries.
- I'm generally unhappy with [this commit](235cedcb3e).  It is... untidy to provide a telemetry object via `setGlobal`, but I don't have a ready alternative.  Is `distDir` strictly required when creating a new Telemetry object?  I didn't dig too deep here.

As noted, there are a lot of changes, so it'd be great if a reviewer could:

- [ ] pull down the branch and try to break it
- [ ] check the Zipkin traces and identify possible regressions in the functionality

Closes #22570
Fixes #22574
2021-03-10 21:00:20 +00:00
Jan Potoms
1ebc9bbb5e
Load next.config.js async (#22578)
Just the conversion of config loading from sync to async from https://github.com/vercel/next.js/pull/22153

cc @Timer
2021-02-27 00:29:32 +00:00
JJ Kasper
51e0752f87
Add generating static 500 status page (#22139)
This adds generating a static 500 status page when a `pages/500.js` file is added similar to how we handle generating static 404 pages when `pages/404.js` is present. This allows showing a customized error page when a 500 error occurs in an optimal way.
2021-02-22 16:29:50 +00:00
Tim Neutkens
3cef2e10ed
Delay server compilation until client compilation is done in development (#22187)
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-19 11:10:19 +01:00
Guy Bedford
bddb02286f
feat: webpack inlining with configuration for v4 / v5 (#20598) 2021-01-13 20:59:08 -05:00
JJ Kasper
c98db81bf4
Remove import type syntax from core files (#20379)
This removes `import type` usage from our core files since `import type` requires a higher TypeScript version than currently expected.

Fixes: https://github.com/vercel/next.js/issues/19300
2020-12-28 22:04:51 +00:00
JJ Kasper
42385f48db
Ensure i18n custom routes resolves correctly (#19766)
This ensures i81n custom routes are resolving correctly, it also corrects redirects for default locales when `locale: false` is not used, and this also simplifies the resolving for custom routes with i18n in `next-server` adding types for the i18n config. This also adds additional tests to prevent regression and starts leveraging a shared `NextConfig` type to allow us to share types for config values. 

Fixes: https://github.com/vercel/next.js/issues/19521
Fixes: https://github.com/vercel/next.js/issues/19403
Fixes: https://github.com/vercel/next.js/issues/19302
Fixes: https://github.com/vercel/next.js/issues/19227
2020-12-04 10:14:55 +00:00
Guy Bedford
8221c180a5
ncc 0.25.0 upgrade and fixes (#18873)
This upgrades to ncc@0.25.0 and fixes the previous bugs including:

* ncc not referenced correctly in build
* Babel type errors
* node-fetch, etag, chalk and raw-body dependencies not building with ncc - these have been "un-ncc'd" for now. As they are relatively small dependencies, this doesn't seem too much of an issue and we can follow up in the tracking ncc issue at https://github.com/vercel/ncc/issues/612.
* `yarn dev` issues

Took a lot of bisecting, but the overall diff isn't too bad here in the end.
2020-11-06 02:33:14 +00:00
Joe Haddad
9a13dd3466
Enable i18n feature flag (#18303) 2020-10-27 10:30:34 -05:00
JJ Kasper
5a8398e34d
Remove unstable_ prefix from unstable_blocking (#18276)
This removes the `unstable_` prefix to prepare for stabilizing the feature
2020-10-27 04:01:37 +00:00
JJ Kasper
b2d1d87e7f
Add initial changes for i18n support (#17370)
This adds the initial changes outlined in the [i18n routing RFC](https://github.com/vercel/next.js/discussions/17078). This currently treats the locale prefix on routes similar to how the basePath is treated in that the config doesn't require any changes to your pages directory and is automatically stripped/added based on the detected locale that should be used. 

Currently redirecting occurs on the `/` route if a locale is detected regardless of if an optional catch-all route would match the `/` route or not we may want to investigate whether we want to disable this redirection automatically if an `/index.js` file isn't present at root of the pages directory. 

TODO: 

- [x] ensure locale detection/populating works in serverless mode correctly
- [x] add tests for locale handling in different modes, fallback/getStaticProps/getServerSideProps

To be continued in fall-up PRs

- [ ] add tests for revalidate, auto-export, basePath + i18n
- [ ] add mapping of domains with locales
- [ ] investigate detecting locale against non-index routes and populating the locale in a cookie

x-ref: https://github.com/vercel/next.js/issues/17110
2020-10-07 21:11:01 +00:00
Joe Haddad
c351f6154b
Improve server performance by skipping decode/re-encode (#17323)
Prior to this pull request, Next.js would immediately decode all URLs sent to its server (via `path-match`).

This was rarely needed, and Next.js would typically re-encode the incoming request right away (see all the `encodeURIComponent`s removed in PR diff). This adds unnecessary performance overhead.

Long term, this will also help prevent weird encoding edge-cases like #10004, #10022, #11371, et al.

---

No new tests are necessary for this change because we've extensively tested these edge cases with existing tests.
One test was updated to reflect that we skip decoding in a 404 scenario.

Let's see if all the existing tests pass!
2020-09-24 06:05:40 +00:00
JJ Kasper
f46ddc6933
Correct page path for GS(S)P data refreshing (#16939)
This makes sure to the page path is the expected version to trigger refreshing on the client and adds additional tests to make sure it is working properly with these page variants. 

Closes: https://github.com/vercel/next.js/issues/16938
2020-09-08 16:00:05 +00:00
JJ Kasper
4685332353
Add automatic reloading when editing GS(S)P methods (#16744)
This adds initial support for reloading the page when `getStaticProps`, `getStaticPaths`, or `getServerSideProps` were changed for a page by triggering a reload when the server output for a page has changed but the client output has not since these methods aren't included in the client output. 

Closes: https://github.com/vercel/next.js/issues/13949
2020-09-02 16:57:21 +00:00
Joe Haddad
33176806f3
Remove unused dependency (#16168) 2020-08-13 13:40:08 -04:00
JJ Kasper
8a489e24bc
Add initial handling for dynamic route href resolving and rewrites on the client (#15231)
Co-authored-by: Tim Neutkens <timneutkens@me.com>
2020-08-13 14:39:36 +02:00
Tim Neutkens
e72ac76422
Remove tslint disables (#16116)
We no longer use tslint so these comments don't do anything.
2020-08-12 14:39:07 +00:00
jiangtao
35e4a370d9
fixed issue with runtime-config returning undefined when building or in a development environment (#15777)
fix https://github.com/vercel/next.js/issues/7713#issuecomment-643632270
2020-08-04 16:47:37 +00:00
Joe Haddad
e818389999
Add support for fallback: 'blocking' (#15672)
By popular request, this pull request adds support for returning `fallback: 'blocking'` from `getStaticPaths`.

This new mode will cause unknown paths to be rendered on-demand ("SSR") without the static (placeholder) fallback.

This feature is **currently experimental and should not be used in production yet**. It's currently flagged behind `unstable_`:

```
fallback: 'unstable_blocking'
```

TODO:

- [x] Next.js tests
- [ ] Add Vercel support
- [ ] Vercel tests

---

Fixes #15637
2020-08-04 15:10:31 +00:00
Tim Neutkens
210a6980d2
Solve large portion of webpack 5 deprecation warnings (#15806)
- Using `namedChunks` where possible, this will also allow for faster access to the chunks as we no longer have to look them up like we did before using `find`
- Using the new asset hooks introduced in the latest webpack beta
- Using the new externals function signature
2020-08-03 12:26:23 +00:00
Joe Haddad
cac35c389a
Fix hot reloader edge case with broken webpack plugins (#15659) 2020-07-29 23:44:25 -04:00
Joe Haddad
13a971c8b5
Do not duplicate compilation errors (#15299)
* Do not duplicate compilation errors

* Add tests

* Attempt to fix test

* Fix test
2020-07-19 14:09:41 -04:00
Wayne Warner
7834766907
Fix DevServer#close doesn't close all resources (#15247) 2020-07-17 09:07:48 +00:00
Tim Neutkens
a33bb5bec5
Fix app-document-import-order test for webpack 5 (#15224) 2020-07-17 10:38:06 +02:00
Tim Neutkens
14babe5b3f
Make type checking compatible with webpack 5 (#15158)
Co-authored-by: Joe Haddad <joe.haddad@zeit.co>
2020-07-15 13:56:27 -04:00
Luis Alvarez D
6639b8f2af
Find pages/_document.js chunk in development (#15172)
Chunks already have a normalized path.

Not sure if there are other chunks that require this change, I did a global search and didn't find similar cases.
2020-07-15 05:28:47 +00:00
Jan Potoms
d2699be6e8
Trailing slash basepath (#14781)
Fixes https://github.com/vercel/next.js/issues/14757

Since https://github.com/vercel/next.js/pull/15041 has been sorted out, this can now be fully fixed
There was also a bug in the dev server that causes redirect loops
2020-07-13 14:59:40 +00:00
JJ Kasper
2d9d649d49
Add handling for custom-routes with basePath (#15041)
This adds handling for custom-routes with `basePath` to automatically add the `basePath` for custom-routes `source` and `destination` unless `basePath: false` is set for the route. 

Closes: https://github.com/vercel/next.js/issues/14782
2020-07-12 19:03:49 +00:00
Tim Neutkens
ab4a90b0d9
Add filename to error in webpack 5 (#14977)
Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>
2020-07-09 14:31:06 +02:00
Tim Neutkens
c5cc6072d0
Use single webpack runtimeChunk for Node.js compilation (#14722)
Webpack 5 supports a single runtimechunk for the Node.js compilation, this solves sharing modules between entrypoints.
2020-07-01 15:34:00 +00:00
Tim Neutkens
260707de68
Add etag support for getServerSideProps/getStaticProps pages (#14760)
Fixes #11711

Also cleaned up some extra code.

This was already supported on the Vercel edge network.
2020-07-01 14:59:18 +00:00
Joe Haddad
1d0e7a8a9a
Enable Optional Catch-All by Default (#14687)
Closes #14682
2020-06-29 18:50:32 +00:00
Tim Neutkens
ae542b8525
Use entry option instead of custom make hook (#14527)
Simplifies on-demand-entries a bit.
2020-06-26 04:26:09 +00:00
Tim Neutkens
6c2ce70608
Remove buildId from server-side files (#14413)
Gets rid of the custom function for naming files by removing buildId from the file paths.
2020-06-22 21:12:36 +00:00
Tim Neutkens
bef9b56109
Update filename generation for client-side compilation (#14279)
Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)

Follow up of these PRs:
https://github.com/vercel/next.js/pull/13759
https://github.com/vercel/next.js/pull/13870
https://github.com/vercel/next.js/pull/13937
https://github.com/vercel/next.js/pull/14130
https://github.com/vercel/next.js/pull/14176
https://github.com/vercel/next.js/pull/14268


Fixes #6303
Fixes #12087 
Fixes #1948
Fixes #4368
Fixes #4255
Fixes #2548
2020-06-20 19:59:47 +00:00
Tim Neutkens
ea6b8dac40
use requirePage for /_document and /_app (#14176) 2020-06-15 10:41:17 -04:00
Tim Neutkens
76fddcd7ef
Use chunkhash instead of buildId for pages (#13937)
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2020-06-11 10:57:24 +02:00
Joe Haddad
91664d1d02
Eliminate public folder enumeration (#14042)
This builds off of @timneutkens's PR instead of updating it because it's his `canary` branch.

Updated to still `stat`, as it's the only way to test the difference between a file and directory. 

---

Closes #13506
Fixes #12235
2020-06-10 20:35:34 +00:00
Jan Potoms
9da99bcb66
Prepare custom routes loading for adding additional routes (#13857)
Extracted from https://github.com/vercel/next.js/pull/13333, the same exact code lives in that PR as well, but we can merge this separately if it makes reviewing https://github.com/vercel/next.js/pull/13333 easier

This PR does 3 things 
- deduplicate code from build and next-dev-server that loads custom routes from next.config.js  (`loadCustomRoutes`)
- in `loadCustomRoutes`, load these rewrites, headers and redirects configs concurrently instead of sequentially.
- in next-server, make `this.customRoutes` always defined, this allows us to remove the big `if` around its initialization code in `generateRoutes`, which in turn makes it possible to reuse this code for other routing than user defined routes, which is how https://github.com/vercel/next.js/pull/13333 adds its redirects.
2020-06-09 20:16:23 +00:00