Since we're always using `ReadableStream`, we should just get rid of `ResultPiper`.
This also lets us replace things like `bufferedReadFromReadableStream` with a `TransformStream` that does the same thing, so that it's `TransformStream`s all the way down.
Finally, we can get rid of the one-off call to `renderToReadableStream` and just use `renderToStream` whenever we're rendering a concurrent tree.
Instead of branching rendering based on Node.js and browser/web runtimes, we should just use the web version for now, which can run as-is on versions >=16.5.0 of Node.js, polyfilling `ReadableStream` on older versions when necessary.
There are a few potential downsides to this, as React is less able to optimize flushing and execution. We can revisit that in the future though if desired.
* fix: update default redirect status to 307 to match docs
* fix: update default redirect status to 307 to match docs
Co-authored-by: JJ Kasper <jj@jjsweb.site>
We currently have inconsistencies when working with URLs in the Edge Functions runtime, this PR addresses them introducing a warning for inconsistent usage that will break in the future. Here is the reasoning.
### The Browser
When we are in a browser environment there is a fixed location stored at `globalThis.location`. Then, if one tries to build a request with a relative URL it will work using that location global hostname as _base_ to construct its URL. For example:
```typescript
// https://nextjs.org
new Request('/test').url; // https://nextjs.org/test
Response.redirect('/test').headers.get('Location'); // https://nextjs.org/test
```
However, if we attempt to run the same code from `about:blank` it would not work because the global to use as a base `String(globalThis.location)` is not a valid URL. Therefore a call to `Response.redirect('/test')` or `new Response('/test')` would fail.
### Edge Functions Runtime
In Next.js Edge Functions runtime the situation is slightly different from a browser. Say that we have a root middleware (`pages/_middleware`) that gets invoked for every page. In the middleware file we expose the handler function and also define a global variable that we mutate on every request:
```typescript
// pages/_middleware
let count = 0;
export function middleware(req: NextRequest) {
console.log(req.url);
count += 1;
}
```
Currently we cache the module scope in the runtime so subsequent invocations would hold the same globals and the module would not be evaluated again. This would make the counter to increment for each request that the middleware handles. It is for this reason that we **can't have a global location** that changes across different invocations. Each invocation of the same function uses the same global which also holds primitives like `URL` or `Request` so changing an hypothetical `globalThis.location` per request would affect concurrent requests being handled.
Then, it is not possible to use relative URLs in the same way the browser does because we don't have a global to rely on to use its host to compose a URL from a relative path.
### Why it works today
We are **not** validating what is provided to, for example, `NextResponse.rewrite()` nor `NextResponse.redirect()`. We simply create a `Response` instance that adds the corresponding header for the rewrite or the redirect. Then it is **the consumer** the one that composes the final destination based on the request. Theoretically you can pass any value and it would fail on redirect but won't validate the input.
Of course this is inconsistent because it doesn't make sense that `NextResponse.rewrite('/test')` works but `fetch(new NextRequest('/test'))` does not. Also we should validate what is provided. Finally, we want to be consistent with the way a browser behaves so `new Request('/test')` _should_ not work if there is no global location which we lack.
### What this PR does
We will have to deprecate the usage of relative URLs in the previously mentioned scenarios. In preparation for it, this PR adds a validation function in those places where it will break in the future, printing a warning with a link that points to a Next.js page with an explanation of the issue and ways to fix it.
Although middleware changes are not covered by semver, we will roll this for some time to make people aware that this change is coming. Then after a reasonable period of time we can remove the warning and make the code fail when using relative URLs in the previously exposed scenarios.
Part of #31506, this PR moves the code of middleware handling from the base server to the node server.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
Production middlewares will only expose env vars that are statically analyzable, as mentioned here: https://nextjs.org/docs/api-reference/next/server#how-do-i-access-environment-variables
This creates some incompatibility with `next dev` and `next start`, where all `process.env` data is shared and can lead to unexpected behavior in runtime.
This PR fixes it by limiting the data in `process.env` with the inferred env vars from the code usage. I believe the test speaks for itself 🕺
<!--
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
-->
It adds AbortController and AbortSignal Web runtimes APIs to be used by the user at Edge Functions.
For doing that it delegates into `abort-controller` dependency that has been frozen to prevent any modification.
Co-authored-by: Zhang Zhi <20026577+fytriht@users.noreply.github.com>
# Description
The redirect responses do not contain a message body. This is in conflict with the RFCs (below) and causes Traefik (a reverse proxy) to invalidate the responses. In this pull request, I add a response body to the redirect responses.
This PR is similar to https://github.com/vercel/next.js/pull/25257, it appears that there are some other locations where redirection is handled incorrectly in next.js.
# References
- https://datatracker.ietf.org/doc/html/rfc7230#section-3.3
> All 1xx (Informational), 204 (No Content), and 304 (Not Modified) responses must not include a message-body. All other responses do include a message-body, although the body may be of zero length.
- https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.3
> The server's response payload usually contains a short hypertext note with a hyperlink to the different URI(s).
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
**Note**: This PR is applying again changes landed #31935 that were reverted from an investigation.
This PR fixes#30398
By default Next will polyfill some fetch APIs (Request, Response, Header and fetch) only if fetch is not found in the global scope in certain entry points. If we have a custom server which is adding a global fetch (and only fetch) at the very top then the rest of APIs will not be polyfilled.
This PR adds a test on the custom server where we can add a custom polyfill for fetch with an env variable. This reproduces the issue since next-server.js will be required without having a polyfill for Response which makes it fail on requiring NextResponse. Then we remove the code that checks for subrequests to happen within the **sandbox** so that we don't need to polyfill `next-server` anymore.
The we also introduce an improvement on how we handle relative requests. Since #31858 introduced a `port` and `hostname` options for the server, we can always pass absolute URLs to the Middleware so we can always use the original `nextUrl` to pass it to fetch. This brings a lot of simplification for `NextURL` since we don't have to consider relative URLs no more.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Errors have helpful link attached, see `contributing.md`
This PR fixes#30398
By default Next will polyfill some fetch APIs (Request, Response, Header and fetch) only if fetch is not found in the global scope in certain entry points. If we have a custom server which is adding a global fetch (and only fetch) at the very top then the rest of APIs will not be polyfilled.
This PR adds a test on the custom server where we can add a custom polyfill for fetch with an env variable. This reproduces the issue since next-server.js will be required without having a polyfill for Response which makes it fail on requiring NextResponse. Then we remove the code that checks for subrequests to happen within the **sandbox** so that we don't need to polyfill `next-server` anymore.
The we also introduce an improvement on how we handle relative requests. Since #31858 introduced a `port` and `hostname` options for the server, we can always pass absolute URLs to the Middleware so we can always use the original `nextUrl` to pass it to fetch. This brings a lot of simplification for `NextURL` since we don't have to consider relative URLs no more.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [x] Errors have helpful link attached, see `contributing.md`
This ensures type checking passes correctly for middleware types when `skipLibCheck: false` is set in `tsconfig.json`. This also moves the `middleware-types` to be an isolated test to ensure it isn't relying on any monorepo dependencies.
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
Fixes: https://github.com/vercel/next.js/issues/31992#issuecomment-984174872
* fix(types): add missing ua type for NextRequest
add missing ua string type on NextRequest["ua"], returns full agent as string
* Update UserAgent type
Co-authored-by: JJ Kasper <jj@jjsweb.site>
This allows TypeScript users to have type safety for middleware functions.
- Closes#30490
Co-authored-by: Lee Robinson <9113740+leerob@users.noreply.github.com>
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
To run middleware we are using a **sandbox** that emulates the web runtime and keeps a module cache. This cache is shared for all of the modules that we run using the sandbox while there are some module-level APIs that must be scoped depending on the module we are running.
One example of this is `fetch` where we want to always inject a special header that indicate the module that is performing the fetch and use it to avoid getting into infinite loops for middleware. For those cases the cached implementation will be the first one that instantiates the module and therefore we can actually get into infinite loops. This is the reason why #31800 is failing.
With this PR we refactor the sandbox so that the module cache is scoped per module name. This means that one execution of a middleware will preserve its cache only for that module so that each execution will still have its own `fetch` implementation, fixing this issue. Also, with this refactor the code is more clear and we also provide an option to avoid using the cache.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
Previously `response.cookie(name, value, options)` would mutate the passed in `options` which lead to unexpected behaviour as described in #31666.
This PR clones the `options` argument before mutating it.
## Bug
- [x] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [x] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [x] Make sure the linting passes by running `yarn lint`
Since `localhost` is actually an alias for `127.0.0.1` that points to loopback, we should take that into consideration at `NextURL` when we handle local URLs.
The implementation is based on [is-localhost-url](https://github.com/Kikobeats/is-localhost-url); I added some tests over local URLs variations present at the library to ensure other variations are working fine.
Additionally, I refactor some things over the code to avoid doing the same twice and added some legibility that is always welcome when you are working with URLs stuff.
closes https://github.com/vercel/next.js/issues/31533
This PR removes the class wrapper that we have for `TextEncoder` and `TextDecoder`. Since this is merged we will be using directly the Node version in the sandbox.
closes: https://github.com/vercel/next.js/issues/30353
According with spec, `'about:client'` is the default value is the user doesn't provide it.
It needs to add a test there, looks like there no unit tests for these classes 🤔
This fixes our `basePath` detection/replacing server-side as we were incorrectly considering `/docss` a match for a `basePath` of `/docs` which caused us to have an unexpected value in the `normalizeLocalePath` function.
- Fixes#22429
- Regression introduced in #17757
- Fixes: https://github.com/vercel/next.js/issues/31423
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
The `resource` argument[^1] in fetch may also be an instance of URL (or any other stringifiable value) but the sandbox variant of middlewares doesn't support that.
```js
export async function middleware(req, ev) {
await fetch(new URL('https://www.googleapis.com/oauth2/v3/certs'), {
redirect: 'manual',
method: 'GET',
})
return new Response(JSON.stringify({}), { status: 200 });
}
```
This is fixing the use of e.g. URL instance in `fetch`.
```
TypeError: initurl.startsWith is not a function
at getFetchURL (/my-next-app/node_modules/next/dist/server/web/sandbox/sandbox.js:246:17)
at fetch (/my-next-app/node_modules/next/dist/server/web/sandbox/sandbox.js:77:29)
at Object.middleware [as handler] (webpack-internal:///./pages/_middleware.js:86:15)
at async adapter (webpack-internal:///./node_modules/next/dist/server/web/adapter.js:30:22)
at async DevServer.runMiddleware (/my-next-app/node_modules/next/dist/server/next-server.js:430:26)
at async DevServer.runMiddleware (/my-next-app/node_modules/next/dist/server/dev/next-dev-server.js:394:28)
at async Object.fn (/my-next-app/node_modules/next/dist/server/next-server.js:807:34)
at async Router.execute (/my-next-app/node_modules/next/dist/server/router.js:211:32)
at async DevServer.run (/my-next-app/node_modules/next/dist/server/next-server.js:1115:29)
at async DevServer.run (/my-next-app/node_modules/next/dist/server/dev/next-dev-server.js:440:20)
```
[^1]: https://developer.mozilla.org/en-US/docs/Web/API/fetch#parameters
When libraries are required outside of the middleware function context and they do checks such as `a instanceof Uint8Array` since the constructors are different between the two contexts they'll always yield false.
This is a problem for libraries validating user input as well as the WebCryptoAPI polyfill used outside of Edge Functions.
- Fixes#30477
- Fixes#30911
This is only a problem for the sandbox runtime, not when ran inside an Edge Function.
Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>
With this PR we are updating the way we check the usage of `eval` and other dynamic code evaluation (like `new Function`) for middleware. Now instead of simply showing a warning it will behave differently depending on if we are building or in development.
- Development: we replace the dynamic code with a wrapper so that we print a warning only when the code is used. We don't fail in this scenario as it is possible that once the application is built the code that uses `eval` is left out.
- Build: we detect with tree shaking if the code that will be bundled into the middleware includes any dynamic code and in such scenario we make the build fail as don't want to allow it for the production environment.
Closes#30674
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
- Code splitting should be disabled for the server-web build. Done via `ServerlessPlugin`.
- ~Target can't be `web`, `webworker` is better.~ Using `web` and `es6` for now, still not ideal.
- https://github.com/acornjs/acorn/issues/970
## Bug
- [ ] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `yarn lint`
## Bug
- [x] Related issues linked using `fixes #number`
- [x] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
Fixes#30430
There's some more discussion in the issue, but in summary:
- web `Headers` implementation combines all header values with `', '`
- For `Set-Cookie` headers, you're supposed to set them as separate values, not combine them
- web `Headers` forbids the use of `Cookie`, `Set-Cookie` and some more headers, so they don't have custom implementation for those, and still joins them with `,`
- We currently just split them using `split(',')`, but this breaks when the header contains a date (expires, max-age) that also includes a `,`
I used this method to split the Set-Cookie header properly: https://www.npmjs.com/package/set-cookie-parser#splitcookiestringcombinedsetcookieheader as suggested [here](https://github.com/whatwg/fetch/issues/973#issuecomment-559678813)
I didn't add it as a dependency, since we only needed that one method and I wasn't sure what the process is for adding dependencies, so I just added the method in the middleware utils
This PR adds support for [Middleware as per RFC ](https://github.com/vercel/next.js/discussions/29750).
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes
