### What
Optimizing the static generation for dynamic metadata routes
If you're not using `generateSitemaps()` or `generateSitemaps()`, you
don't need to change any file conventions.
If you're using multi sitemap routes, make sure the returned `id`
properties from `generateSitemaps()` don't need to contain `.xml`, since
we'll always append one for you.
Analyzing the exports of metadata routes and determine if we need to
make them as dynamic routes.
### Why
Previously, users are struggling with the multi routes of sitemap or
images.
For sitemap, the `.xml` extension in url doesn't get appended
consistently to the multi sitemap route between dev and prod.
For image routes, the generated image routes are always dynamic routes
which cannot get static optimized.
The reason is that we need to always generate a catch-all route (such as
`/icon/[[...id]]` to handle both single route case (e.g. without
`generateImageMetadata`, representing url `/icon`) or multi route (e.g.
with `generateImageMetadata`, representing url `/icon/[id]`), only
catch-all routes can do it. This approach fail the static optimization
and make mapping url pretty difficult as parsing the file to check the
module exports has to be done before it.
#### Benifits
For image routes urls, this approach could help on static generation
such as single `/opengraph-image` route can be treated as static, and
then it can get static optimized if possible.
**Before**: `/opengraph-image/[[...id]]` cannot be optimized
**After**: single route `/opengraph-image` and multi-route
`/opengraph-image/[id]` are both possible to be statically optimized
For sitemap, since we removed appending `.xml` for dynamic routes, it’s
hard for users to have `/sitemap.xml` url with dynamic route convention
`sitemap.js` . But users desire smooth migration and flexibility.
**Before**: In v15 rc we removed the `.xml` appending that `sitemap.js`
will generate url `/sitemap` makes users hard to migrate, as users need
to re-submit the new sitemap url.
**After**: Now we'll consistently generate the `.xml`. Single route will
become `/sitemap.xml`, and multi route will become `/sitemap/[id].xml`.
It's still better than v15 as the urls generation is consistent, no
difference between dev and prod.
Here's the url generation comparsion
#### Before
All the routes are dynamic which cannot be optimized, we only had a
hacky optimization for prodution build multi-routes sitemap routes
| | only default export | `export generateImageMetadata()` | `export
generateSitemaps()` |
| -- | -- | -- | -- |
| opengraph-image.js | /opengraph-image/[[...id]] |
/opengraph-image[[...id]]/ | /opengraph-image/[[...id]] |
| sitemap.js | /sitemap/[[...id]] | /sitemap/[[...id]] | dev:
`/sitemap/[[...id]]` prod: `/sitemap/[id]` |
#### After
Most of the single route will are to get statically optimized now, and
the multi-routes sitemap are able to get SSG now
| | only default export | `export generateImageMetadata()` | `export
generateSitemaps()` |
| -- | -- | -- | -- |
| opengraph-image.js | /opengraph-image | /opengraph-image/[id] | - |
| sitemap.js | /sitemap.xml | - | /sitemap/[id].xml |
Next.js will have less overhead of mapping urls, we can easily multiply
the urls generation simply based on file conventions.
x-ref: feedback from #65507Closes#66232
### What
Keep `test/e2e/app-dir/metadata-dynamic-routes/index.test.ts` with
successful build cases, move the dev error tests into separate test
### Why
x-ref:
https://github.com/vercel/next.js/actions/runs/9429301722/job/25975574075?pr=66286
Before the moving the tests, the error is flaky with turbopack since the
error will fail the hmr. Error observed with turbopack when seeing build
failed cases. So I moved the tests into the separate dev tests, running
inside sandboxes. Then each error test doesn't effect each other.
```
⨯ ./app/metadata-base/unset/icon--metadata.js:1:1
Module not found: Can't resolve './icon.tsx'
> 1 | import { generateImageMetadata } from "./icon.tsx"
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2 | import { fillMetadataSegment } from 'next/dist/lib/metadata/get-metadata-route'
3 |
4 | const imageModule = { generateImageMetadata }
```
### What
Remove the auto appending `.xml` extension to the sitemap routes when
it's a dynamic route.
### Why
Previously we were adding `.xml` to `/[...paths/]sitemap` routes, but
the bad part is when you use it to generate multiple sitemaps with
`generateSitemaps` in format like `/[...paths/]sitemap.xml/[id]`, which
doesn't look good in url format and it can be inferred as xml with
content-type. Hence we don't need to add `.xml` in the url.
Before this change it could also result into the different url between
dev and prod:
dev: `/sitemap.xml/[id]`
prod: `/sitemap/[id].xml`
Now it's going to be aligned as `/sitemap/[id]`. Users can add extension
flexiblely.
Closes NEXT-3357
### What?
generateSitemaps function returns a 404 for /sitemap/[id].xml in
production
### Why?
While finding the correct sitemap partition from the array, we check the
param against the id. Which works in dev because id and param are both
without trailing .xml. But it fails in production as param has a
trailing .xml (/sitemap/[id] works in production because it falls back
to dynamic loading and param and id are both without .xml)
### How?
If we are in production environment, check the id with a trailing .xml
because that's whats returned from generateStaticParams, an array of
__metadata_id__ with trailing .xml
Fixes#61969
---------
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
### What
We should respect the `trailingSlash` config for metadata canonical url,
this PR is adding the handling for strip or keep the trailing slash for
canonical url. Passing down trailingSlash config to metadata resolving
to decide how we handle it.
### Why
The tricky one was `/` pathname, when visiting the origin directly, that
it will always have at least `/` in the URL instance. But for the
default `origin`, it shouldn't show the `/` if the `trailingSlash`
config is `false`. Also it should show trailing slash for all pathnames
if that config is enabled.
BTW there's a `__NEXT_TRAILING_SLASH` env but since we're using the
fixed nextjs runtime module, so this can't be dynamically replaced in
the metadata resolving modules. So we didn't use it
Fixes#54070
Closes NEXT-2424
### What
Fixes the string id that broken when sitemap is optimized to static
route.
### Why
When sitemap is optimized to static route in production, the route
argument is changed from `[[...__metadata_id__]]` to
`[__metadata_id__]`, so the type of it is also changed from array to
string that should reflect in the loader code.
Fixes#60894
Closes NEXT-2154
### What
Fix bad detection of dynamic route of sitemap metadata route, the swc
AST check should process when the text are detected. But prevuious if
there's text with `generateSitemap` such as comment but not the actual
export it will fail.
### How
Add both checks on metadata loader side and detection helper side
* Only call `generateSitemaps` helper when the export existed
* Fix the helper detection logic (major part of this PR)
Fixes#59698Closes#60344
Closes NEXT-2007
Reduce the confusiong of the the logging of pages, make it easier to understand
* Removing the trailing `/page`, `/route` suffix
* Removing the internal segment like `[[...__metadata__]]`
#### After
```
○ Compiling / ...
○ Compiling /dynamic/[slug]/icon ...
```
#### Before
```
○ Compiling /page ...
○ Compiling /(group)/dynamic/[slug]/icon-ahg52g/[[...__metadata_id__]]/route ...
```
Closes NEXT-1701
### What?
Wraps up metadata-dynamic-routes tests fixes for the turbopack. There is 1 remaining failing test due to lacks of supporting `import.meta.url` which need to be addressed separately.
I spent amount of time why turbopack cannot find the route for the dynamic metadata for a certain route. In the end, found there are mismatching expectations for the route due to different hash for the certain route. We do use the same djb2 hash between next.js and turbopack both, so it was quite confusing why we don't get deterministic hash.
After trying some experiments, found out root cause was how 2 different runtimes handle overflow for given type of numbers. In rust + turbpack we use u32 and do 32-bit hash calculation for given string, while in js we implicitly used number type as is, in result overflow occurs with default 53-bit float.
Originally I tried to adjust hash in turbopack side to preserve js hash as-is, but so far I found it was non trivial to do so as rust there's no out of the box types we can coerce to the js number type. In result, unlike other fixes in turbopack this PR changes how js hash is being generated. I hope this woulndn't be a breaking changes; expect so since this is a metadata specific hash that we do not have written spec for it.
Closes WEB-1890
Displaying hints of "missing default export" if you didn't properly export the `default` handler for og image
```
▲ Next.js 14.0.1-canary.2
- Local: http://localhost:3000
✓ Ready in 1089ms
✓ Compiled /opengraph-image/[[...__metadata_id__]]/route in 211ms (44 modules)
⨯ Error: Default export is missing in "/Users/huozhi/workspace/next.js/test/e2e/app-dir/metadata-dynam
ic-routes/app/opengraph-image.tsx"
at eval (webpack:///app/opengraph-image.tsx?3407:11:1)
at (app-metadata-route)/../../../../packages/next/dist/build/webpack/loaders/next-metadata-route-lo
ader.js?page=%2Fopengraph-image%2F%5B%5B...__metadata_id__%5D%5D%2Froute&isDynamic=1!./app/opengraph-im
age.tsx?__next_metadata_route__ (/Users/huozhi/workspace/next.js/test/e2e/app-dir/metadata-dynamic-rout
es/.next/server/app/opengraph-image/[[...__metadata_id__]]/route.js:362:1)
at __webpack_require__ (/Users/huozhi/workspace/next.js/test/e2e/app-dir/metadata-dynamic-routes/.n
ext/server/webpack-runtime.js:33:42)
```
### Story
Since we introduced `ImageResponse` into `next/server` export, there're a few libraries relying on `next/server` that accidentally ended up with bundling og image into the bundle. As og package is quite large that could easily raise the size concern for middleware, edge runtime routes.
### Struggles
We've done optimizations. The tree-shaking strategies are tricky, we tried modularize imports and also optimize cjs require/exports to make sure you're not including og package into bundle if it's not being used. However, it's still not 100% can handle all the bundle optimization cases, such as `import {..} from "next/server.js"` could also ended up with the cjs bundle that failed the tree-shaking.
### Move on
So we decide to move og `ImageResponse` into a separate export `next/og`
Closes NEXT-1660
This PR adds the optional `limit` parameter on String.prototype.split uses.
> If provided, splits the string at each occurrence of the specified separator, but stops when limit entries have been placed in the array. Any leftover text is not included in the array at all.
[MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/split#syntax)
While the performance gain may not be significant for small texts, it can be huge for large ones.
I made a benchmark on the following repository : https://github.com/Yovach/benchmark-nodejs
On my machine, I get the following results:
`node index.js`
> normal 1: 570.092ms
> normal 50: 2.284s
> normal 100: 3.543s
`node index-optimized.js`
> optmized 1: 644.301ms
> optmized 50: 929.39ms
> optmized 100: 1.020s
The "benchmarks" numbers are :
- "lorem-1" file contains 1 paragraph of "lorem ipsum"
- "lorem-50" file contains 50 paragraphes of "lorem ipsum"
- "lorem-100" file contains 100 paragraphes of "lorem ipsum"
Re-export the original route segment config in metadata routes loader, so they can be picked up by app route module wrapper
Closes#54057
Closes NEXT-1645
`isAppLayer` condition was missing `app-metadata-route` layer, made it
as a util now like other webpack layer utils, add metadata route layer
to the group. Then `React.cache` can be available there.
Also update regex to be compatible across platform
Fixes#55561
Closes NEXT-1635
In dev mode, we're using a catch-all route for dynamic metadata routes, e.g. page path `/twitter-image` would become `/twitter-image/[[...metadata_id...]]/route` as a dynamic custom app route.
But we're missing to convert it in filesystem scanning for routing purpose, adding the metadata related normalizing logic for app page to align with other places.
A continuation of #51864 with RedirectType used in an e2e test and some
ts-ignores removed from the codebase. Happy to split it into two PRs but
I left comments in the two consequential files
---------
In the past few rounds of improving metadata image routes bundling, we have improved the bundling strategy and also updated [the usage tutorial of using custom fonts in og image routes](https://vercel.com/docs/functions/edge-functions/og-image-generation/og-image-examples) which should load the font in the image route handler.
Adding some tests to ensure custom fonts are working with metadata
Closes#48081
Closes#48479.
Couldn't find the source for the Next.js beta docs, so I didn't add documentation.
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
After we separating the metadata routes to a separate layer, we didn't apply the webpack alias rules properly to it as it's should still be treated as pure "server" side
This PR fixes the aliasing for that new metadata layer and make it working properly with "server-only"
Fixes#52390
## What?
Currently we use 3 separate webpack compilers:
- server
- client
- edge
All of these were creating their own input filesystem (which is used to
read file, stat, etc.). Changing them to share a single inputFileSystem
allows the `cachedFileSystem` to be reused better, as `stat` and
`readFile` can be cached across the 3 compilers.
For the page on vercel.com we've been testing this shaves off 300-400ms
on a cold compile (no cache, deleted `.next`).
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: Shu Ding <g@shud.in>
The next-metadata-route-loader emitted content with `"next/server"` is still using cjs version which is not get tree-shaked on edge runtime, This PR adds a new esm entry for it and mapped it that on edge runtime so unused exports can be tree-shaked. This case happened when users are using ImageResponse from `"@vercel/og"` instead of `"next/server"`.
2nd change is adding another alias to map `@vercel/og` package to vendored og package inside next.js, this way we could merge the two package instead of bundled both of them
Fixes: NEXT-1303
changed `require.reoslve("next/dist/xxx")` to `${NEXT_PROJECT_ROOT}/xxx}` to avoid to be traced by nft.
## What?
Removes `experimental.appDir` this was leftover from when I flipped the
switch.
Kept the config file as in the future we might add future flags and
such. It also helps that it has the types comment included so you always
get types.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Follow up of #48867
- Statically optimize dynamic generated sitemap routes
- Previously the generated sitemap urls looks bit off
(`/route/sitemap.xml/[id]`), we polish it into `/route/sitemap/[id].xml`
in this PR
* For sitemap if they're not using dynamic routes generation `generateSitemaps`, should optimize them as static sitemap
* For icons and social images, if they're not using `generateImageMetadata`, should optimize them as static path
Closes NEXT-1071
Fixes#48991
### What?
In #48928 we decided to error for the missing `id` from `generateImageMetadata` and `generateSitemaps` for better dev DX. This PR also refactors the metadata image urls generation that assumbling the utils together
* Fix the `generateImageMetadata` for non dynamic routes and related
`param` matching (Found during development)
* Fix dynamic routes with number suffix `(\d)` (Fixes#48689)
### Why
Default font file of `@vercel/og` is not loaded, because the og package is bundled by webpack and we should external it so that `fs.readFileSync` is bundled and manged that can't be traced by nft.
### How
This PR externals `@vercel/og` so that they don't need to be bundled and files can be properly traced
Closes NEXT-1047
Fixes#48704
### What
For dynamic routes you might have different sitemap for different params
* Unloack using `sitemap.[ext]` in your app everywhere
* Support `generateSitemaps()` to create multiple sitemaps at the same time
### How
* Change the metadata regex to allow use sitemap in every routes
* Similar approach to `generateImageMetadata`, we make `sitemap.js` under dynamic routes to a catch all routes, and it can have multiple routes
Closes NEXT-1054
Use should only need to configure one `metadataBase` as fixed domain or
url but metadata image routes can still work properly with `VERCEL_URL`
in preview deployments
If you configured `new URL('https://mydomain.com')`, it will work for
canonical url all the time since it's for SEO.
For preview deployments metadata image routes will work with deployment
url as they're always bound with deployment.
For production deployments metadata images routes can be alias to the
configured `metadataBase` as they could be the expected exposed domain
Follow up for #47910
link NEXT-887
### What?
* Support `generateMetadata(props)` to dynamically generate multiple
metadata images at the same time
```js
// /app/opengraph-image.tsx
import { ImageResponse } from 'next/server';
export async function generateImageMetadata({params}) {
const images = await ...;
return images.map((img, idx) => ({
size: { width: 1200, height: 600 },
alt: img.text,
contentType: 'image/png',
id: idx,
}));
}
export default async function ({params, id}) {
const text = await getTextFor(id);
return new ImageResponse(
(
<div
style={{...}}>
{text}
</div>),
{ width: 1200, height: 600 },
);
}
```
### How?
Use `<metadata image>/[[...__metadata_id__]]/route.js` to catch all
metadata images id, and then use this `params.__metadata_id__` as id
argument for dynamic generate image.
If there's param, then we create `<metadata image>/<id>`, if there's
only 1 static image without dynamic `generateImageMetadata` then we keep
use `<metadata image>`
Closes NEXT-896
### What?
The change in #47985 breaks the URLs of static image files like
`/(group)/opengraph-image.png` to `/opengraph-image.png-012345`.
References from `/` are also broken.
### Why?
This is because only `opengraph-image.ts` and `opengraph-image.tsx` are
considered.
### How?
In this Pull Request, we are trying to solve the problem by including
similar support for `opengraph-image.png` and `opengraph-image.jpeg`.
---------
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
### What
When using dynamic metadata image rouets (such as `opengraph-image.js`)
under group routes, the generated urls were still normalized. In this
case it might have conflicts with those ones not under group routes. For
instance `app/(post)/opengraph-image.js` could have same url with
`/app/opengraph-image.js`. In reality we want them to be different
route, unlike layout or pages.
### How
So when we found `()` or `@` signs from the metadata image urls, we'll
generate a unqiue suffix (`-\d{6}`) and append to the generated url. So
they can be isolated from the ones are not under special convention
routes.
Closes NEXT-937
Should fill dynamic routes url with params when generate the metadata
image urls.
For `/(group)`, we need to normalize the path first;
For dynamic routes `/[slug]`, we need to fill the params.
Change the image module from `() => image` to `(prop) => image(proop)`
so we can fill the runtime `params` into url
Closes NEXT-932
---------
For backward compatibility, we only handle `favicon.ico` file to
generate `/favicon.ico` route and link tag. If you want to use other
extension such as `png`, use `icon(\d)?.[ext]`
### What?
- Provide a default `metadataBase`
- Always resolve urls that could be resolved as absolute with
`metadataBase`, e.g. tw/og urls, canonical urls
- Give a warning in dev mode if user doesn't provide one in dev
- Error if you don't have it but it's required in production
On production it will leverage `VERCEL_URL` if users expose it to the
deployment
### Why?
OG image urls are required to be absolute urls instead of relative urls.
For metadata image conventions we let users don't have to provide
`metadataBase` explicitly when they expect it should be the origin of
their next app.
### How?
Closes NEXT-887
---------
### What?
Followup for #47630 , use ts namespace to group MetadataRoute.
`MetadataRoute['xxx']` -> `MetadataRoute.xxx`
### Why?
namespace is convenient to write for these grouped types such as
`MetadataRoute.Robots`. And `MetadataRoute` itself is not actually a
type. So it can be conveniently typed by ts users.
### How?
Closes NEXT-912
---------
### What?
This PR vendors @vercel/og and export `ImageResponse` from
`next/server`. When you render a opengraph image the below code snippets
will be legit:
```tsx
import { ImageResponse } from 'next/server'
export default function og() {
return new ImageResponse(<div style={{ width: '100%', height: '100%' }}>hello</div>)
}
```
### Why?
To make development more easier, user can directly use `@vercel/og`
Image Response with nextjs instead of install it and use it. This makes
building metadata icons, og or twitter images more convenient.
### How?
Closes NEXT-899
