Fixes WEB-412.
Previously, references to styled-jsx inserted by the swc transform would be bundled, which duplicated the copy required by next, which is external. This marks the dependency is to be resolved as external always.
Test Plan: Verified in a small app that `<style jsx>` nodes result in a server-rendered response with the appropriate `<style>` tag in the head.
This implements a debug mode for spawning Node processes across multiple surfaces including `evaluate` and `get_renderer_pool`.
It:
* automatically limits concurrency to 1 process per pool
* passes `--inspect-brk` to spawned Node process
* sets the timeout to effectively-infinite duration
Test Plan: set the value to true for the call to `get_renderer_pool` in render_static.rs and attached to a debuggable server renderer process.
This:
* Implements a basic http server to mock returning a stylesheet from the Google Fonts API (**Note**: This importantly does *not* mock returning the font resources themselves, which are currently loaded by the browser. We should implement something to do this when we implement caching arbitrary http resources traced from `url()` and this is no longer loaded by the test browser)
* Adds an integration test that asserts the basic shape of the JS object returned by font functions
* ~Adds an integration test that asserts the browser correctly loads a font for the ascii unicode range, along with the appropriate `font-display`, variant, etc.~ Unfortunately `document.fonts` is not reliable with different font-display loading patterns as any of them can fall back.
As noted, this does not mock responses from Google for the font resources themselves, which are currently loaded by the test browser. This means that we'll be dependent on this external service for passing integration tests until we implement caching of `url()`s in Turbopack. We should monitor the reliability of this test.
https://github.com/vercel/next.js/pull/44668 refactored Next's use of `AsyncLocalStorage`, and installs a "polyfill" of the API by patching the node import onto `globalThis.` Importantly, it's then used in the module scope during imports, so we need to install the polyfill early in the app rendering startup.
Fixes https://github.com/vercel/turbo/issues/3319
This adds two basic tests for next/dynamic:
* one ensures that SSR supports dynamic components;
* the other ensures that SSR doesn't run when `ssr: false`.
This PR adds the next/dynamic SWC transform to the repo. This is copied
over from the implementation in
[](f70a6bfb86/packages/next-swc/crates/core/src/next_dynamic.rs),
with an additional mode for Turbopack.
For now, it applies it to all sources files (excluding node_modules, but
including embedded modules).
I've refactored the Next transforms logic to make it more easily
extendable in the future, and avoid creating more Vcs than absolutely
necessary.
This PR is the last item in the list of what's needed to support
next/dynamic in development.
The `processed_assets` `IndexSet` was performing deduplication on the `AssetVc`, which is the same between static (`import ... from "foo"`) and dynamic (`import("foo")`) imports of the same specifier. But, the _reference_ to those assets is different, and generates different chunking semantics. In order for both to succeed, we need to process both assets.
Coupled with vercel/turbo#3193's ability to load chunks on the server side, this fixes fixes WEB-381.
add the minimum to run a simple raw-loader (see test case)
* add `experimental.turbopackWebpackLoaders` to next.config.js
* key is extension like `".mdx"`, value is an array of loaders like
`["mdx-loader"]`
This PR separates Turbopack's `runtime.js` into two:
- the Browser/DOM implementation, which uses `<script>` and `<link>`
tags for dynamically loading chunks;
- the Node.js/CommonJS implementation, which uses `require` instead.
~~The two runtimes share a lot of code, but I'm not sure what's the best
approach for sharing this logic, or even if we should, so for now they
are two separate entities.~~
The shared code between the two runtimes is now in a separate file to
avoid duplication.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Addresses an issue currently affecting non-variable fonts in which
`next-dev` errors for non-variable Google fonts. This issue can be
replicated by using `next dev --turbo` and using a Google font that
isn't variable, like Hind.
The problem is resolved by unwrapping axes only if the font is variable.
Relevant issue: vercel/next.js#44282
Co-authored-by: Will Binns-Smith <wbinnssmith@gmail.com>
This is a straightforward chore PR to move next_client and next_server modules
around, in order to make navigation and imports more predictable between them.
I'm also planning to introduce a next_shared::context in a further PR, hence the
changes.
Based on vercel/turbo#2968
This builds upon vercel/turbo#2968 and @ForsakenHarmony's work on data routes to
enable page data HMR.
Page data HMR is a bit more clever than it is in Next.js as we won't
re-render a Node.js result for each page file update. Instead, thanks to
the `StripPageDefaultExport` transform, there are three versions of the
page chunks:
* client-side (strips page data exports);
* server-side (full);
* data server-side (strips page default export).
Instead of subscribing to the full server-side result, on hydration, the
client-side page separately subscribes to:
* client-side updates (already the case);
* data server-side updates (new).
This means that updating something that only affects the page component
will only cause a client-side update and **no Node.js re-rendering**,
while updating something that only affects the data will only cause a
server-side update.
~~I'm marking this as a draft for now as there are still a few areas to
test/investigate:~~
- [x] When something that is used in both the default page export and
data exports is changed, this will cause *two* HMR updates: one data
update, and one client-side chunk update. **The same case breaks in
Next.js, where we will receive a client-side update, but no server-side
update, ending up with an incorrect result.**
- [x] Differences between `getStaticProps/getServerSideProps`, as well
as `getInitialProps` (need to talk with @timneutkens about this) (see
https://github.com/vercel/next.js/pull/44523)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
## The original transform
The original behavior of the Next SSG transform is to remove `getServerSideProps`, `getStaticProps`, and `getStaticPaths` from Next.js page files so they can be bundled for the client. This is what enables the following code to work properly without advanced tree shaking:
```jsx
import db from "db";
import PostCounter from
export default function Home(props) {
return <>{props.posts.length} post(s)</>;
}
const postsPromise = db.getPosts();
export async function getStaticProps() {
return {
props: {
posts: await postsPromise,
},
};
}
```
The transform is able to remove both `getStaticProps` and all its exclusive, transitive dependencies, so this is what the client would see:
```jsx
import PostCounter from "components/PostCounter";
export var __N_SSG = true;
export default function Home(props) {
return __jsx(PostCounter, {
count: props.posts.length
});
}
```
## Adding the inverse operation
However, to support proper HMR for these data exports, we need to be able to execute somewhat of an inverse operation: remove the default component export, but preserve all the rest. This allows Turbopack to bundle server-side only declarations, only re-rendering when one of these changes, instead of re-rendering on any server-side and client-side change.
From our module above, the updated transform is now also able to generate the following:
```jsx
import db from "db";
const postsPromise = db.getPosts();
export async function getStaticProps() {
return {
props: {
posts: await postsPromise
}
};
}
```
As you can see, this module is no longer importing the `PostCounter`, which means re-rendering will not invalidate when that counter changes. However, if the "db" module changes, we will still be able to detect a change and re-render.
## Other notes
* I renamed the transform from "next_ssg" to "next_transform_strip_page_exports". It's much more verbose, but hopefully also much clearer about what it does at a glance.
* I took the liberty to clean up and comment some parts of the transform to make it more easily understandable (at least for someone like me, who hasn't written a lot of SWC code). I also fixed a few bugs and edge cases.
* I brought over the tests from the transform in the Next.js and added a couple of them.
* For now, only the `StripDataExports` filter is used. A future PR will build on this and @ForsakenHarmony's https://github.com/vercel/turbo/pull/2949 PR to actually implement SSR/SSG HMR.
## Reviewing guide
1. The crux of the change is the move (and refactor) of the next ssg transform from https://github.com/vercel/turbo/pull/2968/files#diff-133d73657647ed793ca4782a73552650b32ad565094b1e0faf452ad58705499b to https://github.com/vercel/turbo/pull/2968/files#diff-d6442fa6af9b66e581f062739dd6de2419f5e8f6f3d97cfa63518c72b0a9ee3e.
2. I also added the [errors.rs](https://github.com/vercel/turbo/pull/2968/files#diff-0f308375da4179c0ea5a0fcbd99593b56d6020cd7dec1694ed08f392f1637c09) and [fixture.rs](https://github.com/vercel/turbo/pull/2968/files#diff-6e23f34483fc17a27dfc630edb455ea95e28d5cb350468c2ba01384fbc3c116a) tests. I adapted fixture.rs to execute on the two transform filters: data exports and default export.
3. Most of the tests in `tests/` are copied from https://github.com/vercel/next.js/tree/canary/packages/next-swc/crates/core/tests. The changes I made are:
i. https://github.com/vercel/turbo/pull/2968/files#diff-774abee6a1bf139c9ce55e226bf15b52e56ea091170ee5d6295c191fd8d793c7: made this one symmetric for both strip data and strip default transforms.
ii. https://github.com/vercel/turbo/pull/2968/files#diff-4792266a264dc67da93748d2c6522917f860527a689891bd5f8f4add9841e7f6 wasn't supported before AFAIK.
iii. https://github.com/vercel/turbo/pull/2968/files#diff-a3b12adbce1dec40f39ff8af13ffecbe7f6963e21bc402cef6332ecf5018877e similar to i.
Currently when an individual `ContentSource` inside a `CombinedContentSource` requests additional vary data, the combined source will treat that source as the winner of the query. But, sometimes the vary data is necessary to determine that the content is not found by this source. And if that's the case, we'll hit a 404 when we should actually keep iterating.
This is exactly the case we're going to hit with redirect/rewrites. Redirects need to be the very first content source, and I also need query/header/cookie/hostname data in order to determine if a redirect applies. Without this, **every** request becomes a 404.
- The 10th attempt used to day "trying again" and then didn't try again because the iterator ended…
- Swaps a nested `.map() == Some(bool)` for a `and_then`
- Extracts into a helper function so code can early return without an `Option<Result>` wrapper.
- Adds `details` to `SourceMapContentSource`
- Fixes `ty` of `SourceMapContentSource`
- Adds default `children` trait method that returns empty children
Any `404.js` page would inject a `[...]` path into the server generation. The problem comes from the `pathname_for_path` function, which will extract the extension from that path. In this case, it leads to a basename of `[...` and an extension of `]`, breaking the `regular_expression_for_path` dynamic route parser.
Unfortunately, our integration tests timeout when I try to write a basic test case for this, so it's not possible to test at the moment. I've manually verified.
Fixes WEB-214
pass the dependencies reported by postcss to turbo-tasks-fs. We don't
have a good mechanism to track dependencies back-timed, so we start
watching some as soon we know about the dependencies (after postcss
compilation)
tailwind reports dependencies as directory + glob, so our globbing need
some improvements to handle these globs reported by tailwind.
e.g. tailwind will rerun when any file changed from the tailwind config
`content` globs.
fixes WEB-356
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
add counting seconds when something is happening. Only starts to show
when compilation takes more than 1s
Co-authored-by: Alex Kirszenberg <alex.kirszenberg@vercel.com>
A follow up to vercel/turbo#2944, which incorrectly rewrote the `new URL("…", import.meta.url)` to `new URL("…", location.origin)` during server rendering (which obviously doesn't have a `location` object).
This PR accomplishes 2 things:
- Splits dev server address binding from setting up the handler function
- This is necessary so that we can determine the bound server address during devs retry attempts.
- Propagates the server address through the `ExecutionEnvironment` to the `UrlAssetReference`
I chose to pipe the server address through the `ExecutionEnvironment` instead of the `EnvironmentIntention`, because I don't fully understand the differences between them. It seems like we want it in the intention for the types of rendering, but a server address is really tied to the node process, so I don't know.
Fixes WEB-318
Fixes WEB-341
This incorporates a digest of the font request into the name of the
font-family defined in the stylesheet, used in the css class, and
returned to the JavaScript importer. This makes font-family names
returned from `@next/font/google` in stylesheets unguessable.
Note that this implementation uses 8-character long hex digests compared
to the 6-character version in Next.js+webpack.
Test Plan: imported Open Sans as a symbol, verified that stylesheet is
updated to reference both `@font-face` definitions and the css class
rules with `font-family: '__Open_Sans_3b020fa1';`. Verified that
exported js object `style.fontFamily` references the same.
This adds the ability to add Source transforms on assets, which modify
the source code of an asset before parsing.
This adds postcss via node.js as first source transform.
Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
HMR is handled via an `eval(newCode)`, and that eval will inherit the filepath of the currently running file (in our case, the chunk file that contains the runtime code). To properly trace, we need to know both the chunk and the item ids, so we trace via the correct item's source map.
Thankfully, we can use the (underspecified) [`//# sourceUrl` ](https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/javascript/source-maps#use--sourceurl-to-name-evaluated-files-in-the-sources-tool) meta comment to name the file we're currently evaluating. That'll get used as the stack frame's file. We can then encode the chunk path and item id into the filename, and let next's stack tracing code handle the rest.
This also fixes an off-by-one on the column. Source maps are 0-based, but the line/column of a file as displayed in an editor or dev tools are 1-based.
