Commit graph

115 commits

Author SHA1 Message Date
Alex Kirszenberg
839b55ff06 Add support for pages/_app (vercel/turbo#442)
This adds preliminary support for pages/_app. There's still some work to do here to support things like `App. getInitialProps`, etc., but that will come in later PRs. I've created issues for these for now.

One issue remaining is that renaming `pages/_app.js` does not switch to the default _app asset. The server needs to be restarted for this to happen. Similarly, adding an _app.js file does not switch from the default asset to it. I'm guessing there's something wrong with the way I set up the import mappings, and they aren't being invalidated properly?

fixes vercel/turbo#424
2022-10-07 11:36:44 +00:00
Tobias Koppers
39663ebc78 pass request info to content source (vercel/turbo#458)
allow the ContentSource to declare which info is required
2022-10-07 13:35:35 +02:00
Tobias Koppers
e8359f7964 refactor AssetContext (vercel/turbo#465)
remove context_path from AssetContext
remove most with_* methods from AssetContext
add ResolveOrigin trait
use ResolveOrigin to pass along context path and context for resolving
ecmascript and css module assets implement ResolveOrigin

Noteable change: Many places use `origin_path` instead of `context_path` which means it points to the issuer module path instead of the issuer module directory.
2022-10-07 13:34:31 +02:00
Will Binns-Smith
e4f92b7206 Automatic downleveling to browser targets using swc_preset_env (vercel/turbo#380)
Closes vercel/turbo#440

This uses swc's preset_env to automatically downlevel code according to the environment's browser targets, and sets next-dev to use a limited, modern target.

To do:
* [x] Add snapshot test
* [x] turbotrace test failures — looks like this has something to do with the Buffer module? Hitting swc's "multiple constructors" not implemented: f655488cfa/crates/swc_ecma_transforms_compat/src/es2015/classes/mod.rs (L545)
* [x] ~Benchmark downleveling node_modules (probably on front) and make a decision re: downleveling everything vs. just workspaces~ Filed as vercel/turbo#457
2022-10-07 06:54:21 +00:00
Tobias Koppers
d49a9c3b9d move ProcessEnv into turbo-tasks-env (vercel/turbo#462)
make ProcessEnv a trait to allow other implemenetations of it
2022-10-06 20:36:36 +02:00
LongYinan
8c80ba4f78 Migrate to pnpm (vercel/turbo#336)
### turbo tracing in pnpm projects

How pnpm link packages:

![image](https://user-images.githubusercontent.com/3468483/191022172-962af095-9b44-4b69-8e2f-1e8d440d1c31.png)

After turbo tracing:

```
dist
├── crates
│   └── turbopack
│       └── tests
│           └── node-file-trace
│               ├── integration
│               └── node_modules
│                   └── better-sqlite3 -> ../../../../../node_modules/.pnpm/better-sqlite3@7.6.2/node_modules/better-sqlite3
└── node_modules
    └── .pnpm
        ├── better-sqlite3@7.6.2
        │   └── node_modules
        │       ├── better-sqlite3
        │       │   ├── build
        │       │   │   └── Release
        │       │   └── lib
        │       │       └── methods
        │       └── bindings -> ../../bindings@1.5.0/node_modules/bindings
        ├── bindings@1.5.0
        │   └── node_modules
        │       ├── bindings
        │       └── file-uri-to-path -> ../../file-uri-to-path@1.0.0/node_modules/file-uri-to-path
        └── file-uri-to-path@1.0.0
            └── node_modules
                └── file-uri-to-path
```

No more **global .pnpm store**. The directory link will follow the original path created by pnpm.
2022-10-05 16:42:55 +00:00
Tobias Koppers
a8b0fd4f4b add layer to ChunkingContext and include that in module id and chunk paths (vercel/turbo#449)
this allows multiple versions of modules to co-exist in one chunking context e. g. RSC layer and SSR layer
2022-10-02 06:30:57 +00:00
Leah
2ffcbe38e0 simpler runtime inclusion (vercel/turbo#438) 2022-09-30 18:03:27 +00:00
Tobias Koppers
b9638d91d5 change WrapperAsset to some more general purpose VirtualAsset (vercel/turbo#447)
Wrapper is a bit of a weird name since it's not actually wrapping the existing asset. Actually it's just a virtual asset with a path below the existing path.
2022-09-30 11:58:35 +00:00
Tobias Koppers
1a5f27bd88 fixes some nitpicks (vercel/turbo#448) 2022-09-30 11:52:36 +00:00
Tobias Koppers
797bbc1a2f Refactor nodejs rendering to be more isolated and reusable (vercel/turbo#428)
(Prerequirement for `app` support)
2022-09-30 08:13:37 +00:00
Leah
9fd9163096 use embed macro (vercel/turbo#437) 2022-09-29 21:36:48 +00:00
Leah
fb76744de1 fix asset paths in server rendered pages (vercel/turbo#436)
Currently, when a server rendered page imports an asset (e.g. an image) there's a hydration mismatch
2022-09-29 17:14:15 +00:00
Tobias Koppers
3ad7c51768 use embed_file (vercel/turbo#432) 2022-09-28 19:32:09 +00:00
Tobias Koppers
a3dd9915d7 Use ValueToString for AssetReference instead of description() (vercel/turbo#430)
(Prerequirement for `app` support)
2022-09-28 19:02:39 +00:00
LongYinan
402c64be00 Introduce AssetContent to handle symlink assets (vercel/turbo#406)
* Fix DiskFileSystem::read_link

* pnpm-like integration test

* Introduce AssetContent to handle symlink assets

* Fix read_link on Windows

* Run clippy fix

* Rename `path` to `target`

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>

* Split Windows specified code

* Add comments about Redirect is only represent Directory

* Handle symlink while reading content

* Clippy fix

* Revert previous changes in FileSystemPathVc::get_type

* Fix Unix read_link

* cleanup

* handle symlink while resolving native bindings

* Make LinkContent::Link contains only target

* Add LinkType to represent link type

* Cleanup VersionedAsset

* Cleanup LinkType

* Normalize the LinkContent::target

* Comments

* Revert special case workaround for sharp on Windows

* comments

* node_native_binding follow file link

* Apply CR suggestion

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
2022-09-28 22:44:32 +08:00
Tobias Koppers
3263f5e5e6 replace json with serde_json (vercel/turbo#398)
get rid of `json` in favor of only `serde_json` for consistency and less dependencies
2022-09-28 09:02:40 +00:00
Justin Ridgewell
25ca6b2e2c Implement .env loading (vercel/turbo#409)
This implements `.env` loading by taking advantage of the [dotenvy](https://docs.rs/dotenvy/latest/dotenvy/) crate. Unfortunately, this isn't as featureful as the npm `dotenv`, lacking `FOO=${BAR:-default}` default support (this might be important), `FOO=${MAYBE_UNDEFINED:-${BAR-:default}}` nested fallbacks, and `` FOO=`backtick` `` support.

This is then converted into a series of assignments for the client code. This is run before any of the user code.

```js
const env = process.env;

env["FOO"] = "bar";
//...
```

- - -

TODO:
- [x] ~~Replace `process.env.FOO`/`process.env["FOO"]` during chunk compilation~~
  - using a runtime module speeds up HMR because the code doesn't need to be recompiled.
- [x] I don't actually know how to test server rendering, so I assume it works but haven't verified.
2022-09-27 20:33:45 +00:00
Leah
b7e9d4e3f0 more type safe next-dev builder (vercel/turbo#388) 2022-09-21 17:14:24 +00:00
Leah
620a6e0aaf small fixes / improvements (vercel/turbo#341)
Many small things I found

The most important is probably the typescript transform

The remaining bits should hopefully be self-explanatory from the commit messages
2022-09-21 15:47:23 +00:00
Leah
647a87f8f0 better ssr error message (vercel/turbo#387) 2022-09-20 18:57:49 +00:00
LongYinan
b9c36e7748 Align eslint and prettier rules with turborepo (vercel/turbo#397) 2022-09-20 07:33:15 +00:00
Tobias Koppers
a9b5ada56a fail-safe bindings code for SSR (vercel/turbo#391) 2022-09-19 19:11:14 +02:00
Tobias Koppers
1f7df032e2 benchmarks need more time and skip faster when failing (vercel/turbo#393) 2022-09-19 19:01:18 +02:00
Tobias Koppers
65be8a0804 handle exceptions during waiting for events (vercel/turbo#383) 2022-09-19 15:22:05 +00:00
Alex Kirszenberg
e35ad2f2dc Use React.memo in the test/bench app (vercel/turbo#384)
With React.memo:
```
bench_hmr_to_commit/Turbopack CSR/30000 modules
                        time:   [50.608 ms 51.659 ms 52.553 ms]
```

Without React.memo:
```
bench_hmr_to_commit/Turbopack CSR/30000 modules
                        time:   [853.47 ms 1.0191 s 1.1873 s]
                        change: [+1543.4% +1872.7% +2207.8%] (p = 0.00 < 0.05)
                        Performance has regressed.
```

Since we're only ever editing the top-level triangle in our HMR benchmarks, we're incurring the time it takes for React to re-render the whole tree, which is a function of the number of components in said tree. By using `React.memo`, we can skip updating children components during HMR.
2022-09-19 15:00:22 +00:00
Tobias Koppers
00b7a92a39 measure HMR to eval and HMR to commit independent from each other (vercel/turbo#382)
HMR to commit include react-refresh time to apply the update in the react component tree
2022-09-19 14:48:18 +02:00
Tobias Koppers
6d1a83384a Benchmark summarize script, workflow and stability (vercel/turbo#381) 2022-09-19 09:24:55 +00:00
Tobias Koppers
36c3d2d80f SourceMap fixes and improvements (vercel/turbo#374)
moves the logic of creating SourceMap assets into the asset reference. This avoids depending on the chunk items in the references() method directly. It also avoids calling CodeVc::source_map() until the source map is read

avoid circular dependency in call graph

It also avoids checking `has_source_map()` and just inserts potential Source Maps assets for every chunk item. Checking `has_source_map()` seems unnecessary work to do for all chunk items, when we can just send an empty source map.

only expose SourceMaps for chunk items when HMR is enabled
2022-09-16 15:08:00 +00:00
Tobias Koppers
7795fafdc7 add typescript support (vercel/turbo#370)
picked typescript transform from https://github.com/vercel/turbo-tooling/pull/341

add resolve options context as global resolve config

enable typescript only for next-dev

move emulating logic from environment to resolve options context

Co-authored-by: Leah <8845940+ForsakenHarmony@users.noreply.github.com>
2022-09-16 09:22:37 +00:00
Tobias Koppers
86f1831b06 fix compare action (vercel/turbo#373) 2022-09-16 08:06:54 +02:00
Will Binns-Smith
cf1eadb5ac Implement styled-jsx with swc's styled_jsx (vercel/turbo#367)
This reverts commit ccf6c112d541f0a1a9f4e691f6220363550aaf01.
2022-09-15 13:51:45 -07:00
Tobias Koppers
22e996b90c ensure that HMR is really happening while measuring (vercel/turbo#365)
no need to test all module counts
2022-09-15 09:54:07 +00:00
Will Binns-Smith
b6d38cd4fd Revert "Implement styled-jsx with swc's styled_jsx" (vercel/turbo#366)
Revert "Implement styled-jsx with swc's styled_jsx (vercel/turbo#354)"

This reverts commit 255a38b07c98252bfe442ee363f07924da45288a.
2022-09-15 02:24:08 +02:00
Tobias Koppers
f4e28a07ef enable react refresh for vite (vercel/turbo#364)
It was doing a full refresh before
2022-09-15 00:04:49 +00:00
Will Binns-Smith
d4192f4bd5 Implement styled-jsx with swc's styled_jsx (vercel/turbo#354)
This implements support for styled-jsx in next-dev using swc's styled_jsx crate.

It's only applied in next-dev, and is only applied as a transform to app code, much like the react-refresh transform.

To do:
* [x] The transform doesn't seem to be applied. Pass the added test.

Test Plan: `cargo test -p next-dev --
test_crates_next_dev_tests_integration_turbopack_basic_styled_jsx
--nocapture`
2022-09-14 23:46:18 +00:00
Tobias Koppers
826998cf07 increase benchmark timeouts to be able to benchmark larger module counts (vercel/turbo#362) 2022-09-14 23:17:14 +00:00
Alex Kirszenberg
50aa9cc261 Initial support for getStaticProps (vercel/turbo#345)
Remaining questions:
* Should we have some static analysis for `getStaticProps` instead of looking into exports at runtime?
* For now, the output of `getStaticProps` (if defined) will always trump the value passed in as `data`. If we consider `data` to be the cached output of `getStaticProps` (in the future, as this is not yet implemented), this logic should be adapted.
2022-09-14 21:31:14 +00:00
Will Binns-Smith
eb93985d95 Benchmarks: allow simultaneously installing multiple packages from npm (vercel/turbo#339)
Previously, we ran multiple `npm install` operations in serial using multiple calls to `install_from_npm`. Instead, this allows us to express dependencies all at once as a single command to the npm cli, which should reduce the time we spend installing from npm and updating package.json.

Test Plan: Manually confirmed that package.json was updated correctly.  `cargo bench`.
2022-09-12 12:34:26 -07:00
Will Binns-Smith
aaa640def6 Benchmark Webpack (vercel/turbo#338)
This adds webpack 5 to the benchmark suite.

Test Plan: Manually confirmed package.json updates and webpack config written to temp dir correctly. `cargo bench`.
2022-09-12 11:09:26 -07:00
Will Binns-Smith
361ca5eca8 Split benchmark code into more modules (vercel/turbo#337)
This splits the benchmark code into more modules. Notes:

* ~Moved/left `get_bundlers()` and `get_module_counts()` to/in mod.rs. In particular, moving `get_bundlers()` to either bundle.rs or util.rs would lead to a circular dependency. These both also rely on env var configuration, so I figured this was a reasonable place for them.~
* The Bundler trait has its own module (not moved to util), since it's a top-level concern and not really a miscellaneous utility.
* Each bundler has its own module file.

Test Plan: `TURBOPACK_BENCH_BUNDLERS=all cargo test --benches -p next-dev -- --nocapture` and verify same output as before change.
2022-09-09 16:35:12 -07:00
Tobias Koppers
ecaf59b3e6 improve startup metric measurement (vercel/turbo#331)
This fixes the `startup` benchmark metric to show the time until first render, instead of being equal to hydration.

It also skips `hydration` for CSR bundlers since it's equal first render for them.

It now correctly measures the initial SSR as `startup`, so CLI start until page visible:
![image](https://user-images.githubusercontent.com/1365881/189152420-396b181b-5a3e-4902-881d-7c247fa43bd8.png)
2022-09-08 16:00:15 +00:00
Alex Kirszenberg
0a112ccb14 Fix Next.js 11 React version (vercel/turbo#333) 2022-09-08 15:18:53 +00:00
Tobias Koppers
378adc7112 cleanup nitpicks (vercel/turbo#330) 2022-09-08 14:54:09 +00:00
Tobias Koppers
6f312afedc allow to pass module counts list via env var (vercel/turbo#332)
add custom benchmark workflow

run large counts in CI
2022-09-08 16:46:19 +02:00
Alex Kirszenberg
a45572b642 Ensure pages close before app does (vercel/turbo#329) 2022-09-08 15:02:48 +02:00
Alex Kirszenberg
6f0fc67cab Stop processes gracefully when possible, add drop guard to pages (vercel/turbo#328) 2022-09-08 12:00:19 +00:00
Will Binns-Smith
dd217c71b9 Benchmark Parcel (vercel/turbo#322)
* Benchmark Parcel

* add Parcel to the CI benchmarks

* move some turbopack dependencies to the bundler as they conflict with other bundlers

Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
2022-09-08 13:08:25 +02:00
Will Binns-Smith
abcdd78623 Benchmark Next.js 11 and 12 (vercel/turbo#304)
This implements benchmark support for Next.js 12. Next.js (the tool) expects to be able to resolve from the `next` package in the cwd, so it must be installed alongside the other node_modules in the test.  `prepare` was added to the Bundler trait to handle this case.

Test Plan: `TURBOPACK_BENCH_ALL=all cargo bench -p next-dev`


Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
2022-09-08 05:00:32 +00:00
Tobias Koppers
512539821d fix runtime code for server rendered content source (vercel/turbo#315)
move transition into turbopack
2022-09-07 15:56:10 +02:00