Follow-up to #37551
Implements prefetching for the new router.
There are multiple behaviors related to prefetching so I've split them out for each case. The list below each case is what's prefetched:
Reference:
- Checkmark checked → it's implemented.
- RSC Payload → Rendered server components.
- Router state → Patch for the router history state.
- Preloads for client component entry → This will be handled in a follow-up PR.
- No `loading.js` static case → Will be handled in a follow-up PR.
---
- `prefetch={true}` (default, same as current router, links in viewport are prefetched)
- [x] Static all the way down the component tree
- [x] RSC payload
- [x] Router state
- [ ] preloads for the client component entry
- [x] Not static all the way down the component tree
- [x] With `loading.js`
- [x] RSC payload up until the loading below the common layout
- [x] router state
- [ ] preloads for the client component entry
- [x] No `loading.js` (This case can be static files to make sure it’s fast)
- [x] router state
- [ ] preloads for the client component entry
- `prefetch={false}`
- [x] always do an optimistic navigation. We already have this implemented where it tries to figure out the router state based on the provided url. That result might be wrong but the router will automatically figure out that
---
In the first implementation there is a distinction between `hard` and `soft` navigation. With the addition of prefetching you no longer have to add a `soft` prop to `next/link` in order to leverage the `soft` case.
A heuristic has been added that automatically prefers `soft` navigation except when navigating between mismatching dynamic parameters.
An example:
- `app/[userOrTeam]/dashboard/page.js` and `app/[userOrTeam]/dashboard/settings/page.js`
- `/tim/dashboard` → `/tim/dashboard/settings` = Soft navigation
- `/tim/dashboard` → `/vercel/dashboard` = Hard navigation
- `/vercel/dashboard` → `/vercel/dashboard/settings` = Soft navigation
- `/vercel/dashboard/settings` -> `/tim/dashboard` = Hard navigation
---
While adding these new heuristics some of the tests started failing and I found some state bugs in `router.reload()` which have been fixed. An example being when you push to `/dashboard` while on `/` in the same transition it would navigate to `/`, it also wouldn't push a new history entry. Both of these cases are now fixed:
```
React.startTransition(() => {
router.push('/dashboard')
router.reload()
})
```
---
While debugging the various changes I ended up debugging and manually diffing the cache and router state quite often and was looking at a way to automate this. `useReducer` is quite similar to Redux so I was wondering if Redux Devtools could be used in order to debug the various actions as it has diffing built-in. It took a bit of time to figure out the connection mechanism but in the end I figured out how to connect `useReducer`, a new hook `useReducerWithReduxDevtools` has been added, we'll probably want to put this behind a compile-time flag when the new router is marked stable but until then it's useful to have it enabled by default (only when you have Redux Devtools installed ofcourse).
> ⚠️ Redux Devtools is only connected to take incoming actions / state. Time travel and other features are not supported because the state sent to the devtools is normalized to allow diffing the maps, you can't move backward based on that state so applying the state is not connected.
Example of the integration:
<img width="1912" alt="Screen Shot 2022-09-02 at 10 00 40" src="https://user-images.githubusercontent.com/6324199/188637303-ad8d6a81-15e5-4b65-875b-1c4f93df4e44.png">
WIP.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Co-authored-by: Shu Ding <3676859+shuding@users.noreply.github.com>
Expand explanation of new router.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Expanded app-render.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Fixes the hash being overwritten when you open a page.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
* inject the link tags before page content in layout router
* fix flight handling during client navigation
* fix buffering
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Renames todos to split them / make it easier to find what has to be done for the new router.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Handles the case where an error is introduced which causes a Fast Refresh and then it's fixed.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
Handles the case where you navigate between routes in `pages` and `app`.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
- Remove cache value that was incorrectly nested deeper
- Remove extra useEffect (already applied during hydration based on the `useReducer` input)
- Add dynamic parameter name into the tree
Follow-up to #37551, cleans up some code and prepares for catch-all and optional catch-all routes.
## Client-side router for `app` directory
This PR implements the new router that leverages React 18 concurrent features like Suspense and startTransition.
It also integrates with React Server Components and builds on top of it to allow server-centric routing that only renders the part of the page that has to change.
It's one of the pieces of the implementation of https://nextjs.org/blog/layouts-rfc.
## Details
I'm going to document the differences with the current router here (will be reworked for the upgrade guide)
### Client-side cache
In the current router we have an in-memory cache for getStaticProps data so that if you prefetch and then navigate to a route that has been prefetched it'll be near-instant. For getServerSideProps the behavior is different, any navigation to a page with getServerSideProps fetches the data again.
In the new model the cache is a fundamental piece, it's more granular than at the page level and is set up to ensure consistency across concurrent renders. It can also be invalidated at any level.
#### Push/Replace (also applies to next/link)
The new router still has a `router.push` / `router.replace` method.
There are a few differences in how it works though:
- It only takes `href` as an argument, historically you had to provide `href` (the page path) and `as` (the actual url path) to do dynamic routing. In later versions of Next.js this is no longer required and in the majority of cases `as` was no longer needed. In the new router there's no way to reason about `href` vs `as` because there is no notion of "pages" in the browser.
- Both methods now use `startTransition`, you can wrap these in your own `startTransition` to get `isPending`
- The push/replace support concurrent rendering. When a render is bailed by clicking a different link to navigate to a completely different page that still works and doesn't cause race conditions.
- Support for optimistic loading states when navigating
##### Hard/Soft push/replace
Because of the client-side cache being reworked this now allows us to cover two cases: hard push and soft push.
The main difference between the two is if the cache is reused while navigating. The default for `next/link` is a `hard` push which means that the part of the cache affected by the navigation will be invalidated, e.g. if you already navigated to `/dashboard` and you `router.push('/dashboard')` again it'll get the latest version. This is similar to the existing `getServerSideProps` handling.
In case of a soft push (API to be defined but for testing added `router.softPush('/')`) it'll reuse the existing cache and not invalidate parts that are already filled in. In practice this means it's more like the `getStaticProps` client-side navigation because it does not fetch on navigation except if a part of the page is missing.
#### Back/Forward navigation
Back and Forward navigation ([popstate](https://developer.mozilla.org/en-US/docs/Web/API/Window/popstate_event)) are always handled as a soft navigation, meaning that the cache is reused, this ensures back/forward navigation is near-instant when it's in the client-side cache. This will also allow back/forward navigation to be a high priority update instead of a transition as it is based on user interaction. Note: in this PR it still uses `startTransition` as there's no way to handle the high priority update suspending which happens in case of missing data in the cache. We're working with the React team on a solution for this particular case.
### Layouts
Note: this section assumes you've read [The layouts RFC](https://nextjs.org/blog/layouts-rfc) and [React Server Components RFC](https://reactjs.org/blog/2020/12/21/data-fetching-with-react-server-components.html)
React Server Components rendering leverages the Flight streaming mechanism in React 18, this allows sending a serializable representation of the rendered React tree on the server to the browser, the client-side React can use this serialized representation to render components client-side without the JavaScript being sent to the browser. This is one of the building blocks of Server Components. This allows a bunch of interesting features but for now I'll keep it to how it affects layouts.
When you have a `app/dashboard/layout.js` and `app/dashboard/page.js` the page will render as children of the layout, when you add another page like `app/dashboard/integrations/page.js` that page falls under the dashboard layout as well. When client-side navigating the new router automatically figures out if the page you're navigating to can be a smaller render than the whole page, in this case `app/dashboard/page.js` and `app/dashboard/integrations/page.js` share the `app/dashboard/layout.js` so instead of rendering the whole page we render below the layout component, this means the layout itself does not get re-rendered, the layout's `getServerSideProps` would not be called, and the Flight response would only hold the result of `app/dashboard/integrations/page.js`, effectively giving you the smallest patch for the UI.
---
Note: the commits in this PR were mostly work in progress to ensure it wasn't lost along the way. The implementation was reworked a bunch of times to where it is now.
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
