Updates the way filenames are generated for browser compilation.
Notably:
- All entry bundles now have hashes in production, this includes pages (previously pages used a buildId in the path)
- The AmpFiles no longer depends on hardcoded bundle names, it uses the buildManifest instead (internals)
- All cases where we match the page name from the chunk/entrypoint name now use the same function `getRouteFromEntrypoint` (internals)
- In development we no longer include the "faked" `buildId` set to `development` for page files, instead we just use the `/_next/static/pages` path (was `/_next/static/development/pages`). This was changed as it caused unneeded complexity and makes generating the bundles easier (internals)
- Updated tons of tests to be more resilient to these changes by relying on the buildManifest instead of hardcoded paths (internals)
Follow up of these PRs:
https://github.com/vercel/next.js/pull/13759https://github.com/vercel/next.js/pull/13870https://github.com/vercel/next.js/pull/13937https://github.com/vercel/next.js/pull/14130https://github.com/vercel/next.js/pull/14176https://github.com/vercel/next.js/pull/14268Fixes#6303Fixes#12087Fixes#1948Fixes#4368Fixes#4255Fixes#2548
This builds off of @timneutkens's PR instead of updating it because it's his `canary` branch.
Updated to still `stat`, as it's the only way to test the difference between a file and directory.
---
Closes#13506Fixes#12235
Extracted from https://github.com/vercel/next.js/pull/13333, the same exact code lives in that PR as well, but we can merge this separately if it makes reviewing https://github.com/vercel/next.js/pull/13333 easier
This PR does 3 things
- deduplicate code from build and next-dev-server that loads custom routes from next.config.js (`loadCustomRoutes`)
- in `loadCustomRoutes`, load these rewrites, headers and redirects configs concurrently instead of sequentially.
- in next-server, make `this.customRoutes` always defined, this allows us to remove the big `if` around its initialization code in `generateRoutes`, which in turn makes it possible to reuse this code for other routing than user defined routes, which is how https://github.com/vercel/next.js/pull/13333 adds its redirects.
Disambiguate between pages/index.js and pages/index/index.js so that they resolve differently.
It all started with a bug in pagesmanifest that propagated throughout the codebase. After fixing pagesmanifest I was able to remove a few hacks here and there and more logic is shared now. especially the logic that resolves an entrypoint back into a route path. To sum up what happened:
- `getRouteFromEntrypoint` is the inverse operation of `getPageFile` that's under `pages/_document.tsx`
- `denormalizePagePath` is the inverse operation of `normalizePagePath`.
Everything is refactored in terms of these operations, that makes their behavior uniform and easier to update/patch in a central place. Before there were subtle differences between those that made `index/index.js` hard to handle.
Some potential follow up on this PR:
- [`hot-reloader`](https://github.com/vercel/next.js/pull/13699/files#diff-6161346d2c5f4b7abc87059d8768c44bR207) still has one place that does very similar behavior to `getRouteFromEntrypoint`. It can probably be rewritten in terms of `getRouteFromEntrypoint`.
- There are a few places where `denormalizePagePath(normalizePagePath(...))` is happening. This is a sign that `normalizePagePath` is doing some validation that is independent of its rewriting logic. That should probably be factored out in its own function. after that I should probably investigate whether `normalizePagePath` is even still needed at all.
- a lot of code is doing `.replace(/\\/g, '')`. If wanted, that could be replaced with `normalizePathSep`.
- It looks to me like some logic that's spread across the project can be centralized in 4 functions
- `getRouteFromEntrypoint` (part of this PR)
- its inverse `getEntrypointFromRoute` (already exists in `_document.tsx` as `getPageFile`)
- `getRouteFromPageFile`
- its inverse `getPageFileFromRoute` (already exists as `findPageFile ` in `server/lib/find-page-file.ts`)
It could be beneficial to structure the code to keep these fuctionalities close together and name them similarly.
- revise `index.amp` handling in pagesmanifest. I left it alone in this PR to keep it scoped, but it may be broken wrt nested index files as well. It might even make sense to reshape the pagesmanifest altogether to handle html/json/amp/... better
**First, apologies for a second PR on the same issue but I was working on this already so I thought I'd push it and let you decide which you want to merge.**
The PR is related to [13466](https://github.com/vercel/next.js/issues/13466).
Based on my research, the error happens if the options are empty, null, or undefined. That's why I have decided that the most proper check would be using the! post-fix expression operator may assert that its operand is non-null and non-undefined. ``if (options == null)``
(Optional)
I have also added a warning, which warns the user if the passed "dev" variable is not a boolean.
It's my first PR on the "packages" part of the repo so I'd be glad to receive all kinds of critics. If you want me to change or remove anything, I'm open to suggestions.
---
Fixes#13466
Was going through _document and noticed some variable shadowing going on. Added a rule for it to our eslint configuration and went through all warnings with @Timer.
This code existed mostly to work around webpack 2 (yes 2.x) limitations where it crashed in certain cases where files didn't exist anymore. We have tests for that behavior and latest webpack has fixed these. Hence why this can be removed 👍
This removes `fork-ts-checker-webpack-plugin` and instead directly calls the TypeScript API.
This is approximately 10x faster.
Base build: 7s (no TypeScript features enabled)
- `fork-ts-checker-webpack-plugin@3.1.1`: 90s, computer sounds like an airplane
- `fork-ts-checker-webpack-plugin@4.1.6`: 84s, computer did **not** sound like an airplane
- `fork-ts-checker-webpack-plugin@5.0.0-alpha.14`: 90s, regressed
- `npx tsc -p tsconfig.json --noEmit`: 12s (time: `18.57s user 0.97s system 169% cpu 11.525 total`)
- **This PR**: 22s, expected to get better when we run this as a side-car
All of these tests were run 3 times and repeat-accurate within +/- 0.5s.
As discussed this adds bundling of `.env` files in `serverless` mode so that the environment values are also available when deploying with this target
closes: https://github.com/zeit/next.js/issues/13332
* fix(debugging): do not pass NODE_OPTIONS='--inspect' to subprocesses
fixes#11030
* fix(debugger): use a regex to remove bad NODE_OPTIONS flags
Co-authored-by: Alec Larson <alec.stanford.larson@gmail.com>
* Rename getServerProps to getServerSideProps
* Remove unstable_ prefix from new methods
* Add error when legacy methods are detected
* Add legacy methods for babel transform
* Add unstable_getServerSideProps also
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Update types import
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Add calling getStaticPaths in development before showing fallback
* Move staticPathsWorker to next-dev-server
* Make sure to clear require cache in worker process
* bump
* Remove staticPathsCache member
* Update numWorkers for staticPathsWorker
* Clean up landed experimental flags
* Remove check for experimental flags from build too
* Remove /_errors/404 in favor of /404
* Remove unneeded check for pathname
* Update test paths
* Fix test
* Update test
* Remove test for disabled config
* Set pages404 always to true
Co-authored-by: Joe Haddad <timer150@gmail.com>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
* checkpoint: api impl
* Add support for tryGetPreviewData
* snapshot: server(less) support
* Add X-Prerender-Bypass-Mode header support
* Pass preview data to getStaticProps call
* add TODO
* setPreviewData
* 100k iterations
* Handle jwt error
* Write out preview values
* forgot file
* set preview props
* Send preview props
* add preview props
* Pass around more data
* update yarn lock
* Fail on Invalid Prerender Manifest
* Make Missing Prerender Manifest Fatal
* fix ts errors
* fix test
* Fix setting cookies + maxage
* Secure is not needed as we encrypt necessary data
* Set on domain root
* Set cookie max ages
* Render a fallback on-demand for non-dynamic pages
* Test preview mode
* remove old build
* remove snapshots
* Add serverless tests
* use afterAll
* Remove object assigns
* fix cookie spread
* add comment
* Update next-server routes order for expected priority
* Update router to allow disabling page routes
* Fix headers having check: true behavior when they should not
Co-authored-by: Joe Haddad <timer150@gmail.com>
* Update to use existing util to de-dupe path check
* Update error message for requested/resolved mismatch
* Use correct dataRoute value for prerender manifest
* Fix pageUrl having double slash on Windows
* Implement experimental pages/404.js for custom 404 page
* Make sure to show error for getInitialProps in pages/404 in dev mode also
* Update routes-manifest tests for new value
* Make sure page404 is boolean in routes-manifest
* Rename variables for consistency
* Make sure to only use 404 page for 404 error
* Update serving of files from public folder to handle special chars
* Update tests and match handling in dev mode
* Fix windows public file handling
* Remove colon test to make git on windows happy
* Remove un-used files
* Add missing await
* Add check: true behavior to custom routes
* Update adding dev routes
* Add checking of pages and dynamic routes for check: true
* Fix hasPage binding
* Add tests for check: true behavior
* Update regex checking
* Make changes based on review
* Update to handle rewrites in serverless loader
* Update to not change req.url
* Make sure to always parse dynamic route params
* Export all of pathToRegexp from path-match
Co-authored-by: Joe Haddad <timer150@gmail.com>
* De-dupe pagesManifest
* Update handleApiRequest a bit
* Simplify handleApiRequest a bit
* Update packages/next/next-server/server/next-server.ts
Co-authored-by: Joe Haddad <joe.haddad@zeit.co>
* Add checking of custom routes for invalid fields
* Remove un-used test imports
* Mentioned statusCode can be undefined in error message
* Update test
* Update invalid routes output
* Add checking to make sure source/destination start with slash
* Update import
* Fix next-news inconsistent return type
* Add failing test for static file rewrite
* Revert "Fix next-news inconsistent return type"
This reverts commit b66e76fb86061e45741c3c4bb9296a0874900f76.
* Add failing test for double redirect
* Fix bug in matcher when having multiple redirects
* Remove custom traversal in favor of router handling it
* Update next-server.ts
* Update router.ts
* Temporarily disable test
* Don't deeply resolve redirects
* Support combined parameters + query passing
* Make sure params are correctly passed in
* Add test for hash in route
* add initial custom-routes handling
* Add tests for custom-routes
* Handle chained redirects, separate dev custom
routes reading, and add version to routes manifest
* Handle no routes manifest
* Swap build custom routes calling
* Add flatten-routes
* Add flattening of custom routes
* Re-work implementation to follow routes top-down
* Add regex field to routes-manifest
* Fix path-to-regexp match breaking after upgrade
* Fix duplicate const from merge
* Add some changes from review
* Don't make path-match strict
* add default custom route values
* Update routes-manifest
* Update options for path-match
* Remove todo
* Add test for rewrite with params
* Only use strict mode for custom routes
* Update dynamic-routing test
* Move getCustomRoutes to prepare
* Remove extra change
* Update handling for error-in-error test
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Apply suggestions from review
* Update slice change
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Apply suggestions from review
* Fix TypeScript error
* Fix getCustomRoutes in dev mode
* Apply suggestions from code review
* Update slice
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Delete un-used test page
* Add test for param overwriting
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Add extra check to param test
* Add a configuration flag to disable integrated type checker
* Add tests for typescript-transpileonly
* Restore removed argument
* Make output more coherent
* Split transpileOnly into ignoreDevErrors and ignoreBuildErrors
* Minor stylistic change
* Add tracking src dir usage to telemetry
* Move isSrcDir back to eventVersion
* Move spinner back
* Add test for isSrcDir telemetry
* Add test for dev mode
* Remove experimental.publicDirectory config
* Error when public is set as an output dir
* Remove experimental.publicDirectory checks
* Update publicFiles checking in next build
* Convert Dev Server to TypeScript
This converts the Next.js Development Server to TypeScript in prep for upcoming changes.
* Convert remaining necessary
* Fix some type errors
* Adjust types
* Use Better Telemetry Directory
So, as it turns out, storing in `node_modules` is a bad idea.
Both npm and Yarn will remove additional files when you run `npm install` or `yarn install`.
Instead, we'll store this inside of Next.js' `distDir`. This should also be cached by users, if it's not, it probably won't be any worse as compared to `node_modules`.
* Fix directory name
* Fix build setup
* Record when export session is started
* Move more into branch
* initial commit for SPRv2
* Add initial SPR cache handling
* update SPR handling
* Implement SPR handling in render
* Update tests, handle caching with serverless next
start, add TODOs, and update manifest generating
* Handle no prerender-manifest from not being used
* Fix url.parse error
* Apply suggestions from code review
Co-Authored-By: Joe Haddad <joe.haddad@zeit.co>
* Replace set with constants in next-page-config
* simplify sprStatus.used
* Add error if getStaticProps is used with getInitialProps
* Remove stale TODO
* Update revalidate values in SPR cache for non-seeded routes
* Apply suggestions from code review
* Remove concurrency type
* Rename variable for clarity
* Add copying prerender files during export
* Add comment for clarity
* Fix exporting
* Update comment
* Add additional note
* Rename variable
* Update to not re-export SPR pages from build
* Hard navigate when fetching data fails
* Remove default extension
* Add brackets
* Add checking output files to prerender tests
* Adjust export move logic
* Clarify behavior of export aggregation
* Update variable names for clarity
* Update tests
* Add comment
* s/an oxymoron/contradictory/
* rename
* Extract error case
* Add tests for exporting SPR pages and update
/_next/data endpoint to end with .json
* Relocate variable
* Adjust route building
* Rename to unstable
* Rename unstable_getStaticParams
* Fix linting
* Only add this when a data request
* Update prerender data tests
* s/isServerless/isLikeServerless/
* Don't rely on query for `next start` in serverless mode
* Rename var
* Update renderedDuringBuild check
* Add test for dynamic param with bracket
* Fix serverless next start handling
* remove todo
* Adjust comment
* Update calculateRevalidate
* Remove cache logic from render.tsx
* Remove extra imports
* Move SPR cache logic to next-server
* Remove old isDynamic prop
* Add calling App getInitialProps for SPR pages
* Update revalidate logic
* Add isStale to SprCacheValue
* Update headers for SPR
* add awaiting pendingRevalidation
* Dont return null for revalidation render
* Adjust logic
* Be sure to remove coalesced render
* Fix data for serverless
* Create a method coalescing utility
* Remove TODO
* Extract send payload helper
* Wrap in-line
* Move around some code
* Add tests for de-duping and revalidating
* Update prerender manifest test
* Match public files priority in dev
* Remove un-needed old public file handling
* Run public file priority test for non-dev also
* Make sure to enable publicDirectory
* Add checking for conflicting pages in dev and during build
* Add error for public dir middleware conflict
* Add err.sh for _next conflict
* Move up public dir conflict checking
* Only run _next conflict check in dev when path contains it
* Fail build on duplicate pages
This will fail the `next build` command when a duplicate page is found.
In development, we'll emit a warning instead of crashing the dev server.
* Add test for warning in development
* Only issue a warning
* Fix production test
* Fix development test
* Remove useless arg
* Warn in development, too
* Dynamic Routes: Change impl from $param to [param]
* Update expected test snapshot
* Update test to use new syntax
* Update test file
* Test more behavior
* Update route sorter for new param syntax
* Update dynamic routing tests
* Update danging test file
* Tweak test
* Fix dev and update tests
* Move client-side dev JS to dev folder
* Move eventsource polyfill
* Move source-map-support
* Move error boundary
* Deprecate Container in _app
* Make initialRender check better
* Remove unused code
* Only support one subscription as there is only one
* Don’t spread object
* Shorten property name
* Add container in development too
* Simplify query update logic
* Revise dynamic route generation
This implements a new tree-based route sorting algorithm that uses a Depth-First-Traversal approach to correctly sort the routes.
This provides better clarity over a `.sort()` based approach and will scale well as we add new features in the future.
* Update import
* Run prettier over packages/**/*.js
* Run prettier over packages/**/*.ts
* Run prettier over examples
* Remove tslint
* Run prettier over examples
* Run prettier over all markdown files
* Run prettier over json files
* Update escape string regexp operators
* temp
* Extract getRouteRegex func
* First iteration of dynamic routing for production only
* Correctly order prod
* Add serverless support
* Single line it
* noop routes
* Format doc
* Fix dynamic routing for dev
* Add flag for dynamic routing
* Update packages/next-server/lib/router/router.ts
Co-Authored-By: JJ Kasper <jj@jjsweb.site>
* remove example
* Add router tests
* Format code
* Sort routes
* Update to not use posix path methods
We also close the connection when the window is in the background and re-connect when it is brought to the foreground. This prevents us from using up too many connections.
* Remove client bundles for AMP pages
after build since they are not used
* Remove trailing white space
* Use async-sema to limit removing AMP client bundles
* Bring AMP client bundle removing
semaphore concurrency down to 20
* Don't check blocked pages when
deleting AMP client bundles
* Update client bundle removing for AMP pages
* Add error handling for removing client AMP pages
* rethrow error unless ENOENT during
deleting AMP client pages
* Handle error during removing AMP client
pages the same during dev
* Fix throwing instead of rejecting
* Make sure next/config is set before requiring page
* Update error check
* return on reject
* Fix next/config
* Only refresh the page when the active
page is updated in AMP mode
* Update handling of page reload to make sure it
still refreshes after a change to another page
* Update checking to be more accurate
* Fix amp-dev not being loaded without
experimental.amp and remove next.config from amp example
* Remove old with-amp example and
rename experimental-amp to with-amp
* update example name
Co-Authored-By: ijjk <jj@jjsweb.site>
* Update comment wording
Co-Authored-By: ijjk <jj@jjsweb.site>
* Use document for reload to keep scroll position
Co-Authored-By: ijjk <jj@jjsweb.site>
* fallback to reloading on error
Co-Authored-By: ijjk <jj@jjsweb.site>
* Update with-amp example readme
* Add WithAmp to enable AMP support for
pages instead of .amp.js
* Update handling for exporting AMP
* Fix ampPath in export for / path and
revert isAmp logic to handle right
* Update amphtml test suite
* Add handling for noDirtyAmp during
export and update amp-export test suite
* Update serverless and export-default-map
test suites
* Update require-page tests
* Do not clear the console
Its rude to clear the console, you may be sharing output with other processes even in tty mode.
* Remove unused dependency
* Dedupe and cleanup dev output without clearing
* use logError
* Remove exit handler
* Add next helper
* Add log helpers
* Switch store to log helpers and a shallow object compare
* Update other files to use new logging utility
* request => build
* Update ready on messages
* Use case insensitive matching
* Add checking of react versions to make sure it
meets the minimum set in peerDependencies
* Simplify react check
* Update error wording
Co-Authored-By: timneutkens <tim@timneutkens.nl>
* Add err.sh
* Update test-production circleci job name
* Add react error message to next-dev-server
* Update test for new wording
* Add support for .amp.js pages and
resolving /page?amp=1 to page.amp.js
* Update amp tests
* Update example and clean up amp page resolving
* Add nested amp test
* page => normalizedPage
* Add type to page options
* Add handling of amp with all pageExtensions
and normalize page
* Make sure findPageFile only falls back to
amp if enabled
* Show a better error when someone throws undefined
* Update error wording
Co-Authored-By: ijjk <22380829+ijjk@users.noreply.github.com>
* Update error wording in test
Co-Authored-By: ijjk <22380829+ijjk@users.noreply.github.com>
* Update test and add check for statusCode
before updating error
