rsnext/packages/create-next-app/helpers/examples.ts

132 lines
3.6 KiB
TypeScript
Raw Normal View History

/* eslint-disable import/no-extraneous-dependencies */
import got from 'got'
import tar from 'tar'
import { Stream } from 'stream'
import { promisify } from 'util'
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
import { join } from 'path'
import { tmpdir } from 'os'
import { createWriteStream, promises as fs } from 'fs'
const pipeline = promisify(Stream.pipeline)
export type RepoInfo = {
username: string
name: string
branch: string
filePath: string
}
export async function isUrlOk(url: string): Promise<boolean> {
const res = await got.head(url).catch((e) => e)
return res.statusCode === 200
}
export async function getRepoInfo(
url: URL,
examplePath?: string
): Promise<RepoInfo | undefined> {
const [, username, name, t, _branch, ...file] = url.pathname.split('/')
const filePath = examplePath ? examplePath.replace(/^\//, '') : file.join('/')
if (
// Support repos whose entire purpose is to be a NextJS example, e.g.
// https://github.com/:username/:my-cool-nextjs-example-repo-name.
t === undefined ||
// Support GitHub URL that ends with a trailing slash, e.g.
// https://github.com/:username/:my-cool-nextjs-example-repo-name/
// In this case "t" will be an empty string while the next part "_branch" will be undefined
(t === '' && _branch === undefined)
) {
const infoResponse = await got(
`https://api.github.com/repos/${username}/${name}`
2020-05-18 21:24:37 +02:00
).catch((e) => e)
if (infoResponse.statusCode !== 200) {
return
}
const info = JSON.parse(infoResponse.body)
return { username, name, branch: info['default_branch'], filePath }
}
// If examplePath is available, the branch name takes the entire path
const branch = examplePath
? `${_branch}/${file.join('/')}`.replace(new RegExp(`/${filePath}|/$`), '')
: _branch
if (username && name && branch && t === 'tree') {
return { username, name, branch, filePath }
}
}
export function hasRepo({
username,
name,
branch,
filePath,
}: RepoInfo): Promise<boolean> {
const contentsUrl = `https://api.github.com/repos/${username}/${name}/contents`
const packagePath = `${filePath ? `/${filePath}` : ''}/package.json`
return isUrlOk(contentsUrl + packagePath + `?ref=${branch}`)
}
feat(cli): support examples without `package.json` (#37910) Previously our CLI assumed that when using an `--example` flag, the example contains a `package.json` file. There are examples though, like `with-docker-compose` which does not have one. In that case, we can simply clone the directory and show a success message. <details> <summary> Full list of examples without <code>package.json</code>: </summary> <ul> <li>custom-server</li> <li>page-transitions</li> <li>parameterized-routing</li> <li>reproduction-template</li> <li>with-chakra-ui-typescript</li> <li>with-cookie-auth</li> <li>with-docker-compose</li> <li>with-dotenv</li> <li>with-facebook-chat-plugin</li> <li>with-firebase-authentication</li> <li>with-firebase-authentication-serverless</li> <li>with-glamorous</li> <li>with-global-stylesheet</li> <li>with-global-stylesheet-simple</li> <li>with-graphql-faunadb</li> <li>with-markdown</li> <li>with-material-ui</li> <li>with-next-auth</li> <li>with-next-i18next</li> <li>with-next-multilingual</li> <li>with-next-routes</li> <li>with-now-env</li> <li>with-pretty-url-routing</li> <li>with-prisma</li> <li>with-redux-toolkit</li> <li>with-redux-toolkit-typescript</li> <li>with-sentry-simple</li> <li>with-strict-csp-hash</li> <li>with-styled-jsx-postcss</li> <li>with-sw-precache</li> <li>with-typescript-styled-components</li> <li>with-universal-configuration-build-time</li> <li>with-universal-configuration-runtime</li> </ul> </details> Fixes #37884 In a follow-up PR, we can try to detect if an example contains a README.md file only, and if the example was moved, we could either try getting it from the new location or at least show a better message. (Eg.: using a magic comment that we can parse) ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-06-23 19:28:14 +02:00
export function existsInRepo(nameOrUrl: string): Promise<boolean> {
try {
const url = new URL(nameOrUrl)
return isUrlOk(url.href)
} catch {
return isUrlOk(
`https://api.github.com/repos/vercel/next.js/contents/examples/${encodeURIComponent(
nameOrUrl
)}`
)
}
}
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
async function downloadTar(url: string) {
const tempFile = join(tmpdir(), `next.js-cna-example.temp-${Date.now()}`)
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
await pipeline(got.stream(url), createWriteStream(tempFile))
return tempFile
}
export async function downloadAndExtractRepo(
root: string,
{ username, name, branch, filePath }: RepoInfo
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
) {
const tempFile = await downloadTar(
`https://codeload.github.com/${username}/${name}/tar.gz/${branch}`
)
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
await tar.x({
file: tempFile,
cwd: root,
strip: filePath ? filePath.split('/').length + 1 : 1,
filter: (p) =>
p.startsWith(
fix: create-next-app copies files it shouldn't (#43131) <!-- Thanks for opening a PR! Your contribution is much appreciated. To make sure your PR is handled as smoothly as possible we request that you follow the checklist sections below. Choose the right checklist for the change that you're making: --> Closes #43130 This fixes the bug where the CLI copies files it shouldn't, by adding a `/` separator at the end of the directory name. Given `examples/next-prisma-starter` and `examples/next-prisma-starter-websockets`, only files inside the `examples/next-prisma-starter` will now be copied. Here is a repo created by the CLI after this fix, using ```bash # Inside packages/create-next-app pnpm build node dist/index.js --example https://github.com/trpc/trpc --example-path examples/next-prisma-starter ``` https://github.com/juliusmarminge/create-next-app-bugfix ## Bug - [x] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have a helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have a helpful link attached, see `contributing.md` ## Documentation / Examples - [x] Make sure the linting passes by running `pnpm build && pnpm lint` - [ ] The "examples guidelines" are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
2022-11-22 01:51:57 +01:00
`${name}-${branch.replace(/\//g, '-')}${
filePath ? `/${filePath}/` : '/'
}`
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
),
})
await fs.unlink(tempFile)
}
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
export async function downloadAndExtractExample(root: string, name: string) {
if (name === '__internal-testing-retry') {
throw new Error('This is an internal example for testing the CLI.')
}
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
const tempFile = await downloadTar(
'https://codeload.github.com/vercel/next.js/tar.gz/canary'
)
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
await tar.x({
file: tempFile,
cwd: root,
strip: 3,
filter: (p) => p.includes(`next.js-canary/examples/${name}/`),
fix(cli): do not throw error when extracting examples in Node 18+ (#40182) `create-next-app` currently cannot extract examples/git repos in Node 18+, because of an issue with `node-tar` (https://github.com/npm/node-tar/issues/321). The files are extracted, but an error is thrown that the stream has been prematurely closed. To prevent `create-next-app` from not being able to finish, ~for now, we can swallow this error, and hopefully `node-tar` will be patched soon.~ we can save the tar in the `tmp` folder and extract it from there as suggested: https://github.com/vercel/next.js/pull/40182#pullrequestreview-1095216714 I cannot reproduce this on earlier Node.js versions. More context: https://github.com/vercel/next.js/issues/39321#issuecomment-1235642058 Fixes #39321 ## Bug - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Errors have helpful link attached, see `contributing.md` ## Feature - [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. - [ ] Related issues linked using `fixes #number` - [ ] Integration tests added - [ ] Documentation added - [ ] Telemetry added. In case of a feature if it's used or not. - [ ] Errors have helpful link attached, see `contributing.md` ## Documentation / Examples - [ ] Make sure the linting passes by running `pnpm lint` - [ ] The examples guidelines are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing.md#adding-examples)
2022-09-05 18:11:12 +02:00
})
await fs.unlink(tempFile)
}