Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
import { IncrementalCache } from './incremental-cache'
|
|
|
|
|
|
|
|
interface CachedRedirectValue {
|
|
|
|
kind: 'REDIRECT'
|
|
|
|
props: Object
|
|
|
|
}
|
|
|
|
|
|
|
|
interface CachedPageValue {
|
|
|
|
kind: 'PAGE'
|
|
|
|
html: string
|
|
|
|
pageData: Object
|
|
|
|
}
|
|
|
|
|
|
|
|
export type ResponseCacheValue = CachedRedirectValue | CachedPageValue
|
|
|
|
|
|
|
|
export type ResponseCacheEntry = {
|
|
|
|
revalidate?: number | false
|
|
|
|
value: ResponseCacheValue | null
|
|
|
|
}
|
|
|
|
|
2021-07-22 23:04:58 +02:00
|
|
|
type ResponseGenerator = (
|
|
|
|
hasResolved: boolean
|
|
|
|
) => Promise<ResponseCacheEntry | null>
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
|
|
|
|
export default class ResponseCache {
|
|
|
|
incrementalCache: IncrementalCache
|
2021-07-22 23:04:58 +02:00
|
|
|
pendingResponses: Map<string, Promise<ResponseCacheEntry | null>>
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
|
|
|
|
constructor(incrementalCache: IncrementalCache) {
|
|
|
|
this.incrementalCache = incrementalCache
|
|
|
|
this.pendingResponses = new Map()
|
|
|
|
}
|
|
|
|
|
|
|
|
public get(
|
|
|
|
key: string | null,
|
|
|
|
responseGenerator: ResponseGenerator
|
2021-07-22 23:04:58 +02:00
|
|
|
): Promise<ResponseCacheEntry | null> {
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
const pendingResponse = key ? this.pendingResponses.get(key) : null
|
|
|
|
if (pendingResponse) {
|
|
|
|
return pendingResponse
|
|
|
|
}
|
|
|
|
|
2021-07-22 23:04:58 +02:00
|
|
|
let resolver: (cacheEntry: ResponseCacheEntry | null) => void = () => {}
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
let rejecter: (error: Error) => void = () => {}
|
2021-07-22 23:04:58 +02:00
|
|
|
const promise: Promise<ResponseCacheEntry | null> = new Promise(
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
(resolve, reject) => {
|
|
|
|
resolver = resolve
|
|
|
|
rejecter = reject
|
|
|
|
}
|
|
|
|
)
|
|
|
|
if (key) {
|
|
|
|
this.pendingResponses.set(key, promise)
|
|
|
|
}
|
|
|
|
|
|
|
|
let resolved = false
|
2021-07-22 23:04:58 +02:00
|
|
|
const resolve = (cacheEntry: ResponseCacheEntry | null) => {
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
if (key) {
|
|
|
|
// Ensure all reads from the cache get the latest value.
|
|
|
|
this.pendingResponses.set(key, Promise.resolve(cacheEntry))
|
|
|
|
}
|
|
|
|
if (!resolved) {
|
|
|
|
resolved = true
|
|
|
|
resolver(cacheEntry)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// We wait to do any async work until after we've added our promise to
|
|
|
|
// `pendingResponses` to ensure that any any other calls will reuse the
|
|
|
|
// same promise until we've fully finished our work.
|
|
|
|
;(async () => {
|
|
|
|
try {
|
|
|
|
const cachedResponse = key ? await this.incrementalCache.get(key) : null
|
|
|
|
if (cachedResponse) {
|
|
|
|
resolve({
|
|
|
|
revalidate: cachedResponse.curRevalidate,
|
|
|
|
value: cachedResponse.value,
|
|
|
|
})
|
|
|
|
if (!cachedResponse.isStale) {
|
|
|
|
// The cached value is still valid, so we don't need
|
|
|
|
// to update it yet.
|
|
|
|
return
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
const cacheEntry = await responseGenerator(resolved)
|
|
|
|
resolve(cacheEntry)
|
|
|
|
|
2021-07-22 23:04:58 +02:00
|
|
|
if (key && cacheEntry && typeof cacheEntry.revalidate !== 'undefined') {
|
Replace `withCoalescedInvoke` with `ResponseCache` (#26997)
By itself, `withCoalescedInvoke` with a separate `this.incrementalCache.set(...)` isn't really suitable for streaming responses. Since streaming is asynchronous, updating the cache separately introduces a gap where another origin request for the same resource could be made.
This could potentially be addressed by moving the cache update, but then `IncrementalCache` itself would need to be made to support streaming, in addition to the many other responsibilities it has. In this case, it seemed best to just use composition to add another caching layer in front of it, which is a familiar and understandable concept. Eventually, we might want to move this cache to the HTTP layer, which will also be simpler with this change.
As an added bonus, `renderToResponseWithComponents` becomes significantly simpler, and we delete some duplication.
2021-07-12 21:47:39 +02:00
|
|
|
await this.incrementalCache.set(
|
|
|
|
key,
|
|
|
|
cacheEntry.value,
|
|
|
|
cacheEntry.revalidate
|
|
|
|
)
|
|
|
|
}
|
|
|
|
} catch (err) {
|
|
|
|
rejecter(err)
|
|
|
|
} finally {
|
|
|
|
if (key) {
|
|
|
|
this.pendingResponses.delete(key)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
})()
|
|
|
|
return promise
|
|
|
|
}
|
|
|
|
}
|