请求拦截
一旦启用了请求拦截,每个请求都会停滞,除非它被 continue、respond 或 abort 处理。
下面是一个简单的请求拦截器示例,它会中止所有图片请求:
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on('request', interceptedRequest => {
if (interceptedRequest.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort();
else interceptedRequest.continue();
});
await page.goto('https://example.com');
await browser.close();
多个拦截处理器与异步判定
默认情况下,如果在 request.abort、request.continue 或 request.respond 中任意一个已被调用之后再次调用它们,Puppeteer 将抛出 Request is already handled! 异常。
始终假定某个未知的处理器可能已经调用了 abort/continue/respond。即使你的处理器是你注册的唯一一个,第三方软件包也可能会注册它们自己的处理器。因此,在调用 abort/continue/respond 之前,务必使用 request.isInterceptResolutionHandled 检查判定状态。
重要的是,当你的处理器正在等待某个异步操作时,拦截的判定可能已被另一个监听器处理。因此,request.isInterceptResolutionHandled 的返回值只有在同步代码块中才是安全的。务必同步执行 request.isInterceptResolutionHandled 和 abort/continue/respond。
下面的示例演示两个同步处理器如何协同工作:
/*
This first handler will succeed in calling request.continue because the request interception has never been resolved.
*/
page.on('request', interceptedRequest => {
if (interceptedRequest.isInterceptResolutionHandled()) return;
interceptedRequest.continue();
});
/*
This second handler will return before calling request.abort because request.continue was already
called by the first handler.
*/
page.on('request', interceptedRequest => {
if (interceptedRequest.isInterceptResolutionHandled()) return;
interceptedRequest.abort();
});
下面的示例演示异步处理器如何协同工作:
/*
This first handler will succeed in calling request.continue because the request interception has never been resolved.
*/
page.on('request', interceptedRequest => {
// The interception has not been handled yet. Control will pass through this guard.
if (interceptedRequest.isInterceptResolutionHandled()) return;
// It is not strictly necessary to return a promise, but doing so will allow Puppeteer to await this handler.
return new Promise(resolve => {
// Continue after 500ms
setTimeout(() => {
// Inside, check synchronously to verify that the intercept wasn't handled already.
// It might have been handled during the 500ms while the other handler awaited an async op of its own.
if (interceptedRequest.isInterceptResolutionHandled()) {
resolve();
return;
}
interceptedRequest.continue();
resolve();
}, 500);
});
});
page.on('request', async interceptedRequest => {
// The interception has not been handled yet. Control will pass through this guard.
if (interceptedRequest.isInterceptResolutionHandled()) return;
await someLongAsyncOperation();
// The interception *MIGHT* have been handled by the first handler, we can't be sure.
// Therefore, we must check again before calling continue() or we risk Puppeteer raising an exception.
if (interceptedRequest.isInterceptResolutionHandled()) return;
interceptedRequest.continue();
});
如需更细粒度的内省(参见下文"协作式拦截模式"),你还可以在使用 abort/continue/respond 之前同步调用 request.interceptResolutionState。
以下是使用 request.interceptResolutionState 重写上述示例的代码:
/*
This first handler will succeed in calling request.continue because the request interception has never been resolved.
*/
page.on('request', interceptedRequest => {
// The interception has not been handled yet. Control will pass through this guard.
const {action} = interceptedRequest.interceptResolutionState();
if (action === InterceptResolutionAction.AlreadyHandled) return;
// It is not strictly necessary to return a promise, but doing so will allow Puppeteer to await this handler.
return new Promise(resolve => {
// Continue after 500ms
setTimeout(() => {
// Inside, check synchronously to verify that the intercept wasn't handled already.
// It might have been handled during the 500ms while the other handler awaited an async op of its own.
const {action} = interceptedRequest.interceptResolutionState();
if (action === InterceptResolutionAction.AlreadyHandled) {
resolve();
return;
}
interceptedRequest.continue();
resolve();
}, 500);
});
});
page.on('request', async interceptedRequest => {
// The interception has not been handled yet. Control will pass through this guard.
if (
interceptedRequest.interceptResolutionState().action ===
InterceptResolutionAction.AlreadyHandled
)
return;
await someLongAsyncOperation();
// The interception *MIGHT* have been handled by the first handler, we can't be sure.
// Therefore, we must check again before calling continue() or we risk Puppeteer raising an exception.
if (
interceptedRequest.interceptResolutionState().action ===
InterceptResolutionAction.AlreadyHandled
)
return;
interceptedRequest.continue();
});
协作式拦截模式
request.abort、request.continue 和 request.respond 可以接受一个可选的 priority,以便在协作式拦截模式下工作。当所有处理器都使用协作式拦截模式时,Puppeteer 保证所有拦截处理器都会按注册顺序运行并被等待。拦截将按优先级最高的判定结果处理。以下是协作式拦截模式的规则:
- 所有判定都必须向
abort/continue/respond提供一个数字形式的priority参数。 - 如果任一判定未提供数字形式的
priority,则遗留模式生效,协作式拦截模式不生效。 - 在拦截判定最终确定之前,异步处理器会先完成。
- 优先级最高的拦截判定"胜出",即拦截最终将按照被赋予最高优先级的判定来中止、响应或继续。
- 若出现平局,则
abort>respond>continue。
为了标准化,在指定协作式拦截模式的优先级时,请使用 0 或 DEFAULT_INTERCEPT_RESOLUTION_PRIORITY(从 HTTPRequest 导出),除非你有明确的理由使用更高的优先级。这样可以优雅地让 respond 优先于 continue、abort 优先于 respond,并允许其他处理器协作工作。如果你确实想使用不同的优先级,那么高优先级会胜过低优先级。允许使用负优先级。例如,continue({}, 4) 会胜过于 continue({}, -2)。
为了保持向后兼容,任何在不指定 priority(遗留模式)的情况下处理拦截的处理器都会导致立即判定。要使协作式拦截模式生效,所有判定都必须使用 priority。实际上,这意味着你仍然必须检查 request.isInterceptResolutionHandled,因为你无法控制的处理器可能已经在未指定优先级(遗留模式)的情况下调用了 abort/continue/respond。
在下面的示例中,由于至少有一个处理器在处理拦截时省略了 priority,遗留模式占据主导,请求被立即中止:
// Final outcome: immediate abort()
page.setRequestInterception(true);
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Legacy Mode: interception is aborted immediately.
request.abort('failed');
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Control will never reach this point because the request was already aborted in Legacy Mode
// Cooperative Intercept Mode: votes for continue at priority 0.
request.continue({}, 0);
});
在下面的示例中,由于至少有一个处理器未指定 priority,遗留模式占据主导,请求被继续:
// Final outcome: immediate continue()
page.setRequestInterception(true);
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to abort at priority 0.
request.abort('failed', 0);
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Control reaches this point because the request was cooperatively aborted which postpones resolution.
// { action: InterceptResolutionAction.Abort, priority: 0 }, because abort @ 0 is the current winning resolution
console.log(request.interceptResolutionState());
// Legacy Mode: intercept continues immediately.
request.continue({});
});
page.on('request', request => {
// { action: InterceptResolutionAction.AlreadyHandled }, because continue in Legacy Mode was called
console.log(request.interceptResolutionState());
});
在下面的示例中,由于所有处理器都指定了 priority,协作式拦截模式生效。continue() 胜出,因为它的优先级高于 abort()。
// Final outcome: cooperative continue() @ 5
page.setRequestInterception(true);
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to abort at priority 10
request.abort('failed', 0);
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to continue at priority 5
request.continue(request.continueRequestOverrides(), 5);
});
page.on('request', request => {
// { action: InterceptResolutionAction.Continue, priority: 5 }, because continue @ 5 > abort @ 0
console.log(request.interceptResolutionState());
});
在下面的示例中,由于所有处理器都指定了 priority,协作式拦截模式生效。respond() 胜出,因为它的优先级与 continue() 相同,但 respond() 优于 continue()。
// Final outcome: cooperative respond() @ 15
page.setRequestInterception(true);
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to abort at priority 10
request.abort('failed', 10);
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to continue at priority 15
request.continue(request.continueRequestOverrides(), 15);
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to respond at priority 15
request.respond(request.responseForRequest(), 15);
});
page.on('request', request => {
if (request.isInterceptResolutionHandled()) return;
// Cooperative Intercept Mode: votes to respond at priority 12
request.respond(request.responseForRequest(), 12);
});
page.on('request', request => {
// { action: InterceptResolutionAction.Respond, priority: 15 }, because respond @ 15 > continue @ 15 > respond @ 12 > abort @ 10
console.log(request.interceptResolutionState());
});
协作式请求继续
Puppeteer 要求显式调用 request.continue(),否则请求将挂起。即使你的处理器本意是不采取任何特殊操作,即"选择退出",也必须调用 request.continue()。
随着协作式拦截模式的引入,协作式请求继续出现了两种使用场景:无主见的继续和有主见的继续。
第一种情况(常见)是,你的处理器打算不对请求做任何特殊处理。它对后续操作没有主见,只打算默认继续,和/或将决定权交给可能另有主见的其他处理器。但如果不存在其他处理器,我们必须调用 request.continue(),以确保请求不会挂起。
我们称之为无主见的继续,因为其意图是:如果没有其他人有更好的想法,就继续该请求。对于此类继续,请使用 request.continue({...}, DEFAULT_INTERCEPT_RESOLUTION_PRIORITY)(或 0)。
第二种情况(少见)是,你的处理器确实有自己的主见,并打算通过覆盖其他地方发出的较低优先级的 abort() 或 respond() 来强制继续。我们称之为有主见的继续。在这些罕见情况下,如果你打算指定一个覆盖性的继续优先级,请使用自定义优先级。
总而言之,请仔细考虑你对 request.continue 的使用是仅作为默认/放行行为,还是属于你处理器的预期使用场景。对于范围内的使用场景,考虑使用自定义优先级;否则使用默认优先级。请注意,你的处理器可能同时存在有主见和无主见的情况。
软件包维护者升级到协作式拦截模式
如果你是软件包维护者,且你的软件包使用了拦截处理器,你可以更新这些拦截处理器以使用协作式拦截模式。假设你已有如下处理器:
page.on('request', interceptedRequest => {
if (request.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort();
else interceptedRequest.continue();
});
要使用协作式拦截模式,请升级 continue() 和 abort():
page.on('request', interceptedRequest => {
if (request.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort('failed', 0);
else
interceptedRequest.continue(
interceptedRequest.continueRequestOverrides(),
0,
);
});
经过这些简单的升级,你的处理器现在改为使用协作式拦截模式。
不过,我们推荐一个稍微更稳健的方案,因为上述做法会引入几个细微的问题:
- 向后兼容性。 如果任何处理器仍使用遗留模式的判定(即未指定优先级),那么即使你的处理器先运行,该处理器也会立即处理拦截。这可能会给你的用户带来令人不安的行为:用户只是升级了你的软件包,却突然发现你的处理器不再处理拦截,而是另一个处理器占据了优先权。
- 硬编码优先级。 你的软件包用户无法为你的处理器指定默认判定优先级。当用户希望根据使用场景调整优先级时,这一点就变得很重要。例如,某个用户可能希望你的软件包具有高优先级,而另一个用户可能希望它具有低优先级。
为解决这两个问题,我们推荐的做法是从你的软件包中导出一个 setInterceptResolutionConfig()。然后用户可以调用 setInterceptResolutionConfig() 来显式激活你软件包中的协作式拦截模式,从而避免因拦截判定方式发生变化而感到意外。他们还可以选择使用 setInterceptResolutionConfig(priority) 指定适合其使用场景的自定义优先级:
// Defaults to undefined which preserves Legacy Mode behavior
let _priority = undefined;
// Export a module configuration function
export const setInterceptResolutionConfig = (priority = 0) =>
(_priority = priority);
/**
* Note that this handler uses `DEFAULT_INTERCEPT_RESOLUTION_PRIORITY` to "pass" on this request. It is important to use
* the default priority when your handler has no opinion on the request and the intent is to continue() by default.
*/
page.on('request', interceptedRequest => {
if (request.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort('failed', _priority);
else
interceptedRequest.continue(
interceptedRequest.continueRequestOverrides(),
DEFAULT_INTERCEPT_RESOLUTION_PRIORITY, // Unopinionated continuation
);
});
如果你的软件包需要对判定优先级进行更细粒度的控制,请使用如下配置模式:
interface InterceptResolutionConfig {
abortPriority?: number;
continuePriority?: number;
}
// This approach supports multiple priorities based on situational
// differences. You could, for example, create a config that
// allowed separate priorities for PNG vs JPG.
const DEFAULT_CONFIG: InterceptResolutionConfig = {
abortPriority: undefined, // Default to Legacy Mode
continuePriority: undefined, // Default to Legacy Mode
};
// Defaults to undefined which preserves Legacy Mode behavior
let _config: Partial<InterceptResolutionConfig> = {};
export const setInterceptResolutionConfig = (
config: InterceptResolutionConfig,
) => (_config = {...DEFAULT_CONFIG, ...config});
page.on('request', interceptedRequest => {
if (request.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
) {
interceptedRequest.abort('failed', _config.abortPriority);
} else {
// Here we use a custom-configured priority to allow for Opinionated
// continuation.
// We would only want to allow this if we had a very clear reason why
// some use cases required Opinionated continuation.
interceptedRequest.continue(
interceptedRequest.continueRequestOverrides(),
_config.continuePriority, // Why would we ever want priority!==0 here?
);
}
});
上述方案在确保向后兼容性的同时,还允许用户在启用协作式拦截模式时调整你的软件包在判定链中的重要性。在用户将其代码及所有第三方软件包完全升级为使用协作式拦截模式之前,你的软件包仍会按预期工作。如果任何处理器或软件包仍在使用遗留模式,你的软件包也可以继续以遗留模式运行。