* Add crawler blocking for fallback: true * update test * Update check * Add note to docs * use browser agent for non-crawler test