crawler

Intelligent web scraping with a tiered approach: use requests for static pages, Playwright for dynamic pages, and Bright Data API as a paid last-resort anti-bot bypass. Use when the user needs to: (1) scrape/crawl web pages, (2) extract content from URLs, (3) download articles or web content, (4) convert web pages to PDF, (5) handle blocked or protected websites. Triggers: "crawl", "scrape", "fetch URL", "get content from".