Async

​

Asynchronous Operations

Saldor provides asynchronous API endpoints for handling long-running crawl jobs. This allows you to initiate a crawl job and retrieve the results later, which is particularly useful for large websites or complex analyses.

​

Initiating an Async Crawl

To start an asynchronous crawl job, use the crawl_async method:

​

Checking Job Status

You can check the status of an async job using the get_job_status method:

​

Best Practices

1. Polling Interval: When checking job status, implement a reasonable polling interval (e.g., every 5-10 seconds).

2. Error Handling: Always implement proper error handling for network issues or unexpected API responses.

3. Timeout Handling: Consider implementing a timeout mechanism for jobs that take longer than expected to complete.