Add Crawler Rate Limit #191

Open

opened

It is beneficial to sometimes have rate limits that a Crawler should run with so it doesn't overwhelm targets and/or complies with robots limits. (from discord user jeremygiberson)

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests