A rust ported robotstxt

[Folyd]

I have ported a Rust version of this library recently. The Rust version keeps the same behavior of the original library, provides a consistent API and passed 100% C++ testing cases via Rust FFI.

Also, Rust version licensed with Apache 2.0 too, and I have preserved existing copyright notices and licensing within the appropriate source files.

Here is my repository: https://github.com/Folyd/robotstxt

[nzrsky]

Btw, I made a performance comparison between this port and the C++ version (including my own C++ fork) https://github.com/nzrsky/robotstxt/blob/master/COMPARISON.md

[garyillyes]

@Folyd We'd like to highlight community ports. Could you run this through https://github.com/google/robotstxt-spec-test/ and report back?