AI Bot Directory / Omgili

What is Omgili?

Omgili is Webz.io's bulk crawler that collects web content for training AI models. It doesn't drive traffic back to your site directly, but the content it collects shapes what future models know — and say — about you.

How to spot Omgili in your logs

Look for this token in the request User-Agent (case-insensitive):

omgili

Should you allow or block Omgili?

Blocking it keeps your content out of future training data, at the cost of future models knowing less about you. Most brands chasing AI visibility leave it allowed.

To block it anyway, add this to your robots.txt:

User-agent: Omgili
Disallow: /

Note: robots.txt is a request, not a lock — well-behaved crawlers honor it; verify with your own traffic data.