Rate-limiting of identifier event feeds
In some cases, an identifier may match an unusually large number of events coming into the Flare platform. When this happens, there’s a risk that too many noisy or irrelevant events could flood the identifier’s event feed, which may cause performance issues when loading the feed.To prevent this, we’ll apply a rate limit to the number of events added to that identifier’s feed.
If an identifier generates more than 1,000 events per hour in any data category, we’ll apply a rate limit. This means any events beyond the 1,000-per-hour limit will be dropped and won’t appear in the identifier’s event feed. The limit resets every hour.
This rate limit only applies to newly discovered events. When a new identifier is created, we’ll backfill its event feed using historical data — and that process is not rate-limited.
Each identifier is treated separately. So if you have duplicate identifiers or the same identifier used in different tenants, each one will be rate limited separately.
- You will see notification in the Flare application, highlighting which identifier is being rate-limited.
- You will receive an email from Flare support, informing you which identifier is being rate-limited.
If you have an identifier that is being rate-limited it is likely that the identifier needs tuning. You can reach out to your SE (if you are in trial/POC) or your CSM (if you are a customer) to set up a call to tune your identifiers.
However here are some questions to consider when trying to make your identifiers more specific:
- Is the term you are using very generic? eg., a keyword identifier for GitHub. This will get a lot of hits and likely most of those events are noise. Consider choosing a more specific term.
- Are you only interested in specific severities? eg., you don't care about low severity events for this identifier.
- Are you seeing a lot of noisy events coming from Look-alike domains?
- If you have a short domain then there is a risk that the number of permutations we send for look-alike detection is vast. This will result in a lot of irrelevant look-alike domain events.
- In this case consider removing the ‘low’ severity filter from their identifier, this will filter a lot of the look-alike event noise.