Data Fields

31 min

The document below details the various fields that are extracted from the data that Flare collects. These fields can be used with the Lucene Query Syntax.

Example Queries

Description	Query
Activities that have the domain name acme.com in their title	title:"acme.com"
Listings that have a price between $100 and $200	price:>100 AND price:<200
Retrieve an actor and their publications by their username	author_name:lapifia
Specifying multiple keywords	your_organisation AND fraud
Using time field	metadata.estimated_created_at: >= now-1d
Actors from specific threads	_index:forum_topic AND title:actor_name AND keyword

Levels of support

Some documents have three levels of support. Each of them is documented in the following table.

Level	Description
1	Base attributes of the entity. As they are part of the document’s identity, fields on this level have the highest chance to be present.
2	Addition to the level 1 attributes. Fields on this level add more value from an analysis point of view. As they are not part of the document’s minimal identity, they can be omitted in some cases
3	Complementary information. Fields on this level may be unavailable for some sources or unnecessary for basic analysis. Their presence depends on the source.

Common fields

Metadata

Each document contains a metadata field with subfields describing the origin of the data and some other attributes to describe the source of the document.

Name	Description
_id	Unique ID of the document. Usually composed of the source and an internal ID at the source.
metadata.source	Name of the source or website from which the raw data was downloaded from. Since a site can have multiple domains that change over time, using names such as dream or wall_st is more reliable.
metadata.first_crawled_at	Date the document was first found. Useful for having an approximate date of the creation of a document.
metadata.last_crawled_at	Date the document was last seen. Useful for having an approximate date of deleting a document on a site.
metadata.estimated_created_at	Some documents may have their exact date of creation. This field estimates the creation date using the date provided by the source if possible or metadata.first_crawled_at. This field therefore provides a better accuracy for the creation date than metadata.first_crawled_at.
author_name	Name of the author associated with the document in question. This may be the seller name for a listing, the repository host on GitHub, etc.

Features

Documents also contain a features field with subfields describing extracted keywords that might identify digital identities.

Name	Description
features.urls	List of URLs extracted from the document. This can be URLs from any scheme, including HTTP, FTP, SSH, etc. The URL needs to have a scheme or path to appear in this field; domains without a scheme or a path will only appear in features.domains.
features.emails	List of email addresses extracted from the document.
features.emails.domains	List the domain of an email address extracted from the document.
features.emails.local_parts	List the local part of an email address extracted from the document (what is before the @ character).
features.domains	List of domains extracted from the document. The list contains every valid variant of a domain, such that a document with the www.flared.io domain to have a list with both www.flared.io and flared.io.
features.reversed_domains	List of domains extracted from the document and the components reversed. A document with www.flared.io will have a list with both io.flared.www and io.flared. This field is useful to index and quickly find documents matching a domain’s subdomains with suffix wildcard query: features.reversed_domains:io.flared.*
features.ip_addresses_cidr	List of IP addresses extracted from the document. Support both IPv4 and IPv6. The field type supports queries with CIDRs such as features.ip_addresses_cidr:127.0.0.0/8.
features.btc_addresses	List of Bitcoin public addresses extracted from the document.
features.cc_numbers	List of potential Credit Card numbers extracted from the document. While the numbers have not been validated, each number matches a known BIN and has a valid Luhn identification number.

Documents

Seller

Seller fields level 1

Name	Description
about	Descriptive or biographical text of the seller.
created_at	Creation date of the seller’s account.
username	Pseudonym of the seller.

Seller fields level 2

Name	Description
feedbacks	List of comments or ratings posted on the seller’s page.
contact_info	Seller’s information.
rating	Rating (out of 5 stars usually) given to the seller.
rating_count	Number of ratings given to the seller.
rating_score	Ratings given to the seller. Usually the sum of positive and negative feedback.
ship_from	Regions from which the seller delivers the goods. Use ship_from_norm for normalized region across data sources.
ship_to	Regions where the seller delivers the goods. Use ship_to_norm for normalized region across data sources.
public_pgp_fingerprint	PGP key impression list associated with the seller.
public_pgp_uid	List of PGP key IDs associated with the seller.
transactions_count	Number of sales made by the seller.

Seller fields level 3

Name	Description
last_7_days_rating	Rating average for the last 7 days given to the seller.
last_7_days_rating_count	Number of ratings given to the seller in the last 7 days.
last_month_rating	Rating average for the last month given to the seller.
last_month_rating_count	Number of ratings given to the seller during the last month.
transactions_amount	Total amount of sales made by the seller in USD.
finalize_early_enabled	Boolean indicating whether the seller requires payments at the time of purchase or via an escrow.
title	Title of the seller on the market. Generally an internal title earned by the experience or reputation of a seller.
last_active_at	Last activity date of the seller.

Listing

Listing fields level 1

Name	Description
title	Ad Title.
description	Description of the ad.
seller_id	Internal ID of the seller.
seller_name	Descriptive name of the seller.
creation_date	Creation date of the ad.
price	Price of the ad in USD.
currency	Price’s currency.

Listing fields level 2

Name	Description
category_id	Internal ID of the category of the ad. Use category for normalized ID across all data sources.
category_name	Descriptive name of the category of the ad. Use categoy_name_norm for normalized name across all data sources.
ship_to	List of regions where the goods can be delivered. Use ship_to_norm for normalized regions across data sources.
ship_from	Regions from where the goods are shipped. Use ship_from_norm for normalized regions across data sources.
rating	Rating given to the merchandise or the seller.
rating_count	Number of ratings given to the merchandise or the seller.
feedbacks	List of comments or ratings posted on the ad. See below for the fields at Feedback.
sold_count	Number of sales of the ad.

Listing fields level 3

Name	Description
escrow	Boolean indicating whether the transaction is via an escrow managed by the market platform.
last_7_days_rating	Rating for the last 7 days given to the merchandise or the seller.
last_7_days_rating_count	Number of ratings given to the merchandise or the seller in the last 7 days.
last_month_rating	Rating for the last month given to the merchandise or the seller.
last_month_rating_count	Number of ratings given to the merchandise or the seller in the last month.
Shipping_options	List of shipping options available for the listing.
stock_count	Remaining quantity of the goods.
view_count	Number of times the listing has been seen.

Forum profile

Forum fields level 1

Name	Description
username	Username of the user.
personal_text	Descriptive or biographical text.
registered_at	Approximate date of account registration.

Forum fields level 2

Name	Description
contact_info	Profile’s contact info.
last_posted_at	Approximate date of the last message.
last_active_at	Approximate date of the last connection.
public_pgp_fingerprint	Fingerprint list of the PGP keys associated with the account.
public_pgp_uid	List of PGP key IDs associated with the account. Usually contains a list of email addresses associated with the key.
rating	User rating.
rating_pos	Number of positive ratings given to the user.
rating_neg	Number of negative ratings given to the user.
signature	Signature attached to each user’s message.

Forum fields level 3

Name	Description
age	Age of the user.
avatar	URL to the user’s avatar.
comments_count	Number of comments from the user.
following_category_ids	Categories followed by the user.
following_user_ids	Other users followed by the user.
location	Location of the user. Unlikely to be accurate. Use location_norm for normalized value across data sources.
posts	List of posts left on the user profile by other users. See below for the fields at Forum Post.
posts_count	Number of messages from the user.
realname	Real name of the user. Unlikely to be an actual real name.
timezone_offset	Time zone used by the user.
title	Title of the user (member, moderator, administrator, etc.).
tags	Tags associated with the user.
website	Website URL of the user.

Forum topic

Forum_topic fields level 1

Name	Description
author_id	Internal ID of the original author of the thread.
author_name	Nickname of the original author of the thread.
posted_at	Date of creation of the thread.
title	Title of the thread.

Forum_topic fields level 2

Name	Description
category_id	Internal ID of the thread’s category. Use category_id_norm for normalized value across data sources.
category_name	Name of the thread’s category. Use category_name_norm for normalized value across data sources.
category_path	Internal ID of the thread’s category and parent's categories.

Forum_topic fields level 3

Name	Description
first_post_preview	Preview of the first post content.
last_reply_at	Date of the last reply in the thread.
profile_id	Related profile if the thread is on a profile page.
tags	Tags associated with the thread.

Forum post

Forum_post fields level 1

Name	Description
author_id	Internal ID of the author of the message.
author_name	Nickname of the author of the message.
content	Content of the message.
posted_at	Date of creation of the message.
parent_post_id	Parent post to which this post reply.
topic_id	Internal ID of the thread containing the message.
topic_title	Title of the thread containing the message.

Forum post fields level 2

Name	Description
There are no support level 2 fields for this type of document.

Forum_post fields level 3

Name	Description
post_title	Title of the post.
rating_pos	Number of positive ratings given to the user.

Driller

Driller fields

Name	Description
html_url	URL of the original document.
cache_url	Cache URL from Google.
is_dork	Whether the document came from a dork query.
title	Title of the document.
author_id	Internal ID of the author of the message.
author_name	Nickname of the author of the message.
author_email	Email of the author of the message.
created_at	Creation date of the document.
project_name	Project’s name.
content	Content of the document.
size	Size in bytes.
snippets	Snippets of the document.
sha	SHA of the document.
is_truncated	Whether the document is truncated.
host	Host URL of the document (ex: Github.com).
fileype	Type of the file.
filename	Name of the file.
dirpath	Dirpath of the document.
user	Document’s user.
issue	Document’s issue.
project	The project that the document comes from.
commit	Document commit.
code	Extracted code from the document.

Paste

Paste documents are mostly raw data coming from paste sites such as Pastebin.

Paste fields

Name	Description
author_id	Internal ID of the author.
author_name	Username of the author.
content	Raw content of the paste document. Can be truncated if actual content is too large.
expire_at	Expiration date of the paste document. Documents remain in Flare's database even after the expiration date.
is_truncated	Large documents are truncated in Flare's database. If True, content is a truncated version of the actual document.
posted_at	Date the paste document was created.
size	Actual size of the paste document (bytes).
syntax	Document’s syntax defined by the author.
title	Title given to the paste document.
title_en	English title given to the paste document.

Domain

Domain fields

Name	Description
Name	Domain’s name.
registered_at	Certificate registration date.
feed	Certificate Transparency feed name.
identifier_domain	Domain Identifier.
cert_data	Contains details about the issued certificates and the issuers.
subject	Domain SSL certificate’s subject.
issuer	Certificate issuer.

Host

Host fields

Name	Description
vulnerabilities	The name of a vulnerability present on the host. (e.g. CVE-2018-15919)
country_code	The country code of where the IP is located. e.g. country_code:US
service	Service running on an IP (e.g. http, ssh, ftp, mongodb, elastic, etc).
tags	Other tag that give information about what this IP is doing. Possible values: cloud,cdn,starttls,self-signed,database,vpn,honeypot,iot,devops,videogame,ics,compromised,cryptocurrency,medical,tor, proxy

Leak

Leak fields

Name	Description
links	[Optional]
name	The email address associated with the leak.
passwords	List of passwords leaked associated with the email.
For each entry in password list
domain	[Optional] Domain name of breached company.
extra	[Optional] Details on the intrusion method if available.
hash	Clear password or hash.
hash_type	[Optional] Hash function.
id	Unique series of numbers for the leaked credential.
imported_at	Timestamp at which the password was added to our database. This does not correspond to the date at which the password was leaked.
breached_at (under source)	[Optional] Date of the breach (not always available).
description (under source)	Description of the breach in English.
description_fr (under source)	Description of the breach in French.
hash_description (under source)	[Optional] Description of the hash function of the source.
hash_type (under source)	[Optional] Hash function of the source if the leaked passwords are the same type. If not, the hash type will be “none”.
id (under source)	Internal source name and year of breach.
leaked_at (under source)	Date when the data became publicly available.
name (under source)	[Optional] Name of the company that was breached.
related_urls (under source)	[Optional] News articles related to the breach, if available.
url (under source)	[Optional] Breached website URL (original source not always available).
source_id	Internal source name.
source_params - line	Line number where the leaked password was found.

Inner objects

Feedbacks

Feedback are usually posted by buyers of an ad and is an essential part of the black market reputation system.

Feedbacks fields

Name	Description
feedbacks.rated_at	Date of the rating or comment.
feedbacks.rating	Rating given by the buyer.
feedbacks.comment	Comment posted by the buyer.
feedbacks.username	Full or partial pseudonym of the buyer.
feedbacks.purchase_amount	Price of the purchase associated with the comment.
feedbacks.purchase_currency	Currency used by purchase_amount.
feedbacks.author_id	Internal ID of the author.

User

User fields

Name	Description
email	Email address
company	Company name
full_name	Full name
location	Location
followers_count	Number of followers

Contact_info

Contact_info fields

Name	Description
contact_info.fdw_forum	FDW Forum internal id.
contact_info.jabber	Jabber address.
contact_info.email	Email address.
contact_info.skype	Skype username.
contact_info.icq	ICQ.
contact_info.irc	IRC nickname.
contact_info.ricochet	Ricochet.
contact_info.bitmessage	Bitmessage.
contact_info.btc	Bitcoin wallet.
contact_info.telegram	Telegram.
contact_info.discord	Discord.

Shipping_options

Shipping_options options fields

Name	Description
shipping_options.description	Description of the option.
shipping_options.price	Price of the option.
shipping_options.currency	Currency of the price of the option.

Project

Project fields

Name	Description
owner_name	Name of the author.
owner_id	Internal ID of the author.
owner_type	Author’s association with the document.
tags	Associated tags.
last_activity_at	Approximate date of the last activity.
language	Author’s language.
followers_count	Number of followers.
forks_count	Number of forks there are of this repository in the whole network.
is_fork	Identifies if the repository is a fork.

Commit

Commit fields

Name	Description
committer_name	Name of the committer.
committer_id	Internal ID of the committer.
committer_email	Email of the committer.
author_email	Email of the author.
sha	SHA.

Data access and privacy safeguards

Some sensitive data elements are subject to privacy safeguards that support compliance with GDPR and other privacy regulations. As a result, certain data fields may appear masked or obfuscated, or may not be visible in your environment. This is expected behavior and is intended to protect the privacy and security of the individuals and organizations involved. These safeguards apply only to stealer logs, leaked credentials, and PII, which are the most sensitive data elements. All other data sources, including dark web, ransomware, and Telegram, remain fully accessible. If broader access is needed for your use case, completing domain authorizationdomain authorization typically provides largely unrestricted access to events involving your domain. Your Flare customer success manager can help review your current configuration and answer any questions.

Product Specifications

Fraudster Jargon

Asset Relations