DataHub API

Ingest data from various sources for contextualized analysis of your cloud.

The DataHub API allows you to import data from internal and external sources into the DoiT Console. You can leverage this functionality to monitor your internal resource usage, optimize resource allocation and manage operational expenses.

πŸ‘

API Reference

The DataHub API reference is available at <https://developer.doit.com/reference/datahubevents>.

Rate limits

To ensure the stability and performance of the system, the DataHub API imposes the following limits:

  • Up to 255 metrics per request.
  • Up to 1,000 requests per minute.

Data latency

Once an event is published, the ingested data will be available in Cloud Analytics within 15 minutes.

Events schema

PropertyTypeDescription
cloud (required)stringThis field is used to identify the data provider. Choose a human-readable value so you can easily recognize the dataset. For example, Datadog, Databricks, Snowflake, etc. The default value is ingestion-api.
idstringA unique identifier for the event. If not set, a UUIDv4 is generated at ingestion time.
DataHub uses the event ID to handle duplicate data (newer data overwrites existing ones). You also need the event ID to delete an incorrect ingestion (an event can be deleted 90 minutes after ingestion).
dimensions.typestringPossible values: fixed, label, project_label
dimensions.keystringIf the type is fixed, the key must be one of the allowed keys for fixed dimensions.
dimensions.valuestringThe value of the dimension.
time (required)string (RFC3339)The time that the event occurred.
metrics.typestringFor integration with billing data. Possible values: cost, usage, or custom metric.
metrics.valuenumberValue for the metric.

Allowed keys for fixed dimensions

Event dimension keyReport field
billing_account_idBilling Account ID
countryCountry
regionRegion
zoneZone
pricing_unitUnit
project_idProject/Account ID
project_nameProject/Account Name
project_numberProject/Account Number
service_descriptionService
service_idService ID
sku_descriptionSKU
sku_idSKU ID
operationOperation
resource_idResource
global_resource_idGlobal Resource

πŸ‘

Note

See Standard dimensions for details about the report fields.

Publish event

Sample request

πŸ“˜

your-api-key

Replace "your-api-key" with your actual API key as explained at Getting Started

curl --request POST \
--url https://api.doit.com/datahub/v1/events \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <your-api-key>' \
--data '{
"events": [
    {
      "cloud": "Datadog",
      "id": "12345678-1234-1234-1234-123412341234",
      "dimensions": [
        {
          "key": "service_description",
          "value": "Devices",
          "type": "fixed"
        },
        {
          "key": "region",
          "value": "us-east-1b",
          "type": "fixed"
        },
        {
          "key": "zone",
          "value": "us-east-1b",
          "type": "fixed",
        },
        {
         "key": "env",
         "value": "prod",
         "type": "label",
        },
        {
         "key": "team",
         "value": "alpha",
         "type": "label",
        },
        {
         "key": "department",
         "value": "rnd",
         "type": "project_label",
        }
       ],
      "time": "2024-02-23T12:00:00.00Z",
      "metrics": [
        {
          "value": 352,
          "type": "cost",
         },
         {
          "value": 36,
          "type": "usage",
         }
        ]
      }
  ]
}
'