DataHub API
Ingest data from various sources for contextualized analysis of your spend.
The DataHub API allows you to import data from internal and external sources into the DoiT Console. You can leverage this functionality to monitor your internal resource usage, optimize resource allocation and manage operational expenses.
API Reference
The DataHub API reference is available at <https://developer.doit.com/reference/datahubevents>.
Rate limits
To ensure the stability and performance of the system, the DataHub API imposes the following limits:
- Up to 255 events per request.
- Up to 1,000 requests per minute.
Data latency
Once an event is published, the ingested data will be available in Cloud Analytics within 15 minutes.
Events schema
| Property | Type | Description |
|---|---|---|
| provider (required) | string | This field is used to identify the data provider. Choose a human-readable value so you can easily recognize the dataset. For example, Datadog, Databricks, Snowflake, etc. The default value is ingestion-api.Allowed characters: alphanumeric (0-9,a-z,A-Z), underscore (_), space, dash (-) |
| id | string | A unique identifier for the event. If not set, a UUIDv4 is generated at ingestion time. DataHub uses the event ID to handle duplicate data (newer data overwrites existing ones). You also need the event ID to delete an incorrect ingestion (an event can be deleted 90 minutes after ingestion). |
| dimensions.type | string | Possible values: fixed, label, project_label |
| dimensions.key | string | If the type is fixed, the key must be one of the allowed keys for fixed dimensions. |
| dimensions.value | string | The value of the dimension. |
| time (required) | string (RFC3339) | The time that the event occurred. |
| metrics.type | string | For integration with billing data. Possible values: cost, usage, or custom metric. |
| metrics.value | number | Value for the metric. |
Allowed keys for fixed dimensions
fixed dimensionsEvent dimension key | Report field |
|---|---|
| billing_account_id | Billing Account ID |
| country | Country |
| is_marketplace | Marketplace |
| operation | Operation |
| pricing_unit | Unit |
| project_id | Project/Account ID |
| project_name | Project/Account Name |
| project_number | Project/Account Number |
| region | Region |
| resource_id | Resource |
| resource_global_id | Global Resource |
| service_description | Service |
| service_id | Service ID |
| sku_description | SKU |
| sku_id | SKU ID |
| zone | Zone |
Note
See Standard dimensions for details about the report fields.
Publish event
Sample request
your-api-key
Replace "your-api-key" with your actual API key as explained at Getting Started
curl --request POST \
--url https://api.doit.com/datahub/v1/events \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <your-api-key>' \
--data '{
"events": [
{
"provider": "Datadog",
"id": "12345678-1234-1234-1234-123412341234",
"dimensions": [
{
"key": "service_description",
"value": "Devices",
"type": "fixed"
},
{
"key": "region",
"value": "us-east-1b",
"type": "fixed"
},
{
"key": "zone",
"value": "us-east-1b",
"type": "fixed",
},
{
"key": "env",
"value": "prod",
"type": "label",
},
{
"key": "team",
"value": "alpha",
"type": "label",
},
{
"key": "department",
"value": "rnd",
"type": "project_label",
}
],
"time": "2024-02-23T12:00:00.00Z",
"metrics": [
{
"value": 352,
"type": "cost",
},
{
"value": 36,
"type": "usage",
}
]
}
]
}
'
Updated 3 days ago