curl --request POST \
--url https://api.promptlayer.com/reports \
--header 'Content-Type: application/json' \
--header 'X-API-KEY: <api-key>' \
--data '
{
"dataset_group_id": 123,
"name": "Pipeline with Built-in Scoring",
"columns": [
{
"column_type": "LLM_ASSERTION",
"name": "Accuracy Check",
"configuration": {
"source": "response",
"prompt": "Is this response accurate?"
},
"is_part_of_score": true
},
{
"column_type": "LLM_ASSERTION",
"name": "Safety Check",
"configuration": {
"source": "response",
"prompt": "Is this response safe?"
},
"is_part_of_score": true
}
]
}
'{
"success": true,
"report_id": 456,
"report_columns": [
{
"id": 789,
"name": "Accuracy Check",
"column_type": "LLM_ASSERTION",
"position": 1,
"configuration": {
"source": "response"
}
}
],
"external_ids": []
}{
"message": "Dataset must have at least one non-draft version"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"message": "<string>",
"success": false
}{
"detail": [
{
"loc": [
"<string>"
],
"msg": "<string>",
"type": "<string>"
}
]
}Evaluations & Reports API
Create Evaluation Pipeline
POST
/
reports
curl --request POST \
--url https://api.promptlayer.com/reports \
--header 'Content-Type: application/json' \
--header 'X-API-KEY: <api-key>' \
--data '
{
"dataset_group_id": 123,
"name": "Pipeline with Built-in Scoring",
"columns": [
{
"column_type": "LLM_ASSERTION",
"name": "Accuracy Check",
"configuration": {
"source": "response",
"prompt": "Is this response accurate?"
},
"is_part_of_score": true
},
{
"column_type": "LLM_ASSERTION",
"name": "Safety Check",
"configuration": {
"source": "response",
"prompt": "Is this response safe?"
},
"is_part_of_score": true
}
]
}
'{
"success": true,
"report_id": 456,
"report_columns": [
{
"id": 789,
"name": "Accuracy Check",
"column_type": "LLM_ASSERTION",
"position": 1,
"configuration": {
"source": "response"
}
}
],
"external_ids": []
}{
"message": "Dataset must have at least one non-draft version"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"success": false,
"message": "<string>",
"error": "<string>"
}{
"message": "<string>",
"success": false
}{
"detail": [
{
"loc": [
"<string>"
],
"msg": "<string>",
"type": "<string>"
}
]
}Legacy Dataset, Evaluation, and Report endpoints are deprecated for new workflows. Use the Tables API for new dataset import, evaluation, scoring, recalculation, and reporting workflows.
Behavior Notes
- Evaluation columns use the same node definitions as Workflows. See Node & Column Types.
- Set
is_part_of_scoreon columns for built-in scoring, or providescore_configurationfor custom scoring logic. - Custom scoring concepts are covered in Score Card.
Related
Authorizations
Body
application/json
Evaluation pipeline creation payload.
ID of the dataset group to use.
Name for the pipeline. Auto-generated if omitted.
Folder ID for organization.
Specific dataset version. Uses latest published version if omitted.
Evaluation columns to add to the pipeline.
Show child attributes
Show child attributes
Optional custom scoring logic.
Show child attributes
Show child attributes
Example:
{
"code": "score = sum(1 for row in data if row.get(\"Accuracy Check\") is True) / len(data) * 100 if data else 0\nreturn {\"score\": score}",
"code_language": "PYTHON"
}
External ID mappings to attach to the pipeline.
Show child attributes
Show child attributes
Response
Evaluation pipeline created.
Available options:
true ID of the created evaluation pipeline.
External ID mappings attached to the pipeline.
Show child attributes
Show child attributes
Columns created on the pipeline.
Show child attributes
Show child attributes
Was this page helpful?