Engines endpoint
Engines are the foundations for machine learning Models in Data Science Workspace. They contain machine learning algorithms that solve specific problems, feature pipelines to perform feature engineering, or both.
Look up your Docker registry
Your Docker registry credentials are required in order to upload a packaged Recipe file, including your Docker host URL, username, and password. You can look up this information by performing the following GET request:
API Format
GET /engines/dockerRegistry
Request
curl -X GET https://platform.adobe.io/data/sensei/engines/dockerRegistry \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}'
Response
A successful response returns a payload containing the details of your Docker registry including the Docker URL (host
), username (username
), and password (password
).
{ACCESS_TOKEN}
is updated.{
"host": "docker_host.azurecr.io",
"username": "00000000-0000-0000-0000-000000000000",
"password": "password"
}
Create an Engine using Docker URLs docker-image
You can create an Engine by performing a POST request while providing its metadata and a Docker URL that references a Docker image in multipart forms.
API Format
POST /engines
Request Python/R
curl -X POST \
https://platform.adobe.io/data/sensei/engines \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}' \
-H 'content-type: multipart/form-data' \
-F 'engine={
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "Python",
"algorithm": "Classification",
"artifacts": {
"default": {
"image": {
"location": "v1rsvj32smc4wbs.azurecr.io/ml-featurepipeline-pyspark:1.0",
"name": "An additional name for the Docker image",
"executionType": "Python"
}
}
}
}'
name
description
type
algorithm
artifacts.default.image.location
artifacts.default.image.executionType
Request PySpark/Scala
When making a request for PySpark recipes, the executionType
and type
is “PySpark”. When making a request for Scala recipes, the executionType
and type
is “Spark”. The following Scala recipe example uses Spark:
curl -X POST \
https://platform.adobe.io/data/sensei/engines \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}' \
-H 'content-type: multipart/form-data' \
-F 'engine={
"name": "Spark retail sales recipe",
"description": "A description for this Engine",
"type": "Spark",
"mlLibrary":"databricks-spark",
"artifacts": {
"default": {
"image": {
"name": "modelspark",
"executionType": "Spark",
"packagingType": "docker",
"location": "v1d2cs4mimnlttw.azurecr.io/sarunbatchtest:0.0.1"
}
}
}
}'
name
description
type
mlLibrary
databricks-spark
.artifacts.default.image.location
artifacts.default.image.executionType
Response
A successful response returns a payload containing the details of the newly created Engine including its unique identifier (id
). The following example response is for a Python Engine. All Engine responses follow this format:
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde32",
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "Python",
"algorithm": "Classification",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-01T00:00:00.000Z",
"artifacts": {
"default": {
"image": {
"location": "v1rsvj32smc4wbs.azurecr.io/ml-featurepipeline-pyspark:1.0",
"name": "An additional name for the Docker image",
"executionType": "Python",
"packagingType": "docker"
}
}
}
}
Create a feature pipeline Engine using Docker URLs feature-pipeline-docker
You can create a feature pipeline Engine by performing a POST request while providing its metadata and a Docker URL that references a Docker image.
API format
POST /engines
Request
curl -X POST \
https://platform.adobe.io/data/sensei/engines \
-H 'Authorization: Bearer ' \
-H 'x-gw-ims-org-id: 20655D0F5B9875B20A495E23@AdobeOrg' \
-H 'Content-Type: application/vnd.adobe.platform.sensei+json;profile=engine.v1.json' \
-H 'x-api-key: acp_foundation_machineLearning' \
-H 'Content-Type: text/plain' \
-F '{
"type": "PySpark",
"algorithm":"fp",
"name": "Feature_Pipeline_Engine",
"description": "Feature_Pipeline_Engine",
"mlLibrary": "databricks-spark",
"artifacts": {
"default": {
"image": {
"location": "v7d1cs2mimnlttw.azurecr.io/ml-featurepipeline-pyspark:0.2.1",
"name": "datatransformation",
"executionType": "PySpark",
"packagingType": "docker"
},
"defaultMLInstanceConfigs": [ ...
]
}
}
}'
type
algorithm
fp
(feature pipeline).name
description
mlLibrary
databricks-spark
.artifacts.default.image.location
artifacts.default.image.executionType
artifacts.default.image.packagingType
docker
.artifacts.default.defaultMLInstanceConfigs
pipeline.json
configuration file parameters.Response
A successful response returns a payload containing the details of the newly created feature pipeline Engine including its unique identifier (id
). The following example response is for a PySpark feature pipeline Engine.
{
"id": "88236891-4309-4fd9-acd0-3de7827cecd1",
"name": "Feature_Pipeline_Engine",
"description": "Feature_Pipeline_Engine",
"type": "PySpark",
"algorithm": "fp",
"mlLibrary": "databricks-spark",
"created": "2020-04-24T20:46:58.382Z",
"updated": "2020-04-24T20:46:58.382Z",
"deprecated": false,
"artifacts": {
"default": {
"image": {
"location": "v7d1cs3mimnlttw.azurecr.io/ml-featurepipeline-pyspark:0.2.1",
"name": "datatransformation",
"executionType": "PySpark",
"packagingType": "docker"
},
"defaultMLInstanceConfigs": [ ... ]
}
}
}
Retrieve a list of Engines
You can retrieve a list of Engines by performing a single GET request. To help filter results, you can specify query parameters in the request path. For a list of available queries, refer to the appendix section on query parameters for asset retrieval.
API Format
GET /engines
GET /engines?parameter_1=value_1
GET /engines?parameter_1=value_1¶meter_2=value_2
Request
curl -X GET \
https://platform.adobe.io/data/sensei/engines \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}'
Response
A successful response returns a list of Engines and their details.
{
"children": [
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde31",
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "PySpark",
"algorithm": "Classification",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-01T00:00:00.000Z"
},
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde32",
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "Python",
"algorithm": "Classification",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-01T00:00:00.000Z"
},
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde33",
"name": "Feature Pipeline Engine",
"description": "A feature pipeline Engine",
"type": "PySpark",
"algorithm":"fp",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-01T00:00:00.000Z"
}
],
"_page": {
"property": "deleted==false",
"totalCount": 100,
"count": 3
}
}
Retrieve a specific Engine retrieve-specific
You can retrieve the details of a specific Engine by performing a GET request that includes the ID of the desired Engine in the request path.
API Format
GET /engines/{ENGINE_ID}
{ENGINE_ID}
Request
curl -X GET \
https://platform.adobe.io/data/sensei/engines/22f4166f-85ba-4130-a995-a2b8e1edde32 \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}'
Response
A successful response returns a payload containing the details of the desired Engine.
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde32",
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "PySpark",
"algorithm": "Classification",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-01T00:00:00.000Z",
"artifacts": {
"default": {
"image": {
"location": "v7d1cs2mimnlttw.azurecr.io/ml-featurepipeline-pyspark:0.2.1",
"name": "file.egg",
"executionType": "PySpark",
"packagingType": "docker"
}
}
}
}
Update an Engine
You can modify and update an existing Engine by overwriting its properties through a PUT request that includes the target Engine’s ID in the request path and providing a JSON payload containing updated properties.
The following sample API call will update an Engine’s name and description while having these properties initially:
{
"name": "A name for this Engine",
"description": "A description for this Engine",
"type": "Python",
"algorithm": "Classification",
"artifacts": {
"default": {
"image": {
"executionType": "Python",
"packagingType": "docker"
}
}
}
}
API Format
PUT /engines/{ENGINE_ID}
{ENGINE_ID}
Request
curl -X PUT \
https://platform.adobe.io/data/sensei/engines/22f4166f-85ba-4130-a995-a2b8e1edde32 \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}' \
-H 'content-type: application/vnd.adobe.platform.sensei+json;profile=engine.v1.json' \
-d '{
"name": "An updated name for this Engine",
"description": "An updated description",
"type": "Python",
"algorithm": "Classification",
"artifacts": {
"default": {
"image": {
"executionType": "Python",
"packagingType": "docker"
}
}
}
}'
Response
A successful response returns a payload containing the Engine’s updated details.
{
"id": "22f4166f-85ba-4130-a995-a2b8e1edde32",
"name": "An updated name for this Engine",
"description": "An updated description",
"type": "Python",
"algorithm": "Classification",
"created": "2019-01-01T00:00:00.000Z",
"createdBy": {
"displayName": "Jane Doe",
"userId": "Jane_Doe@AdobeID"
},
"updated": "2019-01-02T00:00:00.000Z",
"artifacts": {
"default": {
"image": {
"executionType": "Python",
"packagingType": "docker"
}
}
}
}
Delete an Engine
You can delete an Engine by performing a DELETE request while specifying the target Engine’s ID in the request path. Deleting an Engine will cascade delete all MLInstances which reference that Engine, including any Experiments and Experiment runs belonging to those MLInstances.
API Format
DELETE /engines/{ENGINE_ID}
{ENGINE_ID}
Request
curl -X DELETE \
https://platform.adobe.io/data/sensei/engines/22f4166f-85ba-4130-a995-a2b8e1edde32 \
-H 'Authorization: Bearer {ACCESS_TOKEN}' \
-H 'x-api-key: {API_KEY}' \
-H 'x-gw-ims-org-id: {ORG_ID}' \
-H 'x-sandbox-name: {SANDBOX_NAME}'
Response
{
"title": "Success",
"status": 200,
"detail": "Engine deletion was successful"
}