Skip to main content

HTTP API - Commands & Pipeline

Since Version: 1.0

Also see

Commands and pipelines can be executed using a HTTP API which is similar to RESTful endpoints. The HTTP request methods POST and GET, the HTTP header Content-Type and the request body define, how commands and pipelines are executed and which parameters and message body values are set.

The return value depends on the command or the pipeline and is mostly a JSON document.

It's important to understand, that you have two options to call the backend using the HTTP API:

  • A single Command
  • A chain of Commands using a Pipeline script

Below, these two options will be explained in more detail.

HTTP API for single commands calls​

You can call a single command using a HTTP request with the endpoint 

/api/v3/command/<name>

Whereas <name> needs to be replaced by the name of the command. The optional request parameters will become the command parameters. See here for a reference of all built-in commands and their parameters. Using the request header Content-Type and the request body, you can optionally set the message body for the command.

HTTP Methods​

In order to execute a command on the backend, you need to send a HTTP request to the endpoint /api/v3/command/<name> by using one of these HTTP options:

HTTP MethodContent-TypeRequest BodyMessage BodyExecution
is set tois set tois set towill becomeis
GETNoneNoneNoneA single command given by <name> with parameters set from request parameters and a null body. See Example.
POSTapplication/jsonA JSON data documentA parsed JSON data objectA single command given by <name> with parameters set from request parameters and the JSON object in the body. See Example.
POSTAnyAny (binary) data format as specified by the Content-Type Note: Additionally the header Content-Length is mandatory in this case.Content Object referencing the data in the request body as a stream.A single command given by <name> with parameters set from request parameters and a content object in the body. This is the most effective way of uploading large data to a command. See Example
POSTmultipart/form-dataOne or more parts with name="file" in the content disposition header. See how HTTP multipart works.Content Collection with each entry referencing a part data.A single command given by <name> with parameters set from request parameters and a content collection in the body. See Example

Example 1: No body​

In case you would like to execute a single command without a body, you could run it like this:

curl -u 'username:password' \
  'https://hub-try.pipeforce.org/api/v3/command/message.send?key=slack&payload=HELLO'

This example simply sends a new message to the message broker with key slack and with message payload HELLO.

Example 2: With JSON in body​

In case you would like to execute a single command with a JSON document as body data, you could execute this curl:

curl -u 'username:password' \
  -H 'Content-Type: application/json' \
  -X POST 'https://hub-try.pipeforce.org/api/v3/command/property.put?key=some/key' \
  -d '{"name": "someValue"}'

Example 3: With binary data in body​

In case you would like to execute a single command with a binary data in the body, you could execute this curl, which uploads the content of a local PDF file to a command:

curl -u 'username:password' \
  -H 'Content-Type: application/pdf' \
  -X POST 'https://hub-try.pipeforce.org/api/v3/command/property.attachment.put?key=some/key&name=my.pdf' \
  --data-binary "@my.pdf"

Example 4: With multipart body​

In case you would like to execute a single command with multiple parts (files) uploaded and provided in the body, you can run this:

curl -u 'username:password' \
  -X POST 'https://hub-try.pipeforce.org/api/v3/command/property.attachment.put?key=some/key'  \
  -F file=@my1.pdf \
  -F file=@my2.pdf

This example will upload two files my1.pdf and my2.pdf and add them as attachments to the property with key some/key. For more details, how multipart HTTP POST requests work, see the Mozilla Docs.

HTTP API for adhoc pipeline calls​

Another option to call the backend is to define a chain of commands to be executed in an adhoc pipeline and run this by sending it via HTTP POST request to 

/api/v3/pipeline

Again by using the request header Content-Type and the request body, you can optionally set the body data for the pipeline.

HTTP Methods​

In order to execute a pipeline script on the backend, you need to send via HTTP POST request to the endpoint /api/v3/pipeline by using one of these HTTP options:

HTTP MethodContent-TypeRequest BodyMessage BodyExecution
is set tois set tois set towill becomeis
POSTapplication/jsonA JSON pipeline document.null or the value set by the body section of the pipeline.A pipeline set as JSON from the request body.
POSTapplication/yamlA YAML pipeline document.null or the value set by the body section of the pipeline.A pipeline set as YAML from the request body. See examples with no body and embedded body.
POSTapplication/x-www-form-urlencodedThe pipeline as URL encoded request query string whereas the key of each parameter is the command name and the value defines the parameters to the command. Key and value must be separated by a colon :. Multiple command parameters are separated by semicolon ;. Example (before url encoding): log=message:'Hello World!';level:INFOnullA pipeline set as URL encoded query string from the request body. See example with url encoded query.
POSTNoneA YAML pipeline document (same as with application/yaml).null or the value set by the body section of the pipeline.A pipeline set as YAML from the request body. Any other format in the body will throw an 400 bad request error.
POSTmultipart/form-dataOne part with name="pipeline" with a YAML pipeline and one or more parts with name="file" in the content disposition header. See how HTTP multipart works.Content Collection referencing all fileparts.A pipeline set as YAML from the pipeline part and with a content collection in the body created from all file parts of the request body. See Example.
Note

Using the HTTP methd GET with the /pipeline endpoint is not possible and will cause an error.

Example 5: No body​

In case you would like to execute a pipeline YAML script without a message body, you can run this:

curl -u 'username:password' \
  -X POST "https://hub-try.pipeforce.org/api/v3/pipeline" \
  -H "Content-Type: application/yaml" \
  --data-binary @- << EOF
pipeline:
 - drive.read:
     path: /my.pdf
 - pdf.stamp:
     text: "Hello World!"
 - drive.save:
     path: /my-stamped.pdf
EOF

Example 6: With body embedded​

In case you would like to execute a pipeline YAML script with message body, which is embedded inside the YAML, you can run this:

curl -u 'username:password' \
  -X POST "https://hub-try.pipeforce.org/api/v3/pipeline" \
  -H "Content-Type: application/yaml" \
  --data-binary @- << EOF
pipeline:
  - log:
      message: "BODY: #{body.text}"

body: {"text": "Hello World!"}
EOF

Example 7: With url encoded pipeline​

In this example, a simple adhoc pipeline is called using an url encoded query string.

Suppose you have a pipeline that looks like this:

pipeline:
  - datetime:
      format: dd.MM.YY
  - set.body:
      value: "Today is: #{body}"

You can rewrite it to a query string like this (not url encoded):

datetime=format:dd.MM.YY&set.body=value:"Today is: #{body}"
  • The query parameter key becomes the command name (for example datetime).
  • The query parameter value specifies the parameters to the command in the format of <key>:<value> pairs. The value part of such a parameter can optionally be put into 'single' or "double" quotes. Examples:
    • log=message:HELLO WORLD
    • log='HELLO WORLD'
    • log="HELLO WORLD"
  • Multiple <key>:<value> pairs must be separated by ;. Example:
    • log='HELLO WORLD';level=INFO
  • Multiple commands must be separated by &. Examples:
    • log='HELLO WORLD';level=INFO&datetime
    • log='HELLO WORLD';level=INFO&datetime=format:dd.MM.YY
  • Finally, values need to be URL encoded. Example:
    • datetime=format:dd.MM.YY encodes to datetime=format%3Add.MM.YY

For more details about the application/x-www-url-encoded content type in the HTTP standard see for example: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST

After the values have been url encoded, the query string looks like this:

datetime=format%3Add.MM.YY&set.body=value%3A%20%22Today%20is%3A%20%23%7Bbody%7D%2

Below you can find the execution examples using this url encoded query string:

curl -u 'username:password' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -X POST 'https://hub-<your-domain>/api/v3/pipeline' \
  -d 'datetime=format%3Add.MM.YY&set.body=value%3A%20%22Today%20is%3A%20%23%7Bbody%7D%22'

Example 8: With multipart body​

Let's assume you would like to execute a pipeline script and additionally processing one or more files in this same pipeline which must be placed in the body.

Here’s an example of an HTTP multipart request as defined by the HTTP specification. This example does multiple steps in one request: It uploads a PDF, puts a new text on this PDF and finally stores it at server side:

curl -u 'username:password' \
  -X POST 'https://hub-try.pipeforce.org/api/v3/pipeline' \
  -F 'pipeline="pipeline:
        - pdf.stamp:
            text: \"Hello World!\"
        - drive.save:
            path: /my-stamped.pdf "' \
  -F 'file=@/my1.pdf' \
Note

Note that the pipeline has the part name pipeline and one or more files must all have the name file.

HTTP API for persisted pipeline calls​

It is also possible to load and execute a pipline stored in the property store by sending a HTTP POST request to 

/api/v3/pipeline:<key>

Whereas <key> must be replaced by the property key of the pipeline to be executed.

Instead of using the property key, you can also set the uuid of the pipeline property:

/api/v3/pipeline:uuid:<uuid>

Whereas <uuid> must be replaced by the uuid of the pipeline property to be loaded and executed. This approach has the advantage that it will work even if the pipeline has been moved / renamed to a different key.

Any request parameter given will be set as variables to the pipeline.

In case there is a body in the request, it will be set as body to the pipeline. If header ContentType is set to application/json the body will be parsed to a JSON object and provided as such as pipeline body. Otherwise, the body will be provided as content object.

HTTP Methods​

In order to execute a persisted pipeline on the backend, you need to send a HTTP POST request to the endpoint /api/v3/pipeline:<key> or /api/v3/pipeline:uuid:<uuid> by using one of these HTTP options:

HTTP MethodContent-TypeRequest BodyMessage BodyExecution
is set tois set tois set towill becomeis
POSTapplication/jsonA JSON data document.The JSON data document parsed to a JSON instance.The persisted pipeline loaded from the property store using its key or uuid.
POSTAnyAny data.The data from the body provided as a content objectThe persisted pipeline loaded from the property store using its key or uuid.
POSTNoneNonenullThe persisted pipeline loaded from the property store using its key or uuid.
PUTapplication/jsonA JSON pipeline.NoneThe pipeline given as JSON in the request body will be converted and stored as YAML property with key given by <key> from request path. In case a pipeline with such key already exists, updates the existing one.
PUTapplication/yamlA YAML pipeline.NoneThe pipeline given as YAML in the request body will be stored as YAML property with key given by <key> from request path. In case a pipeline with such key already exists, updates the existing one.
PUTapplication/x-www-form-urlencodedAn url encoded pipeline URI.NoneThe pipeline given as url encoded pipeline URI in the request body will be converted and stored as YAML property with key given by <key> from request path. In case a pipeline with such key already exists, updates the existing one.

Example 9: Execute persisted pipeline​

In this example a pipeline persisted under key global/app/myapp/pipeline/mypipeline will be loaded and executed at the backend without any body an request parameters:

curl -u 'username:password' \
  -X POST 'https://hub-<your-domain>/api/v3/pipeline:global/app/myapp/pipeline/mypipeline'

Example 10: Store a pipeline​

In this example a new persisted pipeline will be created under key global/app/myapp/pipeline/newpipeline.

curl -u 'username:password' \
  -h 'Content-Type: application/yaml' \
  -X PUT 'https://hub-<your-domain>/api/v3/pipeline:global/app/myapp/pipeline/newpipeline' \
  --data-raw 'pipeline:
    - log:
        message: "HELLO WORLD!"'

Report an Issue​

Your help is needed!

In case you're missing something on this page, you found an error or you have an idea for improvement, please click here to create a new issue. Another way to contribute is, to click Edit this page below and directly add your changes in GitHub. Many thanks for your contribution in order to improve PIPEFORCE!