Workflow Webhooks

Overview

Workflow Webhooks enable two-way communication between Clear Ideas AI Workflows and external systems. With webhooks, you can trigger workflows automatically when events occur in other applications, and send workflow results to external systems when processing completes.

Webhooks open up powerful automation possibilities:

• Automatically process form submissions through AI workflows
• Trigger content generation when new records are created in your CRM
• Send AI-generated content to your CMS or publishing platform
• Update external databases with workflow results
• Connect workflows to Zapier, Make, or other automation platforms

Incoming Webhooks

Incoming webhooks allow external systems to trigger your AI workflows by sending HTTP requests to a unique URL. Authentication is required for all incoming webhook requests.

How Incoming Webhooks Work

1. Enable webhook triggers - Turn on "Enable Webhook Triggers" in your workflow options
2. Configure variable mappings - Set up how incoming data maps to workflow variables
3. Create an access key - Generate a webhook access key with receive_events scope
4. Get your webhook URL - Clear Ideas generates a unique URL for your workflow
5. Send authenticated requests - External systems POST JSON data with proper authentication
6. Automatic processing - Clear Ideas validates the data, maps it to workflow variables, and starts the workflow job

Enabling Webhook Triggers

Before you can receive webhook requests, you must enable webhook triggers for your workflow:

1. Open your workflow and navigate to the Workflow Options section
2. Check the Enable Webhook Triggers option
3. Save your changes
4. A new Webhook Triggers section will appear where you can configure variable mappings and view your webhook URL

Creating a Webhook Access Key

Incoming webhooks require authentication using a webhook access key. To create one:

1. Navigate to Settings > Access Keys
2. Click New Access Key
3. Select Webhook as the key type
4. Add the receive_events scope
5. Give your key a descriptive name
6. Click Create Key and copy the key immediately (it will only be shown once)

For more details on managing access keys, see Access Keys.

Webhook URL

After enabling webhook triggers, you'll see your unique webhook URL in the Webhook Triggers section. The URL follows this format:

https://api.clearideas.ca/webhook/workflow/{workflow-id}

Use this URL in your external systems to trigger the workflow.

Variable Mappings

Variable mappings define how data from the incoming request body maps to your workflow variables. You configure these mappings in the Webhook Triggers section of your workflow.

For each mapping, specify:

• Payload Key - The field name in the incoming JSON (supports dot notation for nested fields)
• Variable Name - The workflow variable to assign the value to
• Data Type - The expected type (string, number, boolean, or JSON)
• Required - Whether this field must be present in the request

Example: If your external system sends:

{
  "customer": {
    "name": "Acme Corp",
    "email": "contact@acme.com"
  },
  "request_type": "proposal"
}

You would configure mappings like:

Authentication

All incoming webhook requests must be authenticated. Clear Ideas supports the following authentication methods:

Webhook Access Key

Include your webhook access key as a Bearer token in the Authorization header. This approach is well-suited for server-to-server integrations where a dedicated service is triggering workflows.

Authorization: Bearer webhook_abc123.secretkey

The access key must have the receive_events scope to trigger workflows.

OAuth2 Bearer Token

If you have an OAuth2 access token from an authenticated Clear Ideas session, you can use it to trigger webhooks. This is useful when building applications where users are already authenticated with Clear Ideas.

Authorization: Bearer your-oauth2-token

OAuth2 tokens are validated first, with fallback to access key authentication if the token is not a valid OAuth2 token.

Request Format

Incoming webhook requests must follow these requirements:

• Use the POST HTTP method
• Include the Content-Type: application/json header
• Include valid authentication in the Authorization header
• Send a valid JSON body

Example request using curl:

curl -X POST https://api.clearideas.ca/webhook/workflow/{workflow-id} \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer webhook_abc123.secretkey" \
  -d '{"customer_name": "Acme Corp", "request_type": "proposal"}'

Response

When a webhook request is received and processed successfully, Clear Ideas returns a response indicating that the workflow job has been created:

{
  "success": true,
  "jobId": "job-123456",
  "message": "Workflow job created"
}

If authentication fails, you will receive a 401 Unauthorized response. If validation fails (such as missing required fields), you will receive a 400 Bad Request response with details about what went wrong.

Default Sites

You can configure default sites for webhook-triggered jobs in the Webhook Triggers section. When a webhook triggers a workflow, the job will use these sites for context unless the incoming request specifies otherwise. This is useful when your workflow needs access to specific knowledge bases.

Outgoing Webhooks

Outgoing webhooks send workflow results to external systems when a workflow step completes. You create outgoing webhooks by adding a Webhook Step to your workflow.

How Outgoing Webhooks Work

1. Add a webhook step - Create a new step and select "Webhook Step" as the type
2. Configure the destination - Set the URL, HTTP method, and headers
3. Define variable mappings - Map workflow outputs to the outgoing request body
4. Run the workflow - When the webhook step executes, Clear Ideas sends the results
5. External processing - Your external system receives and processes the data

Creating a Webhook Step

To add an outgoing webhook to your workflow:

1. Open your workflow and click Create New Step
2. Select Webhook Step from the step type options
3. Configure the webhook settings in the step editor

Webhook Configuration

When editing a webhook step, you configure the following settings:

URL - The destination endpoint where the webhook will send data. This is required.

HTTP Method - The HTTP method to use for the request. Options include GET, POST, PUT, PATCH, and DELETE. POST is the default and most commonly used for webhooks.

Timeout - How long to wait for a response before timing out (in milliseconds). The default is 30,000ms (30 seconds).

Retries - How many times to retry the request if it fails. The default is 3 retries.

Retry Delay - How long to wait between retry attempts (in milliseconds). The default is 1,000ms (1 second).

Request Headers

You can add custom headers to your webhook request in the Headers section. Common uses include:

• Authorization - Add authentication credentials for the destination system using Bearer your-token format
• Custom headers - Any headers required by your destination API

When you have variable mappings configured, Clear Ideas automatically adds Content-Type: application/json to your headers.

Example: To authenticate with an external API, add a header:

Variable Mappings

Variable mappings define how workflow outputs map to the outgoing request body. Each mapping specifies:

• Payload Key - The field name in the outgoing JSON
• Workflow Variable Key - The source of the data (workflow variable, step output, or final output)
• Data Type - How to format the value (string, number, boolean, or JSON)
• Required - Whether this mapping must have a value

Available sources include:

• Workflow variables: {{variable:variableName}} or just variableName
• Step outputs: {{step-1}}, {{step-2}}, etc.
• Named outputs: {{summary}}, {{analysis}}, etc. (if steps have output variable names)
• Previous step output: {{step-previous}}
• Dot notation for nested data: {{step-1.address.city}}

Example: To send workflow results to your CMS:

This produces a JSON payload like:

{
  "content": "Content from step 1...",
  "title": "My Document",
  "metadata": {
    "author": "John Smith"
  }
}

Use Cases

Trigger Workflows from Form Submissions

You can connect form tools like Typeform, JotForm, or custom forms to trigger AI processing automatically:

1. Create a workflow that processes form data with appropriate variables
2. Enable webhook triggers and configure variable mappings for your form fields
3. Create a webhook access key with the receive_events scope
4. Configure your form tool to POST to the webhook URL with the access key when forms are submitted
5. Form responses will automatically trigger AI processing and store results

Send AI Content to Your CMS

You can automatically publish AI-generated content to your content management system:

1. Create a content generation workflow that produces your desired output
2. Add a webhook step at the end configured for your CMS API endpoint
3. Add an Authorization header with your CMS API credentials
4. Map the workflow output to your CMS content fields
5. When the workflow completes, content will be automatically created in your CMS

Connect to Automation Platforms

You can integrate Clear Ideas with Zapier, Make, or similar platforms:

1. Use incoming webhooks to receive triggers from automation platforms
2. Use outgoing webhook steps to send results back to these platforms
3. Build complex multi-system automations with AI processing in the middle

CRM Integration

You can process new leads or opportunities with AI automatically:

1. Configure your CRM to send webhook notifications when records are created or updated
2. Create a workflow with variable mappings for the CRM data fields
3. Set up an incoming webhook to receive and process the data
4. Optionally add an outgoing webhook step to update the CRM with AI-generated insights

Best Practices

Security

You should always use authentication for incoming webhooks in production environments. Webhook access keys with the receive_events scope provide a secure way to authenticate requests from external systems. Keep your webhook URLs and access keys confidential, and rotate keys periodically to maintain security. If you suspect a key has been compromised, revoke it immediately through the Access Keys settings page.

For outgoing webhooks, use HTTPS endpoints whenever possible to ensure data is encrypted in transit. Store API credentials securely and use the Authorization header to authenticate with destination systems.

Variable Mappings

When configuring variable mappings, use clear and consistent naming between systems to make troubleshooting easier. Document your mappings for team reference so others understand how data flows between systems. Test your configuration with sample data before going live to catch any issues early. For optional fields, configure appropriate default values to handle cases where data may be missing.

Error Handling

Monitor your workflow jobs for webhook-related failures by checking the job details page. Failed webhook steps will show error information that helps identify what went wrong. Set up notifications to be alerted when jobs fail so you can respond quickly. When integrating with external systems, ensure those systems can handle potential errors gracefully and consider implementing idempotency for retry scenarios.

Performance

Keep webhook payloads reasonably sized to avoid timeout issues and reduce processing time. When receiving webhooks, configure your external systems to process them asynchronously when possible. Set appropriate timeout values based on how long your destination systems typically take to respond. For high-volume scenarios, consider implementing rate limiting on your side to avoid overwhelming destination systems.

Troubleshooting

Incoming Webhook Not Triggering

If your incoming webhook is not triggering the workflow, first verify that the webhook URL is correct and includes the proper workflow ID. Check that you have enabled webhook triggers in the workflow options. Ensure your authentication credentials are valid by confirming the access key has not been revoked and has the receive_events scope. Verify that the request body contains valid JSON and matches the expected structure defined in your variable mappings. Review the access key's "Last Used" date in Settings to see if requests are reaching Clear Ideas at all.

Outgoing Webhook Not Delivering

If your outgoing webhook is not delivering data to the destination, verify that the destination URL is correct and reachable from the internet. Check that any required authentication headers are configured correctly in the webhook step. Review the response from the destination system in the job step details for error messages. Ensure there are no network or firewall issues blocking the request. If the destination system is returning errors, check their documentation for required headers or request format requirements.

Variable Mapping Issues

If data is not mapping correctly between systems, ensure your payload keys match the exact path in the JSON using dot notation for nested fields (for example, customer.name rather than just name). Verify that the data types you have selected match what the source or destination expects. Check for typos in variable names and payload keys, as these are case-sensitive. Test your mappings with simple data first before adding complexity.

Authentication Errors

If you receive 401 Unauthorized errors on incoming webhooks, first verify that your credentials are valid. For access keys, check that the key has not expired or been revoked and that it has the receive_events scope required for webhook triggers. For OAuth2 tokens, ensure the token is still valid and has not expired. In both cases, confirm you are including the credentials in the Authorization header with the correct Bearer prefix. If using an older access key, consider creating a new one to ensure it has the correct permissions.