Implementing Agentic Actions With Third-Party Integrations

The first wave of RAG-enabled AI applications generally focused on “read” operations, ingesting and retrieving data from external data sources to enhance LLM responses. In the previous chapters of this tutorial series, we focused on building processes that facilitated the following:

• Data ingestion and retrieval of data from users' third-party platforms like Google Drive and Notion

• Permissions and access control of those data artifacts from platforms like Google Drive and Dropbox

• Scalability of the permissions flow with respect to latency and performance as the number of integrations scale

That said, the next wave It’s become increasingly useful for AI chatbots to be able to use external context, as well as able to take action on behalf of users. In this tutorial, we focused on “write” operations, showcasing a RAG-enabled application that can read data from third-party integrations and perform actions in those integrations.

Tutorial Overview

To build out “agentic actions,” we implemented an AI Agent with tool use. For the purposes of this tutorial, we created two sets of tools that allow Parato to:

1. Write Slack messages in a designated channel

2. Create new contacts in Salesforce

In its simplest form, we can prompt our AI Agent to use the tool to perform a simple action like sending a Slack message:

An important part of implementing tools for an AI Agent involves explicitly describing tool specifications - the purpose of the tool, the required inputs, when to use the tool, etc. When done correctly, we can even chain together tools and actions, creating more complex interactions like this:

Users can also chain a RAG action where we read data from Salesforce with an agentic action and subsequently write a new contact to Salesforce. In this example, we can see Parato chain events in this order:

1. Checks permissions of the user to ensure the user has permissions to read and write to Salesforce contacts (implicit in the queryEngineTool)

2. Queries our Pinecone vector database for embeddings from Salesforce related to the user prompt

3. Retrieves the title of “Marvin Valasquez” from the retrieved vector embedding

4. Creates a Salesforce contact “draft” with the user provided information and the retrieved information

5. Creates the Salesforce contact after confirmation is given

You may have noticed that in both Slack and Salesforce prompts, a draft was created to preview the action. This is an intentional behavior.

Because “write” operations modify the underlying data in your third party platforms, which has larger consequences than a “read” operation, we needed guardrails in place to be transparent on what data will be written (imagine sending a Slack message with sensitive information on accident via chatbot). In our implementation, we required confirmation from the user before performing the action.

Like always, permissions were an important consideration for Parato. Actions triggered on Parato’s chat interface will inherit the user’s permissions to the third-party integration. This means users without write access in Salesforce will not be able to create new records via Parato; users without permissions to a Slack channel will not be able to post via Parato.

Now that we’ve gone over Parato’s new features, let’s dive into the high level architecture and design.

Tutorial Architecture

The Parato AI Agent introduced in this tutorial (which we’ll call Parato v3.0) is layered on top of the existing Parato features.

To recap Parato’s v2.5 architecture on RAG-enabled read actions:

• Users send prompts asking for information on topics where third party data is relevant (for example, a prompt inquiring information from a Google Drive file)

• Permissions are assessed using an Okta FGA graph which models third-party permissions (i.e. Google Drive’s read/write permissions for files and folders)

• For permitted documents, we search our Pinecone vector database for relevant data to enhance Parato’s response

The new architecture introduced in Parato v3.0 is a design pattern for “write” actions.

In this pattern, users can trigger an action with a prompt like “send a Slack message.” Parato v3.0 uses LlamaIndex’s Function Tools to define actions that our LLM can take - in this tutorial, we created four actions: draft a Slack message, send a Slack message, draft a Salesforce contact, and create a Salesforce contact.

In steps 3-4 we used a Human-in-the-Loop (HITL) pattern where a user confirms an action after seeing the preview of their intended action before any action takes places. Only after confirmation do steps 5-8 trigger, where code in the defined Function Tool is run to send a POST request. Paragon handles integration-specific details, like authentication and webhook listeners, and forwards the request to our third party platform to write the data.

You may have noticed, unlike with “read” operations, our FGA graph is not being used for actions to assess permissions. Because Parato needs to use the third-party API to write data, we pass on the credentials of the user to the third-party. If the user is not authorized to perform the action, an error is returned on the third-party server side, which is surfaced back to the chat interface.

That’s the high level architecture of Parato v3.0. Although we split out the architecture diagrams for read versus write operations, chaining actions together (remember the create Salesforce contact example where we needed to retrieve Salesforce data first) can involve using both patterns in the same prompt. Let’s dive deeper into the code and workflows where we implemented these new features.

Implementing Third-Party Actions

For performing actions, we first implemented logic in Parato’s application layer defining the Function Tools our AI Agent can use. Then we took the outputs of those Function Tools and sent them to the corresponding third-party API.

From Reading to Writing

For reading and querying third-party data, Parato utilized Paragon to ingest data, Okta FGA to model permissions, and Pinecone to search for relevant vector embeddings.

For writing and performing actions on third-party data, Parato utilized Llamaindex’s Agent and Tool abstractions to describe and define agentic actions, and Paragon to interact with third-party APIs.

Examining the agentic actions in our application layer, we performed the following steps:

1. Defined Function Tools that map data from a natural language prompt to our application’s desired schema, and created a draft for review:

• Using the “draftSalesforceContact” Function Tool as an example, we were explicit on what the purpose of the tool is and when the tool should be used

• We also described each input that was needed for the function - first name, last name, email, and title

• Lastly, we emphasized in the description that a confirmation is needed to actually send the data to Salesforce

const draftSalesforceContact = FunctionTool.from(
        ({ first_name, last_name, email, title }: { first_name: string, last_name: string, email: string, title: string}) => {
            return "Salesforce contact first name: " + first_name + "\n" +
                "Salesforce contact last name: " + last_name + "\n" +
                "Salesforce contact email: " + email+ "\n" +
                "Salesforce contact title: " + title;
        },
        {
            name: "draftSalesforcecontact",
            description: "Use this function to draft a contact record in Salesforce. This is a required function step" +
                "before creating a contact record in Salesforce. Prompt confirmation from " +
                "user to trigger the Salesforce record confirm and send step. This function does not create the contact record" +
                "in Salesforce. This function only drafts a contact record and prompts for confirmation",
            parameters: {
                type: "object",
                properties: {
                    first_name: {
                        type: "string",
                        description: "First name of Salesforce contact",
                    },
                    last_name: {
                        type: "string",
                        description: "Last name of Salesforce contact",
                    },
                    email: {
                        type: "string",
                        description: "Email of Salesforce contact",
                    },
                    title: {
                        type: "string",
                        description: "Title of Salesforce contact",
                    },
                },
                required: ["first_name", "last_name", "email", "title"],
            },
        }
    );

2. Defined Function Tools that trigger after the HITL confirmation, taking the mapped data and sending it in a POST request:

• Again, we were extremely explicit in our tool description, so that our Agent can use the tool at the correct time

• Both the “draft” and “create” Function Tools involve similar inputs, but we only want this “confirmAndCreateSalesforce” step to trigger after a draft is created

const confirmAndCreateSalesforcecontact = FunctionTool.from(
        async({ confirmation, first_name, last_name, email, title }: { confirmation: string, first_name: string, last_name: string,
            email: string, title: string}) => {
            console.log("Confirmed Salesforce contact Creation: " + confirmation);
            const response = await createSalesforcecontact({first_name, last_name, email, title}, signJwt(userId));
            if(response.statusCode){
                return "Successfully created Salesforce contact";
            }
            return "Salesforce contact not created successfully";
        },
        {
            name: "confirmAndCreateSalesforcecontact",
            description: "Use this function to create a Salesforce contact record only after a draft has been created. Do" +
                "not use this function if an affirmative confirmation is not given. Do not use this function if" +
                "a draft Salesforce contact record has not been created",
            parameters: {
                type: "object",
                properties: {
                    confirmation: {
                        type: "string",
                        description: "affirmative confirmation to create Salesforce contact record"
                    },
                    first_name: {
                        type: "string",
                        description: "First name of Salesforce contact",
                    },
                    last_name: {
                        type: "string",
                        description: "Last name of Salesforce contact",
                    },
                    email: {
                        type: "string",
                        description: "Email of Salesforce contact",
                    },
                    title: {
                        type: "string",
                        description: "Title of Salesforce contact",
                    },
                },
                required: ["confirmation", "first_name", "last_name", "email", "title"],
            },
        }
    );

It’s quite amazing to see how the function tool descriptions can create code-like behavior: The "draft" step triggers and passes data to the "confirm and create" step, and the "confirm and create" step sends the POST request only if an affirmative confirmation is given. It’s like writing functions, objects, and conditionals using plain English.

After the “confirm and create” tool is triggered, we move to the third-party API interaction, where we’ll forward a POST request from Parato to the third-party.

From Your Application to the Third-Party

After the POST request is triggered by the "confirm and create" step, this is where Paragon fits into the Parato AI Agent workflow. Paragon fits in as a middleware that takes care of third-party specific details like authentication and webhook triggers. Anything between your application and the third-party integration can be handled with lower touch using Paragon’s embedded integration platform.

To illustrate, each integration has their own specific way of handling authentication (JWT, OAuth, access tokens, refresh tokens) and how data needs to be formatted (request parameters, request body, headers). With Paragon, we can send data and authentication in the same way (using a signed JWT to Paragon endpoints or a Proxy API) no matter which integration we’re working with. Notice how the Slack and Salesforce requests look almost identical using Paragon’s endpoint, allowing for faster development for new integrations and Function Tools.

export async function sendSlack(message: string, jwt: string) {
  return await fetch(process.env.SEND_SLACK_ENDPOINT ?? "", {
      method: "POST",
      body: JSON.stringify({ message: message }),
      headers: {
        "Content-Type": "application/json",
        Authorization: "bearer " + jwt,
      },
    })
  .then((response) => response.json())
  .catch((error) =>
    console.log("Error sending Slack message: " + message + " - " + error),
    );
}

export async function createSalesforcecontact(contact: {first_name: string, last_name: string, email: string, title: string}, jwt: string) {
    return await fetch(process.env.CREATE_SALESFORCE_contact_ENDPOINT ?? "", {
        method: "POST",
        body: JSON.stringify(contact),
        headers: {
            "Content-Type": "application/json",
            Authorization: "bearer " + jwt,
        },
    })
        .then((response) => response.json())
        .catch((error) =>
            console.log("Error creating Salesforce contact: " + error)
        );
}

In our Salesforce action, when the request endpoint is triggered, passing along the mapped contact data as a request body, our Paragon workflow checks Salesforce’s API to see if the contact has already been created, and then creates the contact in Salesforce if the duplicate check passed or passes an error message back to user if the check failed.

Once a new Salesforce contact is posted, we wanted Parato to immediately ingest the new data into our Pinecone vector database, so information about that new contact can be retrieved in subsequent queries. Webhook triggers were set up on Paragon’s platform, sending the new contact events to Parato’s backend whenever a new contact was created.

Now we can see a real-time feedback loop, where once Parato creates a Salesforce contact, other users immediately have that data available, and can trigger other actions in other integrations!

Wrapping Up

In this tutorial, we implemented third-party actions into Parato’s core functionality. This transforms Parato from a simple context-aware knowledge chatbot to an AI assistant that can save your users’ time by automating tasks across their third-party platforms. While you’re helping your customers save time, use tools like Paragon and LlamaIndex to save yourself some time building integration-specific processes for your RAG application. Paragon is experienced with helping AI companies with their integration use cases and can help bring your team’s AI capabilities to market faster and reliably.