Ingest your users’ 3rd party data
Ingest your users’ 3rd party data
Ingest large volumes of data, both structured and unstructured, from your users' external applications and files.
Ingest large volumes of data, both structured and unstructured, from your users' external applications and files.
How this works in Paragon
How this works in Paragon
Ingest all of your users' existing 3rd party data, and stream new updates in real-time.
Ingest all of your users' existing 3rd party data, and stream new updates in real-time.
Historical ingestion
Real-time ingestion
Historical ingestion
When a user first authenticates, run an ingestion job to pull in all their existing data.
Process millions of requests concurrently w/ fan outs
Enable users to map fields between 3rd party data and your app
Supports usage of batch APIs
Programatically create new fields in the 3rd party app
Paragraph
Workflow Builder
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
define(
integration: IGoogledriveIntegration,
context: IContext<InputResultMap>,
connectUser: IConnectUser<IPersona<typeof personaMeta>>,
) {
const triggerStep = new IntegrationEnabledStep();
const getAllFilesStep = integration.actions.googleDriveSearchFolders(
{
parentId: `${context.getInput(sharedInputs.folder_to_share)}`,
},
{
autoRetry: true,
description: 'Get all files',
},
);
const fanOutFilesStep = new FanOutStep({
description: 'Fan out files',
iterator: getAllFilesStep.output.result,
});
const integrationRequestStep = new IntegrationRequestStep({
description: 'Get file permissions',
method: 'GET',
url: `/files/${fanOutFilesStep.output.instance.id}/permissions`,
params: { ['']: '' },
headers: {},
});
const mapStep = new FanOutStep({
description: 'Fan out permissions',
iterator: integrationRequestStep.output.response.body.permissions,
});
const ifelseStep = new ConditionalStep({
if: Operators.StringContains(mapStep.output.instance.type, 'domain'),
description: 'Is the permission a group/domain?',
});
const integrationRequestStep1 = new IntegrationRequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'List users in domain',
method: 'GET',
url: `admin/directory/v1/users`,
params: { ['']: '' },
headers: {},
});
const requestStep = new RequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'Send each file and associated permissions to my API',
url: `https://yourapp.com/files`,
method: 'GET',
params: { ['']: '' },
headers: {},
});
triggerStep
.nextStep(getAllFilesStep)
.nextStep(
fanOutFilesStep.branch(
integrationRequestStep
.nextStep(
mapStep.branch(ifelseStep.whenTrue(integrationRequestStep1)),
)
.nextStep(requestStep),
),
);
/**
* Pass all steps used in the workflow to the `.register()`
* function. The keys used in this function must remain stable.
*/
return this.register({
triggerStep,
getAllFilesStep,
fanOutFilesStep,
integrationRequestStep,
mapStep,
ifelseStep,
integrationRequestStep1,
requestStep,
});
}
Historical ingestion
Real-time ingestion
Historical ingestion
When a user first authenticates, run an ingestion job to pull in all their existing data.
Process millions of requests concurrently w/ fan outs
Enable users to map fields between 3rd party data and your app
Supports usage of batch APIs
Programatically create new fields in the 3rd party app
Paragraph
Workflow Builder
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
define(
integration: IGoogledriveIntegration,
context: IContext<InputResultMap>,
connectUser: IConnectUser<IPersona<typeof personaMeta>>,
) {
const triggerStep = new IntegrationEnabledStep();
const getAllFilesStep = integration.actions.googleDriveSearchFolders(
{
parentId: `${context.getInput(sharedInputs.folder_to_share)}`,
},
{
autoRetry: true,
description: 'Get all files',
},
);
const fanOutFilesStep = new FanOutStep({
description: 'Fan out files',
iterator: getAllFilesStep.output.result,
});
const integrationRequestStep = new IntegrationRequestStep({
description: 'Get file permissions',
method: 'GET',
url: `/files/${fanOutFilesStep.output.instance.id}/permissions`,
params: { ['']: '' },
headers: {},
});
const mapStep = new FanOutStep({
description: 'Fan out permissions',
iterator: integrationRequestStep.output.response.body.permissions,
});
const ifelseStep = new ConditionalStep({
if: Operators.StringContains(mapStep.output.instance.type, 'domain'),
description: 'Is the permission a group/domain?',
});
const integrationRequestStep1 = new IntegrationRequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'List users in domain',
method: 'GET',
url: `admin/directory/v1/users`,
params: { ['']: '' },
headers: {},
});
const requestStep = new RequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'Send each file and associated permissions to my API',
url: `https://yourapp.com/files`,
method: 'GET',
params: { ['']: '' },
headers: {},
});
triggerStep
.nextStep(getAllFilesStep)
.nextStep(
fanOutFilesStep.branch(
integrationRequestStep
.nextStep(
mapStep.branch(ifelseStep.whenTrue(integrationRequestStep1)),
)
.nextStep(requestStep),
),
);
/**
* Pass all steps used in the workflow to the `.register()`
* function. The keys used in this function must remain stable.
*/
return this.register({
triggerStep,
getAllFilesStep,
fanOutFilesStep,
integrationRequestStep,
mapStep,
ifelseStep,
integrationRequestStep1,
requestStep,
});
}
Historical ingestion
Real-time ingestion
Historical ingestion
When a user first authenticates, run an ingestion job to pull in all their existing data.
Process millions of requests concurrently w/ fan outs
Enable users to map fields between 3rd party data and your app
Supports usage of batch APIs
Programatically create new fields in the 3rd party app
Paragraph
Workflow Builder
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
define(
integration: IGoogledriveIntegration,
context: IContext<InputResultMap>,
connectUser: IConnectUser<IPersona<typeof personaMeta>>,
) {
const triggerStep = new IntegrationEnabledStep();
const getAllFilesStep = integration.actions.googleDriveSearchFolders(
{
parentId: `${context.getInput(sharedInputs.folder_to_share)}`,
},
{
autoRetry: true,
description: 'Get all files',
},
);
const fanOutFilesStep = new FanOutStep({
description: 'Fan out files',
iterator: getAllFilesStep.output.result,
});
const integrationRequestStep = new IntegrationRequestStep({
description: 'Get file permissions',
method: 'GET',
url: `/files/${fanOutFilesStep.output.instance.id}/permissions`,
params: { ['']: '' },
headers: {},
});
const mapStep = new FanOutStep({
description: 'Fan out permissions',
iterator: integrationRequestStep.output.response.body.permissions,
});
const ifelseStep = new ConditionalStep({
if: Operators.StringContains(mapStep.output.instance.type, 'domain'),
description: 'Is the permission a group/domain?',
});
const integrationRequestStep1 = new IntegrationRequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'List users in domain',
method: 'GET',
url: `admin/directory/v1/users`,
params: { ['']: '' },
headers: {},
});
const requestStep = new RequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'Send each file and associated permissions to my API',
url: `https://yourapp.com/files`,
method: 'GET',
params: { ['']: '' },
headers: {},
});
triggerStep
.nextStep(getAllFilesStep)
.nextStep(
fanOutFilesStep.branch(
integrationRequestStep
.nextStep(
mapStep.branch(ifelseStep.whenTrue(integrationRequestStep1)),
)
.nextStep(requestStep),
),
);
/**
* Pass all steps used in the workflow to the `.register()`
* function. The keys used in this function must remain stable.
*/
return this.register({
triggerStep,
getAllFilesStep,
fanOutFilesStep,
integrationRequestStep,
mapStep,
ifelseStep,
integrationRequestStep1,
requestStep,
});
}
Historical ingestion
Real-time ingestion
Historical ingestion
When a user first authenticates, run an ingestion job to pull in all their existing data.
Process millions of requests concurrently w/ fan outs
Enable users to map fields between 3rd party data and your app
Supports usage of batch APIs
Programatically create new fields in the 3rd party app
Paragraph
Workflow Builder
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
define(
integration: IGoogledriveIntegration,
context: IContext<InputResultMap>,
connectUser: IConnectUser<IPersona<typeof personaMeta>>,
) {
const triggerStep = new IntegrationEnabledStep();
const getAllFilesStep = integration.actions.googleDriveSearchFolders(
{
parentId: `${context.getInput(sharedInputs.folder_to_share)}`,
},
{
autoRetry: true,
description: 'Get all files',
},
);
const fanOutFilesStep = new FanOutStep({
description: 'Fan out files',
iterator: getAllFilesStep.output.result,
});
const integrationRequestStep = new IntegrationRequestStep({
description: 'Get file permissions',
method: 'GET',
url: `/files/${fanOutFilesStep.output.instance.id}/permissions`,
params: { ['']: '' },
headers: {},
});
const mapStep = new FanOutStep({
description: 'Fan out permissions',
iterator: integrationRequestStep.output.response.body.permissions,
});
const ifelseStep = new ConditionalStep({
if: Operators.StringContains(mapStep.output.instance.type, 'domain'),
description: 'Is the permission a group/domain?',
});
const integrationRequestStep1 = new IntegrationRequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'List users in domain',
method: 'GET',
url: `admin/directory/v1/users`,
params: { ['']: '' },
headers: {},
});
const requestStep = new RequestStep({
autoRetry: false,
continueWorkflowOnError: false,
description: 'Send each file and associated permissions to my API',
url: `https://yourapp.com/files`,
method: 'GET',
params: { ['']: '' },
headers: {},
});
triggerStep
.nextStep(getAllFilesStep)
.nextStep(
fanOutFilesStep.branch(
integrationRequestStep
.nextStep(
mapStep.branch(ifelseStep.whenTrue(integrationRequestStep1)),
)
.nextStep(requestStep),
),
);
/**
* Pass all steps used in the workflow to the `.register()`
* function. The keys used in this function must remain stable.
*/
return this.register({
triggerStep,
getAllFilesStep,
fanOutFilesStep,
integrationRequestStep,
mapStep,
ifelseStep,
integrationRequestStep1,
requestStep,
});
}
Orchestrate extensible data ingestion pipelines
Orchestrate extensible data ingestion pipelines
Pre-process and support dynamic, user-specified settings in your ingestion workflows.
Pre-process and support dynamic, user-specified settings in your ingestion workflows.
Clean, chunk, and index unstructured data
Pre-process and chunk data with our highly extensible Function Step that supports importing JS libraries and npm modules.
Clean, chunk, and index unstructured data
Pre-process and chunk data with our highly extensible Function Step that supports importing JS libraries and npm modules.
Clean, chunk, and index unstructured data
Pre-process and chunk data with our highly extensible Function Step that supports importing JS libraries and npm modules.
Clean, chunk, and index unstructured data
Pre-process and chunk data with our highly extensible Function Step that supports importing JS libraries and npm modules.
Let users configure what data to share
Easily surface configuration options with User Settings in the embeddable Connect Portal.
Let users configure what data to share
Easily surface configuration options with User Settings in the embeddable Connect Portal.
Let users configure what data to share
Easily surface configuration options with User Settings in the embeddable Connect Portal.
Let users configure what data to share
Easily surface configuration options with User Settings in the embeddable Connect Portal.
Purpose built for high volume data ingestion
Purpose built for high
volume sync
Whether you’re ingesting terrabytes of unstructured data, Paragon’s workflow engine can handle it.
1,600
Requests per second
2TB
Data per day
Workflow engine
Run syncs and automations at scale on our Workflow Engine. Workflows can retry from errors automatically, replay from the original request payload, and show input and output for every running step.
1,600
Requests per second
2TB
Data per day
Workflow engine
Run syncs and automations at scale on our Workflow Engine. Workflows can retry from errors automatically, replay from the original request payload, and show input and output for every running step.
1,600
Requests per second
2TB
Data per day
Workflow engine
Run syncs and automations at scale on our Workflow Engine. Workflows can retry from errors automatically, replay from the original request payload, and show input and output for every running step.
Standalone queue
Main queue
Data sync
Dedicated queues
Install the Paragon SDK from npm and embed any integration into your app. Fully compatible with Next.js, React, and Vue apps.
Standalone queue
Main queue
Data sync
Dedicated queues
Install the Paragon SDK from npm and embed any integration into your app. Fully compatible with Next.js, React, and Vue apps.
Standalone queue
Main queue
Data sync
Dedicated queues
Install the Paragon SDK from npm and embed any integration into your app. Fully compatible with Next.js, React, and Vue apps.
Horizontal scaling
Use Git to version-control and introduce code review to your workflows and integration config.
Horizontal scaling
Use Git to version-control and introduce code review to your workflows and integration config.
Horizontal scaling
Use Git to version-control and introduce code review to your workflows and integration config.
Smart rate limits
Smart rate limits
Smart rate limits
Monitor every ingestion job
Monitor every ingestion job
Get full observability into every run, and get ahead of any potential errors with our event webhooks.
Get full observability into every run, and get ahead of any potential errors with our event webhooks.
Detailed view of every workflow run
Stream errors in real-time to your own monitoring platform (ie. Sentry, Datadog, etc.)
Customer dashboard for monitoring sync job statuses for each of your customers
Build automations in minutes
Join 100+ SaaS companies that are scaling their integration roadmaps with Paragon.
Build automations in minutes
Join 100+ SaaS companies that are scaling their integration roadmaps with Paragon.
Build automations in minutes
Join 100+ SaaS companies that are scaling their integration roadmaps with Paragon.
Build automations in minutes
Join 100+ SaaS companies that are scaling their integration roadmaps with Paragon.
The embedded integration platform for developers
Integrations
© 2024 Paragon Inc. All Rights Reserved
The embedded integration platform for developers
Integrations
© 2024 Paragon Inc. All Rights Reserved
The embedded integration platform for developers
Integrations
© 2024 Paragon Inc. All Rights Reserved