​
What Are Sources?

Sources (or data sources) are the starting points for your organization’s data integration journey. They represent the origins of data that can be ingested, transformed, and utilized to power analytics, applications, and business operations. Nekt allows you to connect to a variety of sources, making it easy to centralize and leverage your data effectively.

Sources can include a wide range of systems, from 3rd party APIs and SaaS platforms to flat files, databases, data warehouses, and data lakes.

​
Types of Sources

Sources can be broadly categorized based on the type of data or system they represent:

  • Databases: Structured data from relational databases like MySQL and PostgreSQL, or NoSQL databases like MongoDB and Cassandra.
  • Cloud Storage: Data stored in cloud services such as Amazon S3, Google Cloud Storage, or Azure Blob Storage.
  • Applications: Data from enterprise applications such as ERP systems, CRM platforms (e.g., Salesforce), and marketing tools.
  • Flat Files: Structured and semi-structured data in formats like CSV, Excel, JSON, or XML.
  • Data Lakes and Warehouses: Centralized repositories such as AWS Lake Formation, Snowflake, and Google BigQuery that aggregate data from multiple sources.

These categories provide flexibility in managing data across diverse formats and platforms, ensuring seamless integration into your workflows.

​
Connecting to Sources

Nekt supports a wide range of source connections. To see the full list of supported sources, visit the Add Source page.

If the source you need isn’t listed, let us know via Slack, and we’ll work with you to prioritize its addition.

​
Retries and Error Management

Data pipelines can sometimes fail due to system instabilities or temporary issues. Nekt allows you to configure retry parameters for each Source, Transformation, or Destination, ensuring reliable pipeline execution without unnecessary resource usage.

​
Retry Parameters:

  1. Number of Retries: Specify the number of retry attempts before marking a run as failed.
  2. Retry Delay: Set the delay (in seconds) between retry attempts.
  3. Max Consecutive Failures: Define the maximum number of consecutive failures allowed before inactivating the pipeline.
    • Use Case: This setting is ideal for preventing repeated failures during off-hours or weekends, saving resources and avoiding unnecessary alerts.

These parameters can be managed in the settings tab of each Source, providing flexibility in how you handle pipeline stability and recovery.

​
Why use Sources in Nekt?

Integrating Sources into Nekt allows your organization to:

  • Centralize Data: Gather data from multiple origins into a single platform for analysis and insights.
  • Improve Data Accessibility: Enable teams to access and work with data from various systems without silos.
  • Streamline Workflows: Automate data ingestion and ensure reliable delivery to transformations and destinations.

With robust support for diverse source types and flexible error management, Nekt makes it easy to integrate data into your workflows while ensuring reliability and efficiency.

​
Need Help?

Have questions or need assistance with setting up a source? Reach out via Slack or contact our support team, and we’ll help you get started.

OnboardingCSV