​
What are Transformations?

Transformations in Nekt enable you to refine and combine the data stored in your Catalog. They allow you to clean data, rename columns, merge tables, and apply custom processing logic, empowering you to create tailored integrations, perform advanced analyses, and derive actionable insights.

The Transformations page is your central hub for managing all your transformations. From here, you can:

  • Activate or deactivate transformations.
  • Trigger executions manually.
  • Review transformation details, logs, and performance metrics.
  • Edit or delete transformations as needed.

​
Creating Transformations with Jupyter Notebooks

Nekt makes it simple to create, test, and deploy transformations using Jupyter Notebooks. Whether you’re new to data engineering or an experienced user, our templates provide an easy starting point for building transformations.

​
Step 1: Generate a Token

To allow Jupyter Notebooks to access your data:

  1. Navigate to the Add Transformation page.
  2. Select the data tables you want to include in your transformation.
  3. Generate a token with appropriate access permissions.
    • You can create multiple tokens, each with specific access levels for better security.

Generate a Token

​
Step 2: Develop and Test Your Transformation

Use one of Nekt’s pre-configured templates to start building your transformation in Jupyter Notebooks. These templates provide ready-made setups for different use cases. Choose the template that best suits your environment and data transformation needs:

  • Google Colab Notebook: Run a Jupyter notebook in the cloud with minimal setup. Requires a Google account.
  • GitHub Codespaces: Use a cloud-hosted Jupyter notebook powered by GitHub for easy setup. Requires a GitHub account.
  • Local Dev Container: Set up a Jupyter notebook on your local machine using an isolated environment with all dependencies pre-installed.
  • Local Jupyter Notebook: Manually configure a Jupyter notebook on your local machine. Best for users with advanced knowledge of Python environments.

How to Start:

  1. Copy your token and paste it into the template as instructed.
  2. Add the INPUT_TABLES list, which specifies the tables for the transformation.
  3. Develop and test your transformation code within the notebook environment.

View Templates

​
Step 3: Add Your Transformation to Nekt

Once your transformation is ready:

  1. Run and test your transformation in Jupyter to ensure everything works as expected.
  2. Navigate back to the Add Transformation page in Nekt.
  3. Copy your transformation code into Nekt and configure the necessary trigger preferences.
  4. Save the transformation to make it available for execution.

Add Transformation

​
Best Practices for Transformations

  1. Use Templates: Start with a pre-configured template to reduce setup time and ensure best practices.
  2. Test Thoroughly: Validate your transformation logic in Jupyter before deploying it to Nekt.
  3. Iterate and Refine: Start simple and scale complexity as your data needs grow.
  4. Monitor and Optimize: Use logs and performance metrics in Nekt to identify bottlenecks and optimize resource usage.

​
Need Help?

If you encounter challenges during the transformation process, our team is ready to assist. Reach out through our support channels for help at any stage.

OverviewAdd Transformation