Qlik Talend Cloud just got a boost with two exciting new features! First, is the introduction of Cross Project Pipelines, enabling modular design, increased reusability, and better alignment with data mesh principles. Second, the AI Processor in Qlik Talend Cloud now supports Snowflake Cortex AI functions.
The new Qlik Talend Cloud features we’ve chosen to highlight are two of the capabilities that have been tested over the last few weeks.
Cross Project Pipelines
Since its inception, Qlik Talend Cloud pipelines have offered straightforward design metaphor. Often, you’d create a pipeline for a single data source that continually landed, merged and transformed data changes into a single target, such as a cloud data warehouse or lake. As time progressed the ability to add multiple data sources to a pipeline was introduced, and dedicated replication tasks with multiple targets followed a short time later.
Qlik Talend Cloud Data Pipelines
However, many customers gave feedback that they’d like pipelines to be more modular, especially as projects became bigger and more complex. Modularity would not only increase component reusability, but also enable pipelines to be segregated by business domain. In addition, pipeline development would be more flexible while adhering to the best data-design practices.
Cross Project Pipelines are now generally available in all tenants. You can split complex pipelines consisting of multiple ingestion and transformation tasks into components that can be reused by other projects providing greater design flexibility and simplified pipeline management. In addition, Cross Project Pipelines can be segregated by data domain to encourage Business Domain Data Product or Data Mesh design principles.
AI Processor Snowflake Support
At the end of 2024, Qlik released an AI processor that allowed you call native Databricks AI functions in a Transformation Flow without the need to hand code SQL. Databricks AI functions are a set of built-in SQL functions that allow you to apply AI directly to your data within SQL queries. This means you can use powerful AI models for tasks like sentiment analysis, text generation, and more, all from your Qlik Talend Cloud pipelines.
While many of Qlik Databricks customers were overjoyed, the Snowflake proponents felt very left out, regularly commenting that Snowflake Cortex offered similar features too. Those comments were frequently followed by the question of “When will Qlik’s AI processor support Snowflake too?” and here it is, users have been heard, and now the AI processor also supports Snowflake Cortex AI functions!
In the screenshot below you can see the AI processor in action and follow the link for more information about Snowflake Cortex LLM functions.
For information about Qlik™, click here: qlik.com.
For specific and specialized solutions from QQinfo, click here: QQsolutions.
In order to be in touch with the latest news in the field, unique solutions explained, but also with our personal perspectives regarding the world of management, data and analytics, click here: QQblog !