QQsemaphors™ is a materialized concept of modularized Qlik™ processing, which allows quick activation and deactivation of processing modules in order to increase the speed of testing and (modular) processing of Qlik™ processing scripts.
There will be 2 independent sets, 1 for manually ordered processing from the interface and one for automatic, programmatically run processing and a switching mechanism between the 2 variants.
Below, we present possible examples of the interface.
The entire data processing package is separated into parts and modules, calling native script but also various other QQtools components (SIRQ™, QQvalidator™, QQslicer™, QQchapters™, etc.), depending on the specific processing needs.
We combine this concept with an approach (at least temporarily, during development) whereby each module or package of modules saves intermediate data in QVD format
(QVD is a Qlik™ format that is extremely efficient in terms of occupied space but also in terms of processing speed, providing a factor of 10x improvement).
Thus, the results of one module or bundle of modules can be taken over by subsequent modules without requiring data to be reprocessed by previous modules in order to have the data available in memory-resident form.
In addition, the non-volatile QVD format allows quick scanning and/ or additional testing of continuous intermediate data to fully assess the quality of the processing to date and the exact data content created.
Prior to using QQsemaphors™, Qlik™ script developers used script pages and/ or QVS modules included in the script, but with a lack of immediate visibility of the enable-disable status of each script part can quite often create confusion.
The Qlik™ interface of QQsemaphors™ allows, on the one hand, the immediate visualization of the whole set of states and, on the other hand, the immediate switching on/ off of each module or package of modules, thus avoiding excessive processing of irrelevant parts of the algorithm at the time.
The QQsemaphors™ concept is extremely useful especially for complex data processing in Qlik™, when the whole processing takes a long time and retesting the impact of a change can become too time-consuming, leading to inefficient consumption of the developer’s time. But, the benefits can also become relevant in fast but multi-structured and frequent processing (e.g. Qsaft™) where, although the complete processing may only take a few minutes, but the number of tests and reloads is relatively large, the time savings can become significant percentages of the total tuning and testing time.
For information about Qlik™, please visit this site: qlik.com.
For specific and specialized solutions from QQinfo, please visit this page: QQsolutions.
In order to be in touch with the latest news in the field, unique solutions explained, but also with our personal perspectives regarding the world of management, data and analytics, we recommend the QQblog !