Risk matrix automating data solutionsMarch 26, 2020
Model delta detection and reportingJune 26, 2020
QUIPU4 is a platform that supports model-to-model transformation. This is how the product sets itself apart from other commonly accepted ETL tools and other data warehouse automation tools. Although such tools usually work well as far as the transformation and manipulation of data at the entity or attribute level is concerned, they lack the support needed to define and develop transformations at the model level. It is at this higher level that patterns can be found and applied, and therefore can be generated. While at the same time, exceptions to these patterns can still be specified at the entity or attribute level. According to QOSQO, this results in a much more efficient way of developing new data solutions quickly.
Lineage is a hot topic in the data warehouse world and is regarded as an important tool. It shows where data comes from, what happens to it and where it flows to over time. Lineage is mainly used to identify wrong processing or execute an impact analysis when the data warehouse has to process changes to data structures.
QUIPU4’s lineage visualisation supports analysis at both the entity and attribute level.
Using lineage visualisation
Lineage visualisation starts by selecting a data model. Once you’ve selected a data model, a list of entities is displayed (see the example below).
You can drag entities from this list onto the canvas. This opens a functionality that allows you to search for preceding or succeeding entities which data comes from or flows to, respectively. Hover above an attribute to view the lineage. In the example below, you can see the attribute Name.
In this case, the metadata shows the use of an expression that indicates along what path the ETL was executed. In QUIPU4, an expression is the option of adding code manually (e.g. adding calculations, calling up functions, or even executing external programs with parameters). In this case, the expression is: DtExec.exe 'D:\DataWarehouse\SSIS\LoadDataSet.dtsx' 'EmployeeDepartmentHistory', for the experts, the execution of an SQL Server Integration Services package.
The Description adds a summarised description, handy for analysis purposes to quickly understand the main points of this data processing.
In some instances, you might like to view the details of an entity or attribute. By double clicking the entity, the data model opens and the attributes, relations and constraints of this entity are displayed.
The example below shows the Shift entity.
The use of lineage visualisation combined with adding extra metadata and documentation to the data model, makes it possible to perform fast and clear analyses in QUIPU4.
Get in touch!
Contact us if you have any feedback, if you would like to try QUIPU4 or if you would simply like to talk about QUIPU4! You can reach us via firstname.lastname@example.org
or our website https://www.quipu.nl
We look forward to hearing from you!