- Contact us!
- +31 499 577 562
- info@qosqo.nl
Lineage visualisation
June 8, 2020
June 2020
Model-to-model transformation
QUIPU4 is a platform that supports model-to-model transformation. This is how the product sets itself apart from other commonly accepted ETL tools and other data warehouse automation tools. Although such tools usually work well as far as the transformation and manipulation of data at the entity or attribute level is concerned, they lack the support needed to define and develop transformations at the model level. It is at this higher level that patterns can be found and applied, and therefore can be generated. While at the same time, exceptions to these patterns can still be specified at the entity or attribute level. According to QOSQO, this results in a much more efficient way of developing new data solutions quickly.Model changes
Data models evolve in every environment. Dealing with changes to these structures requires a balance between overview and detail. QUIPU4 provides you with the Model Delta Detector module that can distil these changes and then present them in the required format. The module and its output enable you to handle these changes in an efficient and structured way.Model Delta Detector module
Model Delta Detector uses two versions of the same data model. The module uses a referenced data model (often the previous version of the data model) as its basis. Subsequently, the compared data model (often the current version of the data model) is retrieved from the repository. Next, these versions are compared to one another. Incidentally, it is possible to time travel and retrieve every version of the data model and compare them to every other version of that data model. An example of this is displayed in the blue rectangle below.
The module detects changes at the entity level, including constraints and relations, and at the attribute level, including the places where these attributes are used in constraints and relations.
Generate output
All changes are recorded in a single dataset. You can use a template to convert this dataset into any format you desire. To this end, QUIPU4 provides two basic templates (HTML and JSON), but it is possible to customise these templates or create new ones. The blue rectangle below shows the option for selecting a random template.
Generating output starts by executing the Model Delta Detector module.
HTML output
The HTML template output is an overview that can be placed on an intranet or sent to analysts or users. The output begins by counting all changes and naming the compared versions of the data model (Summary menu option).
The Entities menu option provides a summary of the changes at the entity level. Each change is categorised in:
- added: exists in the compared data model, but not in the referenced data model
- changed: exists in both data models, but at least one property has been changed (e.g. the data type of an attribute or the position of an attribute in an index)
- removed: does not exist in the compared data model, but does exist in the referenced data model
- added: exists in the compared data model, but not in the referenced data model
- changed: exists in both data models, but at least one property has been changed (e.g. the data type of an attribute or the position of an attribute in an index)
- removed: does not exist in the compared data model, but does exist in the referenced data model
Finally, you can find a detailed list of changes for each entity by selecting the relevant entity from the list. Here too, the changes are categorised in the way described above, but each category also has a colour (added is green, changed is blue and removed is red).
JSON output
The output from the JSON template can be used to automatically process the changes in the data warehouse. The changes from the dataset are provided at both the entity and attribute levels. Although this creates redundancy in the dataset, it makes processing easier because you do not have to search for the related information. “CMP” refers to the compared data model and “REF” to the referenced data model. An example is given below.
