diagram_canvas_413
A QUIPU Dataflow can be composed visually on the Diagram. A Diagram shows Models (circle symbol), Data Containers (database or file symbols) or Modules (rounded square symbols). The Toolbox in the panel on the right shows all available Modules, whereas the Explorer on the left gives an overview of all configured components (Models, Modules, Data Containers) that exist on all Diagrams.

Flexibility by design

In QUIPU you define flows to interact with datamodels. Each interaction is managed via a variety of modules. The process usually starts with importing model metadata Modules are the flexible components (represented by the blue squares on the diagram) that each define and enable a specific function. All modules are fully configurable.

and more!

Other distinguishable features of the QUIPU framework

Visual data flow design


The canvas displays a graphical visualisation of a user diagram, where various modules are placed, configured and linked to another defining the desired data flow. The toolbox shows all configurations available in the QUIPU4 repository, with both the explorer and toolbox supporting drag and drop functionality. In the explorer overview, all configured objects such as data models and modules are categorised and presented including versioning information. The diagram overview is the starting point to create a new data flow and also gives an overview of all existing data flow diagrams per selected environment.

Visual data lineage


Having insight in data provenance, or also called lineage of data, is a must-have for modern companies. Lineage is key when it comes to providing trust, authenticity and credibility of data, especially when data is being processed in a flow.  QUIPU makes data lineage visible on table level as well as on the level of attributes. Pick any table from a model to start with, and expand the lineage both up- and downstream. 

Multiple environments


In many organisations, code lives not only in the production envrionment but also in one or more other ones such as development, test or (user) acceptance. For data solutions we usually see differences between environments in connections to source systems, to other databases and with other credentials. Moreover, the sources themselves can also differ in structure per environment, impacting the data logistics flow further down the line.  In QUIPU you can now specify one or more enviroments, where each environment has its own coloring in the application to clearly identify each environment. Our powerfull global properties can be now defined per environment. Copying a dataflow to another environment automatically promotes the full data flow with all its objects to another environment and applies the corresponding global properties accordingly,  Any generated code matches the environment specific settings.

JSON editor


QUIPU uses JSON for the configuration of its modules. JSON is a de-facto standard on the internet for exchange of data between web services. The Editor tabs in several Property panels in QUIPU use a JSON Editor.

Scripting


A powerful feature of QUIPU is the option to execute most of its functions directly via the REST API. This opens the possibility to script QUIPU actions and expand the QUIPU capabilities beyond what we can offer. A very useful scenario is where you build your solution by configuring modules and stringing them together using the powerful graphical user interface of QUIPU, test it and then create a script to automatically execute all the steps specified on the diagram for future runs when changes come to your environment. Automating as much manual work as possible, since changes will happen.

REST API


QUIPU functions are exposed in web services that can be accessed via a REST API QUIPU uses the standard web REST API mechanism to expose its services. The QUIPU front end uses this API to offer you a user friendly application to use the QUIPU system, but it is this same mechanism that allows scripts to address QUIPU services directly. We will make a detailed description of the API available in the near future. For now, use the built-in script generator that will create a script for any diagram you have drawn as a quick start for your own scripts.
Login / Register