How to change the default behavior to store information for each object in a different table

I want to set up Orion and Cygnus to store all data in one table.

I know that I need to customize data and table names based on HTTP headers, for example:



I was told in this post to ask a different question.


source to share

1 answer

Cygnus operating notification headers fiware-service

and fiware-servicePath

to compile the names of the various elements of the backend. In particular:

  • MySQL
    • Database
    • are called <fiware-service>

    • table names are called <fiware-servicePath>_<destination>

  • HDFS
    • HDFS paths are created as /user/<your_user>/<fiware-service>/<fiware-servicePath>/<destination>/<destination>.txt

  • CKAN
    • organizations are called <fiware-service>

    • packages / datasets are called <fiware-servicePath>

    • are called <destination>

The default <destination>

is <entityId>_<entityType>

. This can lead, as described in the question, to create a MySQL / HDFS / CKAN resource table for each object being notified.

This default assignment creation can be overridden using Cygnus's advanced grouping based on templates; as the name suggests, this feature relies on looking for (customized) patterns in the data to group the contextual data showing the pattern. This function allows, for example, all objects of a certain type to be stored in one MySQL table; or some objects starting with a prefix are stored together in the HDFS file.

To activate this feature, edit the file /usr/cygnus/conf/matching_table.conf

and add as many matching rules as you need; the syntax of the rule syntax is described here . Basically, the rules say, "After validating the pattern, use this new <destination>

and this new <fiware-servicePath>




So, "store all data in a MySQL table called" my_unique_table "would look like this:






Both rules are valid because MySQL table names are created, as already mentioned, by concatenating <fiware-servicePath>

and <destination>

; in this case, the table name is "my_" + _ + "unique_table" or "my_unique" + "_ table".



All Articles