Clever Geek Handbook

How to change the default behavior to store information for each object in a different table

I want to set up Orion and Cygnus to store all data in one table.

I know that I need to customize data and table names based on HTTP headers, for example:

dbName=<fiware-service-header>
tableName=<fiware-servicePath-header>_<entityId>_<entityType>

I was told in this post to ask a different question.

+3


source to share


1 answer


Cygnus operating notification headers fiware-service

and fiware-servicePath

to compile the names of the various elements of the backend. In particular:

  • MySQL
    • Database
    • are called <fiware-service>

    • table names are called <fiware-servicePath>_<destination>

  • HDFS
    • HDFS paths are created as /user/<your_user>/<fiware-service>/<fiware-servicePath>/<destination>/<destination>.txt

  • CKAN
    • organizations are called <fiware-service>

    • packages / datasets are called <fiware-servicePath>

      Resources
    • are called <destination>

The default <destination>

is <entityId>_<entityType>

. This can lead, as described in the question, to create a MySQL / HDFS / CKAN resource table for each object being notified.

This default assignment creation can be overridden using Cygnus's advanced grouping based on templates; as the name suggests, this feature relies on looking for (customized) patterns in the data to group the contextual data showing the pattern. This function allows, for example, all objects of a certain type to be stored in one MySQL table; or some objects starting with a prefix are stored together in the HDFS file.

To activate this feature, edit the file /usr/cygnus/conf/matching_table.conf

and add as many matching rules as you need; the syntax of the rule syntax is described here . Basically, the rules say, "After validating the pattern, use this new <destination>

and this new <fiware-servicePath>

":

<rule_id>|<list_of_fields_to_be_compared>|<regular_expresion>|<new_destination>|<new_fiware-servicePath>


So, "store all data in a MySQL table called" my_unique_table "would look like this:

<any_unique_number>|<entityId>|.*|unique_table|my_

or

<any_unique_number>|<entityId>|.*|_table|my_unique

Both rules are valid because MySQL table names are created, as already mentioned, by concatenating <fiware-servicePath>

and <destination>

; in this case, the table name is "my_" + _ + "unique_table" or "my_unique" + "_ table".

+1


source






More articles:


All Articles