Feature #226

Recursive UBF field support in UBF

Added by Madars over 6 years ago. Updated about 2 years ago.

Status:ClosedStart date:09/30/2017
Priority:Normal (Code 4)Due date:
Assignee:Einars% Done:

100%

Category:-
Target version:-

Description

We shall design support for embedded UBF field support. And design some kind of API to operate with embedded buffers.

for example:

CBEget(p_ub, {C_BUF1, 2, C_BUF2_UNDEF_BUF1, 1, C_FIELD3, BADFLD}, occ, "HELLO WORLD", len, BFLD_STRING)

Here the path to embedded data would be set through the array of fields:
C_BUF1 -> embedded buffer 1 in the p_ub, occurrence 2
C_BUF2 -> embedded buffer in the C_BUF1, occurrence 1

C_FIELD3 ->String data field in C_BUF2

The string path then would look like: "C_BUF[2].C_BUF2_UNDEF_BUF1[1].C_FIELD3" this would identify the exact field in embedded buffers.

The recursive support shall be made in bridge and boolean expressions too.

This would allow to create the structural mappings of the JSON with embedded objects of multi levels

For example form above, the JSON could look like

{
    "C_BUF1":
        [
                {},
        {},
        {
            "C_BUF2_UNDEF_BUF1": 
                        [
                                {},
                {
                    "C_FIELD3": "HELLO WORLD" 
                }
            ]
        }
    ]
}

The other option would be to keep embeded objects as pointer the parent buffer. But here we have a question about serialization and deseralization. This would also allow faster access to the objects and less memory will have to be moved in case of setting the value.

For serialization we could use new format where we convert the real pointer to internal data block pointers, starting from 1. The structure could be the TLV based.

History

#1 Updated by Madars over 6 years ago

  • Description updated (diff)

#2 Updated by Madars over 6 years ago

  • Description updated (diff)

#3 Updated by Madars over 6 years ago

  • Description updated (diff)

#4 Updated by Madars over 6 years ago

  • Description updated (diff)

#5 Updated by Madars over 6 years ago

  • Description updated (diff)

#6 Updated by Madars over 6 years ago

  • Description updated (diff)

#7 Updated by Madars over 6 years ago

Access to function could be made in following way:


BEchg( (int []){ C_OBJ1, 2, C_OBJ2, 4, C_FIELD } );

in string: "C_OBJ1[2].C_OBJ2[4].C_FIELD" 

#8 Updated by Madars over 6 years ago

Google protocol buffer mapping shall be supported out of the box.
Probably we will need different namespaces for UBF buffers, so that we could load "virtual" field tables from the protobuf.
We need to think the mechanism, how to assign the base numbers for different message types.

#10 Updated by Madars over 6 years ago

New buffer formats:

- "UBF2" - recursive UBF. This shall support PBU3 as sub-field.
- "PBU3"- ProtoBuf UBF 3, the same as UBF2 only instead of field tables use protobuf files. In Exfields we need one field which identifies the Protobuf message name. for example EX_PB_MESSAGE. Enduro/X internal transport would be UBF based. We need API call which would generate PBUF blob and call which could parse PBUF into PBU3.
- PB3 - pure binary protobuf 3 buffer.

PBU3 - would require .proto file loader/flex/bison based which would build the AST. Each message name shall be hashed. Each field in struct shall be hashed too.

We will not provide compatibility with PB2.

We will need a cross functions for: PB3->PBU3. PUB3->PB3, PBU3->JSON, JSON->PBU3. (and macro function PB3->JSON, JSON->PB3)

#11 Updated by Madars about 6 years ago

shall be implemented in libubf. As we will need to allocate inner buffers, default allocators will be NDRX_MALLOC, NDRX_REALLOC, NDRX_FREE.
When libatmi will be initalised, allozators must be switched to tpalloc, tprealloc, tpfree

#13 Updated by Madars over 5 years ago

  • Assignee set to Einars

#14 Updated by Madars almost 4 years ago

We could implement also UBF field type PROTOBUF.

#15 Updated by Lauris about 2 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

Release notes

Recursive UBF buffer support has been added. Introduced new UBF types: BFLD_UBF, BFLD_PTR, BFLD_VIEW.
Added set of recursive access functions, such as Bgetrv(), see man page for full list.

Available from Enduro/X release 8.0.1+.

#16 Updated by Lauris about 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF