- Generated on Tue Feb 22 2022 01:34:36 for ArnLib by
1.8.14
|
ArnLib
4.0.x
Active Registry Network
|
This document describes the general concepts of the ArnLib.
All objects are stored in a tree hierarchy and the naming is similar to typical file systems, e.g. "//Measure/Water/Temperature/value".
To get a handle to a folder, use a path ending with "/", e.g "//Measure/Water/".
Folder names can be empty. In the above example, the first level folder is empty and the second level folder is "Measure". The empty folder name can also be referred as "@". Again, the example can equally be written "/@/Measure/Water/Temperature/value". This "@" is typically used when an empty name is unacceptable, e.g. in the tree viewer of the ArnBrowser tool.
A relative path is also called the local path, e.g. "Sys/Discover/This/Service/value".
Each part in a given path is dynamically added as needed, i.e. any path can be used without explicitly creating each folder in advance.
To access an ARN Data Object one can use ArnM::setValue() and ArnM::valueInt() etc. This is a polled access, and gives no signals / events for changed objects. Also this method is rather slow as it has to locate the object via a path lookup. However its good for application assign object "once".
For continous access to an ARN Data Object its better to use an ArnItem. This will be a handle to the object that give fast access. It will also provide signals for changed object. ArnItem is QObject based and has its charateristics.
Yet another way to access an ARN Data Object, is an ArnBasicItem. This will give a basic handle to the object. It is fast, small and is not based on QObject. As such it can not use signals and slots, but it can provide ArnEvents.
Normally ArnItem should be used, as it has a higher level interface with QObject signals and slots. Typically ArnBasicItem is used when no signal is needed, i.e only using direct access with setValue and toXXX methods. If you need a lot of ArnBasicItems and memory foot print (or speed) is important, You can consider to use ArnBasicItem with ArnEvents even if it will be harder to code.
You can expect ArnBasicItem to be lees than a third of the size of an ArnItem. Tests has shown ArnBasicItem to take half the time assigning an integer, compared to ArnItem.
Mode change is a one direction process. Once a specific mode is set, it can't be reset.
If the ArnItem is in a closed state when the mode change is done, the added modes will be stored and the real mode change is done when the ArnItem is opened to an ARN Data Object.
If the general mode change is done to a shared object, the change of general mode is also done at the server and any connected clients.
The following general modes are available:
Additionally there are some sync modes. These modes are used by the local client session and are not shared with others. The sync modes must be set before the ArnItem is opened to an ARN Data Object.
Following sync_modes are available:
Note: It's convenient to always set all the needed modes before an ArnItem is opened or an ArnItem is used as a template. See ArnItem::setTemplate().
A relative path is also called the local path, e.g. the Discover remote service name at path "Sys/Discover/This/Service/value". The local path is mapped to the absolute path "/Local/". The example is then equal to "/Local/Sys/Discover/This/Service/value". The local path should not be shared as it will contain specific data for its running program.
The exception to not sharing local path is for some kind of remote client that must be able to change an ARN Data Object in the local path at the remoted target. For example this is used to change the Discover remote service name for a target host.
Note: Do always mount the local path of the server at a different path at the client. This is to avoid collision with the client's own local path data.
In the above example, a remote client using ArnClient::addMountPoint("/@HostLocal/", "/Local/") will share and access the Discover remote service name at the path "/@HostLocal/Sys/Discover/This/Service/value".
These rules must not be obeyed, but are recommended, to get the most benefits of the Arn echo system, like the ArnBrowser tool.
A bidirectional ARN Data Object is actually a double object, a twin. Each part has its own path but their life span is depending on each other.
One part is the normal "official" and the other part is provider. The provider has an added "!" to the normal path, e.g. normal = "//Measure/Depth/value", provider = "//Measure/Depth/value!".
Data written to one part ends up in the other. This can be compared to crossing electrical lines from one unit to another unit regarding transmit and receive signals in each unit. When a provider slot is connected to the provider part (ArnItem), the slot will receive "request" data from the normal part. The provider slot processes the request data and writes the result to the same provider part. This way the result will end up in the normal "official" part.
This functionality can typically be used for data validation and limiting.
The crossing property of BiDir can be supressed by using ArnItem::setUncrossed(). Again this can be compared to uncrossed electrical lines from a unit to a "communicator" (modem, switch, hub ...) regarding transmit and receive signals. Not supprisingly this is usually a easier mode when making some kind of Bridge for ARN.
Pipes also use the bidirectional functionality. The two (twin) parts are then named requester and provider.
All data put into a pipe are part of a stream and as such will be fully transfered (synchronized) if they are shared with a server and other clients.
ArnPipe is a specialized class for handling pipes.
It contains logic for handling sequence check and anti congest.
Data stream to and from a pipe can be controlled using ArnItemValve class. Actually ArnItemValve can controll any ArnItemB derived class.
Sequence check is used to make sure everything is received and nothing is lost or comes twice. This might happen when a tcp/ip connection goes up and down.
The sequence check uses a hidden sequence number not visible in the pipe stream. The sequence number is increased for each assignment to the pipe. The sending and checking of this sequence number is activated at each end of the pipe.
When checking is activated and the received sequence number is unexpected, a signal will be generated.
See also ArnPipe::setSendSeq(), ArnPipe::setCheckSeq(), ArnPipe::outOfSequence().
When the pipe is a shared oject, all assignment to the pipe is queued up in a send queue. If there is a disconnect in the tcp/ip, an ArnServer will drop the send queue. But in an ArnClient, this send queue will grow out of control if assignments to the pipe keeps coming. This problem can also arise with a fast rate of status messages on a slow network.
One possibility is to keep track of the connection status, but this involves knowing about which ArnClient (if many) to get status from. It also doesn't handle the problem with a slow network.
A probably better way is to use the Pipe anti congest logic.
We identify messages that can be sent any number of times and are used to check the data flow, resending, status and alike. Typically this can be Heart beat, ping, request update, current time etc. These async messages are assigned using ArnPipe::setValueOverwrite().
A regular expression is needed to identify "equal" async messages, that can be overwritten in the send queue. If async messages are repeatedly assigned to a pipe by ArnPipe::setValueOverwrite(), the send queue will then not grow.
All other messages will be normally assigned to the pipe. But these messages will only be assigned when normal data flow is present. Typically there is some expected feedback message from the receiving part to block uncontrolled assignment from one side of the pipe.
The server must use ArnPersist to support the persistance service. As a standard persist storage, ARN Data Objects are stored in a SQLite database. It's also possible to store each object as a file.
The mount point (path) for collecting the persistent ARN Data objects is set by ArnPersist::setMountPoint(). For server applications this is typically set to "/", which makes all ARN Data Objects potential persistent. In client applications the mount point is typically restricted to Arn::pathLocal, which only saves local ARN Data Objects in the local persist storage.
Any connected client or the server can make an ARN Data Object persistent. Just open an ArnItem to the object and change mode to Save.
When the ARN Data Object is set to Save mode, it's automatically loaded by the ArnPersist. At the server this is instantly done. A client has to wait for the value to get synced from the server. It's convenient to use ArnDepend to get a signal when the value is loaded and ready to use.
When the ARN Data Object is changed, it will be automatically saved by ArnPersist. There is a delay from first change of the object until the saving is done, see ArnItem::setDelay(). This allows for intensive updates of the object without choking down the server with saving operations.
It's possible to mark an object in the SQLite data base as mandatory. In this way the ARN Data Object is set as persistent and gets loaded at start of ArnPersist.
To use the persistent storing of ARN Data Objects in files, the root directory is set by: ArnPersist::setPersistDir(). This can also be combined with support of VCS (version control system). See ArnPersist::setVcs(). Currently there is a support module for git.
In the root directory and below, all (VCS) persistent files are stored. The root directory corresponds to the root in Arn tree.
Example: root directory is set to "/usr/local/arn_persist". There is a file stored at "/usr/local/arn_persist/@/doc/help.xhtml". This file will be mapped to Arn at "//doc/help.xhtml".
Any files stored in the root directory and below, get loaded into their ARN Data Object with mode set as persistent at start of ArnPersist.
The files get updated in a similar way to the data base update.
A fundamental aspect of Arn is that ARN Data Objects can be shared. This is centralized to the ARN Server, which stores all shared objects. It's still a distributed model as each client and server has their own set of ARN Data Objects that operate independent of any connection.
Each ARN Client connects to the ARN Server and decides which part of the ARN Data Object tree to be shared.
ArnClient::addMountPoint("/Share/") will make the tree "/Share/" shared.
This doesn't mean that everything in the shared tree at the server now will be available at the client. The client has to create an ARN Data Object in the shared tree. The client can then decide the exact objects of interest.
ArnItem::Open("/Share/Test/value") will open a shared object in previous example.
Note: Normally "//" or "/@.../" is used for shared. See naming conventions.
The remote tree can be at a different path than the local tree (mount point).
An ArnServer can be created with port set to 0. This will be handled as a dynamic port and the system will assign a free port number to the server. The port number will be taken from a range specified by IANA.
This can typically be used to skip configuring static port numbers and be able to have multiple instanses of the ArnServer on the same machine. As an ArnClient must find its ArnServer, this can be used together with ArnDiscoverRemote / ArnDiscover.
Syncing between client and server is normally handled automatically, but for special needs and reference this chapter gives an idea of the rules. Also this describes the rules when connection is established. After that, normal syncing is done almost symetrically between client and server.
An ARN Data Object with Master Mode is used as default generator of data. Normally the server is the default generator of data. This makes difference when client connects or reconnects to the server. The data from the default generator is then used and synced.
Also to have minimal data exchange when using non BiDirectional ARN Data Object, one should take Master mode into consideration. This is more important for big objects.
When a Null value is synced, the receiver store this as an empty value, i.e. it't not stored as Null which is impossible.
ClientSyncMode can be set with ArnClient::setSyncMode(). Basically this controls if a client ARN Data Object is considered as a Master object (se also Modes ).
ClientSyncMode doesn't affect a pipe. Default mode is StdAutoMaster.
ArnRpc is the basic functionality of RPC (Remote Procedure Call). ArnSapi implements SAPI (Service Application Programming Interface) and is using ArnRpc as its base. It's recommended to use ArnSapi which has a higher level model.
The SAPI works by a model which can be described as RPC by remote signal slots. The provider is usually assumed to wait for a requester to initiate the session and then react to different remote calls from the requester. However, this is full duplex, so any side can make a remote call at any time.
A good example of the usage of SAPI is the "Arn Demo Chat", which is included in the source package of the ArnLib.
ArnRpc uses pipes to communicate. The pipes can be monitored and receive test stimuli from the "Arn Browser" program. The used protocol is XString based and quite easy to handtype when common data types are used. "$help" will give the syntax for the actual custom SAPI.
A SAPI is setup by deriving the ArnSapi class to a new class that defines the custom SAPI. This custom-declared class is included at both the provider and requester ends. The custom SAPI class by itself doesn't implement any services. It's merely a hub for connections to external signals and slots. The base ArnSapi class automatically transfers all custom signal (SAPI) calls to the remote connected ends, which also have the ArnSapi derived class and that emits the transfered signal. See example in ArnSapi Detailed Description.
The provider connects the signals from custom SAPI that are prefixed with "pv_" (as default) to each external slot that implements the services. In the same way the requester connects the signals prefixed with "rq_" to its external "service" slots.
When there is a naming pattern between the SAPI services and the external signals and slots, it's a great convenience to use ArnRpc::batchConnect(), ArnSapi::batchConnectTo() or ArnSapi::batchConnectFrom(). This saves a lot of QObject::connect() calls. Also newly added services in the SAPI, that obey the naming scheme, will automaically be connected to the newly matching external signals and slots for implementation of the service.
An extended feature comparing to normal signals is that the SAPI signals are public and can be called by non-derived classes. This makes it optional to use both signal to signal connections or direct signal calls (emit), when issuing a RPC to the remote side.
The service slot can get the emitting custom SAPI object by using normal QObject::sender() functionality.
Under the hood Qt converts a signal that uses default argument(s) into methods with same name and all variation of the arguments. I.e. One method with all arguments, one with all but the last default argument, and so on until there is no more default arguments. When emitting the signal with some number of arguments, all of the signal methods will be exited.
ArnRpc has to deal with this default argument mechanism, otherwise there would be multiple calling messages for just one original signal emit.
The problem arises when there also can be normal signals that are overloaded, i.e. using same method name but different arguments. ArnRpc has to be able to differentiate between these normal overloaded signals and the default argument signals described earlier.
These are the alternatives, how you can help ArnRpc make your SAPI work:
f( int a, int b); f( int b); f( int c); f( uint a);The RPC calling has a basic format as XString (see Arn::XStringMap). A call message can have 3 possible argument formats: positional, named and typed. The positional format is always possible to use and is most comparable to a standard c++ call.
The method name always come first in the message. After that comes arguments that have the argument data in the value part of its key/value pair. The key part can have the argument type and name, but this depends on the used argument format.
The following RPC data types are available:
| RPC | Qt |
|---|---|
| int | int |
| uint | uint |
| int64 | qint64 |
| uint64 | quint64 |
| bool | bool |
| float | float |
| double | double |
| bytes | QByteArray |
| date | QDate |
| time | QTime |
| datetime | QDateTime |
| list | QStringList |
| string | QString |
Also generic RPC data types can be formed as:
Textual like QColor t<QColor>
Binary like QPoint tb<QPoint>
Only textual types, i.e. those that can be converted to/from a string, are reasonable to be hand typed.
Lets have an example method to see the message when it is called.
Method: void put( QString id, int value);
Get called by: put("level", 123);
Alternatives in positional argument format:
put t<QString>.id=level t<int>.value=123
put string.id=level int.value=123
put string.=level int.=123
put string=level int=123
put level int=123
Alternatives in named argument format:
put id=level value=123
put value=123 id=level
put value=123 dummy=ABC id=level garbage=321
Alternatives in typed argument format:
put id:t<QString>=level value:t<int>=123
put id:string=level value:int=123
put value:int=123 id:string=level
put value:int=123 dummy:bytes=ABC id:string=level
Named and typed argument format can be mixed, but positional format is never mixed.
List (QStringList) can be used. All examples below will get same resulting call.
For a function: void test( QStringList lst, int num)
test list=red green blu int=3
test list.lst=red green blu int.num=3
test list= +=red +==green +=blu int=3
test list=red +=green blu int=3
test lst:list=red green blu num=3
test num=3 lst:list=red green +=blu
For special cases, like empty elements, the += syntax is needed. The example below has a first empty element followed by "green".
test list= += green blue int=2
The built-in call "$help" will give an automatically generated list of the present SAPI with the syntax for each available service. The default argument format is positional. This can be changed to named format by giving "$help named".
For getting a basic understanding of ZeroConfig and further references to relevant documentation, see: http://zeroconf.org/
ARN ZeroConfig is the lowest level support for advertising and discovering services on a local network. The implementation has very few dependencies to the rest of the ArnLib.
ARN ZeroConfig can use a built in implementation of Apple (R) mDns / DNS_SD that has no further dependencies to external libraries. For mDns the low end system abstraction layer has been written to use Qt for portability. The higher level DNS_SD has wrappers written to give a good c++ / Qt API.
It's also possible to use an external DNS_SD library, like Avahi. This gives better performance when many applications uses ZeroConfig on the same machine, as they share cashing etc with a common daemon. However you have to deal with this external dependency.
ARN ZeroConfig implementation has two parts. The ArnZeroConfRegister can be used to advertise any service given a host address and a port number. The other part is the ArnZeroConfBrowser / ArnZeroConfResolve / ArnZeroConfLookup. The browser is used to get a realtime list of available services on the network. The resolver takes a given service and resolves it into its host name and port number. Finally ArnZeroConfLookup takes a given host name and makes a DNS (mDNS) lookup to get its ip-address. Each of these classes are stand alone and has to be combined with glue logic for the complete process.
A ZeroConfig service has a service type, that preferably should be registered at IANA. Examples of service types are "http", "ftp" and "arn". This type is mandatory when advertising a service. Also the service must have a service name.
Service names can be any human readable id. It should be easy to understand, without any cryptic coding. There should not be any attempts to make the service name unique as this is taken care of by the ZeroConfig system. It's common that the service name can be modified by the end user. The default starting name could be some system or product name. Example of service name: "My House Registry".
Services can also have sub types. These are identifiers that can be used to filter out some sub group from a specific service type. All services having the same service type must still have some common protocol even if they belong to different sub types. A service can be advertised with many sub types, but browsing can only be filtered with one sub type or with no filter.
It's possible to add a text record to a service. The format of this record is specified by IANA. The purpose is to store properties by a key / value -pair. For convenience this can be done with ArnZeroConfRegister::setTxtRecordMap() using an Arn::XStringMap.
ARN Discover is the mid level support for advertising and discovering services on a local network. This implementation is only for the "arn" service type and is heavily dependent on the ArnLib. The "arn" service type is approved and registered by IANA.
ARN Discover implementation has two parts. The ArnDiscoverAdvertise can be used to advertise an Arn service given a host address and a port number. The other part is the ArnDiscoverBrowser / ArnDiscoverResolver. The browser is used to get a realtime list of available Arn services on the network. The resolver is for taking a manual resolve when a service name is known in advance.
ARN Discover is designed to minimize external glue logic as these classes do all the common processing. Internally ARN ZeroConfig is used, but focus is on solving Arn specific needs in a powerful, yet flexible manner.
An ARN service needs an ArnDiscover::Type and a service name. The ArnDiscover::Type sets up a coarse division of the applications into the groups "server" and "client". The "client" typically only offer the service of ArnDiscoverRemote.
ARN services can also have groups. These are identifiers that can be used to filter out some sub group. An ARN service can be advertised with many groups, but browsing can only be filtered with one group or with no filter.
It's possible to add a custom property to an ARN service. This can be done with ArnDiscoverAdvertise::setCustomProperties() using an Arn::XStringMap. The propertie has a key / value -pair. The custom property are advised to have a key starting with a capital letter to avoid name collision with the system. The added groups will be set as properties with naming as "group0", "group1" ...
ArnDiscoverBrowser collects found Arn services. Each of these services can automatically be further examined. This is chosen by calling ArnDiscoverBrowserB::setDefaultStopState(), which e.g. tells examination to stop after host name has been found. The service can then manually be ordered for further examination by ArnDiscoverBrowserB::goTowardState(), e.g. examination should now stop after host ip is found.
All the information about a service is stored in ArnDiscoverInfo. Found services can be accessed by index, id or service name. Increasing index, starting at 0, gives a list of services alfabetically sorted by service name. The index is kind of volatile and should be used instantly, not be stored. The id gives a unique number for each service and can be stored. However the service given by the id might dissapear.
ARN Discover Remote is the highest level support for advertising and discovering services on a local network. Its implementation is based on ARN Discover. The added functionality is to have a remote control for both advertising an ArnServer and multiple ArnClient connections. The remote control is done via ARN Data Objects in local path "Sys/Discover/".
ARN Discover Remote has one main class, ArnDiscoverRemote which act as a central point. The ArnDiscoverRemote class also takes an ArnServer and advertises it as a service. For remote control the service name is available at local path "Sys/Discover/This/Service/value".
ArnDiscoverRemote can make an internal ArnServer, when there is no need to access the ArnServer class. This is usually the case in an client application. The ArnServer is then merely used to make the discover functionality remote controlled.
Remote controlled client connections can be added. Each ArnClient is handled by an ArnDiscoverConnector instance, which is made by ArnDiscoverRemote::newConnector(). Connections can be added to ArnDiscoverConnector, both as a direct host list and a discover host.
The discover host is indirerctly set, by adding an ArnDiscoverResolver to ArnDiscoverConnector. A service name can then be resolved into the discover host.
The two connection methods can coexist and as standard the discover host has lower priority number than direct host, i.e. discover host is tried first.
The ArnDiscoverConnector is associated with an id, which should be chosen to describe the client target or its purpose. It's not a host address or necessarily a specific host, as there can be many possible addresses assigned to the ArnDiscoverConnector.
The id will appear as an ARN folder in local path, e.g. when id is "WeatherData-XYZ" the folder path will be "Sys/Discover/Connect/WeatherData-XYZ/". The folder and its sub folders will contain ARN Data Objects to remote control the ArnClient. For a more comprehensive description of these objects, see help discover description.
In the above example, a discover host can be remote controlled by setting the service name in local path "Sys/Discover/Connect/WeatherData-XYZ/DiscoverHost/Service/value", e.g. to "Region Weather XYZ".
Also in the above example, the first direct host can be remote controlled by setting the host name in local path "Sys/Discover/Connect/WeatherData-XYZ/DirectHosts/Host-0/value", e.g. to "localhost".
Normally it's wanted that any remote set values in the local path remains after power cycling. This is supported by the Arn persist system.
Connecting via resolver uses the logic:
1.8.14