XML File Parsing Plugin
- Katya Raykova
The XML input plugin allows to continuously read XML files within a given folder. The plugin can search for a given pattern in the naming of the files and also can be set to read-only files after a given timestamp of the modification or access date. The plugin is able to remember the timestamp of the last read file and therefore detects new files and changes in already uploaded files.
Configuration location:
Console Application | Service Application |
|---|---|
These configs are only used for the Edge instance Luna.Console, executed directly from the [Edge Main Folder]. Click the "Service Application" tab above to check the location for Edge services. |
You can install several SF Edge service instances onto one device - for example to collect data of several machines from a local data collection device. Each service instance needs it's own set of configurations. When using the Senseforce Edge Service Installer, the default configurations are automatically installed for any service. |
The default DataMappings.xml can be used for most use cases.
XmlFileParsingSettings.xml
The XmlFileParsingSettings.xml defines where to find the XML files to read, how often to look for new files, and which timestamp to use to decide if the file has to be uploaded.
Example configuration
Description |
|
FilePath (line 8) | Defines the path where to look for the XML files to read |
FileNameSearchPattern (line 10) | Regex defining the search pattern for the files |
ReadFilesInSubdirectories (line 12) | Defines if only files in the folder given by "FilePath" or also files in subfolders should be considered |
MinimumTimestampToRead (line 16) | Unix timestamp in milliseconds defining the minimum timestamp of the file to be uploaded. (Special cases: -1 -> reading all files matching the search pattern, 0 -> reading all files modified/accessed after Luna startup) |
LastReadInformationSource (line 18) | Defines if the timestamp of the last modification or access should be used as reference (allowed values: LastWriteDateTime, LastAccessDateTime) |
ReadingIntervalSeconds (line 21) | Polling time in seconds which is used to look for new files. (Per interval 1 file is transmitted) |
CheckIfOpenInAnotherProcess (line 24) | Defines whether to check if the file is open in another process or not. If this is activated the file isn't read until it's no longer used by another process. This prevents reading the file before the modification is completed. |
PersistLastId (line 27) | The elemetens FilePath and FileName define where to store the information of the last read timestamp. |
NameAttribute (line 33) | This setting defines the name of an attribute in the data XML files which is only used for the naming, but not as data itself. (see example below) |
<?xml version="1.0" encoding="utf-8" ?>
<XmlFileParsingConfig>
<!-- Definition of files to read -->
<Files>
<!-- First file criteria definition (does not have to be a single file) -->
<File>
<!-- Definition of the file directory path (withoud file name) -->
<FilePath>C:\My\desired\search\path</FilePath>
<!-- Regex used to identify the file(s) to parse -->
<FileNameSearchPattern>*.xml</FileNameSearchPattern>
<!-- Defines if files in subdirectories of "FilePath" should also be read or not -->
<ReadFilesInSubdirectories>false</ReadFilesInSubdirectories>
<!-- Define oldest date of files to read -->
<!-- If set to -1, any file is read. If set to 0, all files since Luna startup are read. Otherwise it is interpreted as Unix timestamp in milliseconds -->
<MinimumTimestampToRead>-1</MinimumTimestampToRead>
<!-- Define which timestampl should be used for "MinimumTimestampToRead" condition (Allowed values: LastWriteDateTime, LastAccessDateTime) -->
<LastReadInformationSource>LastWriteDateTime</LastReadInformationSource>
<!-- Time interval for polling for new files or lines in files (in seconds) -->
<ReadingIntervalSeconds>2</ReadingIntervalSeconds>
<!-- Defines whether to check if the file is open in another process -->
<CheckIfOpenInAnotherProcess>true</CheckIfOpenInAnotherProcess>
<!-- Defines the location of the file where information of last read files and lines in files are stored -->
<PersistLastId>
<FilePath>$(LunaAppDataPath)configurations/InputPlugins/XMLFileParsing</FilePath>
<FileName>PersistTime.xml</FileName>
</PersistLastId>
<!-- Defines which XML attribute contains the name of the variable -->
<NameAttribute></NameAttribute>
</File>
<!-- Add several more instances of File here... -->
</Files>
<!--Specify file for message mapping-->
<Mappings>
<FilePath>$(LunaAppDataPath)configurations/InputPlugins/XmlFileParsing</FilePath>
<FileName>DataMappings.xml</FileName>
</Mappings>
<!--Specify file for logging-->
<Logger>
<FilePath>$(LunaAppDataPath)Logs</FilePath>
<FileName>XmlFileParsing.log</FileName>
<!--LogLevel: Debug,Information,Warning,Error,Critical,None-->
<LogLevel>Information</LogLevel>
</Logger>
</XmlFileParsingConfig>
In the Files section, multiple File entries can be defined.
This plugin preserves the hierarchy information by concatenating the node names of all levels of hierarchy and attribute names separated by an underscore ("_"). So for example, if we would use the configuration file shown above the plugin reads the FilePath setting in line 7 as:
"XmlFileParsingConfig_Files_File_FilePath":"C:\My\desired\search\path"
Underscores in node names or attribute names are allowed to use, but has to be considered that they are also included in the variable name e.g. in the EventDefintion.
Example to "NameAttribute" setting
This setting defines the name of an attribute in the data XML files which is only used for the naming, but not as data itself. To understand what this means consider the following very simple XML file as input data:
<Root>
</Node machine="ASX1000" speed=100>
</Root> "NameAttribute" setting not assigned or empty:
If no value is assigned to the setting "NameAttribute" or if this setting isn't defined at all the XML file example above would be read as:
"Root_Node_machine":"ASX1000"
"Root_Node_speed":"100"So in the EventDefinition.xml the variables "Root_Node_machine" and "Root_Node_speed" can be used as Data Value. For more information on that see Edge Data Routing (EventDefinition.xml).
"NameAttribute" set to "machine":
If we set <NameAttribute>machine</NameAttribute> in the XMLFileParsing settings the example file above would be read as:
"Root_Node_ASX1000_speed":"100"So in the EventDefinition.xml the variable "Root_Node_ASX1000_speed" can be used as a Data Value. For more information on that see Edge Data Routing (EventDefinition.xml).