Logs Generation, Signing & Bundling


#1

Hi,

We are logging data in Json format as mentioned in documents shared by DGCA.

Now the place where I need some clarity is how the signing of log files should happen, can we just sign the data with Private Key generated by OpenSSL and self signed certificate and share the public key and signature with the log file?

In document it’s written that the log files should be bundled, so should we just zip all log files?
It’s all written that the bundled file should again be signed, that doesn’t make any sense. If individual log files are signed why we need to again sign the bundled log file.

Is their any other thing that we need to consider while writing, bundling & signing log files.


#2

Hi cks123,

  1. Log bundle needs to be signed with drone’s private key for which DigitalSky already has the public key (from registration).
  2. Bundling/format of log files can be in any format for now provided it is machine readable.
  3. You need to upload both the bundle and the signature to the DigitalSky, mechanics of which are not defined in the standard, but we will shortly provide the API to upload the log files.

#3

Hi Sidhant,

Thanks for your instant reply.

I had few doubts on the points mentioned by you:

  1. Log bundle needs to be signed with drone’s private key for which DigitalSky already has the public key (from registration).
    Query - You mean we need to sign our data with RFM’s RSA Private Key and x509 Certificate ?

  2. Bundling/format of log files can be in any format for now provided it is machine readable.
    Query - Zip will do?
    As mentioned in the DigitalSky document “Digital Sky - Technology Standards v1.0 - 22nd October 2018”, 4.1.4 section Page24 we don’t need to sign the bundle?

  3. You need to upload both the bundle and the signature to the DigitalSky, mechanics of which are not defined in the standard, but we will shortly provide the API to upload the log files.
    -Looking forward for it.

Also can you please list down any other points that we need to consider.


#4

Here is an example of how logging can be done:

  1. We are logging data in exactly same format as mentioned in “Digital Sky - Technology Standards v1.0 - 22nd October 2018” document at Page No. 28.

  2. Following are the events when we log:

    • Invalid TakeOFF
    • TakeOFF
    • Geofence Breach
    • Time Limit Breach
    • Exceeding Flight Frequency Limit
    • Land
  3. New Log file is created for every flight.

  4. Every log file is named as “FlightLog_PA_ID_20190415180000.json”, where timestamp is start timestamp of that particular flight and “PA_ID” is permission artefact id. The timestamp and PA_ID will help us in linking the log file with permission artefact where timestamp should come between artefact time limit.

  5. Whenever we ask for log files from GCS, all logs are signed and bundled together with their signatures and sent to GCS. The process of fetching bundled log files can be automated as described in the below diagram. Also bundling we are using Tar.

Following is a diagram to demonstrate the same:


Test App to validate NPNT implementation
#5

@cks123 “Digital Sky - Technology Standards v1.0 - 22nd October 2018” document at Page No. 28 is a log schema only and not the example log file. If you have already created an example log file can you share it ?


#6

@cks123 Can we explore some other format than JSON for logging because Json document cannot be appended as flight progresses and we can only write it once. We need to maintain a buffer for it and update that buffer as soon as any logging event occurs but in case of any failure in between the flight, we tend to lose everything. My suggestion would be to store in some other format like CSV. Give your suggestions.


#7

@satyam_amu

We were doing csv logging 4 months back in our initial versions, but when we found that DGCA standard expect us to log in JSON we shifted to JSON format.

We were too facing that issue initially but later we found a way to do it. You should dig in more, you will too find a solution.


#8

If you don’t mind, can you share basic details and how do you overcome this problem?


#9

@cks123 and @satyam_amu Few comments here
JSON stores every element in a tree structure. that means you need to keep the elements in a buffer and write the entire tree at once.
There are few problems associated with it. 1) you write the tree more than once you will develop cascaded elements in a file. 2) you need to dynamically allocate buffer, if you breach geofence for a long time and you will have to deal with memory overflow if you are using a small microcontroller. 3) If you lose the buffer mid air you lose the entire data.

The solution I can think is to divide them in to smaller chunks and then store/write respective elements of that tree. But in this case you might need to deviate and make multiple JSON root elements like “takeoff”, “geofence breach”, “land” etc. as root elements. Is that Okay ? @sid.

CSV looks compelling because you can just keep appending data and don’t need to worry about memory constrain. useful for MCUs.