pcap2json
Home

JSON Fields

By default pcap2json uses its builtin JSON field definition. This can however be overridden with a custom JSON format and file using 

--flow-template '{ custom JSON format }'

option in /opt/fmadio/etc/pcap2json.lua Section "backend"

Default JSON format (2021/7/24)

{
\"timestamp\":@TIMESTAMP@,
\"TS\":@TS@,
\"device\":@DEVICE@,
\"hashHalf\":@HASH_HALF@,
\"hashFull\":@HASH_FULL@,
\"flowCount\":@FLOWCNT@,
\"macSrc\":@MACSRC@,
\"macDst\":@MACDST@,
\"macProto\":@MACPROTO@,
\"vlan0\":@VLAN0@,
\"mpls0TC\":@MPLS0TC@,
\"ipv4Src\":@IPV4SRC@,
\"hostSrc\":@SRCIP_HOSTNAME@,
\"ipv4Dst\":@IPV4DST@,
\"hostDst\":@DSTIP_HOSTNAME@,
\"ipv4Proto\":@IPV4PROTO@,
\"ipv4DSCP\":@IPV4DSCP@,
\"ipv4Frag\":@IPV4FRAG@,
\"portSrc\":@PORTSRC@,
\"portDst\":@PORTDST@,
\"application\":@APPLICATION@,
\"tag0\":@TAG0@,
\"tag1\":@TAG1@,
\"tag2\":@TAG2@,
\"tcpFin\":@TCPFIN@,
\"tcpSyn\":@TCPSYN@,
\"tcpSynAck\":@TCPSYNACK@,
\"tcpSackPerm\":@TCPSYNSACK@,
\"tcpRst\":@TCPRST@,
\"tcpSack\":@TCPSACK@,
\"tcpZeroWindow\":@TCPWINZERO@,
\"totalPackets\":@TOTALPKT@,
\"totalBytes\":@TOTALBYTE@,
\"totalBits\":@TOTALBIT@,
\"totalFCS\":@TOTALFCS@,
\"geoipSrc\":
{
    \"location\":@SRCIP_LOCATION@,
    \"country_name\":@SRCIP_COUNTRY@,
    \"country_iso_code\":@SRCIP_COUNTRY_CODE@,
    \"city_name\":@SRCIP_CITY@,
    \"asn\":@SRCIP_ASN@,
    \"org\":@SRCIP_ORG@,
    \"isp\":@SRCIP_ISP@
},
\"geoipDst\":
{
    \"location\":@DSTIP_LOCATION@,
    \"country_name\":@DSTIP_COUNTRY@,
    \"country_iso_code\":@DSTIP_COUNTRY_CODE@,
    \"city_name\":@DSTIP_CITY@,
    \"asn\":@DSTIP_ASN@,
    \"org\":@DSTIP_ORG@,
    \"isp\":@DSTIP_ISP@
},
\"tcpRTTNet\":@TCP_RTT_NET@,
\"tcpRTTApp\":@TCP_RTT_APP@,
\"tcpWindowMin\":@TCP_WINDOW_MIN@,
\"tcpWindowMax\":@TCP_WINDOW_MAX@,
\"tcpWindowMean\":@TCP_WINDOW_MEAN@,
\"decapType\":@DECAP_TYPE@,
\"decapIPv4Src\":@DECAP_IPV4_SRC@,
\"decapIPv4Dst\":@DECAP_IPV4_DST@,
\"decapIpv4Proto\":@DECAP_IPV4_PROTO@,
\"decapPortSrc\":@DECAP_PORT_SRC@,
\"decapPortDst\":@DECAP_PORT_DST@
}"

Example JSON Flow

{
  "timestamp": 1497015593700,
  "TS": "13:39:53.700.000.000",
  "device": "fmadio100v2-228U",
  "hashHalf": "810a2feb5dcdb1f602f913c7de2e90b789e635e9",
  "hashFull": "a7cbb9296d6c642b34ef295a902a9fda8484a178",
  "flowCount": 36365,
  "macSrc": "7c:e2:ca:bd:97:d9",
  "macDst": "00:0e:52:80:00:16",
  "macProto": "IPv4",
  "vlan0": null,
  "mpls0TC": null,
  "ipv4Src": "202.17.220.140",
  "hostSrc": "202.17.220.140",
  "ipv4Dst": "208.67.222.222",
  "hostDst": "208.67.222.222",
  "ipv4Proto": "ICMP",
  "ipv4DSCP": null,
  "ipv4Frag": null,
  "portSrc": null,
  "portDst": null,
  "application": "(ICMP     0)",
  "tag0": null,
  "tag1": null,
  "tag2": null,
  "tcpFin": null,
  "tcpSyn": null,
  "tcpSynAck": null,
  "tcpSackPerm": null,
  "tcpRst": null,
  "tcpSack": null,
  "tcpZeroWindow": null,
  "totalPackets": 1,
  "totalBytes": 74,
  "totalBits": 592,
  "totalFCS": 0,
  "geoipSrc": {
    "location": [
      139.69,
      35.68999
    ],
    "country_name": "Japan",
    "country_iso_code": "JP",
    "city_name": null,
    "asn": null,
    "org": null,
    "isp": "Research Organization of Information and Systems"
  },
  "geoipDst": {
    "location": [
      -74.4774,
      41.45349
    ],
    "country_name": "United States",
    "country_iso_code": "US",
    "city_name": "Middletown",
    "asn": 36692,
    "org": "OPENDNS",
    "isp": "Cisco OpenDNS"
  },
  "tcpRTTNet": null,
  "tcpRTTApp": null,
  "tcpWindowMin": null,
  "tcpWindowMax": null,
  "tcpWindowMean": null,
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632

Reference

The following is a reference for all the fields

@TIMESTAMP@

Outputs timestamp in MSec epoch time

{
  .
  .
  "timestamp": 1497015593700,
  .
  .
}

TCP Connection

PCAP2JSON can calculate RTT and Byte accurate TCP Window information following is a description of each field. TCP Window sizes are real sizes in bytes that are scaled correctly using the SYN/SYNACK negotiated window scaling factor

@TCP_MODE@

This field indicates which side of the tcp connection the Half Duplex flow is on. Its particularly useful when used in combination with tcpRTTNetHalf to monitor the response time of the server or the WAN latency of a client.

tcpMode = server

Server highlighted in Aqua

tcpMode = client

Client highlighted in Aqua

@TCP_RTT_NET_FULL@

Outputs the full TCP round trip time in milliseconds. . This is a full duplex value, both sides of the TCP connection have the same value. This value approximates an ICMP ping between the two hosts.

Calculation is Full Duplex

Unit is Milliseconds

Show below, the aqua colored paths are used in the calculation. Total caluclation is

tcpNetRTTFull = P2 - P0

Example JSON output, note field tcpRTTNetFull, in this example the Full RTT round trip is 856msec.

Both half duplex hash have the exact same tcpRTTNetFull value.

{
  "timestamp": 1497015595700,
  "TS": "13:39:55.700.000.000",
  "device": "fmadio100v2-228U",
  "hashHalf": "4811607328d46ec497c1add34da011649a73b611",
  "hashFull": "d9cd496276ea9c4a4f2f072e4201560239023bd9",
  "flowCount": 44437,
  "macSrc": "84:78:ac:3b:53:a5",
  "macDst": "68:05:ca:0a:b9:15",
  "macProto": "IPv4",
  "vlan0": 2048,
  "ipv4Src": "212.92.115.17",
  "hostSrc": "212.92.115.17",
  "ipv4Dst": "130.128.48.12",
  "hostDst": "130.128.48.12",
  "ipv4Proto": "TCP",
  "portSrc": 61025,
  "portDst": 3389,
  "application": "(TCP  3389)",
  "tcpRst": 1,
  "tcpMode": "client",
  "tcpRTTNetFull": 856.3,
  "tcpRTTNetHalf": 000.2,
  "totalPackets": 5,
  "totalBytes": 1365,
  "totalBits": 10920,
  "totalFCS": 0,
  "geoipSrcLocation": [
    4.89949,
    52.3824
  ],
  "geoipSrcCountry_name": "Netherlands",
  "geoipSrcCountry_iso_code": "NL",
  "geoipSrcAsn": 43350,
  "geoipSrcOrg": "NForce Entertainment B.V.",
  "geoipSrcIsp": "NFOrce Entertainment B.V.",
  "geoipDstLocation": [
    -115.14099,
    36.12139
  ],
  "geoipDstCountry_name": "United States",
  "geoipDstCountry_iso_code": "US",
  "geoipDstCity_name": "Las Vegas"
}

{
  "timestamp": 1497015595700,
  "TS": "13:39:55.700.000.000",
  "device": "fmadio100v2-228U",
  "hashHalf": "adf86ea334da1be65391e97c4a635c2c50bfcc01",
  "hashFull": "d9cd496276ea9c4a4f2f072e4201560239023bd9",
  "flowCount": 44437,
  "macSrc": "68:05:ca:0a:b9:15",
  "macDst": "00:00:5e:00:01:31",
  "macProto": "IPv4",
  "vlan0": 2048,
  "ipv4Src": "130.128.48.12",
  "hostSrc": "130.128.48.12",
  "ipv4Dst": "212.92.115.17",
  "hostDst": "212.92.115.17",
  "ipv4Proto": "TCP",
  "portSrc": 3389,
  "portDst": 61025,
  "application": "(TCP  3389)",
  "tcpMode": "server",
  "tcpRTTNetFull": 856.3,
  "tcpRTTNetPartial": 856.1,
  "totalPackets": 5,
  "totalBytes": 1838,
  "totalBits": 14704,
  "totalFCS": 0,
  "geoipSrcLocation": [
    -115.14099,
    36.12139
  ],
  "geoipSrcCountry_name": "United States",
  "geoipSrcCountry_iso_code": "US",
  "geoipSrcCity_name": "Las Vegas",
  "geoipDstLocation": [
    4.89949,
    52.3824
  ],
  "geoipDstCountry_name": "Netherlands",
  "geoipDstCountry_iso_code": "NL",
  "geoipDstAsn": 43350,
  "geoipDstOrg": "NForce Entertainment B.V.",
  "geoipDstIsp": "NFOrce Entertainment B.V."
}

@TCP_RTT_NET_PARTIAL@

This calculates the partial RTT value. Note depending on where the capture device is located indicates which side of the RTT it shows. This value is used in conjunction with @TCP_MODE@ to monitor Server response time or Client speed of light latency.

tcpMode = server

In this mode tcpRTTNetPartial is the round trip between request to connect to the server, and the services accept() of the connection. Shown below in aqua

tcpRTTNetPartial = P1 - P0

tcpMode = client

In the client mode, tcpRTTNetPartial is (usually) the speed of light network latency between the Server (e.g. HTTP web server) and the Client a desktop / mobile end user. Its not strictly the case, this is the typical use case as FMADIO Capture device is usually co-located with the Web Server.

tcpRTTNetPartial = P2 - P1

Example JSON below. tcpRTTPartial is different for each half duplex flow.

{
  "timestamp": 1497015595700,
  "TS": "13:39:55.700.000.000",
  "device": "fmadio100v2-228U",
  "hashHalf": "4811607328d46ec497c1add34da011649a73b611",
  "hashFull": "d9cd496276ea9c4a4f2f072e4201560239023bd9",
  "flowCount": 44437,
  "macSrc": "84:78:ac:3b:53:a5",
  "macDst": "68:05:ca:0a:b9:15",
  "macProto": "IPv4",
  "vlan0": 2048,
  "ipv4Src": "212.92.115.17",
  "hostSrc": "212.92.115.17",
  "ipv4Dst": "130.128.48.12",
  "hostDst": "130.128.48.12",
  "ipv4Proto": "TCP",
  "portSrc": 61025,
  "portDst": 3389,
  "application": "(TCP  3389)",
  "tcpRst": 1,
  "tcpMode": "client",
  "tcpRTTNetFull": 856.3,
  "tcpRTTNetPartial": 000.2,
  "totalPackets": 5,
  "totalBytes": 1365,
  "totalBits": 10920,
  "totalFCS": 0,
  "geoipSrcLocation": [
    4.89949,
    52.3824
  ],
  "geoipSrcCountry_name": "Netherlands",
  "geoipSrcCountry_iso_code": "NL",
  "geoipSrcAsn": 43350,
  "geoipSrcOrg": "NForce Entertainment B.V.",
  "geoipSrcIsp": "NFOrce Entertainment B.V.",
  "geoipDstLocation": [
    -115.14099,
    36.12139
  ],
  "geoipDstCountry_name": "United States",
  "geoipDstCountry_iso_code": "US",
  "geoipDstCity_name": "Las Vegas"
}

{
  "timestamp": 1497015595700,
  "TS": "13:39:55.700.000.000",
  "device": "fmadio100v2-228U",
  "hashHalf": "adf86ea334da1be65391e97c4a635c2c50bfcc01",
  "hashFull": "d9cd496276ea9c4a4f2f072e4201560239023bd9",
  "flowCount": 44437,
  "macSrc": "68:05:ca:0a:b9:15",
  "macDst": "00:00:5e:00:01:31",
  "macProto": "IPv4",
  "vlan0": 2048,
  "ipv4Src": "130.128.48.12",
  "hostSrc": "130.128.48.12",
  "ipv4Dst": "212.92.115.17",
  "hostDst": "212.92.115.17",
  "ipv4Proto": "TCP",
  "portSrc": 3389,
  "portDst": 61025,
  "application": "(TCP  3389)",
  "tcpMode": "server",
  "tcpRTTNetFull": 856.3,
  "tcpRTTNetPartial": 856.1,
  "totalPackets": 5,
  "totalBytes": 1838,
  "totalBits": 14704,
  "totalFCS": 0,
  "geoipSrcLocation": [
    -115.14099,
    36.12139
  ],
  "geoipSrcCountry_name": "United States",
  "geoipSrcCountry_iso_code": "US",
  "geoipSrcCity_name": "Las Vegas",
  "geoipDstLocation": [
    4.89949,
    52.3824
  ],
  "geoipDstCountry_name": "Netherlands",
  "geoipDstCountry_iso_code": "NL",
  "geoipDstAsn": 43350,
  "geoipDstOrg": "NForce Entertainment B.V.",
  "geoipDstIsp": "NFOrce Entertainment B.V."
}

@TCP_RTT_APP@

Outputs the Application Request -> Response network round trip in milliseconds

Calculation is Half Duplex

Unit is Milliseconds

Time different between between application Push to Push. e.g. HTTP Get -> 200 OK Response

tcpRTTApp = TS1 - TS0

{
  .
  .
  "tcpRTTNet": 51.583,
  "tcpRTTApp": 118.732,
  .
  .
}

@TCP_WINDOW_MIN@

Outputs the minimum TCP window size in Bytes

Calculation is Half Duplex

NOTE: This does take into account full Window Scaling from the SYN/SYNACK connection. If no SYN/SYNCACK has been processed this filed outputs NULL

{
.
.
  "tcpWindowMin": 1,
  "tcpWindowMax": 4739,
  "tcpWindowMean": 89,
.
.
}

@TCP_WINDOW_MAX@

Outputs the maximum TCP window size in Bytes

Calculation is Half Duplex

NOTE: This does take into account full Window Scaling from the SYN/SYNACK connection. If no SYN/SYNCACK has been processed this filed outputs NULL

{
.
.
  "tcpWindowMin": 1,
  "tcpWindowMax": 4739,
  "tcpWindowMean": 89,
.
.
}

@TCP_WINDOW_MEAN@

Outputs the arithmetic mean TCP window size in Bytes

Calculation is Half Duplex

NOTE: This does take into account full Window Scaling from the SYN/SYNACK connection. If no SYN/SYNCACK has been processed this filed outputs NULL

{
.
.
  "tcpWindowMin": 1,
  "tcpWindowMax": 4739,
  "tcpWindowMean": 89,
.
.
}

De-Encapsulation

PCAP2JSON will de-encapsulation some packets and payloads. Following is a description of each field

@DECAP_TYPE@

If packet de-encapsulation was performance, indicates what kind of encapsulation

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_IPV4_SRC@

For de-encapsulated packets, the IPv4 Source address of the INNER packet

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_IPV4_DST@

For de-encapsulated packets, the IPv4 Destination address of the INNER packer

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_IPV4_PROTO@

For de-encapsulated packets the IPv4 Protocol of the INNER packet

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_IPV4_DSCP@

Version:608+

Value (In Hex)

Example JSON

{
    .
    .
    .
     "decapType": "ICMP Time Exceeded",
     "decapIPv4Src": "130.128.197.9",
     "decapIPv4Dst": "111.87.220.242",
     "decapIpv4Proto": "ICMP",
     "decapIpv4DSCP": "AF13",
     "decapPortSrc": 0,
     "decapPortDst": 0
     .
     .
     .
}

@DECAP_PORT_SRC@

For de-encapsulated packets the Source Port of the INNER TCP or UDP packet

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_PORT_DST@

For de-encapsulated packets the Destination Port of the INNER TCP or UDP packet

{
  .
  .
  "decapType": "ICMP Destination Unreachable",
  "decapIPv4Src": "208.67.222.222",
  "decapIPv4Dst": "202.17.220.140",
  "decapIpv4Proto": "UDP",
  "decapPortSrc": 53,
  "decapPortDst": 64632
  .
  .
}

@DECAP_SRCIP_HOSTNAME@

Uses the GEO IP Mapping to search a a textual description of the De-Encapsulated source IPv4 address. If a mapping is not found, the standard IPV4 numeric address is used.

{
  .
  .
  "decapIPv4Src": "45.0.192.14",
  "decapIPv4Dst": "10.1.119.196",
  "decapHostSrc": "45.0.192.14",
  "decapHostDst": "Reserved Private",
  .
  .
}

@DECAP_SRCIP_LOCATION@

De-encapsulated Source IP GeoIP Location. This includes the Longitude / Latitude value

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_COUNTRY@

De-Encapsulated Source IP GeoIP Country

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_COUNTRY_CODE@

De-Encapsulated Source IP GeoIP 2 letter country code

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_CITY@

De-Encapsulated Source IP GeoIP City

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_ASN@

De-Encapsulated Source IP ASN

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_ORG@

De-Encapsulated Source IP Organization

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_SRCIP_ISP@

De-Encapsulated Source IP ISP Name

"decapGeoipSrc": {
    "location": [
      -51.6571,
      -27.4297
    ],
    "country_name": "Brazil",
    "country_iso_code": "BR",
    "city_name": "Capinzal",
    "asn": 53062,
    "org": "GGNET TELECOMUNICACOES LTDA",
    "isp": "Ggnet Telecomunicacoes Ltda"
  },

@DECAP_DSTIP_HOSTNAME@

See @DECAP_SRCIP_HOSTNAME@ for description

@DECAP_DSTIP_LOCATION@

See @DECAP_SRCIP_LOCATION@ for description

@DECAP_DSTIP_COUNTRY@

See @DECAP_SRCIP_COUNTRY@ for description

@DECAP_DSTIP_COUNTRY_CODE@

See @@DECAP_DSTIP_COUNTRY_CODE@

@DECAP_DSTIP_CITY@

See @DECAP_SRCIP_CITY@ for description

@DECAP_DSTIP_ASN@

See @DECAP_SRCIP_ASN@ for description

@DECAP_DSTIP_ORG@

See @DECAP_SRCIP_ORG@ for description

@DECAP_DSTIP_ISP@

See @DECAP_SRCIP_ISP@ for description

PreviousGeo Map GenerationNextHTTPS/TLS Tunneling

Last updated