Giter Club home page Giter Club logo

systematization-binary-formats's People

Contributors

treiher avatar

Watchers

James Cloos avatar Alexander Senier avatar avatar

systematization-binary-formats's Issues

Bluetooth Low Energy Link Layer

Schemas

LE Uncoded + Advertising Physical Channel PDU / Data Physical Channel PDU

The CRC polynomial is a 24-bit CRC and all bits in the PDU shall be processed
in transmitted order starting from the least significant bit. The polynomial has
the form of x^24 + x^10 + x^9 + x^6 + x^4 + x^3 + x + 1.

The preamble is a fixed sequence of alternating 0 and
1 bits. For packets transmitted on the LE 1M PHY, the preamble is 8 bits; for
packets transmitted on the LE 2M PHY, the preamble is 16 bits.

Multi-octet fields, with the exception of the Cyclic Redundancy Check (CRC)
and the Message Integrity Check (MIC), shall be transmitted with the least
significant octet first.

The preamble is a fixed sequence of alternating 0 and
1 bits. [...] The first bit of
the preamble (in transmission order) shall be the same as the LSB of the
Access Address.

Specification

Bluetooth Core Specification 5.2, Vol. 6, Part B
https://www.bluetooth.com/specifications/archived-specifications/

SLP

Schemas

The three byte offset to next extension indicates the position of the
next extension as offset from the beginning of the SLP message.

The offset value is 0 if there are no extensions following the
current extension.

If the offset is 0, the length of the current Extension Data is
determined by subtracting total length of the SLP message as given in
the SLP message header minus the offset of the current extension.

Specification

https://tools.ietf.org/html/rfc2608

HTTP/2

Schemas

Frame

Specification

https://tools.ietf.org/html/rfc7540

ARP

Schemas

In theory, the length fields (ar$hln and ar$pln) are redundant,
since the length of a protocol address should be determined by
the hardware type (found in ar$hrd) and the protocol type (found
in ar$pro). It is included for optional consistency checking,
and for network monitoring and debugging

Specification

https://tools.ietf.org/html/rfc826
https://en.wikipedia.org/wiki/Address_Resolution_Protocol

X.509

X.509 Schemas

Certificate

Certificate  ::=  SEQUENCE  {
     tbsCertificate       TBSCertificate,
     signatureAlgorithm   AlgorithmIdentifier,
     signature            BIT STRING  }

TBSCertificate

TBSCertificate  ::=  SEQUENCE  {
     version         [0]  Version DEFAULT v1,
     serialNumber         CertificateSerialNumber,
     signature            AlgorithmIdentifier,
     issuer               Name,
     validity             Validity,
     subject              Name,
     subjectPublicKeyInfo SubjectPublicKeyInfo,
     issuerUniqueID  [1]  IMPLICIT UniqueIdentifier OPTIONAL,
                          -- If present, version MUST be v2 or v3
     subjectUniqueID [2]  IMPLICIT UniqueIdentifier OPTIONAL,
                          -- If present, version MUST be v2 or v3
     extensions      [3]  Extensions OPTIONAL
                          -- If present, version MUST be v3 --  }

Version

Version  ::=  INTEGER  {  v1(0), v2(1), v3(2)  }

CertificateSerialNumber

CertificateSerialNumber  ::=  INTEGER

Validity

Validity ::= SEQUENCE {
     notBefore      Time,
     notAfter       Time  }

Time

Time ::= CHOICE {
     utcTime        UTCTime,
     generalTime    GeneralizedTime }

UniqueIdentifier

UniqueIdentifier  ::=  BIT STRING

SubjectPublicKeyInfo

SubjectPublicKeyInfo  ::=  SEQUENCE  {
     algorithm            AlgorithmIdentifier,
     subjectPublicKey     BIT STRING  }

Extensions

Extensions  ::=  SEQUENCE SIZE (1..MAX) OF Extension

Extension

Extension  ::=  SEQUENCE  {
     extnID      OBJECT IDENTIFIER,
     critical    BOOLEAN DEFAULT FALSE,
     extnValue   OCTET STRING
                 -- contains the DER encoding of an ASN.1 value
                 -- corresponding to the extension type identified
                 -- by extnID
     }

AlgorithmIdentifier

AlgorithmIdentifier  ::=  SEQUENCE  {
     algorithm               OBJECT IDENTIFIER,
     parameters              ANY DEFINED BY algorithm OPTIONAL  }
                                -- contains a value of the type
                                -- registered for use with the
                                -- algorithm object identifier value

Name

Name ::= CHOICE { rdnSequence  RDNSequence }

RDNSequence

RDNSequence ::= SEQUENCE OF RelativeDistinguishedName

RelativeDistinguishedName

RelativeDistinguishedName ::=
     SET SIZE (1..MAX) OF AttributeTypeAndValue

AttributeTypeAndValue

AttributeTypeAndValue ::= SEQUENCE {
     type     AttributeType,
     value    AttributeValue }

AttributeType

AttributeType ::= OBJECT IDENTIFIER

AttributeValue

AttributeValue ::= CHOICE {
         printableString         PrintableString (SIZE (1..MAX)),
         utf8String              UTF8String (SIZE (1..MAX)) }

ASN.1 built-in types schemas

CHOICE

SEQUENCE

SEQUENCE OF

SET

In DER, a SET must be encoded in ascending order by tag.

SET OF

For DER encoding, there is a similar requirement that the SET OF must be encoded in ascending order. Because all elements in the SET OF have the same type, ordering by tag is not sufficient. So the elements of a SET OF are sorted by their encoded values, with shorter values treated as if they were padded to the right with zeroes.

INTEGER

BIT STRING

From here:

A BIT STRING of N bits is encoded as N/8 bytes (rounded up), with a one-byte prefix that contains the “number of unused bits,” for clarity when the number of bits is not a multiple of 8.

In DER, the unused bits must all be zero.

UTCTime

From RFC5280:

For the purposes of this profile, UTCTime values MUST be expressed in
Greenwich Mean Time (Zulu) and MUST include seconds (i.e., times are
YYMMDDHHMMSSZ), even where the number of seconds is zero.

Dates imply additional constraints, e.g. that MM is greater than 0 and less than 13.

GeneralizedTime

From RFC5280:

For the purposes of this profile, GeneralizedTime values MUST be
expressed in Greenwich Mean Time (Zulu) and MUST include seconds
(i.e., times are YYYYMMDDHHMMSSZ), even where the number of seconds
is zero. GeneralizedTime values MUST NOT include fractional seconds.

Same schemas as UTCTime.

OBJECT IDENTIFIER

From here:

OIDs are conceptually a series of integers

They are always at least two components long. The first component is always 0, 1, or 2. When the first component is 0 or 1, the second component is always less than 40. Because of this, the first two components are unambiguously represented as 40*X+Y, where X is the first component and Y is the second.

BOOLEAN

OCTET STRING

String types

Specification

CoAP

Schemas

Message

The payload data extends from
after the marker to the end of the UDP datagram, i.e., the Payload
Length is calculated from the datagram size.

Header and Token are followed by zero or more Options (Section 3.1).
An Option can be followed by the end of the message, by another
Option, or by the Payload Marker and the payload.

Following the header, token, and options, if any, comes the optional
payload. If present and of non-zero length, it is prefixed by a
fixed, one-byte Payload Marker (0xFF), which indicates the end of
options and the start of the payload.

Option

Option Delta: 4-bit unsigned integer. A value between 0 and 12
indicates the Option Delta. Three values are reserved for special
constructs:
13: An 8-bit unsigned integer follows the initial byte and
indicates the Option Delta minus 13.
14: A 16-bit unsigned integer in network byte order follows the
initial byte and indicates the Option Delta minus 269.
15: Reserved for the Payload Marker. If the field is set to this
value but the entire byte is not the payload marker, this MUST
be processed as a message format error.

Option Length: 4-bit unsigned integer. A value between 0 and 12
indicates the length of the Option Value, in bytes. Three values
are reserved for special constructs:
13: An 8-bit unsigned integer precedes the Option Value and
indicates the Option Length minus 13.
14: A 16-bit unsigned integer in network byte order precedes the
Option Value and indicates the Option Length minus 269.
15: Reserved for future use. If the field is set to this value,
it MUST be processed as a message format error.

The resulting Option Delta is used as the difference between the
Option Number of this option and that of the previous option (or
zero for the first option). In other words, the Option Number is
calculated by simply summing the Option Delta values of this and
all previous options before it.

Specification

https://tools.ietf.org/html/rfc7252
https://en.wikipedia.org/wiki/Constrained_Application_Protocol#Message_formats

Ethernet

Schemas

Specification

https://en.wikipedia.org/wiki/Ethernet_frame#Structure

L2CAP

Schemas

Basic information frame (B-frame)

All L2CAP layer packet fields shall use little-endian byte
order with the exception of the information payload field. The endianness of
higher layer protocols encapsulated within L2CAP information payload is
protocol-specific.

Group frame (G-frame)

The PSM field is at least two octets in length. All PSM values shall have the
least significant bit of the most significant octet equal to 0 and the least
significant bit of all other octets equal to 1.

Specification

Bluetooth Core Specification 5.2, Vol. 3, Part A
https://www.bluetooth.com/specifications/archived-specifications/

TCP

Schemas

Specification

https://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure
https://tools.ietf.org/html/rfc793

ICMP

Schemas

Destination Unreachable Message, Time Exceeded Message, Source Quench Message

The checksum is the 16-bit ones's complement of the one's
complement sum of the ICMP message starting with the ICMP Type.
For computing the checksum , the checksum field should be zero.
This checksum may be replaced in the future.

Parameter Problem Message

Redirect Message

Echo or Echo Reply Message

Timestamp or Timestamp Reply Message

Information Request or Information Reply Message

Specification

https://tools.ietf.org/html/rfc792

IPv6

Schemas

Individual options may have specific alignment requirements, to
ensure that multi-octet values within Option Data fields fall on
natural boundaries. The alignment requirement of an option is
specified using the notation xn+y, meaning the Option Type must
appear at an integer multiple of x octets from the start of the
header, plus y octets. For example:

 2n     means any 2-octet offset from the start of the header.
 8n+2   means any 8-octet offset from the start of the header, plus
        2 octets.

Variable-length field, of length such that the
complete Hop-by-Hop Options header is an
integer multiple of 8 octets long.

 Reserved            8-bit reserved field.  Initialized to zero for
                     transmission; ignored on reception.

CmprI                4-bit unsigned integer.  Number of prefix octets
                     from each segment, except than the last segment,
                     (i.e., segments 1 through n-1) that are elided.
                     For example, an SRH carrying full IPv6 addresses
                     in Addresses[1..n-1] sets CmprI to 0.

Address[1..n]        Vector of addresses, numbered 1 to n.  Each
                     vector element in [1..n-1] has size (16 - CmprI)
                     and element [n] has size (16-CmprE).  The
                     originator of an SRH places the next (first)
                     hop's IPv6 address in the IPv6 header's IPv6
                     Destination Address and the second hop's IPv6
                     address as the first address in Address[1..n]
                     (i.e., Address[1]).

Specification

https://en.wikipedia.org/wiki/IPv6_packet
https://tools.ietf.org/html/rfc8200
https://tools.ietf.org/html/rfc6554

STUN

Schemas

                   0                 1
                   2  3  4 5 6 7 8 9 0 1 2 3 4 5

                  +--+--+-+-+-+-+-+-+-+-+-+-+-+-+
                  |M |M |M|M|M|C|M|M|M|C|M|M|M|M|
                  |11|10|9|8|7|1|6|5|4|0|3|2|1|0|
                  +--+--+-+-+-+-+-+-+-+-+-+-+-+-+

           Figure 3: Format of STUN Message Type Field

Here the bits in the message type field are shown as most significant
(M11) through least significant (M0). M11 through M0 represent a 12-
bit encoding of the method. C1 and C0 represent a 2-bit encoding of
the class.

Specification

https://tools.ietf.org/html/rfc5389

DTLS 1.3

Schemas

If a connection ID is negotiated, then it MUST be contained in all datagrams.

Because each party sends the value in the "connection_id" extension
it wants to receive as a CID in encrypted records, it is possible for
an endpoint to use a globally constant length for such connection
identifiers. This can in turn ease parsing and connection lookup,
for example by having the length in question be a compile-time
constant. Such implementations MUST still be able to send CIDs of
different length to other parties. Implementations that want to use
variable-length CIDs are responsible for constructing the CID in such
a way that its length can be determined on reception. Note that
there is no CID length information included in the record itself.

struct {
opaque cid<0..2^8-1>;
} ConnectionId;

legacy_record_version This value MUST be set to {254, 253} for all
records other than the initial ClientHello (i.e., one not
generated after a HelloRetryRequest), where it may also be {254,
255} for compatibility purposes. It MUST be ignored for all
purposes. See [TLS13]; Appendix D.1 for the rationale for this.

Specification

https://datatracker.ietf.org/doc/html/draft-ietf-tls-dtls13-41
https://datatracker.ietf.org/doc/html/draft-ietf-tls-dtls-connection-id-09

DHCP

Schemas

The first four octets of the 'options' field of the DHCP message
contain the (decimal) values 99, 130, 83 and 99, respectively (this
is the same magic cookie as is defined in RFC 1497 [17]).

the "DHCP message type" option - must be included in every DHCP
message. This option defines the "type" of the DHCP message.
Additional options may be allowed, required, or not allowed,
depending on the DHCP message type.

Option Overload

This option is used to indicate that the DHCP "sname" or "file"
fields are being overloaded by using them to carry DHCP options. A
DHCP server inserts this option if the returned parameters will
exceed the usual space allotted for options.

If this option is present, the client interprets the specified
additional fields after it concludes interpretation of the standard
option fields.

Specification

https://tools.ietf.org/html/rfc2131

QUIC

Schemas

Specification

https://tools.ietf.org/html/draft-ietf-quic-transport-27#section-17

TLS 1.3

Schemas

struct {
opaque verify_data[Hash.length];
} Finished;

When multiple extensions of different types are present, the
extensions MAY appear in any order, with the exception of
"pre_shared_key" (Section 4.2.11) which MUST be the last extension in
the ClientHello (but can appear anywhere in the ServerHello
extensions block). There MUST NOT be more than one extension of the
same type in a given extension block.

An arbitrary-length run of zero-valued bytes may appear in
the cleartext after the type field. This provides an opportunity
for senders to pad any TLS record by a chosen amount as long as
the total stays within record size limits.

encrypted_record: The AEAD-encrypted form of the serialized
TLSInnerPlaintext structure.

If the corresponding certificate type extension
("server_certificate_type" or "client_certificate_type") was not
negotiated in EncryptedExtensions, or the X.509 certificate type was
negotiated, then each CertificateEntry contains a DER-encoded X.509
certificate.

Specification

https://tools.ietf.org/html/rfc8446

NTP

Schemas

Message

Message Digest (digest): 128-bit MD5 hash computed over the key
followed by the NTP packet header and extensions fields (but not the
Key Identifier or Message Digest fields).

All NTP time values are represented in twos-complement format, with
bits numbered in big-endian (as described in Appendix A of [RFC0791])
fashion from zero starting at the left, or high-order, position.

Reference ID (refid): 32-bit code identifying the particular server
or reference clock. The interpretation depends on the value in the
stratum field. For packet stratum 0 (unspecified or invalid), this
is a four-character ASCII [RFC1345] string[...]
For stratum 1 (reference
clock), this is a four-octet, left-justified, zero-padded ASCII
string assigned to the reference clock.
Above stratum 1 (secondary servers and clients): this is the
reference identifier of the server and can be used to detect timing
loops. If using the IPv4 address family, the identifier is the four-
octet IPv4 address. If using the IPv6 address family, it is the
first four octets of the MD5 hash of the IPv6 address.

A MAC MUST NOT be longer than 24 octets if there is no extension
field present, unless a longer MAC is agreed upon by both client
and server. The client and server can negotiate this behavior
using a previous exchange of packets with an extension field that
defines the size and algorithm of the MAC transmitted in NTP
packets.

If a MAC is not present, one or more extension fields can be
inserted after the header, according to the following rules:
o If the packet includes a single extension field, the length of
the extension field MUST be at least 7 words, i.e., at least
28 octets.
o If the packet includes more than one extension field, the
length of the last extension field MUST be at least 28 octets.
The length of the other extension fields in this case MUST be
at least 16 octets each.

Extension Field

All extension fields are zero-padded to a word (four octets)
boundary.

Specification

https://tools.ietf.org/html/rfc5905
https://tools.ietf.org/html/rfc7822

IPv4

Schemas

The checksum field is the 16 bit one's complement of the one's
complement sum of all 16 bit words in the header. For purposes of
computing the checksum, the value of the checksum field is zero.

The list of options may be terminated with an EOL (End of Options List, 0x00) option; this is only necessary if the end of the options would not otherwise coincide with the end of the header.

Specification

https://en.wikipedia.org/wiki/IPv4#Packet_structure
https://tools.ietf.org/html/rfc791

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.