small changes to field load types (mostly just wider) - see before/after versions of nf2, tf2, qf2

support NYSE change to schema as specified in: https://list.theice.com/t/92262/395348/57007/0/

support NYSE change to schema, format and datacentre as specified in: http://www.nyxdata.com/doc/247075

taq.k reads the new data files, but saves data to the old schema, discarding nano-second timestamp precision. It is a drop-in replacement for the previous version for those who don't want to move to the new schema yet.

tq.q reads and exports ALL fields for trade, quote and nbbo

Timestamp precision extended from millisecond to microseconds (additional 3 digits ignored for now). Ignore 3 additional fields participant timestamp, RRN and TRF
http://www.nyxdata.com/nysedata/default.aspx?tabid=993&id=2784

Support additional exchange codes (12-15 - ZJKY) in taqmaster

Amend taq.k to discard the first line of taqmaster* when it is just lists the record count

Amend taq.k and tq.q to recognise and ignore the new quote fields:

• SIP-generated Message Identifier
• National BBO LULD Indicator tq.q to recognise and ignore the new NBBO fields:
• Limit-Up/Limit-Down Indicator
• Limit-up/Limit-down NBBO Indicator
• SIP-generated Message Identifier and handle deletion of previous NBBO fields:
• Limit-Up/Limit-Down NBBO (UTP) Indicator
• Limit-Up/Limit-Down NBBO (CQS) Indicator

The format change is scheduled to take effect from 2nd December 2013 see: http://www.nyxdata.com/nysedata/Default.aspx?tabID=993&id=2194

Amended taq.k and tq.q to recognise and ignore the new quote fields:

• Short Sale Restriction (SSR) Indicator
• Limit-Up/Limit-Down BBO (UTP) Indicator
• Limit-Up/Limit-Down BBO (CQS) Indicator
• FINRA ADF MPID Indicator and tq.q to recognise and ignore the new NBBO fields:
• Limit-Up/Limit-Down NBBO (UTP) Indicator
• Limit-Up/Limit-Down NBBO (CQS) Indicator

The format change is scheduled to take effect from 1st February 2013 see:
http://www.nyxdata.com/nysedata/default.aspx?tabid=993&id=1771
http://www.nyxdata.com/doc/185107

Amended taq.k and tq.q to recognise (and ignore) the new Quote field RPI (Retail Interest Indicator)

Added tq.q as example of how to load more fields from the FTP files. taq.k is unchanged.

Adjust partitioning used with -par to split at a whole symbol – so any particular symbol will only be in one partition.

Enabled to handle >2billion rows in input file. Use in combination with -par cmd line option and par.txt

Amend the file detection code to pick up the new file names (as well as the old ones) when NYSE changes the filenames for the FTP download on September 17th, 2010.

You are welcome to download and use this code according to the terms of the licence.

Kx Systems recommends you do not link your application to this repository, which would expose your application to various risks:

• This is not a high-availability hosting service
• Updates to the repo may break your application
• Code refactoring might return 404s to your application

Instead, download code and subject it to the version control and regression testing you use for your application.