Datetime handling before Unix epoch about blpapi-http HOT 16 CLOSED

@andybrnr, thanks for reporting this issue. Can you provide the precise HTTP request you are making that replicates the issue? Thanks.

from blpapi-http.

Hi Eric,

Python 3 code I'm using for the request below. I've had no problems with dates after the Unix Epoch, and the standard Python BLPAPI handles pre-epoch dates correctly (haven't tried to replicate this using only blpapi-node yet).

import json
import urllib.request as urllib

histData = {"securities": ['SPX Index'],
            "fields": ['PX_LAST'],
            "startDate": '19270101',
            "endDate": '20150818',
            "periodicitySelection": 'DAILY'}
data = json.dumps(histData)
bin_data = data.encode('utf8')
clen = len(bin_data)
req = urllib.Request('http://{}/request?ns=blp&service=refdata&type=ReferenceDataRequest'.format(host),
                      bin_data,{'Content-Type': 'application/json', 'Content-Length': clen})

from blpapi-http.

@andybrnr, this example errors when I try to replicate it similarly in blpapi-node. I don't believe that ReferenceDataRequest's accept startDate, endDate, nor periodicitySelection. Did you intend for this example to be a HistoricalDataRequest?

from blpapi-http.

If I switch to HistoricalDataRequest, I am unable to replicate any timestamps that are equal to "1969-12-31 23:59:59". In blpapi-node, I modified one of the .js examples and the timestamps returned appeared correct. Additionally, for blpapi-http, I modified one of the shell/curl examples and the values appeared correct?

To clarify when you observe the timestamp of "1969-12-31 23:59:59", is this from the returned JSON string or are you observing this timestamp afterwards in your Python3 code?

from blpapi-http.

Hi Eric,

Sorry, cut and pasted from the wrong block; that was indeed supposed to be a HistoricalDataRequest. The timestamp 1969-12-31 23:59:59 that I get is in the returned JSON string. Server is running on Windows 7 x64; I'll try and get the exact versions of all the dependencies installed tomorrow.

from blpapi-http.

Screenshot:

from blpapi-http.

What is your node --version? I have been using v0.12.7 for my testing. I am wondering if there are different NodeJS environments at play here.

from blpapi-http.

I'm also on v0.12.7. From some googling and stack overflow references, I feel like this might be caused by some JS datetime library buried in the node dependency stack which is expecting seconds instead of milliseconds or a similar issue, but I'm at a bit of a loss here.

from blpapi-http.

@andybrnr, what OS are you running on?

from blpapi-http.

win7x64

from blpapi-http.

The root cause is due to how mktime() behaves on Windows. The documentation for WIN32 mktime indicates that for struct tm* before January 1, 1970, that -1 is returned. Which, when -1 is passed into V8::Date::New() it results in one second behind the Unix epoch (i.e., '1969-12-31 23:59:59').

I need to dig a bit deeper to figure out how we can resolve this in blpapi-node, which blpapi-http depends upon, for Windows. blpapi-node uses the this call here.

from blpapi-http.

Hmm... chalk up another "win" for MS. I did some looking at other classes that might be applicable. Does something like this look workable https://msdn.microsoft.com/en-us/library/38wh24td.aspx ? Irritating to rely on a specific compilation under MSVS, but since it's windows support anyway, maybe it's not the end of the world?

from blpapi-http.

Hi Eric,

I managed to patch this issue by plugging in the library here: http://www.codeproject.com/Articles/4674/Time-bit-Times-for-bit-Windows-Apps

Only change is adding includes for time64.h and time64.c below line 30 in blpapijs.cpp and changing the mktime() call on line 1088 to mktime64(). I made the changes to blpapijs.cpp, used npm rebuild, and then did a fresh make build of blpapi-http, and validated by grabbing SPX since 1928.

Libs seem to be legit for use as no explicit license is specified, but not sure what appropriate protocol is here. I can push a changeset to blpapi-node if you like.

from blpapi-http.

Unfortunately libraries are not legitimate to use if there is no explicit license specified. If no license is specified, default copyright laws apply. In some cases, this can be even more dangerous because the original author retains all rights to the source code and nobody else may reproduce, distribute, or create derivative works of the work.

Don't worry, this issue is still on my radar.

from blpapi-http.

Closing this in favor of bloomberg/blpapi-node#68.

from blpapi-http.

Gotcha. Thanks for the heads up on the library copyright issues when no license is spec'd; still learning how the rules work in open-source land. Appreciate you continuing to look into this.

from blpapi-http.