Integration tests are failing too

http://apis.is/lottery
http://apis.is/lottery/eurojackpot

Maybe it'll get fixed automatically when there's a draw, but maybe there's something that can be done in the endpoint.

I didn't investigate what's going on.

For example (values from the tests, valid numbers that should return results)

http://apis.is/company?socialnumber=4307003590
http://apis.is/company?vsknr=78874

Problem seems (at first glance) to be on their side (rsk.is)

in order to ensure code quality etc.

Note: This relates to the new APIs.is, on the new2 branch

We have to come up with a strategy to prevent the server going down on updates. Has anyone done anything like that in their own projects? Out of many possible ways which do you think is the best one?

Mín hugmynd er að scrape-a bin.arnastofnun.is, þannig að maður geti sótt beygingar á íslenskum orðum. Væri mjög hentugt t.d. þegar maður er með leit eftir götuheitum. Þannig að ef einhver leitar að t.d. "Laugavegi", en maður er með DB í nefnifalli, þ.e. "Laugavegur", þá gæti maður tjekkað hvort að það er result í öðru falli.

Server returns 502 error and burns to the ground when you try to get info for busses that are not running

I was wondering if it might make more sense to have the /bus/realtime API return a flat array with all the buses, instead of an array of a group of bus numbers and each bus with that number

Makes it so actually using the API is more complicated, and if you'd want to group them somehow you could probably do that in your code in the implementation.

That would also make the API more consistent with other endpoints.

Output now:

{
    "results": [
        {
            "busNr": "1",
            "busses": [
                {
                    "unixTime": 1354618575,
                    "x": 64.090668,
                    "y": -21.92904,
                    "from": "Hlemmur",
                    "to": "Klukkuvellir"
                },
                {
                    "unixTime": 1354618575,
                    "x": 64.086315,
                    "y": -21.937813,
                    "from": "Klukkuvellir",
                    "to": "Hlemmur"
                }
            ]
        },
        {
            "busNr": "5",
            "busses": [
                {
                    "unixTime": 1354618563,
                    "x": 64.135644,
                    "y": -21.850126,
                    "from": "Hlemmur",
                    "to": "Elliðabraut / Árvað"
                },
                {
                    "unixTime": 1354618577,
                    "x": 64.143989,
                    "y": -21.913048,
                    "from": "Elliðabraut / Árvað",
                    "to": "Hlemmur"
                }
            ]
        }
    ]
}   

My suggestion:

{
    "results": [
        {
            "busNr": "1",
            "unixTime": 1354618575,
            "x": 64.090668,
            "y": -21.92904,
            "from": "Hlemmur",
            "to": "Klukkuvellir"
        },
        {
            "busNr": "1",
            "unixTime": 1354618575,
            "x": 64.086315,
            "y": -21.937813,
            "from": "Klukkuvellir",
            "to": "Hlemmur"
        }
        {
            "busNr": "5",
            "unixTime": 1354618563,
            "x": 64.135644,
            "y": -21.850126,
            "from": "Hlemmur",
            "to": "Elliðabraut / Árvað"
        },
        {
            "busNr": "5",
            "unixTime": 1354618577,
            "x": 64.143989,
            "y": -21.913048,
            "from": "Elliðabraut / Árvað",
            "to": "Hlemmur"
        }
    ]
}   

The cyclist counter at Suðurlandsbraut has a feed:
http://www.bicyclecounter.dk/BicycleCounter/GetCycleInfo?ran=1379500208853&StationId=235&LaneId=0

Currently calls to the bus endpoints don't return any location/position data

example: http://apis.is/bus/realtime?bussess=1,5

{
  "results": [
    {
      "busNr": "1",
      "busses": [
        {
          "unixTime": 1381691627,
          "from": "Klukkuvellir",
          "to": "Hlemmur"
        },
        {
          "unixTime": 1381691622,
          "from": "Hlemmur",
          "to": "Klukkuvellir"
        },
        {
          "unixTime": 1381691632,
          "from": "Klukkuvellir",
          "to": "Hlemmur"
        },
        {
          "unixTime": 1381691629,
          "from": "Hlemmur",
          "to": "Klukkuvellir"
        }
      ]
    }
  ]
}

did bus.is change the format of their data perhaps?

As seen on the frontpage of reykjavik.is, data is fetched from http://www.loft.rvk.is/xml/svifryk.xml

Currently the docs explain that you can add an accept-version header along with your request.

If I'm not mistaken, none of the endpoints take advantage of this yet, so it might be a good idea to brainstorm a bit about how this should take place.

There's a lot of questions that might be good to think about for deciding on a proper versioning scheme:

• Does the API have to allow developers to omit a version?
• if so, what is the default version? The very first one? the last one? or one that has been branded "stable" somehow?
• Should the API docs encourage you to add a version, so your code won't randomly break?
• Should the version number be global across all endpoints, or endpoint specific? If it's
• Should you be able to pass in the version as part of the post request as well? even in the url? or only as a header?

And then from a code standpoint, it would be ideal that for each endpoint, there would only be one parser, so for additional versions the data just goes through different transforms to make a different JSON scheme..

Although, maybe some of the versions would also change the way that the corresponding parser behaves? eg. it would paginate differently etc etc.. probably best to think about that gradually.

Thoughts?

http://iceland.adagios.org/apis.is/ this has been red for 2 weeks now.

http://apis.is/car?number=1337 shows empty results

If the data is being cached by apis.is, send a caching header accordingly

If the endpoint sends caching information, forward it to the client

Seems to be due to changes in markup at bin.arnastofnun.is

I thought i had opened this issue before, but whatever here it goes:

Strætó API seems to be down. so is currency:

$ curl -X GET "apis.is/bus/realtime?busses=1,5" -H 'Content-Type: application/json'

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /bus/realtime.

Reason: Error reading from remote server

The TV endpoint is undocumented, it could also be more useful if more channels would be added. I will go ahead and start working on that, will also try to add more channels, starting with Skjár 1, then issue a pull request.
Is there any reason for the endpoint being left out of the documentation, or any objections to me working on this?

There seems to be an issue with the jquery module on v0.10 - I'm running that right now and I'm getting the same issue as reported here:
HorseAJ86/node-jquery#52

fix involves one of the following:

• replacing jquery with something else -- i tried jquery-loader and that seemed to work
• waiting for the module to get updated .. the issue seems old at this point
• some other monkeypatching fix? 😦

I don't know where we are in terms of node version support -- maybe it's just me who's seeing the issue?

There are many Icelandic wordbanks and dictionaries scattered around the internets and it would be nice to have endpoints for them at APIs.is , to be enable aggregation, unification, or such.

Some years ago I implemented a search interface for a few of those wordbanks at

http://oss.nemur.net

which is also available as an Android app: https://play.google.com/store/apps/details?id=net.nemur.oss

...and it sorely needs updating as some of the scraped web interfaces have changed or moved.

The search interface uses endpoints implemented in Python hosted on Google App Engine, and so the roundtrip for each request is probably longer than would be for GreenQloud hosted APIs.is endpoints. Since some of the endpoints need updating, it might be just as good to reimplement them as new APIs.is endpoints, where they might benefit from that ecosystem. I'm interested in giving that a try but probably won't do anything about it in the next few months. Here I'll at least put this idea in the open, and it's just great if someone beats me to the implementation.

The Python endpoint implementation is here:
https://code.google.com/p/ordasafnasafn/source/browse/trunk/ordasafnasafn/src/ordasafnasafn.py

and gives those:

Hugtakasafn þýðingamiðstöðvar utanríkisráðuneytisins
http://oss.nemur.net/search?ordasafn=SearchHugtakasafn&q=prammi&exact=false
...the web interface being scraped there is implemented and still maintained by me, and so it was an option to provide a JSON interface directly, but it was just quicker to scrape :)

Orðabanki íslenskrar málstöðvar
http://oss.nemur.net/search?ordasafn=SearchIsmal&q=prammi&exact=true
...here it's XML parsing rather than scraping. I implemented that XML interface, for a now dead project (Dælan), but haven't had anything to do with the rest of the web interface.

Tölvuorðasafn
http://oss.nemur.net/search?ordasafn=SearchTos&q=platform&exact=false

Beygingarlýsing íslensks nútímamáls
http://oss.nemur.net/search?ordasafn=SearchBin&q=prammi&exact=true
...this endpoint is broken after changes in the scraped web site.

Sjávardýraorðabók
http://oss.nemur.net/search?ordasafn=SearchHafro&q=prammi&exact=true
...broken - the website moved here: http://www.hafro.is/undir.php?ID=22&REF=3

Ritmálasafn Orðabókar Háskólans
http://oss.nemur.net/search?ordasafn=SearchRitmalaskra&q=prammi&exact=true

Málfarsbanki Íslenskrar málstöðvar
http://oss.nemur.net/search?ordasafn=SearchMalfar&q=skutla&exact=false

Other wordbanks I would like to add, are:

Skrá um orðasambönd
http://www.lexis.hi.is/osamb/osamb.pl?finna=ok&flyk=ellisto%F0&fofl=no%20kvk

ISLEX
http://islex.hi.is/

Orðaskrá Íslenska stærðfræðifélagsins
http://stae.is/os

Translate the project to english

http://iceland.adagios.org/apis.is/#tab_hours

To night the data source that the cinema endpoint relies on changed. It does not show data from Háskólabíói, Laugarásbíói, Smárabíói, Borgarbíói og Bíó Paradís anymore. @rthor @hugihlynsson do you have any suggestion how we can get the data to merge it into the current api?

Gathers information on current (todays) showtimes in Icelandic cinemas.

@vincentbriglia made a huge pull request with some changes regarding how the code is structured and validated, see here. What do you guys think about implementing some standards into the project?

This endpoint is no longer a part of the project for a reason mistake I gather. Should I send a new pull request or is this something that can be fixed through other means?

If the endpoint was broken or somehow did not meat standards - please let me know so that I can fix it and get it up and running.

In the /lib/helpers.js file - the logVisit method on the exports object dosen't do anything but return true. Is the method meant for later implementation (like db logging) or am I missing some nodejs dark-magic?

Starts here

It seems the server should mostly (only) be handling GET requests.

The examples in the docs all reference POST requests, so I built the integration tests around those, so those should probably also change.

Curl demo show :

curl -X GET 'apis.is/bus/realtime?buses=1,5' -H 'accept-version:1' -H 'Content-Type: application/json'

buses parameter should be busses according to the code behind it.

I noticed that the lib/helpers.js file exports a mysqlTimestamp file that's not being used anywhere, and there's also a file named lib/errors.js which is not being required anywhere

My vote would be to remove them, since I'm not a huge fan of leaving unused code in repos, but maybe there's a reason I'm not a aware of.

sundlaugar.is is a horribad website and this one can almost not be found http://reykjavik.is/desktopdefault.aspx/tabid-3697/6079_view-1520/. I will probably implement this endpoint in the next couple of weeks, do you know any other data source where this data can be scraped from?

Probably wouldn't be a bad idea to make it synchronous.

We have to rethink the project...

I just had a great meeting with @arnorhs and we were discussing the future of the platform. We came up with some interesting thoughts and we truly believe we can grow this project beyond what it is today into something bigger than a simple JSON scraper.

We have to start to decouple the project more than we are currently doing. All the endpoints are hosted in the endpoints directory and there is a circular dependency tree at this point. A cool thing would be to create one npm package for each endpoint and then a pull request into the project would not be needed to update the endpoint itself. The module would need to export its tests and documentation.
By doing this we are decoupling the project so we can start to focus more on the core development and give the power back to the endpoint creators(and the ownership).

When we have split the project into more maintainable parts we can really start to focus on the core itself. Me and @arnorhs discussed some features that would be nice to have in the platform but we definitely have to prioritise them in terms of implementation stages.

Things that could benefit the platform:

Caching layer
Each endpoint can cache its own output or store some values in the caching layer. We currently offer output caching but it does not support caching headers at this point.

Database layer
Each endpoint would be able to store some data in a document storage such as MongoDB. The endpoint could then read data from the database and do something more than just realtime stuff. A cool idea would be to create a hashing helper that could hash a scraped content from a website and look up if the has exists in the database.

Worker and Queue layer
Each endpoint would be able to create a task that would execute on a specific interval or on a event basis. The endpoint could then start to aggregate data outside the requests that are currently happening and could use the database/caching layer to dump the data into.

Pubsub
With a pubsub model we are able to accept connections that go beyond the current connection model. We would be able to offer hooks on specific events or if the connection is long living we could pump out notifications through that (e.g. twitter firehose). For this to be able to work we need a working worker layer and some persistence/queue layer.

Notification layer
When something goes down or a state alters on the main server we could send out emails to the people that subscribe to them

What are your thoughts on this? This is just the initial brain dump after the meeting and I think that we need to have a good discussion before we can make some drastic changes.

Cheers!
@kristjanmik

We should be using @gudmundur's isn2wgs module for converting coordinates in the bus endpoint.

https://npmjs.org/package/isn2wgs

To setup and run apis locally

The /Concerts endpoint down since midi.is changed their layout.
However the good part is that it's now using a REST based service which can be called instead of scraping the site.
http://midi.is/Home/LoadMoreEventsByDate?eventType=Concerts&pageNumber=1
@rthor

After changing the .walk method to be a .walkSync I realized that the func signatures weren't the same, so I thought I'd fix that and do a pull request..

Lo and behold, the lib hasn't been updated in a while, it has tests that can't be run, referencing files that do not exist, etc.. it's really horrible.

I recommend replacing it with built in fs methods.. I'm happy to do so if nobody has objections..

Outputting errors is sometimes bad or lacking info.

In general it's probably bad practice to leave incomplete and unfinished endpoints in the master branch, it might be a good idea to keep them on feature branches until they're ready.

Thoughts?

A better documentation needs to be implemented for further endpoints to be added.