Giter Club home page Giter Club logo

cognicity-reports-detik's Introduction

CogniCity

Open Source GeoSocial Intelligence Framework

####cognicity-reports-detik: Module for cognicity-reports module to collect reports from Detik API.

Build Status

DOI for current stable release v2.0.0: DOI

About

Cognicity-reports-detik is the NodeJS reports module for collecting relevant articles from the Detik API. For detailed framework documentation see http://cognicity.info. This module is not designed to be run standalone but is designed to be run as a submodule of cognicity-reports, which can run just with this submodule alone.

API Documentation

http://cognicity.info/cognicity/api-docs/cognicity-reports-detik/index.html

Dependencies

  • NodeJS version 4.2.1 or compatible

Node Modules

  • Check package.json for details

If you're going to commit changes to the JavaScript, be sure to run 'npm test' first - and fix any issues that it complains about, otherwise the build will fail when you push the commit.

Installation

Please install this as a submodule of cognicity-reports. Please refer to the documentation of that project for further information.

Install the node dependencies for this submodule as listed in package.json using npm: npm install

Platform-specific notes

To build on OS X we recommend using homebrew to install node, npm, and required node modules as follows:

brew install node
npm install

To build on Windows we recommend installing all dependencies and running npm install.

Configuration

App configuration parameters are stored in a configuration file which is parsed by DetikDataSource.js. See sample-detik-config.js for an example configuration.

Detik parameters

  • serviceURL - URL to the Detik service API, including topic number. E.g. https://example.com/latest?topic=2
  • pollInterval - How long to wait between checking the URL for new articles, in milliseconds. E.g. 1000 * 60 * 5 = 5min
  • historicalLoadPeriod - How old an article can be and still be processed, in milliseconds. E.g. 1000 * 60 * 60 = 1hr
  • pg.table_detik - Database table used to store Detik reports
  • pg.table_detik_users - Database tables used to store Detik report counts per user

Development

Testing

To run the full set of tests, run:

npm test

This will run the following tests:

JSHint

JSHint will run on all JavaScript files in the root folder and test folders.

Running the script:

npm run jshint

This will print an error to the screen if any problems are found.

Mocha

Mocha will run all unit tests in the test folder and can be run with the following script:

npm run mocha

The test output will tell you how many tests passed and failed.

Git Hooks

There is a git pre-commit hook which will run the 'npm test' command before your commit and will fail the commit if testing fails.

To use this hook, copy the file from 'git-hooks/pre-commit' to '.git/hooks/pre-commit' in your project folder.

cp git-hooks/pre-commit .git/hooks/

Documentation

To build the JSDoc documentation run the following npm script:

npm run build-docs

This will generate the API documentation in HTML format in the docs folder, where you can open it with a web browser.

Test Coverage

To build test code coverage documentation, run the following npm script:

npm run coverage

This will run istanbul code coverage over the full mocha test harness and produce HTML documentation in the directory coverage where you can open it with a web browser.

Release

The release procedure is as follows:

  • Update the CHANGELOG.md file with the newly released version, date, and a high-level overview of changes. Commit the change.
  • Create a tag in git from the current head of master. The tag version should be the same as the version specified in the package.json file - this is the release version.
  • Update the version in the package.json file and commit the change.
  • Further development is now on the updated version number until the release process begins again.

License

This software is released under the GPLv3 License. See License.txt for details.

cognicity-reports-detik's People

Contributors

benatwork99 avatar matthewberryman avatar tomasholderness avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

cognicity-reports-detik's Issues

Detik API

Hi @talltom ,
We got the reply from Detik about the API question for Surabaya and Bandung:
Based on the agreement with PetaJakarta last time, they opened their flood report API for us. And it was already for entire Indonesia area, so technically we already got what we just ask now.
They can not clip it based on the cities that are requested, but we can freely do it for our PetaBencana.

If we can not see any reports from Surabaya or Bandung, it could be:

  1. there is no flood report on that area
  2. we need to refresh our map so that we can get the updated feeds (?)

p.s I am sorry, I don't understand exactly why we failed to load the data for Bandung and Surabaya since we were accidentally see a report from Kalimantan about 2 months ago.

cc @anexact

Not up-to-date with latest schema changes

2017-01-17T10:41:24.446Z - error: dbQuery: {"text":"INSERT INTO detik.reports (contribution_id, created_at, disaster_type, text, lang, url, image_url, title, the_geom) VALUES ($1, to_timestamp($2), $3, $4, $5, $6, $7, $8, ST_GeomFromText('POINT(' || $9 || ')',4326));","values":[253032,1484649399,"Selasa (17/1/2017) Banjir terjadi di desa Bajang, Balong, Ponorogo, sekitar pukul 15.30 wib. Menurut warga sekitar banjir terjadi akibat hujan deras yang terjadi sejak pukul 14.00 wib. ","id","https://pasangmata.detik.com/contribution/253032","https://images.detik.com/community/pasma/2017/01/17/1484649399271939873.jpg","Desa Bajang Ponorogo Dilanda Banjir Akibat Hujan Deras","111.4544179 -7.9629978"]}, error: bind message supplies 8 parameters, but prepared statement "" requires 9

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.