uscdatascience Goto Github PK

nutch-analytics

Nutch Crawl Analysis - Spark based project

ocean_observation_facetview

This is a FacetView setup for ocean observation Crawled Data.

parser-indexer

Metadata Parser and Solr Indexer. For Python equivalent, checkout https://github.com/USCDataScience/parser-indexer-py

parser-indexer-py

Python tools for parsing documents and building the inverted index with enriched metadata. Java version with slightly different features - https://github.com/USCDataScience/parser-indexer

pdftabextract

A set of tools for extracting tables from PDF files helping to do data mining on (OCR-processed) scanned documents.

pdi-topics

LDA Topic Modeling for Polar Data Insights

personaextraction

planetaryir

Information Retrieval for Planetary Science using DeepDive

polar-deep-insights

Conceptual - Temporal - Spatial analysis of the trec polar dataset

polar-domain-discovery

Domain Discovery on Polar Domain

polar.usc.edu

Polar USC activities related to NSF Polar CyberInfrastructure program at the University of Southern California

polardatacollection

Using Google Search API we collect URLs relevant to the Polar Domain for deep insights and intelligent crawling

polarpostprocessing

This code gets connected to Solr DB created for Sparkler Crawled Data to do further data extraction, classification, filtering and insights generation using various Machine Learning models. The ML models are capable of using keywords list from user, extract features from URL content, and classify (score) output and update Solr parameter accordingly. Apache Sparkler Link: https://github.com/USCDataScience/sparkler

scala-json-doclet

Scala Doclet that produces JSON output

sce-domain-discovery

Domain Discovery for the Sparkler Crawl Environment

sentimentanalysisparser

Combines Apache OpenNLP and Apache Tika and provides facilities for automatically deriving sentiment from text.

solrmerge

An open source project to merge Solr cores in an extremely customizable way.

sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

sparkler-jsdriver

sparkler-ui

supervising-ui

Web UI for labelling dataset for supervised learning.

svm-classifier-memex

sweet-neo4j

A ruby parser using linkeddata and RDF to fetch the JPL Sweet ontology and load it into Neo4J for cool graph queries and examination.

tattdl

Tattoo detection and localization

textrest.jl

Language Detection REST Server using MIT Lincoln Lab’s Text.jl library

tika-dl-models

A place to release saved machine learning models for tika-dl

tika-dl4j-spark-imgrec

Image recognition on Spark cluster powered by Deeplearning4j and Apache Tika

tika-dockers

A suite of Machine Learning / Deep Learning Dockerfiles to allow Apache Tika to extract objects and to produce textual captions for images and video

tika-ner-corenlp

Stanford CoreNLP NER addon for Apache Tika's NamerEntityParser

tika-ner-libraries

This is a d3 visualization benchmarking named entities extracted by NLTK and Standford Core NLP