jordiae / clgen Goto Github PK

Fork of https://github.com/fivosts/clgen. While the original software has many features, I just use it to dump C Github code from BigQuery into an SQLite database. With respect to the original code to get Github C repos, I modified it to also include header files (.h) and keep the repo stars as a field. Steps:

1. Clone this repo.
2. Run requirements.apt to install system dependencies (basically, we need an up-to-date version of CMake, Protobuf, etc).
3. Install Python requirements (requirements.txt).
4. Build: mkdir build && cd build && cmake .. && make
5. Get BigQuery credentials. The path to the credentials JSON is specified here: https://github.com/jordiae/clgen/blob/master/model_zoo/github/bq_C_db.pbtxt#L5
6. Run: ./clgen --config model_zoo/github/bq_C_db.pbtxt

(Original README below)

CLgen is an open source application for generating runnable programs using deep learning. CLgen learns to program using neural networks which model the semantics and usage from large volumes of program fragments, generating many-core OpenCL programs that are representative of, but distinct from, the programs it learns from.

TODO!

CLgen is a tool for generating source code. How you use it will depend entirely on what you want to do with the generated code. As a first port of call I'd recommend checking out how CLgen is configured. CLgen is configured through a handful of protocol buffers defined in //deeplearning/clgen/proto. The clgen.Instance message type combines a clgen.Model and clgen.Sampler which define the way in which models are trained, and how new programs are generated, respectively. You will probably want to assemble a large corpus of source code to train a new model on - I have tools which may help with that. You may also want a means to execute arbitrary generated code - as it happens I have tools for that too. :-) Thought of a new use case? I'd love to hear about it!

Presentation slides:

Publication "Synthesizing Benchmarks for Predictive Modeling" (CGO'17).

Jupyter notebook containing experimental evaluation of an early version of CLgen.

My documentation sucks. Don't be afraid to get stuck in and start reading the code!

Copyright 2016-2020 Chris Cummins [email protected].

Released under the terms of the GPLv3 license. See LICENSE for details.