• HerbConstraints.jl#32
• HerbCore.jl#13
• HerbInterpret.jl#22
• HerbGrammar.jl#63
• HerbSearch.jl#64
• HerbSpecification.jl#3
• Check that all README.mds include a badge

Additionally, each PR adds the latest 1.x to the test matrix of each package.

Add an iterator to search that makes use of probabilistic grammars.
It should enumerate expressions with higher probability before expressions with lower probability.

• Make sure the md and ipynb are the up-to-date and the same
• Add automation to only have to update ipynb and automatically generate markdown
• Add automation to workflow

Motivation:

New contributors/students/researchers barely know where and what to write down their implementation.

Goal:

Write proper contribution guidelines:

• update READMEs and define scope for each repo (give proper intuition about what the respective modules are supposed to do)
• how to use Gitjub sub-modules locally within Julia Herb package

This repo contains implementations of search heuristics.
It should support both static and dynamic constraints.

Interfaces:

• Search.jl iterators should take this as an input

Add a function for reading input-output examples to Data.jl

• Functionality for reading string data
• Functionality for reading pixel data
• Functionality for reading robot data
• Data structure for storing input-output problems

HerbSearch.jl/test/realistic_search_test.jl and all test in HerbLearn.jl/test/ take longer time to run. Hence it is not feasible to include in the general test suites. However, they are very valuable to be run from time to time.

Thus, I propose to have a separate test-alike way of running these tests separately.

This repo should contain implementations of search procedures over grammar expressions (BFS, DFS, A*, ...)

Core functionality:

• Depth-first search and breadth-first search implementations
• Write the expansion step in a different file: the input is the current state and the output is the set of possible children

Desired Functionality:

• Nice abstracted search template: state manipulation
• Search algorithms: A*, Metropolis-Hastings, Very Large Neighbourhood Search, Monte Carlo Tree Search, Genetic algorithms

Interfacing with other packages:

• take a grammar definition from Grammar.jl,
• takes Heuristics.jl
• takes ConstraintsLib.jl
• takes Specification.jl
• takes LearningToSearch.jl

@THinnerichs and @jaapdejong15 will make the template, and @piotr-cichon and @nickk2002 will focus on the other search functionalities

Create a function that removes/disables a rule in the grammar.
It is important to make sure that indices of the other rules do not shift, since this would make existing RuleNode trees that do not use the deleted rule change meaning.

This repository should contain the functionality for defining specifications, both the DSL and the task.

Interfacing with other components:

• Grammar.jl should associate these descriptions with symbols in a DSL
• Data.jl should use this language to specify the task

Figure out how to get tests for submodules to run

• Add the basic benchmarks from last year's Bachelor End Project repository to the Benchmarks.jl repository
• Create READMEs with a basic description of the benchmarks

The constraints functionality is currently in ExprRules.jl

Move everything regarding specifying constraints in a new repo Constraints.jl

Interface with others:

• Search.jl should implement all functionality regarding calling/checking constraints
• This package only implements the constraints

• Add contact details for main contributors and maintainers
• add contributors to each sub-module and
• copy the form the sub-modules into the main module (markup file with actual contributions)

As per Herb-AI/HerbGrammar.jl#52, the types and macros related to grammars will be renamed. All mentions of them in tutorials and/or documentation should be renamed.

ProbExprRules.jl adds functionalities for probabilistic grammars, on top of another non-probabilistic grammar.

Move functionalities for specifying the probabilistic grammar to Grammar.jl
Move functionalities for search over probabilistic grammars to Search.jl

look into:
https://github.com/SciML/diffeqpy is a wonderful repo for which we may just copy the entire logic.

Other repo to look into: https://github.com/MilesCranmer/PySR

Unify the paradigms:

• specifications
• input/outputs examples
• probabilistic programs
  (- RL alike synthesis with a reward function?)
  within eval and interpret. We need this as a foundation stone for future students. Functions should throw NotImplementedError

MIT

Add the following:

• Context free grammar type
• Functionality for specifying grammars
  • Adding rules
  • Removing rules
  • Reading grammars from files
  • Parsing macros
  • Basic helper functions
  • Storing grammars to files

Most if not all functionality can be taken from the ExprRules repository.

• Create repository for examples of working with the framework
• Figure out how to best show the examples (e.g. Jupyter notebook / Pluto.jl)
• Create Examples.jl label in Herb.jl

As crucial for many ongoing and coming projects, ML support for Herb.jl is highly important.

This module will focus on:

• Pre-training
• interplay with heuristics

• HerbGrammar.jl
• HerbSearch.jl
• HerbCore.jl
• HerbEvaluation.jl
• HerbData.jl
• HerbConstraints.jl
• Herb.jl

Add context-sensitive grammars to Grammar.jl.
(Parts of) functionality can be copied from ExprRules.
Also see Herb-AI/HerbSearch.jl#4 (comment).

• Add a function expand_heuristic that filters the grammar rules when expanding
• Add a function queue_priority that determines the value of a (partial) AST

Move iterators from ExprRules.jl to create simple enumeration search in Search.jl.

This package should provide the functionality to evaluate candidate programs.
In the beginning, it can only contain single evaluation criteria: run a program on input and see if produces the right output.
We can later add more sophisticated criteria.

Core functionality:

• take a program and examples, run the program on the input, and return the output

The Grammar.jl repository should contain all code related to declaring grammars and basic supporting functions (generating random expressions, getting the number of rules and so on...)

Core functionality:

• specifying grammars: add, remove rules, read grammars from files, parsing macros, basic helper functions, store
  grammars
• core type: context-free grammar

Others:

• different types of grammars: context-free, context-sensitive, probabilistic

Interfacing with other parts:

• no input dependencies from other packages
• Search.jl uses it to define iterators/search procedures

grammar = @cfgrammar begin
    Real = x                      # symbol
    Real = Real * Real            # julia expression
    Real = f(Real)                # function call
    Real = _(Base.rand(1.0:5.0))  # special syntax, eval argument of _() at derivation time
    Real = A | B | cos(Real)        # multiple rules on a single line
    Real = 1 | 2 | 3
    Real = |(4:6)                 # same as Real = 4 | 5 | 6
    Real = |([7,8,9])             # same as Real = 7 | 8 | 9 
end
   

• Herb-AI/HerbConstraints.jl#33
• Herb-AI/HerbInterpret.jl#24
• Herb-AI/HerbBenchmarks.jl@3c08d62 — won't make a PR directly as merging dev to master is currently 60+ commits
• #73

This repo should contain all functionality regarding specifying examples (in whatever form)

core functionality:

• input-output examples
• read examples from files

Additional functionality:

• traces
• specifications

This repo should contain functionality to discover specifications during the search.

Interfaces:

• SpecificationLanguage.jl to express constraints
• Search.jl to discover and prune
• Constraints.jl to express the constraints for Search.jl

Add support for constraints in the search.

This repo should contain functionality for learning to guide synthesis.

Interfacing:

• should produce a heuristic implement in Heuristics.jl?
• the heuristic should be passed to Search.jl

This repo should contain functionalities for program refactoring a la DreamCoder/Knorf.

Interfaces:

• Grammar.jl contains expressions it operates on

See https://github.com/Herb-AI/Search.jl/pull/7/files/f9fc7a131512f76cdf4aaf540f05a063c6b2d1af#r1114333679

Create a new root repo that composes a program synthesiser.

Core functionality:

• install all dependencies
• focus on user interaction, how the user should use the library: init, run, output

Additional functionality:

• benchmarking and basic config for default cases
• template notebooks

Create an extendible expand function

• Expand has an option for finding all holes
• Expand uses an external function for picking the hole

Update: Explicit Hole objects can now be added to the RuleNode tree, eliminating the problem described in the conversation below.

This repo should contain functionality to reason about the Specification of DSL/tasks.
This package should simply translate the specification to off-the-shelf solvers (Z3, Type theory reasoners, whatever we might need).

Interfaces:

• Takes a specification written in SpecificationLanguage.jl
• Should interface with Search.jl by producing constraints expressed in Constraints.jl

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Make Herb Julia-esk, exploiting what Julia is good at (2-3 people, including Sebastijan), i.e. multiple dispatch, non-pythonic code, reduce memory allocation.

• Stochastic search methods (by Piotr & Nikolai) (1-2 people, including Sebastijan)
• other early code (HerbCore, HerbGrammar, (HerbConstraints), ...)

• Make sure probabilities can change during search
• Create a wrapper around an existing grammar and add probabilities to every rule

• Remove #@TODOs from mainly HerbSearch, but check HerbConstraints, HerbCore, and HerbGrammar, too
• remove multi-line comments starting with #
• add documentation to all added functions, especially specific propagate!(...) functions, and getter/setter functions (otherwise they will not show up in the docs

Include a run of Documenter in each package to check for proper cross-references in docstrings

• Herb-AI/HerbCore.jl#15
• HerbConstraints.jl
• HerbInterpret.jl
• HerbGrammar.jl
• HerbSearch.jl
• HerbSpecification.jl

This should be a collection of all random utilities that we need across packages

Currently tests are tied to each sub-module respectively. It would be very beneficial for everybody to add a possibility to run the jointly. This might be a bit more intricate using Julia modules

CI throws Error: Cannot resolve @ref errors for broken crossreferences in multiple repos. For more details, see the individual PRs.

• Herb.jl: PR #62
• Herb-AI/HerbCore.jl#16
• Herb-AI/HerbGrammar.jl#65
• Herb-AI/HerbSpecification.jl#4
• Herb-AI/HerbInterpret.jl#23
• HerbSearch.jl