From a design perspective, the authors seem confused about the purposes and strengths of OpenMP, MPI, CUDA and “kernel”-style programming. OpenMP is designed to support multicore, shared-memory processing on a single machine. MPI supports programming a cluster of machines/nodes, where each has a processor and its own memory. The combination of OpenMP and MPI supports “hybrid” parallel computing—a cluster of machines/nodes, where each node is a multicore machine. GPU programming is good for data parallel programming, where the same set of instructions can be applied to multiple data elements in parallel. What the authors have said about all of these in the paper is confusing, and needs to be rewritten for clarity.

• rewrite openmp
• rewrite mpi
• remove cuda

Methodology, Processors section: The authors state that there are a “handful” of possible
ways to parallelize the algorithm. Please explicitly list these ways before discussing them.
Give the reader a roadmap to follow in your ideas.

Is Cluster Parallelism really better?

• Please rewrite Results section in terms of strong and weak scaling. That will clarify things
• Results: It is absolutely unclear how standard deviation and error are related to parallelism, in the report. The authors want to show that their fast, parallel proposed algorithm is also accurate enough, but this needs to be stated explicitly, and is NOT at all clear from what is presented. In fact, the authors state that “OpenMP results in such low standard error because it works with memory within the program and requires no network communication like MPI.” This seems to imply that network communication introduces error. This needs to be explained.
• Results: Please explain how the kernel (OS) scheduling more threads reduces error. Ideally, a sequential version of the algorithm and a parallel version should reach the same result, but the parallel version will do it much faster. Otherwise, there is a bug in the code somewhere.
• Results: Separate out the discussion of CUDA and its potential from the discussion of actual results. As-written, that is a distraction. Useful to think about for future work.

• Conclusions: First paragraph has at least three different topics. Consider splitting the discussion of these into separate paragraphs.
  • taken care of
• Conclusions: Consider moving the reference to figure 3 and the discussion around it to
  the Results section. Generally, new material should not be introduced in the Conclusion
  section of a paper.
  • this is not new information, but a recap; this was clarified
• Conclusion: Conclusion sections should summarize what was discussed in the paper and
  then describe unanswered questions and potential future work. Consider reorganizing the material in the conclusion to make this division clear.
  • clarified

Explain the inflection is a restriction with the current algorithm, and that future work can reduce this more.

Consider labelling the tables as Tables rather than Figures.

The captions to each of the figures and tables should give the reader some indication of what the data is attempting to show. Does figure 3 give me different information than figure 1? Does table 1 give me different information than table 1? It does not seem to me that they do.

• Why use a bar graph here? Line chart is more typical and expected.
  • switched to line charts

Methodology, Processors section: Comments about CUDA are confusing. It is better to
state what IS being done first, then later discuss options that were considered, but rejected. One thing that a reader might think is maybe to redesign the way matrices are used in order to make it work efficiently with GPUs, if that is important or desired. But there is not enough information here for the reader to decide, since the authors don’t tell us how matrices are actually used.

Future work and unanswered questions in ending of conclusion.

The search space reduction, K2, was a great start but there may still be room to improve with CUDA. Future work contains plans to implement the CUDA accelerations for the matrix operations that happen within the Bayesian network learning algorithm, but is significantly more difficult than the OpenMP and MPI implementation. The motivation for this is that CUDA has the potential to speed the algorithm up by several orders of magnitude.

Methodology: The authors here discuss that they are using synthetic data. This is OK. However, there is no discussion of what specific experiments or tests are being done, what algorithms or mathematical models are being used, how the consensus model is being generated or anything. If this information is in previous paper reference 6, some
of that needs to be repeated here so that the reader knows what is being done, EVEN if
that paper were available—which it is NOT.

• Abstract: Bayesian Networks (BNs) are slow as compared to what?
  • explain that is is NP-hard
• Abstract: K2 introduces what kind of bias?
  • added content
• Abstract: How does computing multiple (a population of?) avoid that bias?
• Abstract: The authors implement two new versions—multicore and cluster/hybrid
  parallel BNs?
  • Are there actually three versions, including a single-core sequential version for comparison?
  • clarified
• Abstract: Results show linear speedup for both multicore and cluster versions—linear speedup as compared to what? Linear speedup in terms of what: Problem size? Number of cores?
  • Already answered

Some of the required discussion of patterns of parallelism is sprinkled in the Results
section, but as a reader, I expect to see this information in Methodology explicitly, or in a separate section, depending on how it is organized. This is such an important idea to communicate that it deserves to be highlighted. Perhaps the confusion would be alleviated if the subsections called “Machines” were titled “Cluster Parallelism” or something similar instead. See relevant material on the topic of patterns by Jim Demmel at UC Berkley or Alan Kaminsky’s Big Computing Big Data book online for ideas on what I mean.

The first paragraph of the conclusion includes what is and is not parallelized.
This should be:

• highlighted in the abstract
  • added
• stated in the Introduction
  • already does so
• described in more detail within Methodology or a separate section
  • already does so

Is the review of Bayesian Networks, OpenMP, MPI and CUDA part of the Introduction?

• I suggest making that a separate section.
  • made it a separate section
• Also, it seems from what is written, that CUDA was not used. If so, don’t include it in the paper, other than to state that future work would incorporate CUDA and GPUs.
  • removed CUDA background

Give more background into the value of this work.
Possibly other papers showing bayesian networks being used for gene correlation (cancer)?

Abstract: It would be helpful to state explicitly that the goal is to propose a new hybrid (cluster and multicore) algorithm for generating consensus BNs that is both accurate and fast. Also explicitly state that one of the goals of the project was to learn OpenMP and MPI, and so the authors highlight the challenges involved.

• Please cite reference 6 explicitly near the end of the Introduction, where the connection to prior work is explained.
  • already is cited
• The Introduction is 2 paragraphs long. The first paragraph has multiple topics. Please use more paragraphs, one per main topic.
• Please explain: added new content
  • What question is being answered in this paper?
    • already explained This project examines the value of Bayesian network learning within a parallel environment in order to reduce the time needed to generate consensus networks using many topological inputs.
  • Why it is interesting?
    • already explained: Inferring relations among genes ... can then be used to make testable hypotheses to determine how gene interactions influence life in organisms or humans. As a result, tests can be performed in the lab with more confidence and a reduced chance of wasting time and resources.
  • Summarize
    • what experiments were done, results, and lessons learned?
    • What are you going to talk about in the rest of the paper?

• Sections should be numbered
  • see articles from a previous issue of the journal, for example

Reflect on the experience including such things as project organization, challenges, and solutions as well as the impact of the project on the students’ education and career outlook. See other student papers from previous issues of the journal, for example. Remove the discussion of these items from the conclusion.

Methodology: If the authors are willing to share their source code, or provide a library for others to use, please add a link to it in the references and cite it here.