hbs-rcs / hbsgrid-docs Goto Github PK

View Code? Open in Web Editor NEW

3.0 5.0 4.0 129.49 MB

Documentation and discussion forum for the HBS grid technology preview

Home Page: https://hbs-rcs.github.io/hbsgrid-docs/

hbsgrid-docs's Introduction

Software Environments and Desktop Integrations for High Performance Computing

Author
Ista Zahn

Updated
2021-10-05

LifeCycle
This software is in development and may have bugs or other issues. Use at your own risk.

This project documents versioned data science environments and utilities available for use on the HBS Grid. These environments provide up-to-date open-source data science stacks including Python, R, Octave, QGIS, and integrate other software including Matlab, Stata, and SAS.

Documentation

Documentation is available in the user guide.

Discussion, feature suggestions etc.

Do you have a use case that doesn't seem well supported? An idea for making these environments more useful or easier to use? Or do you want to connect and share tips with other HBS Grid users? Head on over to the discusison forum!

Issues and bug reports

If you find something that doesn't work or doesn't work as expected please let us know by opening an issue so we can fix it. Telling us when something isn't working is a valuable service and we appreciate any and all bug reports!

hbsgrid-docs's People

Contributors

Stargazers

Watchers

Forkers

izahn williamtarr devbioinfoguy velmela

hbsgrid-docs's Issues

Sample code for MariaDB is out of date

See https://github.com/hbs-rcs/sample_code/blob/master/mariadb/R_MariaDB_fromGrid.R

Obtain and configure hbs sub-domain for github pages site

Although we plan to host the documentation on github, it will be good to use a HBS domain name. This will give our users more confidence that this is the official and supported HBS Grid documentation.

Changing environments / module loads needs rewrite

We have not explained fully to the user on how to change environments, permanently or temporarily. And the docs on doing so in the terminal/shell don't explain the limitations of this method.

We need a more comprehensive way to explain how to change things, why, and what's the effect. A table or decision tree could help significantly, in addition to steps on how to do so.

Corollary: explain how Grid3 works, so that persons understand the above table or decision tree.

-b

Broken R package in Tech Preview

Discussed in https://github.com/hbs-rcs/hbsgrid-docs/discussions/36

^{Originally posted by Econometrica17212 May 3, 2022}
As of today, when I try to use the rayshader package in R, trying to load it crashes my R session without fail. At first I thought it might be due to some conflict I had already loaded, but even if I first load in the rayshader package, R encounters a fatal error before it finishes loading. This is using the most up-to-date version of the tech preview environment. (2022-01)

Enter password to unlock pop-up

Whenever I open JupyterLab, I get a pop-up asking me to enter a password (see screenshot).

Romove disclaimers and re-brand techpreview as grid 3.0

I don't think we need the disclaimers as shown in the bold text below.

"cs_2022.01
After more than 6 months of hard work, the HBS grid technology preview software environment version 2022.01 was released in January 2022! This environment remains a technology preview and any and all use is at your own risk."

Also, throughout there should be some indication that if you need help you can email or call RCS.

Not being able to delete items from the trash

Title^

Desktop UI seems to be reset

When logging into NoMachine, I have icons on my desktop. After activating the Grid3-UI, the desktop icons are no longer displayed.

(I have not yet tried the TweaksUI tool to see if these are turned off)

Migrate "best practices for data transfers"

This page is generic for any file transfers, whether or not the HBSGrid cluster and research storage is involved.

We should probably canvas the User Guide Copying & Transferring Data so that it is current as well. (Aside, why is it titled 'sync' files? How many people actually sync, versus one-way?)

-B

Post READMEs on cluster storage volumes

Post README to help inform users of the purpose of the volumes, as well as features, characteristics; and how to access, transfer data, etc.

DBeaver pop-up

When trying to navigate within DBeaver, I get the following pop-up (and didn't want to inadvertently install anything on the Grid):

Thank you!

Documentation tweaks and improvements

The following issues were noted in discussion this morning:

Remove top-level blog link
Move hbs home link to the top banner
Add more direct links (e.g., to dbeaver connection docs, ssl instructions etc.)
Info boxes should have informative title, e.g., the one at the top of the research data storage page
~~- [ ] Add WRDS connection section to database page~~
Look into fixing stata tmp issue

Let me know if I missed anything @velmela

Also I don't think Katie has a github account, or she is not in the hbs-rcs organization. We should help her set that up so she can edit the docs directly.

Documentation Suggestions - 1

May want to change "These environments are in active development and are currently available as a Technology Preview for testing." to remove the Technology Preview language

Remove requests section and link out to sharepoint

For the account requests and other support for the grid, I think it might be better to point to our current website.

Loading into grid gives a black screen

Hi, I have been trying to access the grid since morning but every time I open it I get a black screen and I am stuck there. I have attached a screenshot of the same

.

Chrome job getting submitted but chrome never opens

Hi, I have not been able to open google chrome on the grid even when I open the application with just 4GB of RAM. I have checked "bjobs" and the job shows up there but the chrome application never launches.

Consider using versioning

Should be easy enough to maintain multiple versions of the documentation: https://squidfunk.github.io/mkdocs-material/setup/setting-up-versioning

Documentation Suggestions - 2

Update the environment versions.

Code examples on Parallelization tutorial might be outdate, garbled, or other problems

Saw this with MATLAB code. Other examples need to be reviewed!

And sorry to say, we might do a pass quickly through all the other examples on the site.

Use github actions to build documentation

Instructions at https://squidfunk.github.io/mkdocs-material/publishing-your-site/#with-github-actions

Not being able to add Dropbox to rcs clone

I am using the instructions given here https://hbs-rcs.github.io/hbsgrid-docs/syncfiles.html and the drive just does not show up in the remotes section

Reading SAS File in JupyterLab on Grid

Hi,

I have a large SAS file that I am trying to read in JupyterLab on the Grid. I keep getting the attached error. I did pip install sas7bdat. Your thoughts would be helpful.

Thanks

Improve login message to better inform HBSGrid users

Current terminal login message is somewhat helpful. Improve the message shown at login to put documentation front-and-center for both terminal (SSH) users as well as NoMachine users (method TBD).

Improve Available Resources utility to show unreserved memory

Discussed in https://github.com/hbs-rcs/hbsgrid-docs/discussions/14

^{Originally posted by Econometrica17212 January 25, 2022}
Hey all,

I'm having trouble getting an interactive job request launched and thought I'd share the issues so that others might learn from my mistakes.

I'm currently trying to launch a short_int job with 500G of RAM and 4 CPUs of RStudio. I'm working with a dataset that is 390GB, so I need the maximum amount of RAM for this. Luckily the operations I'm running shouldn't take up too much more RAM as I'm just doing some basic analysis and regressions, not data manipulation.
When I run this via the tech preview environment using the submission GUI, I get the following error:

Looking at the user guide, there is a video documenting a similar issue where the user requests 10 CPUs and 4 GB RAM. However, the error code is different. The video shows that the usage report demonstrates that no node has the 10 available CPUs, so the job is resubmitted with 2 CPUs and it works.

When I check the HBS Cluster usage monitor, it seems that there is ample room for the job to be run, e.g. on node 13. As a result, I'm a bit confused why I'm getting the error message here. It does also say when running bqueues that there are 28 pending jobs -- could this be why? Am I just behind in a long line of jobs right now? For other context, I am not running any other jobs, pending or running, on any queue, when attempting this.

Hopefully there is a simple explanation here that will help others understand too!

(also worth noting that the button in the first image does not say "Read Documentation," so that could be changed to make it clearer which button to press)