Giter Club home page Giter Club logo

virtual-storage-manager's Introduction

VSM - Virtual Storage Manager

Travis CI: Build Status

Virtual Storage Manager (VSM) is software that Intel has developed to help manage Ceph clusters. VSM simplifies the creation and day-to-day management of Ceph cluster for cloud and datacenter storage administrators.

VSM enables OEMs and system integrators to ensure consistent cluster configuration through the use of pre-defined, standard cluster configurations, and as a result improves ease of cluster installation and operational reliability, and reduces maintenance and support costs.

VSM supports the creation of clusters containing a mix of hard disk drives (HDDs), Solid State storage, and SSD-cached HDDs, and simplifies management of the Ceph cluster using a system to organize servers and storage devices according to performance characteristics, intended use, and failure domain.

The VSM web-based user interface provides the operator with the ability to monitor overall cluster status, manage cluster hardware and storage capacity, inspect detailed operation status of Ceph subsystems, and attach Ceph pools to OpenStack Cinder.

VSM has been developed in Python using OpenStack Horizon as the starting point for the application framework, and has a familiar look and feel for both software developers and OpenStack administrators.

Important Notice and Contact Information

a) Open source VSM does not have a full-time support team and so would not be generally suitable for production use unless you can support it or have support from a third party. Before you use VSM, please understand the need to invest enough effort to learn how to use it effectively and to address possible bugs.

b) To help VSM develop further, please become an active member of the community and consider giving back by making contributions. We intend to make all open source VSM feature proposals public, and do all development publicly.

For other questions, contact [email protected] or [email protected]

Licensing

a) Intel source code is being released under the Apache 2.0 license.

b) Additional libraries used with VSM have their own licensing; refer to NOTICE for details.

Installation & Usage

Please refer to INSTALL.md or INSTALL.pdf to know how to install VSM, and wiki page to know how to get started.

Contributing

Please refer to wiki page to know how to get involved.

Resources

Wiki: (https://github.com/01org/virtual-storage-manager/wiki)

Issue tracking: (https://01.org/jira/browse/VSM)

Mailing list: (http://vsm-discuss.33411.n7.nabble.com/)

*Other names and brands may be claimed as the property of others.

virtual-storage-manager's People

Contributors

danferber avatar eduardegorov avatar iansoft avatar jcalcote avatar jiyou avatar lihaijing avatar lorraine19871987 avatar micheal12345678 avatar oguzy avatar ramkrsna avatar rzarzynski avatar swiftgist avatar yison avatar ywang19 avatar zhangjun-inspur avatar zhubx007 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

virtual-storage-manager's Issues

support to manage volume backup/snapshot

it's valuable to provide volume's snapshot management, some solution to generate snapshots, and save them to NFS. at some time, they might be restored on demand.
Note: this feature doesn't include VM snapshot.

RHEL 7.0 Support

known adaptability issues:
i) RHEL7 uses mariadb so should be 'systemctl start mariadb''
ii) Generally the ifconfig command does not exist on RHEL7. Use the 'ip' command instead. (You can install ifconfig but its use is discouraged)

VSM 2.1 ignores ceph parameters defined in cluster.manifest file during cluster creation.

I defined the the ceph parameter "osd_max_backfills 2" in the cluster.manifest. I restarted the VSM controller on the Management node and VSM agents on the storage nodes. Afterward, i created a cluster through the VSM GUI and found the osd_max_backfills parameter was not included in ceph.conf. Consequently, the osd_max_backfills parameter had the default value of 10

[root@storage1 ~]# ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config show |egrep backfil
"osd_max_backfills": "10",
"osd_backfill_full_ratio": "0.85",
"osd_backfill_retry_interval": "10",
"osd_backfill_scan_min": "64",
"osd_backfill_scan_max": "512",
"osd_kill_backfill_at": "0",
"osd_debug_skip_full_check_in_backfill_reservation": "false",
"osd_debug_reject_backfill_probability": "0",

requires access control for different features

in current feature set, some operations are dangerous, shall we apply an access control mechnism to distinguish different roles?

Generally, there will be 3 kind of roles:
administrator (deploy/build cluster...)
operator (operation and maintain cluster)
demo (read only)

VSM log messages numbered and documented

An operator of field service engineer can inspect VSM logs to determine the timing and sequence of operations and errors reported by VSM. The messages include a message number. The messages are documented in English to facilitate translation.

High availability for VSM controller

As vsm controller is one key component to manage ceph cluster, in production environment, one requirement is to make the controller highly available.

Ceph data collection

An operator can inspect/analyze up to 1 year of Ceph operational data with timestamps. (for what purpose?) (What data should be collected?)

PPA Repo?

The releases here would be easier to install on ubuntu if they were pushed to a PPA. I can set that up but it would be better if the project had an official PPA associated with it.

Trend data

An operator can observe trend data for capacity utilization and performance metrics (public and cluster) for 90 minute, 90 hour, and 90 day intervals

When importing cluster, if there is no OSd and MON sections, the import will fail

we noticed this issue as we were debugging our code on wip-53 dev branch. I put a comment to the vsm-discuss list asking if VSM dev team will be continuing this approach since modern ceph does not require these sections anymore.
I also asked in that thread: Does VSM read the /etc/ceph/ceph.conf file (ever)? If not, then we can contrive the missing sections during processing of our import script and simply concatenate the ceph.conf file with the dynamically discovered missing sections, using that composite as the ceph.conf data we pass to the VSM api.

Support email alerts

An operator can configure email alerts for a) OSDs over near full threshold, b) storage groups over near full threshold, c) servers that have become unavailable, d) OSDs that have been placed in autoout, c) too many incidents of long latency or client timeout

there is no cluster_manifest supplied during the installation

Seems there are two files that can be used:

virtual-storage-manager/source/vsm/bin/cluster_manifest
virtual-storage-manager/source/vsm-deploy/cluster_manifest

None is at the $PATH so whenever cluster_manifest is called, it is giving errors.

Which is the correct file to be used during install.sh and vsm-installer?

Support SNMP alerts

An operator can configure SNMP alerts for a) OSDs over near full threshold, b) storage groups over near full threshold, c) servers that have become unavailable, d)OSDs that have been placed in autoout, c) too many incidents of long latency or client timeout

Don’t start VSM operations under fault conditions

The operator cannot start certain operations if the cluster is temporarily in a fault condition. (need to be more specific about what fault conditions - for instance, want to be able to add a server is Ceph is reporting OSD full)

Allow VSM to Delete Pools

Allows pools to be deleted. This is dangerous in a production environment but useful in a POC environment. Perhaps this feature could be controlled via CLI so the Feature could be disabled when the cluster is in production.

require MDS management

we have MDS status check now, and user also hopes to have MDS management capabilities.

VSM Should Support Multiple Operator Roles

"VSM should support two operator roles:
Admin: Can perform all VSM operations
Operator: Can perform routine maintenance - restart OSDs, remove OSDs, restore OSDs"

Grant specific roles to each operator - for example create an operator with permission to create new storage groups and add servers (less likely to cause data loss or disrupt service) but without permission to remove servers (more likely to disrupt service or cause data loss)

support to configure and manage ceph client

VSM doesn’t configure ceph client while building ceph cluster i.e we would have to configure the clients manually with admin/gatherkeys. It’s not bad but I think it would be great to have an option for configuring client as well with VSM.

list all vsm related service status on UI

occasionally, we see weird behaviors if some vsm processes stop, it's meaningful to show all vsm related processes status on UI to help quick fix.

the feature will add on dashboard to show all vsm critical service status, like vsm daemon, mariadb, keystone, rabbitmq, diamond...

Add complete set of Juno features in Horizon; pluggable settings, etc.

This feature request came from the following vsm-discuss thread:
http://vsm-discuss.33411.n7.nabble.com/VSM-horizon-framework-version-update-td128.html
Version 2.0 currently has been updated to Horizon "Juno", for features that were already in the 1.0 code base when it was on Havana, however, new post-Havana features have not been added - such as pluggable settings:
http://docs.openstack.org/developer/horizon/topics/settings.html#pluggable-settings
Pluggable settings would make extending VSM much simpler for third-parties.

Sequence VSM operations

this ensures that an operator cannot start a new operation while another incompatible operation is under way.

The operator cannot start operations until potentially conflicting operation already in flight are completed successfully and failed gracefully. In one implementation, Ceph executes operations fro a queue.

VSM Needs to Report More Detailed Errors, and Problem Solving Information

This refers specifically to providing more detailed problem solving information for getting VSM set up for the first time.
VSM has insufficient error information available when things go wrong. This starts with little explanation on what the various logs are for and what the messages in them show - but also extends to insufficient detailed error messages and codes available.

Client monitoring enhancements for RBD clients and RADOS gateway

An operator can identify RBD clients that are hot (busy). An operator can identify RADOS Gateway clients that tare hot (busy).

"Would be useful to have a client status page that showed the performance (iOS, mb/s) for each RBD.
Since there can be a lot RBDs, this page would require pagination (e.g. 100 RBDs per page) and sort (e.g. sort by IOPS or sort by mb/s)."

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.