Giter Club home page Giter Club logo

Build Status Documentation Status DOI

CernVM-File System (CernVM-FS)

The CernVM-File System provides a scalable, reliable and low-maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide-distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a FUSE module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs. Internally, CernVM-FS uses content-addressable storage and Merkle trees in order to maintain file data and meta-data. CernVM-FS uses outgoing HTTP connections only, thereby it avoids most of the firewall issues of other network file systems. It transfers data and meta-data on demand and verifies data integrity by cryptographic hashes.

By means of aggressive caching and reduction of latency, CernVM-FS focuses specifically on the software use case. Software usually comprises many small files that are frequently opened and read as a whole. Furthermore, the software use case includes frequent look-ups for files in multiple directories when search paths are examined.

Content is published into /cvmfs by means of dedicated "release manager machines". The release manager machines provide a writable CernVM-FS instance by means of a union file system (e.g., overlayfs) on top of the read-only client. When publishing, the CernVM-FS server tools process new and modified data from the union file system's writable branch and transform the data into the CernVM-FS storage format.

CernVM-FS is actively used by small and large scientific collaborations. In many cases, it replaces package managers and shared software areas on cluster file systems as means to distribute the software used to process experiment data.

Non-exhaustive List of Resources Related to CernVM-FS

  • Official Documentation
    • Aimed at maintainers of CernVM-FS instances, users and developers. Contains many in-depth explanations and a complete list of all (client) configuration parameters in the appendix.
  • Quickstart Guide for Developers
    • Aimed at developers. Includes how to contribute, code style, many short coding examples how to set up a working CernVM-File System, and how to test and debug.
  • cvmfs-contrib
    • Community-contributed packages related to CernVM-FS but not maintained by the CernVM-FS developer team.

How to Get in Touch With Us

  • CernVM Forum For support questions, or problems you encounter.
  • GitHub Issues For bug reporting or feature requests. This issue tracker is used for all CernVM-FS-related repositories.

CernVM File System's Projects

aufs2-standalone icon aufs2-standalone

Clone of the aufs2-standalone upstream repository (aufs.sorceforge.net) with patches for RHEL6

aufs2-util icon aufs2-util

Clone of the aufs2-util upstream repository (aufs.sorceforge.net)

cctools icon cctools

The Cooperative Computing Tools (cctools) enable large scale distributed computations to harness hundreds to thousands of machines from clusters, clouds, and grids.

conveyor icon conveyor

A high-level, job-based, interface for publishing to CernVM-FS repositories

cvmfs_info icon cvmfs_info

Collect and display repository replication status

cvmfsexec icon cvmfsexec

Mount cvmfs repositories as an unprivileged user

docker2cvmfs icon docker2cvmfs

A tool for importing existing Docker images into the cvmfs

minio icon minio

Minio is an open source object storage server compatible with Amazon S3 APIs

monitor-repos icon monitor-repos

A repository containing repositories' configuration for cvmfs-monitor

standalone icon standalone

Build system to creates a tarball with a standalone cvmfs client

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.