Giter Club home page Giter Club logo

filey-diff's Introduction

Filey diff

[Build Status] (http://travis-ci.org/laurilehmijoki/filey-diff)

A Ruby library for comparing file-like objects from various data sources.

Central concepts

Filey

A file-like object. Can be, for example, a file system file or an AWS S3 object.

Data source

Provides Fileys.

The current built-in data sources support Amazon Web Services S3 and file system.

Operations

List outdated files

Given two data sources A and B, list the changed files that A has but B doesn't.

require 'aws-sdk'
require 'filey-diff'

s3 = AWS::S3.new(:access_key_id => 'some-id',
                 :secret_access_key => 'some-secret')
s3_bucket = s3.buckets['your-s3-bucket-name']

s3_data_source = Filey::DataSources::AwsSdkS3.new(s3_bucket)
fs_data_source = Filey::DataSources::FileSystem.new('/tmp/site-root')
Filey::Comparison.list_changed(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} has different contents on local file system than on S3"
}

List missing files

Given two data sources A and B, list the files that A has but B doesn't.

Filey::Comparison.list_missing(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} is missing from S3"
}

List changed files

Given two data sources A and B, list the files on A that have a different MD5 hash than the corresponding file on B.

Filey::Comparison.list_outdated(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} is newer on local file system than on S3"
}

AWS SDK data source

Specifing custom concurrency level

The concurrency level determines the amount of parallel operations that the AwsSdkS3 data source performs against the S3 API.

config = { :concurrency_level => 1000 }
s3_data_source = Filey::DataSources::AwsSdkS3.new(s3_bucket, config)

Example use cases

Arnie has a blog on AWS S3. He has just finished a new post and wants to upload only the new post into S3. With the help of Filey diff Arnie can write a Ruby program that uploads only the new post and nothing else.

License

Copyright (C) 2012 Lauri Lehmijoki

Distributed under the Apache-2.0 license http://www.apache.org/licenses/LICENSE-2.0.html

filey-diff's People

Contributors

laurilehmijoki avatar alexmarchant avatar jordanwhite avatar postmodern avatar stanislas avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.