Giter Club home page Giter Club logo

codec's Introduction

level-codec

Encode keys, values and range options, with built-in or custom encodings.

📌 This module will soon be deprecated, because it is superseded by level-transcoder.

level badge npm Node version npm Test Coverage Status JavaScript Style Guide Backers on Open Collective Sponsors on Open Collective

Usage

If you are upgrading: please see UPGRADING.md.

const Codec = require('level-codec')
const codec = Codec({ keyEncoding: 'json' })
const key = codec.encodeKey({ foo: 'bar' })
console.log(key) // -> '{"foo":"bar"}'
console.log(codec.decodeKey(key)) // -> { foo: 'bar' }

API

codec = Codec([opts])

Create a new codec, with a global options object.

codec.encodeKey(key[, opts])

Encode key with given opts.

codec.encodeValue(value[, opts])

Encode value with given opts.

codec.encodeBatch(batch[, opts])

Encode batch ops with given opts.

codec.encodeLtgt(ltgt)

Encode the ltgt values of option object ltgt.

codec.decodeKey(key[, opts])

Decode key with given opts.

codec.decodeValue(value[, opts])

Decode value with given opts.

codec.createStreamDecoder([opts])

Create a function with signature (key, value), that for each key-value pair returned from a levelup read stream returns the decoded value to be emitted.

codec.keyAsBuffer([opts])

Check whether opts and the global opts call for a binary key encoding.

codec.valueAsBuffer([opts])

Check whether opts and the global opts call for a binary value encoding.

codec.encodings

The builtin encodings as object of form

{
  [type]: encoding
}

See below for a list and the format of encoding.

Builtin Encodings

Type Input Stored as Output
utf8 String or Buffer String or Buffer String
json Any JSON type JSON string Input
binary Buffer, string or byte array Buffer As stored
hex
ascii
base64
ucs2
utf16le
utf-16le
String or Buffer Buffer String
none a.k.a. id Any type (bypass encoding) Input* As stored

* Stores may have their own type coercion. Whether type information is preserved depends on the abstract-leveldown implementation as well as the underlying storage (LevelDB, IndexedDB, etc).

Encoding Format

An encoding is an object of the form:

{
  encode: function (data) {
    return data
  },
  decode: function (data) {
    return data
  },
  buffer: Boolean,
  type: 'example'
}

All of these properties are required.

The buffer boolean tells consumers whether to fetch data as a Buffer, before calling your decode() function on that data. If buffer is true, it is assumed that decode() takes a Buffer. If false, it is assumed that decode takes any other type (usually a string).

To explain this in the grand scheme of things, consider a store like leveldown which has the ability to return either a Buffer or string, both sourced from the same byte array. Wrap this store with encoding-down and it'll select the most optimal data type based on the buffer property of the active encoding. If your decode() function needs a string (and the data can legitimately become a UTF8 string), you should set buffer to false. This avoids the cost of having to convert a Buffer to a string.

The type string should be a unique name.

Contributing

Level/codec is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Donate

To sustain Level and its activities, become a backer or sponsor on Open Collective. Your logo or avatar will be displayed on our 28+ GitHub repositories and npm packages. 💖

Backers

Open Collective backers

Sponsors

Open Collective sponsors

License

MIT © 2012-present Contributors.

codec's People

Contributors

achingbrain avatar cristianossd avatar dcousens avatar dependabot[bot] avatar dominictarr avatar greenkeeper[bot] avatar greenkeeperio-bot avatar hugomrdias avatar juliangruber avatar meirionhughes avatar ralphtheninja avatar vweevers avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codec's Issues

Clarification on buffer : boolean

I'm currently exploring the idea of making the codec typing a little more strict, but need some clarification.

From what I can see; a codec encoder looks like this:

  type CodecEncoder<I> = CodecStringEncoder<I> | CodecBufferEncoder<I>

  interface CodecStringEncoder<I> {
    encode: (val: I) => string;
    decode: (val: string) => I;
    buffer: false;
    type: string
  }

  interface CodecBufferEncoder<I> {
    encode: (val: I) => Buffer;
    decode: (val: Buffer) => I;
    buffer: true;
    type: string
  }

such that:

  • if buffer === false then it is expected that the encoder will

    • encode to a string
    • decode from a string
  • if buffer === true then it is expected that the encoder will

    • encode to a Buffer
    • decode from a Buffer

Is this correct?

cc: @vweevers

number bug

console.log(encode(1000000000)>encode(100000000));
echo false

Version 10 of node.js has been released

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

  • Added the new Node.js version to your .travis.yml

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

More information on this issue

Greenkeeper has checked the engines key in any package.json file, the .nvmrc file, and the .travis.yml file, if present.

  • engines was only updated if it defined a single version, not a range.
  • .nvmrc was updated to Node.js 10
  • .travis.yml was only changed if there was a root-level node_js that didn’t already include Node.js 10, such as node or lts/*. In this case, the new version was appended to the list. We didn’t touch job or matrix configurations because these tend to be quite specific and complex, and it’s difficult to infer what the intentions were.

For many simpler .travis.yml configurations, this PR should suffice as-is, but depending on what you’re doing it may require additional work or may not be applicable at all. We’re also aware that you may have good reasons to not update to Node.js 10, which is why this was sent as an issue and not a pull request. Feel free to delete it without comment, I’m a humble robot and won’t feel rejected 🤖


FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Add buffer decoding to the `json` encoding

When you use the json encoding and store a buffer you get back an object like {type: 'Buffer': data: [...]}. It would be nice to augment the JSON.parse with buffer handling. Is this something that you would all be interested in accepting a PR for?

Update description

"Encoding logic for levelup" is outdated. Update:

  • GH description
  • package.json
  • README.md

update readme

  • Add node badge
  • Fix api style
  • Remove Publishers section (I added it but doesn't really mean much)
  • Fix copyright year
  • levelup -> level-codec

api

This module currently has 2 apis, which differ in the way encodings are inherited. Those two snippets are equivalent:

value = encodeValue(value, [{ valueEncoding: 'hex' }, db.options]);
var codec = new Codec(db.options);
value = codec.encodeValue(value, { valueEncoding: 'hex' })

The first is more generic as it supports inheritance over n encoding options, whereas the second one works exactly with what we need in levelup.

We could change the class api to take n encoding options too:

var codec = new Codec([optionsA, optionsB]);
value = codec.encodeValue(value, [optionsC, optionsD]);

I guess the real question is: Do we want to make this module truly generic, or tie it to it's main use cases: levelup and multilevel.

Input from people having to use levelup's codec logic would help a lot.

@ralphtheninja @dominictarr @substack

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because we are using your CI build statuses to figure out when to notify you about breaking changes.

Since we did not receive a CI status on the greenkeeper/initial branch, we assume that you still need to configure it.

If you have already set up a CI for this repository, you might need to check your configuration. Make sure it will run on all new branches. If you don’t want it to run on every branch, you can whitelist branches starting with greenkeeper/.

We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

Once you have installed CI on this repository, you’ll need to re-trigger Greenkeeper’s initial Pull Request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper integration’s white list on Github. You'll find this list on your repo or organiszation’s settings page, under Installed GitHub Apps.

An in-range update of tape is breaking the build 🚨

The devDependency tape was updated from 4.9.2 to 4.10.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

tape is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build failed (Details).

Commits

The new version differs by 60 commits.

  • 34b1832 v4.10.0
  • 6209882 Merge all orphaned tags: 'v1.1.2', 'v2.0.2', 'v2.1.1', 'v2.2.2', 'v2.3.3', 'v2.4.3', 'v2.5.1', 'v2.6.1', 'v2.7.3', 'v2.8.1', 'v2.9.1', 'v2.10.3', 'v2.11.1', 'v2.13.4', 'v2.14.0', 'v2.14.1', 'v3.6.1'
  • 82e7b26 [Deps] update glob
  • 9e3d25e [Dev Deps] update eslint, js-yaml
  • fd807f5 v1.1.2
  • eddbff5 v2.14.1
  • 6ce09d9 Minor test tweaks due to whitespace differences in v2 vs v4.
  • 71af8ba gitignore node_modules
  • 4c0d9e6 Merge pull request #268 from ljharb/throws_non_function_should_fail
  • d0a675f v3.6.1
  • d22b5fc Minor test tweaks due to output differences in v1 vs v4.
  • 8b3c1b7 Add missing concat-stream devDep
  • 3495543 gitignore node_modules
  • db81846 Merge pull request #268 from ljharb/throws_non_function_should_fail
  • 7ed6651 Minor test tweaks due to whitespace differences in v3 vs v4.

There are 60 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Question on coverage

So, given the current state we have the following results:

screenshot from 2018-07-19 12-08-46

I want to write a test that covers the following line

codec/index.js

Line 97 in 63d29a5

return function () {}

So I add the following test:

  t.test('no keys and no values not decoding', function (t) {
    var decoder = codec.createStreamDecoder({})
    t.equal(decoder(null, Buffer.from('hey')), undefined)
    t.equal(decoder(Buffer.from('hey')), undefined)
    t.end()
  })

The test works (passes) and covers the line correctly. However, for some reason nyc reports new lines in the code that aren't covered, which seems a bit weird to me. How can adding an extra test remove coverage? At worst it it's a bad test that doesn't add coverage, but it should never remove coverage imo. See below:

screenshot from 2018-07-19 12-15-23

What's going on here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.