Giter Club home page Giter Club logo

node-lei-stream's Introduction

NPM version build status Test coverage David deps node version npm download npm license

lei-stream

Read/Write Stream line by line 按行读写流

安装

$ npm install lei-stream --save

需要 Node.js v4.0.0 或更高版本

readLine 按行读取流

使用方法1:

'use strict';

const readLine = require('lei-stream').readLine;

readLine('./myfile.txt').go((data, next) => {
  console.log(data);
  next();
}, function () {
  console.log('end');
});

使用方法2:

'use strict';

const fs = require('fs');
const readLine = require('lei-stream').readLine;

// readLineStream第一个参数为ReadStream实例,也可以为文件名
const s = readLine(fs.createReadStream('./myfile.txt'), {
  // 换行符,默认\n
  newline: '\n',
  // 是否自动读取下一行,默认false
  autoNext: false,
  // 编码器,可以为函数或字符串(内置编码器:json,base64),默认null
  encoding: function (data) {
    return JSON.parse(data);
  }
});

// 读取到一行数据时触发data事件
s.on('data', (data) => {
  console.log(data);
  s.next();
});

// 流结束时触发end事件
s.on('end', () => {
  console.log('end');
});

// 读取时出错
s.on('error', (err) => {
  console.error(err);
});

writeLine

按行写流

'use strict';

const fs = require('fs');
const writeLineStream = require('lei-stream').writeLine;

// writeLineStream第一个参数为WriteStream实例,也可以为文件名
const s = writeLineStream(fs.createWriteStream('./myfile.txt'), {
  // 换行符,默认\n
  newline: '\n',
  // 编码器,可以为函数或字符串(内置编码器:json,base64),默认null
  encoding: function (data) {
    return JSON.stringify(data);
  },
  // 缓存的行数,默认为0(表示不缓存),此选项主要用于优化写文件性能,写入的内容会先存储到缓存中,当内容超过指定数量时再一次性写入到流中,可以提高写速度
  cacheLines: 0
});

// 写一行
s.write(data, () => {
  // 回调函数可选
  console.log('wrote');
});

// 结束
s.end(() => {
  // 回调函数可选
  console.log('end');
});

// 写时出错
s.on('error', (err) => {
  console.error(err);
});

TailStream

监听文件新增内容变化的流(返回的是一个Readable Stream,具体可参考 Node.js API 文档)

'use strict';

const tailStream = require('lei-stream').tailStream;

// 创建流
const s = tailStream('./myfile.txt', {
  position: 'end', // end表示监听之前先定位到文件末尾,否则会先读取出文件之前的所有内容再开始监听
});

// 有新内容会触发data事件
s.on('data', data => {
  console.log(data);
});

License

The MIT License (MIT)

Copyright (c) 2015-2017 Zongmin Lei <[email protected]>
http://ucdok.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

node-lei-stream's People

Contributors

greenkeeperio-bot avatar leizongmin avatar sjfkai avatar tces1 avatar yukrain avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

node-lei-stream's Issues

如何追加写入

你好 麻烦问下
我的需求是在一个大文件中以追加的方式写入数据,我在使用中加上了{flags:'r+'} 但是查看文件还是只有一条数据
在我的文件中 我加入了 {flags:'r+'}


var s = writeLineStream(fs.createWriteStream('./myfile.txt',{flags:'r+'}), {

    newline: '\n',

    encoding: function (data) {

      return JSON.stringify(data);

    },

    cacheLines: 0

  });

在模块文件writeline.js中 我也加入了{flags:'r+'}

In node8,readLine throw error

if the file has lots of lines,such as 5000 lines
and run the script below,node8 will throw error:"RangeError: Maximum call stack size exceeded"

require('lei-stream').readLine('filename').go(function abc (data,next) {
	for(a of ['a']){
		console.log(data);
	}
	next();
});

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

autoNext = false 且文件行数过多时报错

autoNext: false 文件行数为 20000+

RangeError: Maximum call stack size exceeded
    at new TickObject (node.js:482:24)
    at process.nextTick (node.js:500:26)
    at ReadLineStream.next (C:\Git\programming-pearls\chapter1\node_modules\lei-stream\lib\readline.js:74:13)
    at ReadLineStream.<anonymous> (C:\Git\programming-pearls\chapter1\index.js:31:10)
    at emitOne (events.js:77:13)
    at ReadLineStream.emit (events.js:169:7)
    at ReadLineStream.next (C:\Git\programming-pearls\chapter1\node_modules\lei-stream\lib\readline.js:107:8)
    at ReadLineStream.<anonymous> (C:\Git\programming-pearls\chapter1\index.js:31:10)
    at emitOne (events.js:77:13)
    at ReadLineStream.emit (events.js:169:7)

暂时的解决方法:

// 读取到一行数据时触发data事件 
s.on('data', function (line) {
    console.log(line);
    process.nextTick(function () {
        s.next();
    });
});

为什么next函数执行了一定次数之后,程序自动退出了呢?

function readAndAnalysize2(file) {
let lines=0;
// readLineStream第一个参数为ReadStream实例,也可以为文件名
const s = readLine(fs.createReadStream(file), {
// 换行符,默认\n
newline: '\n',
// 是否自动读取下一行,默认false
autoNext: false
});

// 读取到一行数据时触发data事件
s.on('data', (data) => {
    console.log(data);
    lines++;
    if (lines < 20) {
        s.next();
    } else {
        console.log("wait!");
    }
});

// 流结束时触发end事件
s.on('end', () => {
    console.log('end');
});

// 读取时出错
s.on('error', (err) => {
    console.error(err);
});

s.next();

}

上面这个例子,执行了20次之后,程序自动退出了。预期情况应该是程序在空闲中,等待其他逻辑的执行。
源码我没有找到哪里有process.exit()的调用。

index.d.ts文件ts声明有问题

export function readLine(stream: ReadStream, options?: ReadLineStreamOptions): ReadableStream;
这里ReadableStream 应该是ReadLineStream

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.