Light

microsoft / vscode-iconv-lite-umd Goto Github PK

View Code? Open in Web Editor NEW

7.0 15.0 6.0 604 KB

A UMD version of iconv-lite to be used in VSCode.

License: MIT License

JavaScript 100.00%

vscode-iconv-lite-umd's Introduction

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

vscode-iconv-lite-umd's People

Contributors

Stargazers

Watchers

Forkers

gyzerok taffywrinkle claudiusgonzo jeanp413 standardgalactic codeeditorland

vscode-iconv-lite-umd's Issues

How to ensure that install is called before new version is published

@bpasero refs #5 (comment)

@gyzerok oh, it would be great if yarn could run as part of the prepublish to fix that in the future.

I've looked into it a bit. Unfortunately when you run yarn the prepublish hook is called automatically. Which means that if you put yarn inside prepublish, you get an infinite loop of installs.

Personally I've had great experiences publishing to npm from CI. This way you just push the new tag and CI will do the rest: install, build and publish.

If it sounds good I think you can use GitHub Actions to make this work. Unfortunately I can't really make it myself since you need to generate an npm secret to make it work.

Reopen big (400MB .csv) file with encoding

VSCode Version: 1.50.0-insider (user setup)
Commit: 9e505675670d65138405321a60b0df4ddec28799
Date: 2020-09-16T06:49:37.816Z
Electron: 9.3.0
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.19041

Steps to Reproduce:

Open a large (400MB -csv for example) windows 1252 file and click on the UTF-8 at the bottom right. And on reopen with encoding.
Ram grows and finally code crashes with oom :)

I guess it's the encoding guessing algorithm

Does this issue occur when all extensions are disabled?: Yes

Tests from iconv-lite are not passing

I did an experiment where I let all tests from iconv-lite run against this webpacked version. Here are the steps:

copy the test folder of iconv-lite into the root of this project
copy all devDependencies of iconv-lite into the package.json of this project
copy the "test": "mocha --reporter spec --grep ." over into the scripts section of the package.json of this project
remove the UTF-32/UTF-7 exclusions from webpack.config.js
ensure you are not webpacking with target: "node" but the default (web)
run yarn test

The result is a couple of test failures:

1) Full DBCS encoding tests Encode DBCS encoding 'shiftjis':
     Error: Bad argument.
      at convert (node_modules/iconv/lib/iconv.js:103:11)
      at Iconv.convert (node_modules/iconv/lib/iconv.js:64:12)
      at convertWithDefault (test/dbcs-test.js:44:26)
      at Context.<anonymous> (test/dbcs-test.js:230:32)
      at processImmediate (internal/timers.js:456:21)

  2) Full DBCS encoding tests Encode DBCS encoding 'eucjp':
     Error: Bad argument.
      at convert (node_modules/iconv/lib/iconv.js:103:11)
      at Iconv.convert (node_modules/iconv/lib/iconv.js:64:12)
      at convertWithDefault (test/dbcs-test.js:44:26)
      at Context.<anonymous> (test/dbcs-test.js:230:32)
      at processImmediate (internal/timers.js:456:21)

  3) Full DBCS encoding tests Encode DBCS encoding 'gb18030':
     Error: Bad argument.
      at convert (node_modules/iconv/lib/iconv.js:103:11)
      at Iconv.convert (node_modules/iconv/lib/iconv.js:64:12)
      at convertWithDefault (test/dbcs-test.js:44:26)
      at Context.<anonymous> (test/dbcs-test.js:230:32)
      at processImmediate (internal/timers.js:456:21)

  4) Full DBCS encoding tests Encode DBCS encoding 'cp950':
     Error: Bad argument.
      at convert (node_modules/iconv/lib/iconv.js:103:11)
      at Iconv.convert (node_modules/iconv/lib/iconv.js:64:12)
      at convertWithDefault (test/dbcs-test.js:44:26)
      at Context.<anonymous> (test/dbcs-test.js:230:32)
      at processImmediate (internal/timers.js:456:21)

  5) Full DBCS encoding tests Encode DBCS encoding 'big5hkscs':
     Error: Bad argument.
      at convert (node_modules/iconv/lib/iconv.js:103:11)
      at Iconv.convert (node_modules/iconv/lib/iconv.js:64:12)
      at convertWithDefault (test/dbcs-test.js:44:26)
      at Context.<anonymous> (test/dbcs-test.js:230:32)
      at processImmediate (internal/timers.js:456:21)

  6) Generic UTF8-UCS2 tests Return values are of correct types:

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(Buffer.isBuffer(iconv.encode(testString, "utf8")))

      + expected - actual

      -false
      +true
      
      at Context.<anonymous> (test/main-test.js:13:16)
      at processImmediate (internal/timers.js:456:21)

  7) Generic UTF8-UCS2 tests Convert to string, not buffer (utf8 used):

      AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:

  assert.ok(Buffer.isBuffer(res))

      + expected - actual

      -false
      +true
      
      at Context.<anonymous> (test/main-test.js:43:16)
      at processImmediate (internal/timers.js:456:21)

  8) UTF-16 decoder handles very short buffers nice:

      AssertionError [ERR_ASSERTION]: '\u0000' == ''
      + expected - actual

      -�
      
      at Context.<anonymous> (test/utf16-test.js:45:16)
      at processImmediate (internal/timers.js:456:21)

@gyzerok @ashtuchkin this worries me a bit. Some notes:

all tests run green when using target: "node" (not to any surprise)
I am seeing the same test errors when I try [email protected]

I would appreciate some help in fixing these issues as they would probably block the July release where I would like to go back to the webpacked version that does not target node. Thanks.

where is the lib folder

where is the lib folder?

What is this about?

The README.md doesn't even say what this software is about...

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.