qiniu / iconv Goto Github PK
View Code? Open in Web Editor NEWGolang bindings to libiconv - Convert string to requested character encoding
License: Apache License 2.0
Golang bindings to libiconv - Convert string to requested character encoding
License: Apache License 2.0
In iconv.go
// #cgo darwin LDFLAGS: -liconv_darwin_amd64 -L${SRCDIR}
// #cgo windows LDFLAGS: -liconv_windows_amd64 -L${SRCDIR}
filelist in root dir:
iconv.go
...
libiconv_darwin_amd64.a
libiconv_windows_amd64.a
So we do not need to install libiconv.
Also, the program will be built with the the libiconv.a archived in it , instead of running with dynamic libraries.
Refer to github.com/andlabs/ui
感谢大大们提供的绑定,好用的飞起~
在使用ConvString时发现一个问题,当s的长度过长后,转换的结果与预期不符,转换后的字符串的后半段发生了与前半段重复的情况
如原字符串为
......
A
B
C
D
E
F
转换后成了
........
A
B
C
B
C
B
C
看了下源代码
func (cd Iconv) ConvString(s string) string {
var outbuf [512]byte
if s1, _, err := cd.Conv([]byte(s), outbuf[:]); err != nil {
return ""
} else {
return string(s1)
}
}
这里的变量是var outbuf [512]byte,不知道为什么是512呢?
个人觉得
var outbuf []byte
length := bytes.Count([]byte(s), nil)
outbuf = make([]byte, length)
这样应该更好啊。。
对golang以及这里的出发点都不太熟悉。还请大家指导!
package main
import (
"fmt"
"github.com/qiniu/iconv"
)
func main() {
cd, err := iconv.Open("shift-jis", "utf-8")
if err != nil {
fmt.Println("iconv.Open failed!")
return
}
defer cd.Close()
result := cd.ConvString("你好,世界!")
fmt.Println(result)
}
When you run above program, you will get nothing because of "你好,世界!" is not supported by shift-jis.
I think when you are trying to convert some chars unsupported by the target encoding, you should get a error but not just the blank string?
I'm using the example found here (https://github.com/qiniu/iconv/blob/develop/examples/convstr/convstr.go) and I get signal: killed
each time. I have verified my iconv install is fine by calling it directly via command line.
I am on OS X Sierra.
Is there something I am missing?
C:\Users\****\go\src\****\*****>set GOOS=linux
C:\Users\****\go\src\****\*****>set GOARCH=amd64
C:\Users\****\go\src\****\*****>set CGO_ENABLED=0
C:\Users\****\go\src\****\*****>go build
# github.com/qiniu/iconv
..\..\..\pkg\mod\github.com\qiniu\[email protected]\reader.go:10:5: undefined: Iconv
..\..\..\pkg\mod\github.com\qiniu\[email protected]\reader.go:17:19: undefined: Iconv
..\..\..\pkg\mod\github.com\qiniu\[email protected]\reader.go:18:30: undefined: DefaultBufSize
..\..\..\pkg\mod\github.com\qiniu\[email protected]\reader.go:42:14: undefined: EILSEQ
..\..\..\pkg\mod\github.com\qiniu\[email protected]\writer.go:11:5: undefined: Iconv
..\..\..\pkg\mod\github.com\qiniu\[email protected]\writer.go:17:19: undefined: Iconv
..\..\..\pkg\mod\github.com\qiniu\[email protected]\writer.go:18:30: undefined: DefaultBufSize
..\..\..\pkg\mod\github.com\qiniu\[email protected]\writer.go:63:30: undefined: EILSEQ
..\..\..\pkg\mod\github.com\qiniu\[email protected]\writer.go:89:30: undefined: EILSEQ
In your example
import (
"fmt"
"github.com/qiniu/iconv"
)
func main() {
cd, err := iconv.Open("gbk", "utf-8")
if err != nil {
fmt.Println("iconv.Open failed!")
return
}
defer cd.Close()
gbk := cd.ConvString("你好,世界!")
fmt.Println(gbk)
}
there is a missing package main
at the very beggining
sh-3.2# CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build autohome.go
github.com/qiniu/iconv
../../github.com/qiniu/iconv/reader.go:10: undefined: Iconv
../../github.com/qiniu/iconv/reader.go:17: undefined: Iconv
../../github.com/qiniu/iconv/reader.go:18: undefined: DefaultBufSize
../../github.com/qiniu/iconv/reader.go:42: undefined: EILSEQ
../../github.com/qiniu/iconv/writer.go:11: undefined: Iconv
../../github.com/qiniu/iconv/writer.go:17: undefined: Iconv
../../github.com/qiniu/iconv/writer.go:18: undefined: DefaultBufSize
../../github.com/qiniu/iconv/writer.go:63: undefined: EILSEQ
../../github.com/qiniu/iconv/writer.go:89: undefined: EILSEQ
编译命令是 CGO_ENABLED=1 GOOS=windows GOARCH=amd64 go build -o bin/xxxx.exe client
如果 CGO_ENABLED=0 就报错src/common/iconv/reader.go:10: undefined: Iconv
求教这个是哪里有问题?
runtime error: cgo argument has Go pointer to Go pointer
GOOS=windows GOARCH=amd64 go build main.go
on Docker gives the following error when this command is run.
/go/pkg/mod/github.com/qiniu/[email protected]/reader.go:11:11: undefined: Iconv
/go/pkg/mod/github.com/qiniu/[email protected]/reader.go:19:19: undefined: Iconv
/go/pkg/mod/github.com/qiniu/[email protected]/reader.go:21:13: undefined: DefaultBufSize
/go/pkg/mod/github.com/qiniu/[email protected]/reader.go:49:14: undefined: EILSEQ
/go/pkg/mod/github.com/qiniu/[email protected]/writer.go:12:11: undefined: Iconv
/go/pkg/mod/github.com/qiniu/[email protected]/writer.go:19:19: undefined: Iconv
/go/pkg/mod/github.com/qiniu/[email protected]/writer.go:21:13: undefined: DefaultBufSize
/go/pkg/mod/github.com/qiniu/[email protected]/writer.go:70:15: undefined: EILSEQ
/go/pkg/mod/github.com/qiniu/[email protected]/writer.go:102:15: undefined: EILSEQ
Can you check please?
系统环境:mac osx 10.8.2
go get github.com/qiniu/iconv
出现以下报错
Undefined symbols for architecture x86_64:
"_iconv", referenced from:
__cgo_fa90eb1c31fe_C2func_iconv in iconv.cgo2.o
__cgo_fa90eb1c31fe_Cfunc_iconv in iconv.cgo2.o
(maybe you meant: __cgo_fa90eb1c31fe_Cfunc_iconv, __cgo_fa90eb1c31fe_C2func_iconv_open , __cgo_fa90eb1c31fe_Cfunc_iconv_close , __cgo_fa90eb1c31fe_Cfunc_iconv_open , __cgo_fa90eb1c31fe_C2func_iconv_close , __cgo_fa90eb1c31fe_C2func_iconv )
"_iconv_close", referenced from:
__cgo_fa90eb1c31fe_Cfunc_iconv_close in iconv.cgo2.o
__cgo_fa90eb1c31fe_C2func_iconv_close in iconv.cgo2.o
(maybe you meant: __cgo_fa90eb1c31fe_Cfunc_iconv_close, __cgo_fa90eb1c31fe_C2func_iconv_close )
"_iconv_open", referenced from:
__cgo_fa90eb1c31fe_Cfunc_iconv_open in iconv.cgo2.o
__cgo_fa90eb1c31fe_C2func_iconv_open in iconv.cgo2.o
(maybe you meant: __cgo_fa90eb1c31fe_C2func_iconv_open, __cgo_fa90eb1c31fe_Cfunc_iconv_open )
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
package main
import (
"fmt"
"github.com/qiniu/iconv"
)
func main() {
cd, err := iconv.Open("gbk", "utf-8") // convert utf-8 to gbk
if err != nil {
fmt.Println("iconv.Open failed!")
return
}
defer cd.Close()
gbk := cd.ConvString("你好,世界!")
fmt.Println(gbk)
}
ld: warning: object file (/var/folders/5f/_ck73t8x3172pffm63n58_4m0000gn/T/go-link-935370207/000002.o) was built for newer macOS version (12.0) than being linked (11.3)
same as github.com/djimenez/iconv-go ????????
目前只有
// #cgo darwin LDFLAGS: -liconv
建议加上:
// #cgo linux LDFLAGS: -liconv
// #cgo darwin LDFLAGS: -liconv
// #cgo windows LDFLAGS: -liconv
我是windows8,在执行go get github.com/qiniu/iconv时报如下错误
github.com/qiniu/iconv
iconv.go:6:20: fatal error: iconv.h: No such file or directory
compilation terminated.
群里有朋友说是因为系统问题,linux底下就好了,特此问下有没有windows底下的解决办法?
Can you improve your doc?
What iconv.Open("iso8859-1", "utf-8")
means? from iso8859-1 to utf-8 or the opposite?
由于外网被强了,因此克隆后直接使用,但是出现了如下错误:
/tmp/go-build377085664/util/iconv/_obj/iconv.cgo2.o:在函数‘bridge_iconv’中:
util/iconv/iconv.go:16:对‘libiconv’未定义的引用
/tmp/go-build377085664/util/iconv/_obj/iconv.cgo2.o:在函数‘_cgo_f7d217435582_C2func_iconv_close’中:
util/iconv/iconv.go:88:对‘libiconv_close’未定义的引用
/tmp/go-build377085664/util/iconv/_obj/iconv.cgo2.o:在函数‘_cgo_f7d217435582_C2func_iconv_open’中:
util/iconv/iconv.go:110:对‘libiconv_open’未定义的引用
/tmp/go-build377085664/util/iconv/_obj/iconv.cgo2.o:在函数‘_cgo_f7d217435582_Cfunc_iconv_close’中:
util/iconv/iconv.go:163:对‘libiconv_close’未定义的引用
/tmp/go-build377085664/util/iconv/_obj/iconv.cgo2.o:在函数‘_cgo_f7d217435582_Cfunc_iconv_open’中:
util/iconv/iconv.go:181:对‘libiconv_open’未定义的引用
collect2: 错误:ld 返回 1
感觉像是动态库找不到的问题,我发现自己的libconv.so在/usr/local/lib目录下,然后我有专门在/etc/ld.so.conf.d/ 添加了一条记录,ldconfig 之后 重新编译还是失败。
应该怎么做呢?(;′⌒`)
sh-3.2# go get github.com/qiniu/iconv
Undefined symbols for architecture x86_64:
"_libiconv", referenced from:
__cgo_6945b30ee13d_C2func_iconv in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv in iconv.cgo2.o
"_libiconv_close", referenced from:
__cgo_6945b30ee13d_C2func_iconv_close in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv_close in iconv.cgo2.o
"_libiconv_open", referenced from:
__cgo_6945b30ee13d_C2func_iconv_open in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv_open in iconv.cgo2.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
不知道iconv是否是goroutine安全的?
Hello! I'm not sure if it's issue with iconv or Go iteself, but my program randomly crashes. The code:
func main() {
in := make(chan []byte)
dir := os.Args[1]
files, err := ioutil.ReadDir(dir)
if err != nil {
panic(err)
}
go func() {
cd, err := iconv.Open("UTF-8", "CP932")
if err != nil {
panic(err)
}
for data := range in {
cd.ConvString(string(data))
}
}()
iterations := 1000
for i := 0; i < iterations; i++ {
for _, file := range files {
file, err := os.Open(path.Join(dir, file.Name()))
if err != nil {
panic(err)
}
data, err := ioutil.ReadAll(file)
if err != nil {
panic(err)
}
in <- data
file.Close()
}
}
close(in)
}
And after some (random) time executing this code program crashes with:
runtime:objectstart Span weird: p=0xc820880000 k=0x6410440 s.start=0xc820880000 s.limit=0xc820884000 s.state=2
fatal error: objectstart: bad pointer in unexpected span
exit status 2
1: running [Created by runtime.gcBgMarkStartWorkers @ mgc.go:1239]
runtime panic.go:527 throw(0x542a00, 0x2b)
runtime mbitmap.go:217 heapBitsForObject(0xc820880000, 0, 0, 0xc800000000, 0x7fa7d82f41a0)
runtime mgcmark.go:878 scanobject(0xc820770208, #1)
runtime mgcmark.go:689 gcDrain(#1, 0x7d0)
runtime mgc.go:1320 gcBgMarkWorker(0xc820028000)
runtime asm_amd64.s:1696 goexit()
1: runnable [locked] [Created by main.main @ test.go:44]
iconv _cgo_gotypes.go:49 _C2func_iconv(#14, #4, #9, #12, #11, 0xffffffffffffffff, 0, 0)
iconv iconv.go:102 Iconv.DoWrite(#14, #15, #10, #8, 0x1a38, 0x1e04, 0x1a38, #5, 0x200, 0x200, ...)
iconv iconv.go:61 Iconv.Conv(#14, #7, 0x1c34, 0x2000, #5, 0x200, 0x200, 0, 0, 0, ...)
iconv iconv.go:68 Iconv.ConvString(#14, #6, 0x1c34, 0, 0)
main test.go:42 main.func1(#2)
1: runnable
syscall asm_linux_amd64.s:18 Syscall(0x3, 0x3, 0, 0, 0, 0, 0)
syscall zsyscall_linux_amd64.go:296 Close(0x3, 0, 0)
os file_unix.go:126 (*file).close(#3, 0, 0)
os file_unix.go:118 (*File).Close(#13, 0, 0)
main test.go:58 main()
1: syscall [locked]
runtime asm_amd64.s:1696 goexit()
I'm using Go 1.5.2 installed with gvm under Ubuntu Linux 14.04.3 LTS (x86_64). After some googling i found this issue, but it supposedly fixed in 1.5.2.
Any advice?
有什么包需要依赖么
ret, err := C.iconv_open(C.CString(tocode), C.CString(fromcode))
这里两个CString没有释放内存
go get github.com/qiniu/iconv
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_Cfunc_iconv_open': /home/gopath/src/github.com/qiniu/iconv/iconv.go:119: undefined reference to
libiconv_open'
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_Cfunc_iconv_close': /home/gopath/src/github.com/qiniu/iconv/iconv.go:108: undefined reference to
libiconv_close'
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_Cfunc_iconv': /home/gopath/src/github.com/qiniu/iconv/iconv.go:97: undefined reference to
libiconv'
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_C2func_iconv_open': /home/gopath/src/github.com/qiniu/iconv/iconv.go:73: undefined reference to
libiconv_open'
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_C2func_iconv_close': /home/gopath/src/github.com/qiniu/iconv/iconv.go:59: undefined reference to
libiconv_close'
/tmp/go-build074892034/github.com/qiniu/iconv/_obj/iconv.cgo2.o: In function _cgo_6945b30ee13d_C2func_iconv': /home/gopath/src/github.com/qiniu/iconv/iconv.go:45: undefined reference to
libiconv'
collect2: ld 返回 1
ps: 已解决,原来是装了两个libiconv
Hello,
it seems not to be possible to use //TRANSLIT
when converting from one charset to another. It seems //IGNORE
is always the default.
When using iconv on CLI it works fine for me:
$ echo -e "ç ß ∑" | iconv -t ASCII//TRANSLIT -f UTF-8
c ss ?
sh-3.2# go get github.com/qiniu/iconv
Undefined symbols for architecture x86_64:
"_libiconv", referenced from:
__cgo_6945b30ee13d_C2func_iconv in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv in iconv.cgo2.o
"_libiconv_close", referenced from:
__cgo_6945b30ee13d_C2func_iconv_close in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv_close in iconv.cgo2.o
"_libiconv_open", referenced from:
__cgo_6945b30ee13d_C2func_iconv_open in iconv.cgo2.o
__cgo_6945b30ee13d_Cfunc_iconv_open in iconv.cgo2.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
//conv.go
package main
import (
"fmt"
"github.com/qiniu/iconv"
)
func main() {
cd, err := iconv.Open("gbk", "utf-8")
if err != nil {
fmt.Println("iconv.Open failed!")
return
}
defer cd.Close()
gbk := cd.ConvString(你好,世界!你好,
)
fmt.Println(gbk)
}
go build conv.go 可以编译成功!
但使用 go tool 6g -o conv.6 conv.go 编译时
报如下错误
can't find import: "githum.com/qiniu/iconv 不知道为什么?
Go-getting the package results in the following error:
[path-to-mingw-lib]../lib/libiconv.a(localcharset.o):localcharset.c (.text+0x8): undefined reference to `__imp_GetACP'
collect2.exe: error: ld returned 1 exit status
MingW version 5.2.0, Windows 64-bit build, Win10
go run是可以的,go build或者go install 就报这个错误../gopkg.in/iconv.v1/reader.go:10:5: undefined: Iconv
C:\Users\ADMINI1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function 1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function cgo_74e683614c51_C2func_iconv': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:47: undefined reference to
libiconv'
C:\Users\ADMINIcgo_74e683614c51_C2func_iconv_close': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:62: undefined reference to
libiconv_close'
C:\Users\ADMINI1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function 1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function cgo_74e683614c51_C2func_iconv_open': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:78: undefined reference to
libiconv_open'
C:\Users\ADMINIcgo_74e683614c51_Cfunc_iconv': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:105: undefined reference to
libiconv'
C:\Users\ADMINI1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function 1\AppData\Local\Temp\go-build665995231\github.com\qiniu\iconv_obj\iconv.cgo2.o: In function cgo_74e683614c51_Cfunc_iconv_close': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:118: undefined reference to
libiconv_close'
C:\Users\ADMINIcgo_74e683614c51_Cfunc_iconv_open': C:/Go/src/src/github.com/qiniu/iconv/iconv.go:132: undefined reference to
libiconv_open'
collect2.exe: error: ld returned 1 exit status
エラー: プロセスがコード 2 で終了しました
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.