cnbattle / douyin Goto Github PK

View Code? Open in Web Editor NEW

1.1K 29.0 235.0 38.4 MB

抖音推荐/搜索页视频列表视频爬虫方案,基于app(虚拟机或真机) 相关技术 golang adb

License: MIT License

Go 85.07% Makefile 14.93%

golang go douyin

douyin's Introduction

douyin's People

Contributors

Stargazers

Watchers

Forkers

kingking888 masdude hailongeric zhuoziyu awesome-archive melice royswale liyongshen tjtb2018 gc888 secretwiki crazykylin zzy0302 sublime-cn youmuyou zarte howardyan93 magichain pangxiaobin codecopy damonxiao154 88886 dulumao zjz6b6 viyiu linuer shaunstanislauslau bytoola hhy5277 mbyase myhololens senjing333 jvedi crackercat bluejad yamingd aidejiusnxx shuyabin cathing lyrl miltonqiu cyprestar xiaolonglee andy521 wangdongya wyrover gilgameshzzz temptat 872409 riseinrose csu-anzai borisbao a-lone bryanjamiejaramiello0721 xxhantian catmonkeylee lancewalker91 quincyc379 xiaobaojiang csu-xiao-an yuruyigit kylehuang 1514louluo songrat test4cz litttley jhs2jhs lili0710432 kevinsir cjiajiazhuiqiu asdlei99 yale8848 kisskeyoyo wf1314 sujunming suskycode cnhuzi yxb1987 zztong goforkdemo kyxkbbs banben beikee kingsun0 lovethemefromtiga jamesphe keycodinggood pengjinfu kmend cxapython ryanfu l1-j5n whitespur codeeps sunqiang25 eddylapis w08 fyns netskyline chenkaigithub

douyin's Issues

收集测试可用的android版本

exit status 2

exit status 2 是什么问题

mitm-proxy.crt安装问题

您项目中有一个mitm-proxy.crt文件，请问这个是不是可以直接被安装在模拟器中的证书？可是我始终没有找到安装入口，求告知。

Convert Int to String Error

douyin/internal/core/core.go

Line 44 in 89e9eb0

video.DiggCount = string(item.Statistics.DiggCount)

douyin/internal/core/core.go

Line 45 in 89e9eb0

video.CommentCount = string(item.Statistics.CommentCount)

Here should use strconv.Itoa(int) to convert int to string.

模拟器采集

我使用的是夜游模拟器，安装好环境以后，可以正常运行 main.go 也在模拟器中安装了证书，抖音版本是你提供V7.0
但现在打开抖音没办法实现采集。不知道是哪里设置错了

现在还能用吗？我用mimtproxy 行不通了

希望可以增加对搜索列表的抓取，搜索的内容更精准一些

连接代理后，抖音客户端就提示没有网络了，手机浏览其他网页没问题，是抖音机制改了吗？

有没有爬91的？

考虑支持下mac？

抖音检测到了anyproxy，自动断开连接？

[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request p1-dy.byteimg.com
[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request aweme.snssdk.com
[AnyProxy Log][2019-10-03 22:25:03]: will forward to local https server
[AnyProxy Log][2019-10-03 22:25:03]: will forward to local https server
[AnyProxy Log][2019-10-03 22:25:03]: [internal https]proxy server for p1-dy.byteimg.com established
[AnyProxy Log][2019-10-03 22:25:03]: [internal https]proxy server for aweme.snssdk.com established
[AnyProxy ERROR][2019-10-03 22:25:03]: Error: read ECONNRESET
[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request p3-dy.byteimg.com
[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request p9-dy.byteimg.com
[AnyProxy Log][2019-10-03 22:25:03]: will forward to local https server
[AnyProxy Log][2019-10-03 22:25:03]: will forward to local https server
[AnyProxy Log][2019-10-03 22:25:03]: [internal https]proxy server for p3-dy.byteimg.com established
[AnyProxy Log][2019-10-03 22:25:03]: [internal https]proxy server for p9-dy.byteimg.com established
[AnyProxy ERROR][2019-10-03 22:25:03]: Error: read ECONNRESET
[AnyProxy ERROR][2019-10-03 22:25:03]: Error: read ECONNRESET
[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request p9-dy.byteimg.com
[AnyProxy Log][2019-10-03 22:25:03]: received https CONNECT request p1-dy.byteimg.com

Mac OS10.14, anyproxy4.1.0, 抖音8.1.1

10月3号晚上尝试运行，发现连接anyproxy后抖音无法联网，不知道是抖音限制或者anyproxy问题。

建议吧anyproxy换掉，一套go

代码疑问，main.go文件

入口main.go文件，MAIN函数里面，select {}函数实现似乎没有？好像无法正常跑起来吧？

拦截的请求是不是对android7.0以上无效

安卓7.0之后，不在信任用户的证书了，问下这个是怎么弄的。
另外使用模拟器的教程有没有?