28
爬取前端渲染网站网站(vue,react)
source link: https://studygolang.com/articles/31016
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
最近公司写爬虫 但是对于 前端渲染的网站(vue,react)
但是 chromedp selenium等又太重了
于是用puppeteer koa2 写了一个 通用服务
https://github.com/dollarkillerx/marionette
docker 运行
docker run --name marionette -d -p3000:3000 dollarkiller/marionette:latest
简单说一下这个服务的Rest API
GET /ssr?q=http://google.com
返回 respcode, html body, cookie 均为 目标网站的返回
我们现在使用Go来调用下这个API
Go的http client 推荐一下我自己写的urllib
https://github.com/dollarkillerx/urllib
httpCode, bytes, err = urllib.Get("http://0.0.0.0:3000").Querys("q","http://google.com").Byte()
有疑问加站长微信联系
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK