Giter Club home page Giter Club logo

chromedp-undetected's Introduction

Undetected chromedp

Go.Dev reference Go Report Card Unit Tests GitHub

A small library that provides a chromedp context with a browser configured to mimick a regular browser to prevent triggering anti-bot measures. This is not a fool proof method, and how you use it will still dictate whether you will run into anti-bot detection, but at least it won't trigger on all the basic detection tests.

The headless option only works on linux, and requires Xvfb to be installed. Could theoretically work on Mac OS with xquartz but I don't have a Mac to test with, so feel free to PR.

A Docker container example is provided in Dockerfile. The most important things to note is to not use the headless chrome image as base, but to normally install chrome or chromium, and to install xvfb. Note that this image is neither secure nor optimized, and merely serves as an example.

package main

import (
	"time"

	"github.com/chromedp/chromedp"

	cu "github.com/Davincible/chromedp-undetected"
)

func main() {
	// New creates a new context for use with chromedp. With this context
	// you can use chromedp as you normally would.
	ctx, cancel, err := cu.New(cu.NewConfig(
		// Remove this if you want to see a browser window.
		cu.WithHeadless(),

		// If the webelement is not found within 10 seconds, timeout.
		cu.WithTimeout(10 * time.Second),
	))
	if err != nil {
		panic(err)
	}
	defer cancel()

	if err := chromedp.Run(ctx,
		// Check if we pass anti-bot measures.
		chromedp.Navigate("https://nowsecure.nl"),
		chromedp.WaitVisible(`//div[@class="hystericalbg"]`),
	); err != nil {
		panic(err)
	}

	fmt.Println("Undetected!")
}

Based on undetected-chromedriver

Utilities

Some utility functions are included I was missing in chromedp itself.

// BlockURLs blocks a set of URLs in Chrome.
func BlockURLs(url ...string) chromedp.ActionFunc 

// LoadCookies will load a set of cookies into the browser.
func LoadCookies(cookies []Cookie) chromedp.ActionFunc

// LoadCookiesFromFile takes a file path to a json file containing cookies,
// and loads in the cookies into the browser.
func LoadCookiesFromFile(path string) chromedp.ActionFunc

// SaveCookies extracts the cookies from the current URL and appends them to
// provided array.
func SaveCookies(cookies *[]Cookie) chromedp.ActionFunc

// SaveCookiesTo extracts the cookies from the current page and saves them
// as JSON to the provided path.
func SaveCookiesTo(path string) chromedp.ActionFunc

// RunCommand runs any Chrome Dev Tools command, with any params.
// 
// In contrast to the native method of chromedp, with this method you can
// directly pass in a map with the data passed to the command.
func RunCommand(method string, params any) chromedp.ActionFunc

// RunCommandWithRes runs any Chrome Dev Tools command, with any params and
// sets the result to the res parameter. Make sure it is a pointer.
// 
// In contrast to the native method of chromedp, with this method you can
// directly pass in a map with the data passed to the command.
func RunCommandWithRes(method string, params, res any) chromedp.ActionFunc

// UserAgentOverride overwrites the Chrome user agent.
// 
// It's better to use this method than emulation.UserAgentOverride.
func UserAgentOverride(userAgent string) chromedp.ActionFunc

// SendKeys does the same as chromedp.SendKeys excepts it randomly waits 100-500ms
// between sending key presses.
func SendKeys(sel any, v string, opts ...chromedp.QueryOption) chromedp.ActionFunc

chromedp-undetected's People

Contributors

davincible avatar dependabot[bot] avatar

Stargazers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.