Giter Club home page Giter Club logo

local-api-client-python's Introduction

Kameleo Local API Client

With Kameleo, you can easily create multiple virtual browser profiles to work with multiple accounts. It helps you hide your actual timezone, geolocation, language, IP address and creates natural browser fingerprints to prevent detection by anti-bot systems. Kameleo is compatible with Selenium, Playwright, and Puppeteer frameworks for automating web scraping tasks. This Python package provides convenient access to the Local API REST interface of the Kameleo Client. See the article in our knowledge base for Getting Started with Kameleo Automation.

Features

  • Stay completely undetected, so websites won’t be able to detect that you are using automation tools
  • Start unlimited number of profiles with different natural browser fingerprints
  • Use authenticated HTTP/SOCKS/SSH proxies in browsers
  • Create isolated browsing environments simultaneously
  • Use real browser profiles of Chrome, Firefox, Safari and Edge
  • Edit, Import or Export browser cookies
  • Modify WebRTC parameters
  • Modify Geolocation settings
  • Modify Timezone and Language settings
  • Modify WebGL fingerprint
  • Modify 2D Canvas fingerprint
  • Modify Navigator properties
  • Modify Screen resolution

Note: You need Automation package of Kameleo to access the features described below.

Quickstart Guide

1. Install package

pip install kameleo.local_api_client

2. Start the Kameleo.CLI on your computer

./Kameleo.CLI.exe email="[email protected]" password="Pa$$w0rd"

3. Start a browser with out-of-the-box fingerprinting protection

from kameleo.local_api_client import KameleoLocalApiClient
from kameleo.local_api_client.builder_for_create_profile import BuilderForCreateProfile

client = KameleoLocalApiClient()
base_profiles = client.search_base_profiles(
    device_type='desktop',
    browser_product='chrome'
)

# Create a new profile with recommended settings
# for browser fingerprinting protection
create_profile_request = BuilderForCreateProfile \
    .for_base_profile(base_profiles[0].id) \
    .set_name('example profile') \
    .set_recommended_defaults() \
    .build()
profile = client.create_profile(body=create_profile_request)

# Start the browser
client.start_profile(profile.id)

# At this point you can automate the browser with your favorite framework

Automate Kameleo profiles with Selenium

Kameleo gives you the ability to control any supported browser using Selenium. It uses the WebDriver protocol, a W3C specification, and industry-standard to interact with a browser.

You need to install the official Selenium package.

from selenium import webdriver
# Connect to the running browser instance using WebDriver
kameleo_port = 5050
options = webdriver.ChromeOptions()
options.add_experimental_option('kameleo:profileId', profile.id)
driver = webdriver.Remote(
    command_executor=f'http://localhost:{kameleo_port}/webdriver',
    options=options
)

# Use any WebDriver command to drive the browser
# and enjoy full protection from bot detection products
driver.get('https://google.com')

The full example can be found here.

Automate Kameleo profiles with Puppeteer (Chromium-based)

Kameleo lets you control Chromium-based browsers (sorry Firefox fans) using the Pyppeteer library. In this simple example you can see how to connect to the browser that Kameleo starts.

You need to import the Pyppeteer library.

import pyppeteer
# Connect to the browser with Puppeteer through CDP
kameleo_port = 5050
browser_ws_endpoint = f'ws://localhost:{kameleo_port}/puppeteer/{profile.id}'
browser = await pyppeteer.launcher.connect(browserWSEndpoint=browser_ws_endpoint, defaultViewport=False)
page = await browser.newPage()

# Use any Playwright command to drive the browser
# and enjoy full protection from bot detection products
await page.goto('https://google.com')

The full example can be found here.

Automate Kameleo profiles with Playwright

Kameleo allows you to control the browser with the official Playwright package. It works little bit different with Chromium-based browsers and Firefox, so we provide an example for both. Here we showcase how you can connect to the browser that is already started by Kameleo.

You need to import the official Playwright package.

import playwright
from playwright.sync_api import sync_playwright

You can find more details here: Using Kameleo with Playwright framework – Kameleo Support Center.

Chromium-based profiles with Playwright

# Connect to the browser with Playwright through CDP
kameleo_port = 5050
browser_ws_endpoint = f'ws://localhost:{kameleo_port}/playwright/{profile.id}'
with sync_playwright() as playwright:
    browser = playwright.chromium.connect_over_cdp(endpoint_url=browser_ws_endpoint)
    context = browser.contexts[0]
    page = context.new_page()

    # Use any Playwright command to drive the browser
    # and enjoy full protection from bot detection products
    page.goto('https://google.com')

The full example can be found here.

Firefox-based profiles with Playwright

# Connect to the browser with Playwright
kameleo_port = 5050
browser_ws_endpoint = f'ws://localhost:{kameleo_port}/playwright/{profile.id}'
with sync_playwright() as playwright:
    # The exact path to the bridge executable is subject to change. Here, we use %LOCALAPPDATA%\Programs\Kameleo\pw-bridge.exe
    executable_path_example = path.expandvars(r'%LOCALAPPDATA%\Programs\Kameleo\pw-bridge.exe')
    browser = playwright.firefox.launch_persistent_context(
        '',
        # The Playwright framework is not designed to connect to already running
        # browsers. To overcome this limitation, a tool bundled with Kameleo, named
        # pw-bridge.exe will bridge the communication gap between the running Firefox
        # instance and this playwright script.
        executable_path=executable_path_example,
        args=[f'-target {browser_ws_endpoint}'],
        viewport=None)

    # Kameleo will open the a new page in the default browser context.
    # NOTE: We DO NOT recommend using multiple browser contexts, as this might interfere
    #       with Kameleo's browser fingerprint modification features.
    page = browser.new_page()

    # Use any Playwright command to drive the browser
    # and enjoy full protection from bot detection products
    page.goto('https://google.com')

    # Here we need to close the browser object as well, it is not enough just to stop the profile
    client.stop_profile(profile.id)
    browser.close()

The full example can be found here.

Automate mobile profiles

Kameleo can emulate mobile devices in the custom built Chromium.

# Search for a mobile Base Profiles
base_profile_list = client.search_base_profiles(
    device_type='mobile',
    os_family='ios',
    browser_product='safari',
    language='en-us'
)

# Create a new profile with recommended settings
# Choose one of the Base Profiles
# Set the launcher to 'chromium' so the mobile profile will be started in Chroma browser
create_profile_request = BuilderForCreateProfile \
    .for_base_profile(base_profile_list[0].id) \
    .set_name('automate mobile profiles on desktop example') \
    .set_recommended_defaults() \
    .set_launcher('chromium') \
    .build()
profile = client.create_profile(body=create_profile_request)

# Start the profile
client.start_profile_with_options(profile.id, body={
    # This allows you to click on elements using the cursor when emulating a touch screen in the browser.
    # If you leave this out, your script may time out after clicks and fail.
    'additionalOptions': [
        {
            'key': 'disableTouchEmulation',
            'value': True,
        },
    ],
})

# At this point you can automate the browser with your favorite framework

The full example can be found here.

Example codes

Several examples have been prepared in a different repository to showcase the most interesting features. Feel free to create a pull request to add new example codes.

  • Finding base profiles
  • Creating profiles with custom options
  • Updating profiles with new settings
  • How to start a profile
  • Using Selenium with Local API
  • Using Playwright with Kameleo
  • Using Puppeteer with Kameleo
  • How to emulate mobile devices
  • Adding an HTTP, SOCKS or SSH proxy to profile
  • Saving/Loading a browsing session to/from a .kameleo file
  • Modify and Delete browser cookies
  • Start profile with extra WebDriver capabilities
  • How to duplicate virtual browser profiles
  • Refresh the browser of the emulated profiles

Note: If you are interested in more information about Kameleo, or have encountered an issue with using it, please check out our Help Center.

Endpoints

Available API endpoints with exhaustive descriptions and example values are documented on this SwaggerHub page. This package has built-in IntelliSense support in Visual Studio Code, no extra package installation needed.

Package

This package can be found on PyPI here: kameleo.local-api-client.

License

This project is released under MIT License. Please refer the LICENSE.txt for more details.

local-api-client-python's People

Contributors

kameleo-team avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

local-api-client-python's Issues

Multiprocessing

Hi,
Example multiprocessing in python? I only see in one port.
Thanks

Currently having trouble with the local api client whilst attempting to run multiple browsers at once.

I wanted to ask if something like this would work.
(if your local api client, can run multiple browsers at once than control them trough playwright)

from kameleo.local_api_client import KameleoLocalApiClient
from kameleo.local_api_client.builder_for_create_profile import BuilderForCreateProfile
from playwright.async_api import async_playwright
import asyncio

async def start_browser_profile(kameleo_port, profile_number):
client = KameleoLocalApiClient(
endpoint=f'http://localhost:{kameleo_port}/',
retry_total=0
)

base_profiles = client.search_base_profiles(
device_type='desktop',
browser_product='chrome'
)

create_profile_request = BuilderForCreateProfile \
.for_base_profile(base_profiles[profile_number % len(base_profiles)].id) \
.set_recommended_defaults() \
.build()

profile = client.create_profile(body=create_profile_request)
client.start_profile(profile.id)

browser_ws_endpoint = f'ws://localhost:{kameleo_port}/playwright/{profile.id}'
async with async_playwright() as p:
browser = await p.chromium.connect_over_cdp(endpoint_url=browser_ws_endpoint)
context = browser.contexts[0]
page = context.new_page()

# Use any Playwright commands with the page

# Stop the browser profile after use
client.stop_profile(profile.id)

async def main():
# This is the port Kameleo.CLI is listening on. Default value is 5050, but can be overridden in appsettings.json file
kameleo_port = 5050

# Create a list of tasks to start multiple browser profiles concurrently
tasks = []
num_profiles = 3

for i in range(num_profiles):
tasks.append(start_browser_profile(kameleo_port, i))

# Start the tasks concurrently
await asyncio.gather(*tasks)

if __name__ == '__main__':
asyncio.run(main())

List of base profiles

Hi. I have a question about profiles

Based on https://github.com/kameleo-io/local-api-examples/blob/2048fd86cbb542b465854bbccc9d5d47b27f2510/python/create_profile/app.py#L16C1-L16C14

It returns a random list of 25 profiles and I have an issue from time to time where I do not receive the necessary profile that I need created.

For example:

    base_profiles = client.search_base_profiles(
        device_type='desktop',
        browser_product='chrome',
        os_family='windows',
        language='en-us',

This is where I expected to see profile.browser.major == 122 and profile.os.version == '11', but occasionally there is no v122. Do you know how I can be sure I receive it back in 100% cases for the request?

Automate Mobile Profiles

Just fooling around with the mobile profiles, I was wondering if there is any way to connect and control my mobile device with selenium automation.

When I try connecting to the profile id selenium throws the error: "selenium.common.exceptions.SessionNotCreatedException: Message: This profile is using the External Spoofing Engine, so Selenium connection is not allowed.'

Is there any work around you are aware of or other language which makes automation of the mobile device through Kameleo achievable?

Profiles with different Chrome version

I noticed with automation the profiles are being created with the same chrome version, how do I get that to be unique and random?

I'm using your example code but every time Chrome is created. I need different versions

Last tab still opens on profile start

Hello,

I had a question about starting profiles, whether there is a way to open a new tab instead of loading up the previously saved session? Furthermore, if there is a way to do it, can be be done with in a manner where cookies persist across sessions?

Thank you!

Cookies import

I would like to import cookies with my profiles. How could I do that on a larger scale, since i have a lot of profiles?

One of them for example is:

Name: 2429 Cookies: [{"domain":".google.com","expirationDate":1756269164,"httpOnly":false,"name":"1P_JAR","path":"/","secure":true,"session":false,"value":"2023-08-14-09","sameSite":"no_restriction"},{"domain":".google.com","expirationDate":1707559018.287,"httpOnly":false,"name":"AEC","path":"/","secure":true,"session":false,"value":"Ad49MVFK4xJJRq2b9EBcL69RLW_Bd04C9377l4ztNJfMvOHr8l8KHd06Fw","sameSite":"lax"},{"domain":".wildberries.ru","expirationDate":1699955823.408,"httpOnly":false,"name":"BasketUID","path":"/","secure":true,"session":false,"value":"ea807926-b5be-4400-82ec-47dc42555435","sameSite":"lax"},{"domain":".google.com","expirationDate":1707818218.287,"httpOnly":false,"name":"NID","path":"/","secure":true,"session":false,"value":"511=a-fY-e7MJVYOiSRNMxls0RG3lkmQfRz2Nl8YzQRZgFXlngBZYAKcWv4lT-5pSUPABsz6OyhkRl75GlDth3CJfq-z9y2GXIoTRg1g-IXQ9GfziMj_7DyO5bW0vzme4uZwMNb2mPTKYLwrgGlaMPOrDp9t6NkgpJH8_x_KL2eJTqs","sameSite":"no_restriction"},{"domain":".wildberries.ru","expirationDate":1756269164,"httpOnly":false,"name":"wbs","path":"/","secure":true,"session":false,"value":"339aadf2-a949-4df1-aee9-3b206a40f61f.1692007029","sameSite":"lax"},{"domain":".wildberries.ru","expirationDate":1726567029.441,"httpOnly":false,"name":"wbu","path":"/","secure":true,"session":false,"value":"4a86c3cf-644c-4820-b755-923feb22701e.1692007029","sameSite":"lax"},{"domain":".wildberries.ru","expirationDate":1723543024,"httpOnly":false,"name":"wbauid","path":"/","secure":true,"session":false,"value":"9123079461692007024","sameSite":"lax"},{"domain":".wildberries.ru","expirationDate":1723543356.068,"httpOnly":false,"name":"WILDAUTHNEW_V3","path":"/","secure":true,"session":false,"value":"3C11CB17BE3BEE62CA29E5B73D672B8C44784EF440FA3FB753DFA171519A0D8A5F89C316298A846F53EF3AC2CFE94DAF0318C5E6FB328B895C7A1AB81EAD748CA45AD65BB1D26C08DAFC380FBC19EE0FF52F6A2CFF6AB31E5E83697C83A070E24F6CDCE83B104E3D150F8F1840F1702554A7E21763F992433310C40D86D0271EC6273F908846A6FD9F8005AAD45F707F04EE19F7E2FBD165FC57DE962DF1CA2BC3A60E9A8E46127C7BCC1084E34E43BA73938D245A53B336D07B53A4C19E74FE74E8D5F6DBEE64E4DB72426759EE61AD27BCEE35C119A905822740D2690BC10AB579BD1FC57E1C4EEE6F86DC56CD62B7CCFE997904A26617345AA0E76B947F6A30F2A0AF3343379D37AD9055AE5A52715DEB7C91026C99F449F766593C79B46C43F631440C29F2D543470A2345078E6476157ED8","sameSite":"lax"},{"domain":".wildberries.ru","expirationDate":1756269164,"httpOnly":false,"name":"um","path":"/","secure":true,"session":false,"value":"uid%3Dw7TDssOkw7PCu8KwwrPCs8KwwrHCtcK5wrPCuA%253d%253d%3Aproc%3D100%3Aehash%3Dd41d8cd98f00b204e9800998ecf8427e","sameSite":"lax"},{"domain":".wbx-auth.wildberries.ru","expirationDate":1723543358.378,"httpOnly":false,"name":"wbx-validation-key","path":"/v2/auth","secure":true,"session":false,"value":"a4bab65e-2e30-4b43-a69b-53b1ca001f96","sameSite":"lax"},{"domain":""_wba_s","path":"/","secure":false,"session":true,"value":"1","sameSite":"unspecified"},{"domain":"ru-basket-api.wildberries.ru","expirationDate":-11644473600,"httpOnly":true,"name":"routeb","path":"/","secure":false,"session":true,"value":"1696565586.54.333.934027|c4b1652c8c0c161d421c5b735e35af14","sameSite":"unspecified"},{"domain":".wbx-auth.wildberries.ru","expirationDate":1728616411.45763,"httpOnly":true,"name":"w..."}]

Automatic WebGL Metadata Spoofing

This question was asked in our support system. We thought it may worth sharing it

How I can do create profiles without manual webgl? Kameleo have in GUI a "automatic"-function for random webgl options, but how I can set this in API because i cant select automatic in WebglSpoofingOptions?
".set_webgl("noise", WebglSpoofingOptions(vendor="none", renderer="none")) \"

Update proxy on loaded profile

This question was asked in our support system and we thought it worth sharing:

I was wondering if and how can I change the proxy using the update function,
I know of the function but I don't know the Property/format to change the proxy Ip:port of a loaded profile

Python Question

So I am trying to use Kameleo in my python script that goes to a website and logs into it. I am saving the profile and then loading back when executing the script next time. I want a functionality check if the profile was deleted from Kameleo during the last execution of the script and if it still exists then the script will first delete it and then load it again...

Failed to create connection

Hi,

When we start a profile, the browser starts but shortly after it terminates.

An unhandled exception has occurred while executing the request.
Application.Services.ProfileLifetimeHandler.ProfileLifetimeHandlerException: Failed to start profile.
---> PuppeteerSharp.ProcessException: Failed to create connection
---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server
---> System.Net.Http.HttpRequestException: The proxy tunnel request to proxy 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.