emiln / cljck Goto Github PK

An OS-agnostic Clojure library for automating keyboard and mouse actions.

License: GNU General Public License v3.0

Clojure 100.00%

cljck's Introduction

Cljck

An OS-agnostic Clojure library for automating keyboard and mouse actions.

Cljck provides two distinct ways of automating various actions related to keyboard, mouse, and the visual content on your screen.

This is a regular library, which can be used as you write your own Clojure or Java program to solve specific tasks.
The JAR file resulting from building the library can process script files written in EDN, which expose the most generally useful functions of the library in a way where you don't have to compile any code to write small scripts. You simply pass text files to the JAR on the command line.

You may at this point be left with a few questions. Even if you're not, I took the liberty of making some up.

Why didn't you just use AutoIT?

It's not available on my favorite OS and I didn't want to learn a new language.

Do you expect me to learn your strange DSL?

You can use it as a library to write Clojure/Groovy/Java/etc. programs or you can treat it as an interpreter of a simple DSL. If both options sound unappealing, you're probably not the intended consumer of this product.

Requirements

You will need a Java Runtime Environment of version 1.8. Will it work with Java 1.7? I have no idea. Will it work with Java 1.6? Absolutely not.

If you want to build the project yourself you will need Boot.

Motivation

I need to beat friends and family in Clicker Heroes. This is proving an excellent game for driving progressive enhancements of Cljck, as there's always one more thing you could be automating. Ideally it will eventually be possible to handle everything including ascensions, and I can just leave a laptop playing the game indefinitely until my accumulated Hero Souls make my friends weep with frustrated jealousy.

Library Usage

Just require the project from Clojars.

Note: It's not actually on Clojars yet.

DSL Usage

Assuming you have the executable cljck-x.y.z.jar in your current directory, and have written the script click-the-stuff.edn and placed it in the same directory, just run

java -jar cljck-x.y.z.jar click-the-stuff.edn

in your terminal emulator. Passing multiple files will run all the scripts, interleaving their execution in non-deterministic sequence. This is usually perfectly fine, though, and is precisely how I use it.

DSL API

Each EDN file should contain exactly one top-level element. Why not two? Because right now it will only process the last top-level expression found due to laziness on my part.

Every expression should be a vector starting with a keyword and containing zero or more arguments. The keyword will serve as the name of a function to call, and the remaining arguments will be passed to this function. The keywords and their possible argument arities are as follows:

Keyword	Arguments	Description
`:click`		Clicks the left mouse button.
`:if`	`condition` `if-expression` `else-expression`	Evaluates the condition. If it is true, process the if-expression. If not, process the then-expression.
`:move-to`	`x` `y`	Moves the mouse cursor to position [x, y].
`:pointer-near`	`x` `y` `distance`	Evalutes to true if the mouse cursor is currently withing the given distance of the point [x, y].
`:press`	`key-string`	Presses the keyboard key given by the key string. See Oracle's documentation about the specifics of this format.
`:repeat`	`n` `expression+`	One or more expressions should be supplied. These will be repeated n times in the sequence they appear.
`:repeatedly`	`expression+`	One or more expressions should be supplied. These will be repeated indefinitely in the sequence they appear.
`:scroll-down`	`n`	Scrolls down n ticks of the mouse wheel. You can omit the argument, which will be interpreted as meaning 1 tick.
`:scroll-up`	`n`	Scrolls up n ticks of the mouse wheel. You can omit the argument, which will be interpreted as meaning 1 tick.
`:wait`	`n`	Pauses execution of the script for roughly n miliseconds.
`:when`	`condition` `expression+`	Evalutes the condition. If it is true, processes all expressions following the condition. If not, does nothing.

Example

Here's an example using all the keywords, but not making much sense.

[:repeatedly
 [:if [:pointer-near 100 100 50]
  [:move-to 500 500]
  [:move-to 100 100]]
 [:repeat 100
  [:click]
  [:scroll-down 5]]
 [:when [:pointer-near 500 500 50]
  [:scroll-up]]
 [:wait 1000]]

It reads as follows.

Repeat the following indefinitely:

Is the mouse cursor within 50 pixels of the point [100, 100]?

Yes: Move it to [500, 500]. No: Move it to [100, 100].
Repeat 100 times:
- Click the left mouse button.
- Scroll down 5 ticks.
Is the mouse cursor within 50 pixels of the point [500, 500]?

Yes: Scroll up 1 tick.
Wait for 1000 miliseconds.

cljck's People

Contributors

Stargazers

Watchers

cljck's Issues

Update JNativeHook

The author seems to have released a new version, which should no longer print the GPL excerpt to system out.

Allow the processing of multiple script files.

It'd be great to be able to divide your scripts into multiple files and launch any number of them at once. They should interleave as you'd expect (which is to say chaotically) and no synchronization mechanism is intended just yet. The robot will attempt to queue the commands up and to the best of its ability process them as they arrived.

Boot task to create uberjar capable of processing EDN files

There should be a boot task that creates an uberjar. This uberjar should be supplied a file name on the command line and should process this file as EDN commands.

Move the mouse to given position

It should be possible to call something (move-to 100 100) to move the cursor to the absolute position [100, 100]. It should work concurrently with clicks and you'd expect the move to happen in a sequence like the following:

Press button. Release button. Move mouse. Press button. Release button.

That is, movement should not happen during a click (which is both press and release).

Ability to change clicking pace.

While clicking away, it is sometimes necessary to click slower or faster, depending on what the flash game can handle on the given machine you're using. It should be possible to change this without having to stop anything.

Compare area on screen to histogram

In order to write logic for visual content on the screen, a simple start would be the ability to express a condition like:

Take a histogram of the area [x, y, width, height]. Is this within [distance] of [other histogram]?

As histograms were already implemented with #11 this is actually mostly done. A few tasks remain:

Allow script authors to write literal histograms. Perhaps [:histogram 0xffff...].
Allow script authors to generate a histogram of a portion of the screen. Perhaps [:histogram-at x y width height].

Conditionals

It would be great to have simple if-then-else conditionals in the scripting language. The first really useful condition could be something along the lines of:

[:when [:pointer-near 500 500 50]
 [:click]]

This would process the body if the mouse pointer is within 50 pixels of the point [500, 500]. There should be similarl functionality for :if, but taking both and if and an else branch.

Locate upper-left corner of ClickerHeroes window

It would be great to have the bot locate the upper left corner of the Clicker Heroes window. This would allow movement commands relative to the inside of the game frame. I imagine the commands [:relative-move-to 100 100] and [:absolute-move-to 100 100] to replace the current [:move-to 100 100].

Add `with-game-borders` command

The idea is that it'd be nice to tell Cljck about the border of the game window, which would enable relative positions rather than absolute positions. In this fashion people playing the same game on different monitors could share their scripts without having to translate all the coordinates.

I imagine something along the lines of:

(with-game-borders
  (move-to 10% 10%)
  (click))

This should make Cljck prompt the user to click the lower left and then upper right corner of the game window to log the coordinates. Any attempt to use relative coordinates outside of with-game-borders should be an error.

Add "with-killswitch" command

The idea is that you can write a script like

(with-killswitch "shift ESCAPE"
  ...)

which puts the choice of the killswitch key (combination) into the hands of the user.

Add the library on Clojars

Your library looks very cool, yet I would like to use it with Clojure to make a bot too.

Would you mind puting it on clojars so that I can include it in my project ? (Or tell me the step so that I can do it myself)

Thanks

Global keybinding to stop processing events

It's almost impossible to stop the robot if you've set up a lot of movement and clicking. It would be great to have a global keybinding that would immediately stop event processing.

Click x times per second

It should be possible to have cljck repeatedly simulate a mouse click x times per second.

Issue with "kill switch"

Using Escape to kill the bot only seems to work about 10 times on Windows, after which pressing the key has no effect and the terminal may start spouting error messages about being unable to bind... something.

As an angry wife to a hardened Clicker Heroes addict I'd like to be able to regain control of the computer, which is frantically moving and clicking the mouse cursor about.

Fix the removal of gen-class

On the bright side I now recall why it's there.

Process files with scripted events.

It would be neat to be able to save various routines in text files and have cljck process them. The format should be simple data, and EDN seems an excellent fit. It would be simple if the format could be precisely the same as the internal format passed to the event-channel A script could then look as follows:

[:repeat
 [:click]
 [:move-to 500 1200]
 [:click]
 [:click]]

To fit as a single command, I imagine composite events like :repeat will be needed. Repeating stuff over and over again seems to be my primary use case, anyway. This brings the sum total of tasks in this issue to:

Implement a :repeat event that has two function signatures: [& body] and [repetitions & body].
Write a function that processes an EDN file.

Make an 'always on top' GUI with a button to start and stop clicking.

This would make it easier to control the aggressive clicking. A nice additional feature would be an input of some kind allowing you to choose the number of clicks per second.

Remove the GUI

It's not used for anything right now and I have no plans to change that. Removing it could help slim down the JAR slightly.

Write a README

It should touch upon topics like:

What's this project about?
How do I use it?
How do I write script files?

Symbols instead of keywords?

I wonder if perhaps symbols would be easier for people to relate to for the EDN DSL. The current example in the README is

[:repeatedly
 [:if [:pointer-near 100 100 50]
  [:move-to 500 500]
  [:move-to 100 100]]
 [:repeat 100
  [:click]
  [:scroll-down 5]]
 [:when [:pointer-near 500 500 50]
  [:scroll-up]]
 [:wait 1000]]

but it could just as easily be

[repeatedly
 [if [pointer-near 100 100 50]
  [move-to 500 500]
  [move-to 100 100]]
 [repeat 100
  [click]
  [scroll-down 5]]
 [when [pointer-near 500 500 50]
  [scroll-up]]
 [wait 1000]]

or even

(repeatedly
 (if (pointer-near 100 100 50)
  (move-to 500 500)
  (move-to 100 100))
 (repeat 100
  (click)
  (scroll-down 5))
 (when (pointer-near 500 500 50)
  (scroll-up))
 (wait 1000))

Ability to assess similarity of images.

It would be great to be able to express flows like: If this region of the screen is similar to this old version of it, perform some command. One measure for similarity could be simple color histograms. Here's my thought on a first version:

Similarity is calculated by:

Creating 8 buckets for each of red, green, and blue. The first is for values [0 31], the next [32 63] etc.
For each pixel considered, increment the proper bucket for its red, green, and blue color value.
Normalize all buckets relative to the number of pixels considered.
Calculate the absolute differences in normalized buckets values between the two images.
Sum up the differences.

Note that this means you can save a "fingerprint" of an image by saving its bucket values.

Add "while holding" command

It would be nice to be able to write scripts like:

(while-holding "Shift"
  (click))

I suggest the first argument for while-holding would be the same type as the existing key press commands.

Send key presses

To use the abilities with cool downs, you need to be able to simulate key presses.

Ability to scroll

It should be possible to call

[:scroll-up 5]

to scroll five "ticks" up, and similar for :scroll-down. I think it'd be nice to have a no-argument default of 1 so you don't have to supply a 1 all the time, which I imagine to be the typical usage.