Use this VKontakte captcha breaker with 91% accuracy right in browser with chrome extension, Python, JS, 2020

Images in Demo or Sandbox are broken? Solution

• Features
• Installation
  • Windows/Linux/MacOS
  • Android
• Description
• DataMining
• ONNXTroubles
• Results
• Sources

Currently supported actions (on mobile domain m.vk.com only):

• messages ✉️
• likes ❤️
• comments 💬
• adding to friends 🧑‍🤝‍🧑

Please, note that captcha on login page and promocode page is not supported!

Update v1.5: recognition in background, Firefox addon

Update v1.4: added support for other captcha scenarios

Update v1.3: now it works for mobile domain only (m.vk.com)

Google Chrome Extension and Firefox Addon for VKontakte which utilizes trained AI models to predict captcha from image and automatically enters it.

Currently only captcha from messages are supported (no likes or comments for you, my bad)

You can download latest version from: VKCaptchaBreaker.crx

To install the extension, simply follow this tutorial:

• For Chrome: www.cnet.com/how-to/how-to-install-chrome-extensions-manually

• For Firefox: www.muo.com/tag/how-to-install-firefox-add-ons-manually-even-from-github

Finally, activate it:

To install the extension, simply follow this tutorial:

qwikfone.medium.com/how-to-install-chrome-extension-on-android-3cd89f3a771c

Installation and activation example:

If something won't work, just reactivate extension.

Keras implementation of Convolutional Recurrent Neural Network with ONNX.js support. To achieve better accuracy, CRNN utilizes CTC loss and Knowledge distillation with Label smoothing. CRNN comes in two versions: the one BIG with recurrent layers (RNNs) and the other small one without them.

The reason for doing so is that if we want to use trained models on end devices, such as Mobile devices/Client-side browsers, we want to get a model with a minimum number of parameters.

For the BIG version, Google's Inception-like CNN encoder is used with Bidirectional GRUs as RNN decoder. For the small version, the ShuffleNetV2 encoder is used with 1d convs instead of RNN cells.

To parse captcha you can use any URL that Vkontakte provided, e.g. https://vk.com/captcha.php?sid=625944628258&s=1 Here 625944628258 is a random seed. It's important to mention that some seeds will generate old captchas with Cyrillic symbols. Using URLs with different seeds over 1.5 million images was parsed.

At first 5000 captcha's was manually labelled (that process take approximately 1 year). Thus, an initial dataset was created, let's call it Dataset5k. Next, on this dataset, BIG CRNN was trained and achieved 80% accuracy on unseen data. After that using Vkontakte account without phone verification more images were automatically labelled online. Thus, a new dataset was created, and the process continues.

After achieving 99% accuracy with BIG CRNN model, it was used for labelling the rest of 1.5 million images offline. Datasets with 850k images and 1500k images were constructed this way. This huge "filthy" datasets (implying that there are mistakes in predictions) were used for training small pseudoCRNN model.

Unfortunately, in the case of using models on Client-side, e.g. with JavaScript, existing frameworks (Open Neural Network Exchange (ONNX.js)) does not support RNNs. The only one which supports them is Tensorflow.js, but it is slower. It's important to note that RNNs also are quite slow due to its nature.

To mitigate this problem this project introduces a small pseudoCRNN architecture without RNN blocks. Instead of bidirectional Gated Recurrent Units (GRU) as in the BIG version, it uses one-dimensional convolutions. Other troubles:

• Can't use dynamic reshaping, e.g Reshape(None, -1, 32) will crash
• Can't use 1d Convolutions, instead, the following trick was used:

(None, 32, 128) --Reshape--> (None, 32, 1, 128) --Conv2d--> (None, 32, 1, filters) --Reshape--> (None, 32, filters)

• Can't use Droupout, it's important to reconstruct model without this layer
• Can't use Casting operation in ONNX.js, e.g. INT32 -> FLOAT32 conversion
• Can't use Ragged Tensors in ONNX.js, e.g. for CTC algorithm
• Can't use tf.where, because Nonzero is not implemented in ONNX.js

You can check the list of supported operators in ONNX.js HERE

ONNX is a great thing, but its JavaScript Framework needs more attention from Microsoft developers.

As a result, in the next project, Tensorflow.js should be used instead of ONNX.js

After training on 1.5 Million filthy labeled captcha for 300k steps, the student model achieved ~90% accuracy (while teacher accuracy was 99%)

Comparision:

* - means that dataset was made with filthy labels from the teacher

** - after conversion the final size of the model is 1 MB

Thus, you can achieve up to 10 times faster inference time with the small version.

Optimized pseudoCRNN student architecture after convertion to ONNX format:

This project was inspired by number-plate recognition work and uses its modified version of CRNN implementation, @RIA.com.

ONNX.js reference example, created by @elliotwaite.

For small pseudoCRNN CNN encoder, this system uses a modified version of ShuffleNetV2, implemented by @opconty.

Tensorflow Dataset class with caching implementation is the modified version of @krasserm.

For the model's graph visualization was used the Netron application, by @lutzroeder