Giter Club home page Giter Club logo

Comments (5)

constverum avatar constverum commented on May 13, 2024

Yes, this feature is in the TODO. I think the only way to implement this is to compare a checksum. If anyone have other options I'm ready to discuss =)

from proxybroker.

cyian-1756 avatar cyian-1756 commented on May 13, 2024

Yes, this feature is in the TODO.

Well I completely missed it, sorry about that.

I think the only way to implement this is to compare a checksum.

You might also be able to do size and number of files returned as well. Also sending data thru the proxy and seeing if it was changed on it's way to it's destination would be another way to tell if the proxy was messy with the traffic

from proxybroker.

erm3nda avatar erm3nda commented on May 13, 2024

Theres no real way to compare websites 100% because each html will be rendered different in each request. Checksum will fail so often.

An idea will be to save a list of web elements (the html dom names, not their content) and compare the main structure. You will notice changes for ads or div created between direct and proxy requests.

from proxybroker.

Wilbo007 avatar Wilbo007 commented on May 13, 2024

@erm3nda or you could just request a static web page retard

from proxybroker.

synchronizing avatar synchronizing commented on May 13, 2024

@erm3nda I partially agree with this comment as checksum doesn't have to be index.html specific. Another method would be to do a checksum on the website's resources; i.e., its main .css file, js files, etc. Whatever the case -- whether it is checksum or specific HTML -- there are only two ways this can be achieved:

  • Statistically, by verifying > ~10% of proxies respond with the same DOM/checksum request.
  • Request from the local machine that saves either checksum or DOM state to be verified against all proxies. This method, however, exposes the IP of the user using ProxyBroker.

Another thing to mention is the fact that not all websites are rendered the same depending on the language. Take NYTimes.com, for example:

Both websites look fairly similar, but it seems that their HTML is rendered differently. Therefore, I believe, it would be preferred the client serves a proper Accept-Language header and verifies the Content-Language in the checking of the page on top of whatever other mechanism is being done to verify the page.

from proxybroker.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.