kata198 / advancedhtmlparser Goto Github PK

Hi, it seems that in commit ef7b130 renaming of AdvancedTag.classNames to AdvancedTag._classNames was not complete. I ran into an error in function IndexedAdvancedHTMLParser._indexClassName() since tag has no field classNames.
Is this correct? Are there any other not renamed occurrences of classNames?

First, great job. Really like AdvancedHTMLParser.

Second, a request or two: on the xpath front, is it possible to add support for the count() and concat() methods?

Thanks.

First of all, thanks for writing this library, it seems very useful!

I just ran pip install AdvancedHTMLParser with Python 3.7, and it indeed installed AdvancedHTMLParser 9.0.0 (according to AdvancedHTMLParser.version).

However, it only installed these files:

% ls lib/python3.7/site-packages/AdvancedHTMLParser/
Formatter.py
Parser.py
SpecialAttributes.py
Tags.py
Validator.py
__init__.py
compat.py
constants.py
conversions.py
exceptions.py
utils.py

Thus not the xpath subdirectory.

And indeed, I got this error:

>>> e.getElementsByXPath('thead')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/AdvancedHTMLParser/Tags.py", line 1949, in getElementsByXPathExpression
    from . import xpath as axpath
ImportError: cannot import name 'xpath' from 'AdvancedHTMLParser' (/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/AdvancedHTMLParser/__init__.py)

Is it possible that the package description is incomplete and is missing the xpath files?

Hi there!
I was trying to fuzz your interesting library as part of my university testing task, when I encountered interesting detail - there is possibility to open debugger at runtime.
I used the following code:

parser = AdvancedHTMLParser.AdvancedHTMLParser()
parser.parseStr(html)
parser.getHTML()

I started fuzzing it. At some point of fuzzing I figured out that program just stopped. At first, I thought it was some infinite cycle case. However, after some research I realized that it was pdb activation which forced program to wait for key input. The following appeared in console:

> /lib/python3.9/site-packages/AdvancedHTMLParser/Tags.py(1531)innerHTML()
-> return ''.join(ret)
(Pdb)

I traced back to the place where pdb was opened and found the following code snippet (Tags.py:1530):

try:
    return ''.join(ret)
except:
    import pdb; pdb.set_trace()
    return ''.join(ret)

This happened when processing of the following input was underway: ",pMêom><`e<adbiT�L".
This case is quite rare, for sure. Although I recommend that you fix it.

It seems that the attribute-name validation that was added in version 8.1.5 causes the ValidatingAdvancedHTMLParser to fail when it needs to validate HTML that includes embedded svg files.

Take e.g.:

import AdvancedHTMLParser

html = """
<div>
  <svg
    width="100%"
    height="100%"
    viewBox="0 0 100 100"
    version="1.1"
    xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    xml:space="preserve"
    style="fill-rule:evenodd;clip-rule:evenodd;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:1.5;"
  >
    <circle
      cx="50"
      cy="50"
      r="45"
      style="fill:#ff2711;stroke:#1f9f00;stroke-width:6px;"
    />
  </svg>
</div>
"""

parser = AdvancedHTMLParser.ValidatingAdvancedHTMLParser()
parser.parseStr(html)

Running the above with AdvancedHTMLParser 8.1.4 returns no errors, but with 8.1.5 and above it will return:

AdvancedHTMLParser.exceptions.InvalidAttributeNameException: Parsed a tag svg  which contains an invalid attribute, 'xmlns:xlink' = 'http://www.w3.org/1999/xlink' . ( Maybe characters outside quotes in tag? )

In more advanced SVG files the xlink can cause more errors, e.g. when used with a mask, e.g.:

<mask id="mask-2" fill="white" ><use xlink:href="#path-1" ></use></mask>

For now I can downgrade to a lower version, but that's not the best way forward. Can there be a way to except these errors for embedded SVG files?

there is error in importing AdvancedHTMLParser in python 3.7.
i used:
import AdvancedHTMLParser
but i got:

Traceback (most recent call last):
  File "J:\Ext_links.py", line 6, in <module>
    import AdvancedHTMLParser
ImportError: cannot import name AdvancedHTMLParser

and the AdvancedHTMLParser is installed!!

C:\Program Files\Python 3.7\Scripts>pip install AdvancedHTMLParser
Requirement already satisfied: AdvancedHTMLParser in c:\program files\python 3.7\lib\site-packages (8.1.6)
Requirement already satisfied: QueryableList in c:\program files\python 3.7\lib\site-packages (from AdvancedHTMLParser) (3.1.0)

Hi,
Im having problem when i'm trying to get element with classname when there is a "-" or space in class name.

see code sample

`
#!/usr/bin/python

import AdvancedHTMLParser
import requests
r = requests.get('SOMEURL')
parser = AdvancedHTMLParser.AdvancedHTMLParser()
parser.parseStr(r.text)
e1=parser.getElementsByClassName('body-content')

print (e1[0].innerHTML)
`

Hello, thank you for this good project.

is it possible to parse an html that is inside javascript ?
i tried to getElementsByClassName('nc_ensemble') but the TagCollection was empty

<script type="text/javascript">
  var textePage = new Array();
  textePage[0] = new Array();
  textePage[0]['Texte'] = '
  <div class="nc_ensemble">
    <div id="popupCourrierNoble" class="nc_notificationEnsemble nc_notification_apparition" style="display:none;">
      <div class="nc_notificationContent">
          Vous devez être noble de robe ou d\'épée pour accéder à cette fonctionnalité !\n
      </div>
    </div>
  </div>'
</script>

How can i get the src of an image tag (img) using this libary iin two cases:

• when the src is a url
• when the src is a base64 encoded image

thanks

Hi!

Nice library.

I have only one small problem. When i try print tag which contain attribute with None value, print fail.

Here is simple code to reproduce that:

import AdvancedHTMLParser

parser = AdvancedHTMLParser.AdvancedHTMLParser()
parser.parseStr('<div itemscope itemtype="http://schema.org/Order">')
parser.getHTML()

Returnung:

Traceback (most recent call last):
  File "/home/jiri/userconfig/mail/error.py", line 5, in <module>
    parser.getHTML()
  File "/tmp/p/venv/lib/python3.11/site-packages/AdvancedHTMLParser/Parser.py", line 887, in getHTML
    return doctypeStr + rootNode.outerHTML
           ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str

Observed on python 3.11.5 (Arch linux) with up-to-date package from pip.